一、使用autotrain-advanced微调LLAMA-2

AutoTrain是一种无代码工具#xff0c;用于为自然语言处理#xff08;NLP#xff09;任务、计算机视觉#xff08;CV#xff09;任务、语音任务甚至表格任务训练最先进的模型。

一、使用autotrain-advanced微调LLAMA-2

AutoTrain是一种无代码工具用于为自然语言处理NLP任务、计算机视觉CV任务、语音任务甚至表格任务训练最先进的模型。

!pip

如果数据是表格需要指定instructions和responses对应的列名

use_peft:

llama2-autotrain-openassitant--model

TinyPixel/Llama-2-7B-bf16-sharded--data_path

timdettmers/openassistant-guanaco--text_column

text--use_peft--use_int4--learning_rate

3--num_train_epochs

trojrobert/llama2-autotrain-openassistant--block_size

1048

TRL是一个全栈库提供了通过强化学习来训练transformer语言模型一系列工具包括从监督微调步骤SFT、奖励建模步骤RM到近端策略优化PPO步骤。

!pip

timdettmers/openassistant-guanaco​dataset

splittrain)

bnb_4bit_compute_dtypetorch.float16,)​​#

download

TinyPixel/Llama-2-7B-bf16-shardedmodel

AutoModelForCausalLM.from_pretrained(

model_name,

trust_remote_codeTrue)model.config.use_cache

False

AutoTokenizer.from_pretrained(model_name,

trust_remote_codeTrue)tokenizer.pad_token

tokenizer.eos_token

./resultsper_device_train_batch_size

4optim

per_device_train_batch_sizeper_device_train_batch_size,

gradient_accumulation_stepsgradient_accumulation_steps,

optimoptim,

lr_scheduler_typelr_scheduler_type,)

7创建SFTTrainer配置

trainingmodel_to_save.save_pretrained(outputs)

lora_config

LoraConfig.from_pretrained(outputs)tuned_model

lora_config)

return_tensorspt).to(device)outputs

max_new_tokens50)print(tokenizer.decode(outputs[0],

参考文献

https://trojrobert.medium.com/4-easier-ways-for-fine-tuning-llama-2-and-o***r-open-source-llms-eb3218657f6e

[2]

https://colab.research.google.com/drive/1JMEi2VMNGMOTyfEcQZyp23EISUrWg5cg?uspsharing

[3]

https://colab.research.google.com/drive/1ctevXhrE60s7o9RzsxpIqq37EjyU9tBn?uspsharing#scrollTobsbdrb5p2ONa