model

model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
adapter_name_or_path: saves/llama3-8b/lora/sft

method

stage: sft
do_predict: true
finetuning_type: lora

dataset

eval_dataset: identity,alpaca_en_demo # change to my own dataset
template: llama3
cutoff_len: 1024
max_samples: 50
overwrite_cache: true
preprocessing_num_workers: 16

output

output_dir: saves/llama3-8b/lora/predict
overwrite_output_dir: true

eval

per_device_eval_batch_size: 1
predict_with_generate: true
ddp_timeout: 180000000

from llama-factory.

https://github.com/hiyouga/LLaMA-Factory/tree/main/examples#batch-predicting-and-computing-bleu-and-rouge-scores-1

from llama-factory.

model

model_name_or_path: ‘’

method

stage: sft
do_predict: true
finetuning_type: full

dataset

eval_dataset: clinical-trials-v2
template: llama3
cutoff_len: 1024
max_samples: 50
overwrite_cache: true
preprocessing_num_workers: 16

output

output_dir: ‘’
overwrite_output_dir: true

eval

per_device_eval_batch_size: 1
predict_with_generate: true

Still: ValueError: Some keys are not used by the HfArgumentParser: ['ddp_timeout', 'do_eval', 'output_dir', 'overwrite_output_dir', 'per_device_eval_batch_size', 'predict_with_generate']

from llama-factory.