[TASK] Another language support for GoLLIE (more specifically Vietnamese) about gollie HOT 3 OPEN

Hi @NoAtmosphere0!

I believe the easiest way to achieve this would be by fine-tuning one of the GoLLIE checkpoints with a Vietnamese dataset. Both Wikiann and Polyglot NER seem like the best candidates since they use the same labels as CoNLL03. To fine-tune your model with either of these datasets, you should:

1. Duplicate the CoNLL03 config and craft a Wikiann/Polyglot.json file: https://github.com/hitz-zentroa/GoLLIE/blob/main/configs/data_configs/conll03_config.json. Substitute the values in "train_file", "dev_file", and "test_file" with the paths to the datasets in .conll format (.tsv).
2. Modify the generate data script: https://github.com/hitz-zentroa/GoLLIE/blob/main/bash_scripts/generate_data.sh. Delete all the config files and incorporate the ones you produced in step 1. Subsequently, execute the script.
3. Modify the GoLLIE7B config file: https://github.com/hitz-zentroa/GoLLIE/blob/main/configs/model_configs/GoLLIE-7B_CodeLLaMA.yaml. Remove all the tasks and incorporate the ones you've recently made. Change the model from codellama/CodeLlama-7b-hf to HiTZ/GoLLIE-7B.
4. In the output folder, you'll get the new LoRA adapters for GoLLIE. You can use them using the load_model function found here: https://github.com/hitz-zentroa/GoLLIE/blob/main/src/model/load_model.py.

A significant concern here is the proficiency of LLaMA2/CodeLLaMA in Vietnamese. The model might not be very adept for that particular language, and unfortunately, there's a limited selection of multilingual LLMs available.

from gollie.

Hi @ikergarcia1996!

Thank you for your prompt response and helpful instructions. We will follow the steps that you have outlined in your response to train GoLLIE and also keep in mind your concerns about the proficiency of LLaMA2/CodeLLaMA in Vietnamese.

We will keep you updated on our progress by not closing this issue and let you know if we have any questions or need any further assistance. Thanks again for your support!

from gollie.

@NoAtmosphere0 Did you had any progress on that?

from gollie.