agora-x / atom Goto Github PK
View Code? Open in Web Editor NEWa suite of finetuned LLMs for atomically precise function calling 🧪
Home Page: https://discord.gg/GYbXvDGevY
License: MIT License
a suite of finetuned LLMs for atomically precise function calling 🧪
Home Page: https://discord.gg/GYbXvDGevY
License: MIT License
Describe the bug
A clear and concise description of what the bug is and what the main root cause error is. Test very thoroughly before submitting.
Arguments are not being Implemented correctly and training is prohibited
equirement already satisfied: smmap<6,>=3.0.1 in /usr/local/lib/python3.10/dist-packages (from gitdb<5,>=4.0.1->GitPython!=3.1.29,>=1.0.0->wandb->-r requirements.txt (line 14)) (5.0.1)
The following values were not passed to `accelerate launch` and had defaults used instead:
`--num_processes` was set to a value of `1`
`--num_machines` was set to a value of `1`
`--mixed_precision` was set to a value of `'no'`
`--dynamo_backend` was set to a value of `'no'`
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py", line 1282, in _get_module
return importlib.import_module("." + module_name, self.__name__)
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/usr/local/lib/python3.10/dist-packages/transformers/data/__init__.py", line 27, in <module>
from .processors import (
File "/usr/local/lib/python3.10/dist-packages/transformers/data/processors/__init__.py", line 15, in <module>
from .glue import glue_convert_examples_to_features, glue_output_modes, glue_processors, glue_tasks_num_labels
File "/usr/local/lib/python3.10/dist-packages/transformers/data/processors/glue.py", line 30, in <module>
import tensorflow as tf
File "/usr/local/lib/python3.10/dist-packages/tensorflow/__init__.py", line 38, in <module>
from tensorflow.python.tools import module_util as _module_util
File "/usr/local/lib/python3.10/dist-packages/tensorflow/python/__init__.py", line 37, in <module>
from tensorflow.python.eager import context
File "/usr/local/lib/python3.10/dist-packages/tensorflow/python/eager/context.py", line 29, in <module>
from tensorflow.core.framework import function_pb2
File "/usr/local/lib/python3.10/dist-packages/tensorflow/core/framework/function_pb2.py", line 5, in <module>
from google.protobuf.internal import builder as _builder
ImportError: cannot import name 'builder' from 'google.protobuf.internal' (/usr/local/lib/python3.10/dist-packages/google/protobuf/internal/__init__.py)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/content/Atom/Atom/Atom/finetune.py", line 15, in <module>
from transformers import (
File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
File "/usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py", line 1272, in __getattr__
module = self._get_module(self._class_to_module[name])
File "/usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py", line 1284, in _get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.data.data_collator because of the following error (look up to see its traceback):
cannot import name 'builder' from 'google.protobuf.internal' (/usr/local/lib/python3.10/dist-packages/google/protobuf/internal/__init__.py)
Traceback (most recent call last):
File "/usr/local/bin/accelerate", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.10/dist-packages/accelerate/commands/accelerate_cli.py", line 47, in main
args.func(args)
File "/usr/local/lib/python3.10/dist-packages/accelerate/commands/launch.py", line 986, in launch_command
simple_launcher(args)
File "/usr/local/lib/python3.10/dist-packages/accelerate/commands/launch.py", line 628, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python3', 'finetune.py', '--batch-size', '32', '--gradient-accumulate-every', '8', '--output-dir', 'output/atom-65k-pytorch', '--wandb', 'yarn', '--seed', '42', '--max-train-steps', '400', '--warmup-steps', '20', '--learning-rate', '2e-5', '--grad-norm', '--lora', '--model', 'conceptofmind/Yarn-Llama-2-13b-64k', '--yarn-factor', '16.0', '--dataset', 'kye/all-lucidrain-code-python-tokenized-65536-1']' returned non-zero exit status 1.
To Reproduce
Steps to reproduce the behavior:
!git clone https://github.com/Agora-X/Atom.git
%cd Atom
!pip install -r requirements.txt
!accelerate launch finetune.py \
--batch-size 32 \
--gradient-accumulate-every 8 \
--output-dir output/atom-65k-pytorch \
--wandb yarn \
--seed 42 \
--max-train-steps 400 \
--warmup-steps 20 \
--learning-rate 2e-5 \
--grad-norm \
--lora \
--model conceptofmind/Yarn-Llama-2-13b-64k \
--yarn-factor 16.0 \
--dataset kye/all-lucidrain-code-python-tokenized-65536-1
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.