okuvshynov / slowllama Goto Github PK

View Code? Open in Web Editor NEW

423.0 423.0 32.0 1.16 MB

Finetune llama2-70b and codellama on MacBook Air without quantization

License: MIT License

Python 99.47% Shell 0.53%

apple-silicon fine-tuning llama llama2

slowllama's People

Stargazers

Watchers

slowllama's Issues

finetune.py segmentation fault

I am trying to run the finetune.py and getting a seg. fault. Can anyone help. I am on Apple M2 mac mini with 24G memory.

% python finetune.py 
loc("mps_transpose"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/75428952-3aa4-11ee-8b65-46d450270006/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":206:0)): error: 'anec.transpose' op Invalid configuration for the following reasons: Tensor dimensions N1D1C4096H1W32000 are not within supported range, N[1-65536]D[1-16384]C[1-65536]H[1-16384]W[1-16384].
loc("mps_matmul"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/75428952-3aa4-11ee-8b65-46d450270006/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":39:0)): error: 'anec.matmul' op Invalid configuration for the following reasons: Tensor dimensions N1D1C4096H1W32000 are not within supported range, N[1-65536]D[1-16384]C[1-65536]H[1-16384]W[1-16384].
zsh: segmentation fault  python finetune.py

/slowllama/logs/prepare_model.log doesnt exist

Hi, when I try to run the prepare_model.py which is the first script I get
/slowllama/logs/prepare_model.log
does not exist. I cant find that log file anywhere.

run prepare_model.py error

when I use CodeLlama-7b to run prepare_model.py,An exception occurred

RuntimeError: The expanded size of the tensor (32000) must match the existing size (32016) at non-singleton dimension 0. Target sizes: [32000, 4096]. Tensor sizes: [32016, 4096]

Is there a particular dataset format required for finetuning codellama? I have the dataset in the OpenAI suggested format which is basically a jsonl with each entry having messages: [{role: 'system', content: ''}, {role: 'user', content: ''}, {role: 'assistant', content: ''}]} object. Will this format work?

Mojo 🔥?

Now that mojo is available for M1/M2 platforms, have you considered attempting this with mojo for improved performance? (Questionable as to how much I guess with all the shuffling to the ssd?)

https://www.modular.com/blog/mojo-is-now-available-on-mac

Here is a llama2 implementation: https://github.com/tairov/llama2.mojo

Fine-tune other models

Hello,

Can we apply this method to fine-tune models other than llamas and codellama, such as mistral 7b?

Many thanks in advance!

okuvshynov / slowllama Goto Github PK

slowllama's People

Stargazers

Watchers

Forkers

slowllama's Issues

finetune.py segmentation fault

/slowllama/logs/prepare_model.log doesnt exist

run prepare_model.py error

Fine-tuning codellama dataset

Mojo 🔥?

Fine-tune other models

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent