A simple LLM blender, to easily combine the best available open-source large language models into a super one.

• git
• Miniconda

• Clone this repository and navigate into it:

  git clone https://github.com/pandego/blendit.git
  cd blendit

• Create and activate your Python Environment:

  conda env create -f envrionment.yml
  conda activate blendit

• Initiate your project with poetry:

  poetry install

That's it for the Python environment!

In order to blend models you will have to setup a config.yaml file with the instructions for the blend. You can find some examples in src/config/.

• (Optional) If you need to download a set of models to get started with the example above, you can run the following command:

  bash download_models.sh

• Run the follow command to launch the blend, and let it shake 🍸

  sh run_merge.sh

  If you are curious, the most important part of the script is the following:

  mergekit-yaml src/config/example_task_arithmetic.yaml models/merged-models/my-first-blend \
  --verbose \
  --allow-crimes \         # experimental: allows blends of different model architectures
  --no-cuda \              # if you have a big Nvidia GPU, then set this to --cuda
  --copy-tokenizer \       # copies the tokenizer from the base model
  --out-shard-size 1B \    # splits model so it can be load in smaller RAMs (default: parse_kmb("5B"))
  --lazy-unpickle \        # experimental: lowers memory usage
  --write-model-card \
  --low-cpu-memory \
  --random-seed 123 \
  --trust-remote-code \
  --safe-serialization

  You can see all the options by running mergekit-yaml --help.

In order submit your blend to OpenLLM Leaderboard, you need to upload it to HuggingFace first.

• You can do that by running the following script and follow the instructions:

  sh upload_model_to_hf.sh

#TODOs

[ ] Add way to test model locally
[x] Add way to submit it to HF for evaluation
[ ] Wrap it all up in a nice Streamlit App 
[ ] Dockerize it, deploy it!