Audiocraft is a PyTorch library made by Meta for deep learning research on audio generation. At the moment, it contains the code for MusicGen, a state-of-the-art controllable text-to-music model.

Adds generation of songs with a length of over 30 seconds.

Adds the ability to continue songs.

Disables (hopefully) the gradio analytics.

Audiocraft requires Python 3.9, PyTorch 2.0.0. To install Audiocraft, you can run the following:

# Best to make sure you have torch installed first, in particular before installing xformers.
# Don't run this if you already have PyTorch installed.
pip install 'torch>=2.0'
# Then proceed to one of the following
pip install -U audiocraft  # stable release
pip install -U git+https://[email protected]/facebookresearch/audiocraft#egg=audiocraft  # bleeding edge
pip install -e .  # or if you cloned the repo locally

python app.py

Meta provides a simple API and 4 pre-trained models. The pre trained models are:

• small: 300M model, text to music only - 🤗 Hub
• medium: 1.5B model, text to music only - 🤗 Hub
• melody: 1.5B model, text to music and text+melody to music - 🤗 Hub
• large: 3.3B model, text to music only - 🤗 Hub

Needs a GPU! I recommend 12GB of VRAM for the large model.

Note: Please make sure to have ffmpeg installed when using newer version of torchaudio. You can install it with:

apt get install ffmpeg

See after a quick example for using the API.

import torchaudio
from audiocraft.models import MusicGen
from audiocraft.data.audio import audio_write

model = MusicGen.get_pretrained('melody')
model.set_generation_params(duration=8)  # generate 8 seconds.
wav = model.generate_unconditional(4)    # generates 4 unconditional audio samples
descriptions = ['happy rock', 'energetic EDM', 'sad jazz']
wav = model.generate(descriptions)  # generates 3 samples.

melody, sr = torchaudio.load('./assets/bach.mp3')
# generates using the melody from the given audio and the provided descriptions.
wav = model.generate_with_chroma(descriptions, melody[None].expand(3, -1, -1), sr)

for idx, one_wav in enumerate(wav):
    # Will save under {idx}.wav, with loudness normalization at -14 db LUFS.
    audio_write(f'{idx}', one_wav.cpu(), model.sample_rate, strategy="loudness")

@article{copet2023simple,
      title={Simple and Controllable Music Generation},
      author={Jade Copet and Felix Kreuk and Itai Gat and Tal Remez and David Kant and Gabriel Synnaeve and Yossi Adi and Alexandre Défossez},
      year={2023},
      journal={arXiv preprint arXiv:2306.05284},
}

• The code in this repository is released under the AGPLv3 license as found in the LICENSE file.

Copyright (c) Meta Platforms, Inc. and affiliates.