Light

pabannier / encodec.cpp Goto Github PK

View Code? Open in Web Editor NEW

180.0 10.0 12.0 3.94 MB

Port of Meta's Encodec in C/C++

Python 7.71% C++ 80.60% CMake 1.53% C 10.16%

encodec.cpp's Introduction

encodec.cpp

High-performance inference of Meta's Encodec deep learning based audio codec model:

Plain C/C++ implementation without dependencies using ggml

Demo

Here is a demo of running Encodec on a single M1 MacBook Pro:

demo.mp4

Roadmap

Implementation details

The core tensor operations are implemented in C (ggml.h / ggml.c)
The encoder-decoder architecture and the high-level C-style API are implemented in C++ (encodec.h / encodec.cpp)
Basic usage is demonstrated in main.cpp.

Usage

Here are the steps for the bark model.

Get the code

git clone --recurse-submodules https://github.com/PABannier/encodec.cpp.git
cd encodec.cpp

Build

In order to build encodec.cpp you must use CMake:

mkdir build
cd build
cmake ..
cmake --build . --config Release

Using Metal

Offloading to GPU is possible with the Metal backend for MacOS. Performance are not improved but the power consumption and CPU activity is reduced.

cmake -DGGML_METAL=ON -DBUILD_SHARED_LIBS=Off ..
cmake --build . --config Release

Using cuBLAS

The inference can be offloaded on a CUDA backend with cuBLAS.

cmake -DGGML_CUBLAS=ON -DBUILD_SHARED_LIBS=Off ..
cmake --build . --config Release

encodec.cpp's People

Contributors

Stargazers

Watchers

Forkers

btbujiangjun maxmax2016 ishine bachittle dkakaie fish23 uextm pieterscholtz ti3x ezhangle ochafik randydom

encodec.cpp's Issues

Steps needed to support the 48khz model

Hey, Thankyou for all your work, I was wondering can this be extended to support the 48khz model? What all will that require, is it simple? Would love to contribute if someone can provide some guidance.

Thanks!

I can't find ggml_weights. I build ggml, but none of those libs were comparable with Encodec.

Sir, I am trying to compress audio with the examples provided in the short video. I don't find the ggml_model.bin. I would appreciate if you could guide me to correct direction. Thank you

not enough space in context's memory pool

read_wav_from_disk: Number of frames read = 1577459.
ggml_new_object: not enough space in the context's memory pool (needed 39200416, available 39200096)
compress: /mnt/c/prog/fork/encodec.cpp/ggml/src/ggml.c:4858: ggml_new_object: Assertion `false' failed.
Aborted

This looks similar to the unresolved bark.cpp error: PABannier/bark.cpp#122

Now that I have been experimenting, I believe it is due to the size of the file. If it is above 1mb it cannot fit into the buffer. So even if the file is like 10-15 seconds in length, its too big for the buffer.

too many initializers

get the following errors when compiling main and v0.1.0:

It is related to ggml.c compilation, I believe you are using a custom fork with changes to it, and that version is not working, but it was since October 2023 so I'm unsure what the reason is.

Maybe it's because I'm using MSVC and Windows instead of Mac?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.