Comments (3)
We don't currently support it. One way to do so would be to keep half the layers of your model on one GPU, half on another, and spin up the process with two visible GPU's. For eg like here: https://pytorch.org/tutorials/intermediate/model_parallel_tutorial.html
You might have to insert .to("cuda:0/1") calls at the correct places for it to work
from jukebox.
@tmlbl Did you ever figure out a workable solution?
from jukebox.
@prafullasd - we need this to have a chance of running 2 cards - to get 48gb of VRAM.
https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090/
related - #142
Could you please reopen this ticket? as an investigation.
Need to sift through the forks to see if anyone has made any progress.
can use this shell script to find-forks.sh
https://gist.githubusercontent.com/joeytwiddle/abee0610244d283f1ebe2f76bf608086/raw/094ef14c0b83125149698b234e95cb525751f25e/github_get_all_forks.sh
chmod +x find-forks.sh
/find-forks.sh -do
there is actual an official branch / this should be mentioned on readme.
https://github.com/openai/jukebox/tree/multi_gpu_sampling
so did you guys get anywhere with this? I don't have multiple gpus to support
@heewooj - do you know if multiple gpus can access more VRAM? if I buy 2 x 3090 nvidia cards / 24gb - will it give 48gb and allow fine tune of 5 billion parameters?
from jukebox.
Related Issues (20)
- Installation Error,Packages not found HOT 7
- Fine-tuning the prior
- Hudba
- Error
- Just ignore
- Error right at the start of Colab HOT 1
- Changing output permissions
- stuck repeating the last few lines over and over and over and over again
- #0penA1 - #Fu11A1
- Encountering Errors with OpenAI Jukebox in Google Colab HOT 3
- Enhancement feature
- "Installation Error: jukebox package not found on conda channels (Windows 64-bit, Python 3.9.18)" "Unable to Install Jukebox from Conda (conda 23.11.0, Python 3.9.18)" HOT 1
- Syntax error on Colab
- My trackq HOT 1
- Rt
- Te amo
- AROA HOT 1
- Size mismatch
- Ya par tal
- Does anyone use the jukebox on their mac M2?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jukebox.