Comments (5)
Yes, you are correct. In FunCodec and LauraTTS, we didn't include any optimization on attention operation, since the model is very small.
from funcodec.
Sorry, I can't understand your question, please provide more details to specify the question, thanks.
from funcodec.
My bad, let me explain my question.
As far as I know, big GPT-like models always use some optimization in attention like FlashAttention for less GPU mem cost and improve the forward speed. However, I didn't find any like this, only find a "torch.matmul" at here. I'd like to make sure that I'm correct, or there is optimization somewhere else~
from funcodec.
Thanks for your reply!
After reading the code, I still have two little questions:
- Is LauraTTS a totally independent model, which is trained from scratch without any pretrained parameters from LauraGPT?
2.Since I see LauraGPT paper is also referenced in this repo, also taking into consideration that LauraGPT and LauraTTS has highly similar name, what's the relevance between both?
from funcodec.
Thanks for your reply! After reading the code, I still have two little questions:
- Is LauraTTS a totally independent model, which is trained from scratch without any pretrained parameters from LauraGPT?
2.Since I see LauraGPT paper is also referenced in this repo, also taking into consideration that LauraGPT and LauraTTS has highly similar name, what's the relevance between both?
- Yes, LauraTTS is a totally independent model, you can train it from scratch on LibriTTS corpus with the released code.
- The reason of LauraGPT paper referencing this repo is because LauraGPT use the same codec model to tokenize speech signals. In fact, LauraTTS can be treated as a improved version of LauraGPT for TTS task only. There are two main differences between LauraTTS and LauraGPT in terms of TTS task: 1) LauraTTS use two codec groups in LM model rather than one. 2) In LauraTTS the LM and NAR model are jointly trained while they are trained separately in LauraGPT.
from funcodec.
Related Issues (20)
- TypeError: 'NoneType' object is not callable HOT 4
- [bug] encoding阶段生成的codec.txt, 无法直接读取? HOT 1
- Discriminator loss? HOT 5
- LauraTTS模型的训练花了多长时间? HOT 5
- 如何仅用funcodec的4层或者8层量化器进行推理 HOT 2
- zipfile.BadZipFile: File is not a zip file HOT 1
- run.sh: 34: utils/parse_options.sh: Syntax error: Bad for loop variable HOT 1
- How to check progress? HOT 1
- 运行run.sh在stage 4时报错 HOT 2
- Stage 3 HOT 2
- Feature to resume training after stopping? HOT 1
- NET/Plugin : dlerror=libnccl-net.so: cannot open shared object file: No such file or directory No plugin found (libnccl-net.so)
- Out of memory when train on large dataset (librilight)
- Stage 1 can only be run on one gpu card 0 HOT 1
- Model sometimes stops training due to nan inputs HOT 2
- Should the audio be normed to [-1, 1]? HOT 1
- Encountering errors when training LauraGPT with codec model configured to num_quantizers=16, codebook_size=512 HOT 1
- 多机多卡
- 流式推理
- audio_codec-encodec-zh_en-general-16k-nq32ds640-pytorch模型,Huggingface和Modelscope下载的参数不同
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from funcodec.