Comments (17)
@Taka152 NO, the Driver Version is 440.33.01
from lightseq.
env information:
NVIDIA-SMI 455.32.00
Driver Version: 455.32.00
CUDA Version: 11.1
|GPU Name GeForce RTX 3090
from lightseq.
@letiantony After googling it. It seems a problem with this CUDA version, could you try different CUDA versions?
from lightseq.
@Taka152 thanks for the reply, it worked with cuda 10.2,but still hope it support cuda 11.1
from lightseq.
@letiantony Thanks for your feedback, we have reproduced this error on CUDA 11. It seems to be a CUBLAS API deprecated in the newest CUDA. We'll find a way to solve this.
BTW, when you try CUDA 10.2, did you downgrade the driver version or it was still 455.32.00?
from lightseq.
@Taka152 NO, the Driver Version is 440.33.01
I see, I guess it is the driver version, not CUDA that is incompatible with LightSeq. Because CUDA is statically linked in LightSeq lib.
@letiantony Would you mind trying the 455 driver version with CUDA 10.2? That would help a lot to confirm my guess.
from lightseq.
BTW, our test on driver 450 is OK
from lightseq.
@Taka152 Sorry, because of the limits of authority,I can't upgrade or downgrade the version. I just use another GPU to try cuda 10.2.
from lightseq.
@Taka152 Sorry, because of the limits of authority,I can't upgrade or downgrade the version. I just use another GPU to try cuda 10.2.
That's ok, we'll find it
from lightseq.
@Taka152 Hi Ying Xiong. Thank you for making this awesome library available to public. Actually I encountered the same issue as mentioned in this post. Now I am trying to downgrade the graphics driver as well as CUDA to see if it works. For your information, right now I am using NVIDIA RTX 3090 graphics card, and the versions of the graphics driver and CUDA are 460.56 and 11.2 respectively.
from lightseq.
We found on driver greater than 450, some cublas API has changed, so currently lightseq inference can't run driver greater than 450
from lightseq.
We found on driver greater than 450, some cublas API has changed, so currently lightseq inference can't run driver greater than 450
Ying Xiong thanks for your prompt reply. I just checked the supported product list of the 450 driver, and it seems that this driver does not support NVIDIA RTX 3090 card. I was thinking if there would be another lightseq release to resolve the CUBLAS issue on more recent drivers in the near future. Thank you!
from lightseq.
Sure, we'll try to fix it.
from lightseq.
Have you fix this issue?
from lightseq.
Here is the solution.#251 (comment). This problem is not caused by driver or cuda version. It is the card that not support pre-built lightseq.
from lightseq.
How to fix this?
from lightseq.
I had the same issue.
====================START warmup====================
=========lightseq=========
lightseq generating...
Traceback (most recent call last):
File "test/ls_bart.py", line 102, in
main()
File "test/ls_bart.py", line 83, in main
warmup(tokenizer, ls_model, hf_model, sentences)
File "test/ls_bart.py", line 54, in warmup
ls_generate(ls_model, tokenizer, inputs_id)
File "test/ls_bart.py", line 30, in ls_generate
ls_res_ids, ls_time = ls_bart(model, inputs_id)
File "test/ls_bart.py", line 12, in ls_bart
generated_ids = model.infer(inputs)
RuntimeError: [CUDA][ERROR] /tmp/build-via-sdist-t8nsnqiz/lightseq-2.2.1/lightseq/inference/model/encoder.cc.cu(168): CUBLAS_STATUS_NOT_SUPPORTED
NVIDIA-SMI 460.106.00
Driver Version: 460.106.00
CUDA Version: 11.2
|GPU Name GeForce RTX 3090
from lightseq.
Related Issues (20)
- lightseq' Transformer expects an extra layer_norm on both encoder and decoder level
- LLaMA example 结果验证
- 请问lightseq在推理流程中有gemm调参这一步吗?
- Is llama inference available now? HOT 1
- Do you have plans to support token_type_ids?
- llama inference test HOT 3
- 请问lightseq可以支持segmentAnyting的推理优化吗 HOT 1
- 为什么连给的example也有bug? HOT 4
- [Question] gptj, mpt support.
- question about environment
- how to resolve xlm-roberta convert fail
- Can int8 in pre-training large model ???
- lightseq是否支持clip模型的int8量化?
- Is it normal that A10 inference speed is lower than 2080ti? HOT 1
- identifier "__hisnan" is undefined HOT 4
- 要求C++ 17 HOT 1
- Exception: Installed CUDA version 12.3 does not match the version torch was compiled with 12.1, unable to compile cuda/cpp extensions without a matching cuda version.
- 参数格式不对args format wrong
- is Baichuan2 model inference available now?
- [FeedBack][1.0.8][com.miniclip.eightballpool]- HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lightseq.