Comments (19)
prnake
commented on July 20, 2024
1
@volkerha Maybe replace
40 | true,
| ^~~~
| |
| bool
with
c10d::OpType::SEND,
would slove the problem.
Just check GlmOp.h and do little changes.
from glm-130b.
prnake
commented on July 20, 2024
This is a known issue mentioned here, you can try to use g++-7 and g++-9, in most of the case, one of them will work.
from glm-130b.
todiketan
commented on July 20, 2024
Thanks for the reply, I had built with g++-9, and was able to complete until the make -j step but was failing in the above part. But on using g++-7, I am facing a number of issues in the initial cmake itself, will it be a better option to build pytorch instead? And will building a pytorch with g++-9 solve the issue?
from glm-130b.
prnake
commented on July 20, 2024
Thanks for the reply, I had built with g++-9, and was able to complete until the make -j step but was failing in the above part. But on using g++-7, I am facing a number of issues in the initial cmake itself, will it be a better option to build pytorch instead? And will building a pytorch with g++-9 solve the issue?
Yes, that should be the best solution.
from glm-130b.
volkerha
commented on July 20, 2024
I have a similar issue. When running make -j I get the following error:
[100%] Built target pyt_swintransformer
In file included from /home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:17:
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.h: In member function ‘ncclComm* torch_ext::HackNCCLGroup::getcomm(size_t, size_t, const char*)’:
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.h:40:17: error: cannot convert ‘bool’ to ‘c10d::OpType’
40 | true,
| ^~~~
| |
| bool
In file included from /home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.h:21,
from /home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:17:
/opt/conda/lib/python3.8/site-packages/torch/include/c10d/ProcessGroupNCCL.hpp:374:14: note: initializing argument 2 of ‘void c10d::ProcessGroupNCCL::broadcastUniqueNCCLID(ncclUniqueId*, c10d::OpType, const string&, int)’
374 | OpType opType,
| ~~~~~~~^~~~~~
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc: In member function ‘std::vector<at::Tensor> torch_ext::GlmOp::forward(at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor, int64_t)’:
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:148:55: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t’ {aka ‘long unsigned int’} to ‘long int’ [-Wnarrowing]
148 | th::Tensor output_ids = torch::empty({batch_size, beam_width, total_request_output_len},
| ^~~~~~~~~~
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:148:55: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t’ {aka ‘long unsigned int’} to ‘long int’ [-Wnarrowing]
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:150:59: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t’ {aka ‘long unsigned int’} to ‘long int’ [-Wnarrowing]
150 | th::Tensor output_ids_buf = torch::empty({batch_size, beam_width, total_request_output_len},
| ^~~~~~~~~~
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:150:59: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t’ {aka ‘long unsigned int’} to ‘long int’ [-Wnarrowing]
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:152:55: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t’ {aka ‘long unsigned int’} to ‘long int’ [-Wnarrowing]
152 | th::Tensor logits_buf = torch::empty({batch_size, beam_width, vocab_size_},
| ^~~~~~~~~~
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:152:55: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t’ {aka ‘long unsigned int’} to ‘long int’ [-Wnarrowing]
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:154:81: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t’ {aka ‘long unsigned int’} to ‘long int’ [-Wnarrowing]
154 | th::Tensor parent_ids = torch::empty({total_request_output_len, batch_size, beam_width},
| ^~~~~~~~~~
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:154:81: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t’ {aka ‘long unsigned int’} to ‘long int’ [-Wnarrowing]
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:157:35: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t’ {aka ‘long unsigned int’} to ‘long int’ [-Wnarrowing]
157 | torch::empty({batch_size, beam_width}, torch::dtype(torch::kInt32).device(torch::kCUDA).requires_grad(false));
| ^~~~~~~~~~
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:157:35: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t’ {aka ‘long unsigned int’} to ‘long int’ [-Wnarrowing]
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:159:35: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t’ {aka ‘long unsigned int’} to ‘long int’ [-Wnarrowing]
159 | torch::empty({batch_size, beam_width}, torch::dtype(torch::kFloat32).device(torch::kCUDA).requires_grad(false));
| ^~~~~~~~~~
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:159:35: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t’ {aka ‘long unsigned int’} to ‘long int’ [-Wnarrowing]
make[2]: *** [src/fastertransformer/th_op/glm/CMakeFiles/th_glm.dir/build.make:76: src/fastertransformer/th_op/glm/CMakeFiles/th_glm.dir/GlmOp.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:6088: src/fastertransformer/th_op/glm/CMakeFiles/th_glm.dir/all] Error 2
make: *** [Makefile:136: all] Error 2
This is for g++ 9/10.
g++ 7/8 yields a different error about missing torch functions:
[100%] Built target pyt_swintransformer
In file included from /home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:17:0:
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.h: In member function ‘ncclComm* torch_ext::HackNCCLGroup::getcomm(size_t, size_t, const char*)’:
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.h:42:21: error: no matching function for call to ‘torch_ext::HackNCCLGroup::broadcastUniqueNCCLID(ncclUniqueId*, bool, const char*&, size_t&)’
rank);
^
In file included from /home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.h:21:0,
from /home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:17:
/opt/conda/lib/python3.8/site-packages/torch/include/c10d/ProcessGroupNCCL.hpp:372:8: note: candidate: void c10d::ProcessGroupNCCL::broadcastUniqueNCCLID(ncclUniqueId*, c10d::OpType, const string&, int)
void broadcastUniqueNCCLID(
^~~~~~~~~~~~~~~~~~~~~
/opt/conda/lib/python3.8/site-packages/torch/include/c10d/ProcessGroupNCCL.hpp:372:8: note: no known conversion for argument 2 from ‘bool’ to ‘c10d::OpType’
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc: In member function ‘std::vector<at::Tensor> torch_ext::GlmOp::forward(at::Tensor, at::Tensor, at::Tensor, at::Tensor, at::Tensor, int64_t)’:
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:148:55: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t {aka long unsigned int}’ to ‘long int’ inside { } [-Wnarrowing]
th::Tensor output_ids = torch::empty({batch_size, beam_width, total_request_output_len},
^~~~~~~~~~
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:148:55: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t {aka long unsigned int}’ to ‘long int’ inside { } [-Wnarrowing]
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:150:59: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t {aka long unsigned int}’ to ‘long int’ inside { } [-Wnarrowing]
th::Tensor output_ids_buf = torch::empty({batch_size, beam_width, total_request_output_len},
^~~~~~~~~~
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:150:59: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t {aka long unsigned int}’ to ‘long int’ inside { } [-Wnarrowing]
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:152:55: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t {aka long unsigned int}’ to ‘long int’ inside { } [-Wnarrowing]
th::Tensor logits_buf = torch::empty({batch_size, beam_width, vocab_size_},
^~~~~~~~~~
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:152:55: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t {aka long unsigned int}’ to ‘long int’ inside { } [-Wnarrowing]
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:154:81: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t {aka long unsigned int}’ to ‘long int’ inside { } [-Wnarrowing]
th::Tensor parent_ids = torch::empty({total_request_output_len, batch_size, beam_width},
^~~~~~~~~~
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:154:81: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t {aka long unsigned int}’ to ‘long int’ inside { } [-Wnarrowing]
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:157:35: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t {aka long unsigned int}’ to ‘long int’ inside { } [-Wnarrowing]
torch::empty({batch_size, beam_width}, torch::dtype(torch::kInt32).device(torch::kCUDA).requires_grad(false));
^~~~~~~~~~
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:157:35: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t {aka long unsigned int}’ to ‘long int’ inside { } [-Wnarrowing]
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:159:35: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t {aka long unsigned int}’ to ‘long int’ inside { } [-Wnarrowing]
torch::empty({batch_size, beam_width}, torch::dtype(torch::kFloat32).device(torch::kCUDA).requires_grad(false));
^~~~~~~~~~
/home/jovyan/src/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.cc:159:35: warning: narrowing conversion of ‘((torch_ext::GlmOp*)this)->torch_ext::GlmOp::beam_width’ from ‘size_t {aka long unsigned int}’ to ‘long int’ inside { } [-Wnarrowing]
make[2]: *** [src/fastertransformer/th_op/glm/CMakeFiles/th_glm.dir/build.make:76: src/fastertransformer/th_op/glm/CMakeFiles/th_glm.dir/GlmOp.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:6088: src/fastertransformer/th_op/glm/CMakeFiles/th_glm.dir/all] Error 2
make: *** [Makefile:136: all] Error 2
from glm-130b.
todiketan
commented on July 20, 2024
@volkerha I tried installing with c++7, but could not get past cmake stage. Will it be possible for you to share your conda environment in yaml file or something? It would be extremely helpful if you could share your environment details.
from glm-130b.
prnake
commented on July 20, 2024
@volkerha I tried installing with c++7, but could not get past cmake stage. Will it be possible for you to share your conda environment in yaml file or something? It would be extremely helpful if you could share your environment details.
Modifying the compiler version should not cause too many side effects, I recommend try this steps.
- Delete build folder
- Change CX and CXX in environmental variables
- Retry
from glm-130b.
volkerha
commented on July 20, 2024
Maybe replace
40 | true,
| ^~~~
| |
| bool
with
c10d::OpType::SEND
That solved the issue for me 🙏
@todiketan I managed to build with g++ 9. g++ 7 gave me some torch errors as mentioned above. My conda env contains a lot of other dependencies, so I'm not sure how much sense it makes to share it. Also, I have a local cuda-11-6 installation.
from glm-130b.
prnake
commented on July 20, 2024
Maybe replace
40 | true,
| ^~~~
| |
| bool
with
c10d::OpType::SENDThat solved the issue for me 🙏
@todiketan I managed to build with g++ 9. g++ 7 gave me some torch errors as mentioned above. My conda env contains a lot of other dependencies, so I'm not sure how much sense it makes to share it. Also, I have a local cuda-11-6 installation.
Thanks for the reply.
from glm-130b.
jimmycrowoo
commented on July 20, 2024
I meet the same issue.
from glm-130b.
jimmycrowoo
commented on July 20, 2024
@todiketan have you solved the problem
from glm-130b.
todiketan
commented on July 20, 2024
@jimmycrowoo No I was not able to solve the issue with either g++7 or g++9.
from glm-130b.
prnake
commented on July 20, 2024
The easiest way to solve this problem is using docker, due to conda environment varies greatly.
from glm-130b.
jimmycrowoo
commented on July 20, 2024
@todiketan I met the issue with g++-9, and solved the problem with g++-7
from glm-130b.
jimmycrowoo
commented on July 20, 2024
@prnake How can I call the service after running service with default flask app of fasttransformers? Are there some demos? Thanks!
from glm-130b.
prnake
commented on July 20, 2024
@prnake How can I call the service after running service with default flask app of fasttransformers? Are there some demos? Thanks!
You can use scripts in the example directory: https://github.com/THUDM/FasterTransformer/tree/main/examples/pytorch/glm, there are tests and a simple gradio frontend.
from glm-130b.
champson
commented on July 20, 2024
The root cause of this issue is difference pybind11 abi between pytorch and th-glm. Add the pybind11 macro define like https://github.com/pytorch/pytorch/blob/1fae179ee1b59c42c41f9dc7b55a2cba64737adb/torch/utils/cpp_extension.py#L1975 when build can fix it.
from glm-130b.
Rateofteasing
commented on July 20, 2024
with
How did this change? I didn't quite understand it
from glm-130b.
Rateofteasing
commented on July 20, 2024
Maybe replace
40 | true,
| ^~~~
| |
| bool
with
c10d::OpType::SENDThat solved the issue for me 🙏
@todiketan I managed to build with g++ 9. g++ 7 gave me some torch errors as mentioned above. My conda env contains a lot of other dependencies, so I'm not sure how much sense it makes to share it. Also, I have a local cuda-11-6 installation.
大佬怎么解决的我没看懂呢
from glm-130b.
Related Issues (20)
- 关于FT inference benchmark数据的疑问
- 每个token耗时呈脉冲式变化
- GLM-130B文档中描述model weights，GPU内存需要260G，测试demo中实际测试总占用在240G左右，请问是什么原因
- 模型并行集群怎么搭建
- 请问GLM可以在输出内容时，同时输出引用内容的来源吗？
- 模型申请页面无法提交申请 HOT 1
- 基于130B有chat版本开源的计划吗？
- 申请邮件收到的模型下载链接都失效了 HOT 5
- FasterTransformer能否支持Glm6B呢
- glm2-130B will it be made? HOT 1
- 请问，课程链接在哪里？ HOT 1
- RuntimeError: probability tensor contains either `inf`, `nan` or element < 0answers, answers_with_style, blanks = fill_blanks(raw_text, model, tokenizer, strategy)
- 8卡 fastertransformer 推理报错RuntimeError: [FT][ERROR] Assertion fail: /home/young.ruan/FasterTransformer/src/fastertransformer/th_op/glm/GlmOp.h:539
- 执行bash scripts/generate.sh --input-source interactive时出现的错误。大佬救救！ HOT 1
- Clarification Request on GLM-130B Model Architecture and Licensing for Commercial Use
- 有用tensortRT-llm的docker环境跑通模型的吗？求助...
- 请各位大佬伸以援手，我想要在自己本地部署一个该模型，怎么在windows上进行部署？
- 下载到一半就再也下不了了
- error about the GLM-130B’s model checkpoint HOT 1
- Could you offer a download link with Chinese mainland mirror?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from glm-130b.