Comments (4)
Sengxian
commented on June 22, 2024
Hello @LorrinWWW. GLM-130B can still generate fluent sentences in this scenario, you can also generate different sentences by changing the seed. However, we are not sure if it is appropriate, it maybe depends on what you want to do.
from glm-130b.
LorrinWWW
commented on June 22, 2024
Thanks for your answer! Another question is do you have any suggestion on performing batch generation?
from glm-130b.
Sengxian
commented on June 22, 2024
Currently we don't support batch generation, but I'm guessing it's not too hard to modify the autoregressive generation logic and its corresponding strategy in SAT to support batch generation with a fixed context length (or no context).
from glm-130b.
LorrinWWW
commented on June 22, 2024
Thank you very much!
from glm-130b.
Related Issues (20)
- Question about P-Tuning
- 关于Fastertransformer推理的程序
- torch run的问题 HOT 4
- 部署后报错 size mismatch for transformer.word_embeddings.weight: copying a param with shape torch.Size([18816, 12288]) from checkpoint, the shape in current model is torch.Size([150528, 12288]). HOT 5
- V100（8 * 32G）运行报错 HOT 14
- 为什么没有中文说明？ HOT 3
- https://tianqi.aminer.cn/ 天启官网合作咨询验证码打不开，请问如何联系商用 HOT 1
- 想问一下作者，量化成int4 int8 之后为什么模型大小没有变化，都是240g HOT 15
- 请教
- 4*4090gpu for int4 model inference error HOT 1
- question: what does token mean here ?
- 国内模型下载地址 HOT 2
- [ERROR] `bash scripts/generate.sh --input-source interactive` 报错 HOT 7
- 是不是chatglm与这个GLM-130b开源模型中间还有很多问题待解决？ HOT 2
- [HELP] 有人能分享一下量化好的int4 版本的模型吗?
- 关于论文中bf16的一个疑问
- RuntimeError: CUDA error: invalid device ordinal HOT 1
- 如何使用FasterTransformer适配自己的模型 HOT 1
- 现在好像没有ChatGLM-130B开源吧？只有6B， 130B的不是Chat HOT 1
- bash scripts/generate.sh --input-source interactive运行报错 HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from glm-130b.