thudm / codegeex4 Goto Github PK

使用本地模式，模型使用CodeQwen，提示连接成功，但是无法使用，请问如何解决呢

在线模式中，可以

本地模式下就不能选中代码进行解释

ollama从0.2.7开始已经支持function call了， 请求将codegeex4-all-9b发布到ollama平台中；

设置中只有Commit Message Style

Hey there! 👋 Your project CodeGeeX4 caught my eye on the trending list. Incredible job! 💪 Could you share what the most rewarding part of developing this project was? Looking forward to your insights!

Best regards,
@nectariferous
Telegram: t.me/nectariferous

• 🌐 Web: likhon.xyz
  © All rights reserved.

👉 For Promo/Help/Support, DM @nectariferous

do you have a plan to make the vs/jetbrains plugin supporting local deployed CodeGeeX Service

system: 你是审查机器人请给出代码评审，审查包含代码质量、可读性、拼写错误、可维护性、复用性、Bug、命名等。
prompt:

- 你是审查机器人请给出代码评审，审查包含代码质量、可读性、拼写错误、可维护性、复用性、Bug、命名， 

- 不需要审查代码风格、未使用的变量、导入路径, 

- 中文或代码回答

- 你要输出 JSON 文本，格式为：
[
 {
 filename:"app.tsx", 
 
 review:[{code: 代码行号, content:"review 评论，此处可用markdown", score:"严重等级(致命、严重、一般、轻微)"}]
 }
]

- 代码前面附带了代码行号

- 你只需要对修改后，与修改前有差异的代码评论

- 下面是代码数据：

修改前的代码:  
xxxx
 
修改后的代码:
xxxxx

参数：

temperature=0.2,
presence_penalty=1.2,
top_p=0.95

如题

Hey! 👋 I came across your CodeGeeX4 project. It's fantastic! Keep up the good work! 💪 Could you send me more details on Telegram? Also, please review my work and follow me on GitHub @nectariferous. Thanks!

我想使用我自己的代码库，用text类型的文件，来预训练一下。但是现在在create_datasets时报错

train_dataset, eval_dataset = create_datasets(tokenizer, args)

希望官方能提供一个demo，类似starcoder2

报错信息：Plugin 'CodeGeeX' (version '2.13.0-223') is not compatible with the current version of the IDE, because it requires build 241.* or older but the current build is IU-242.20224.300

ollama已经成功启动了codegeex4模型，但是用codegeex去连接，一直提示：连接错误，请确认模型配置

在README_zh.md这个文件中的底部申请链接中，对应到官网开放平台的商用申请表中无CodeGeeX模型

Hi there,

We're trying to reproduce the scores reported on BigCodeBench using v0.1.7post2. As there is no chat template provided inside the HF tokenizer config, I slightly changed the code and used the default chat template. So far, I got 49.0 on Complete and 38.9 on Instruct. The reproduced Instruct performance is a bit lower than your reported one. I doubt you had a customized template during the evaluation. Could you share more details about your setup?

bigcode-project/bigcodebench#19

Cheers

run on ollama, 403 err

能否像CodeQwen1.5-7B-Chat一样，提供tokenizer.json文件？

问题：在输入时，内容会被删除，导致无法继续输入。

参考视频如下：

inputs = tokenizer.apply_chat_template([{"role": "user", "content": "你是谁"}],
                                       add_generation_prompt=True, tokenize=True, return_tensors="pt",
                                       return_dict=True).to(device)
with torch.no_grad():
    print("inputs", inputs)
    outputs = model.generate(**inputs, max_length=512)
    print("outputs", outputs)
    outputs = outputs[:, inputs['input_ids'].shape[1]:]
    print("outputs_part", outputs)
    print(tokenizer.decode(outputs[0], skip_special_tokens=True))

outputs_part tensor([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0]], device='cuda:0')

如果函数有完整的括号，在其内部无法进行补全

如题。代码补全和生成注释报一下错误。

2024-07-10 10:41:34.001 [info] Local mode state: false
2024-07-10 10:41:34.001 [info] CodeGeeX is now active
2024-07-10 10:41:36.106 [info] Registering commands...
2024-07-10 11:12:33.167 [error] AggregateError: 
	at internalConnectMultiple (node:net:1114:18)
	at internalConnectMultiple (node:net:1177:5)
	at Timeout.internalConnectMultipleTimeout (node:net:1687:3)
	at listOnTimeout (node:internal/timers:575:11)
	at process.processTimers (node:internal/timers:514:7)
2024-07-10 11:14:15.027 [info] 10/7/2024, 11:14:15 am [Error] [Add Comment] Internal error occurs
2024-07-10 11:14:37.670 [info] 10/7/2024, 11:14:37 am [Error] [Add Comment] Internal error occurs

请问这个问题如何解决？

The key word suppose to be "type" rather than "typo"

目前demo中的嵌入模型使用的是在线服务，embedding-2模型。但是因为我们公司的开发环境是不联网的，所以想请教一下embedding-2模型是否有开源计划。
或者使用其它的开源嵌入模型会不会影响效果。因为目前我在网上找到的资料，关于RAG基本上都是针对自然语言文本的，不清楚代码和自然语言在向量转换和向量检索方面是否一样。

看演示编辑器插件中可以使用本地模式连接模型工具，但是在idea中安装最新版的插件后没有发现有可以开启本地模式的页面

PyCharm：2024.1
插件：2.13.0 - 223
ollama：0.3
模型：ollama默认的 codegeex4 模型

问题：可以连接模型、聊天，但在编程窗口无法自动补全，提示“服务器错误：Code 500 服务器内部错误”

问题：在webstorm使用codegeex插件时，部分大型项目卡死，如下图，经过测试，会卡两三个小时以上。

受影响版本：如下图：

关闭插件后，项目秒开，经过多轮验证。确定是该插件引起。

项目情况：

项目使用umi/dumi 库，会在src下生成.umi缓存文件，文件数量巨大，尽管webstorm已经设置该目录为禁用索引目录，但是插件仍然对该目录文件进行了遍历索引解析，从而导致卡死。

期望

希望优化此问题。

使用ollama本地运行codegeex4
请求补全的接口curl -s http://localhost:62333/v1/completions -d @/tmp/p3.json|jq .

请求的body

{
  "model": "codegeex4",
  "messages": [
    {
      "role": "user",
      "content": "###LANGUAGE:Python\n###MODE:BLOCK\n<|code_suffix|>\n<|code_prefix|>def parse_nested_parens(paren_string: str) -> List[int]:\n    \"\"\" Input to this function is a string represented multiple groups for nested parentheses separated by spaces.\n    For each of the group, output the deepest level of nesting of parentheses.\n    E.g. (()()) has maximum two levels of nesting while ((())) has three.\n\n    >>> parse_nested_parens('(()()) ((())) () ((())()())')\n   [2, 3, 1, 3]\n\"\"\"\n<|code_middle|>"
    }
  ],
  "temperature": 0.2,
  "top_p": 0.95,
  "max_tokens": 64,
  "presence_penalty": 1,
  "stream": false,
  "stop": []
}

响应 "finish_reason": "load"，没有生成text是何原因？

vsocde的codegeex插件，cmd + ↓ 代表接收下一条建议，但是在所有情况下都会与编辑器、控制台的向下滚动快捷键冲突，导致失效

代码如下:

from transformers import AutoTokenizer
from vllm import LLM, SamplingParams
max_model_len, tp_size = 131072, 1
model_name = "/models/codegeex4-all-9b"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
llm = LLM(
    model=model_name,
    tensor_parallel_size=tp_size,
    max_model_len=max_model_len,
    trust_remote_code=True,
    enforce_eager=True,
)
stop_token_ids = [151329, 151336, 151338]
sampling_params = SamplingParams(temperature=0.95, max_tokens=1024, stop_token_ids=stop_token_ids)

vllm的输出：

Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
INFO 07-11 15:51:44 llm_engine.py:169] Initializing an LLM engine (v0.5.1) with config: model='/models/codegeex4-all-9b', speculative_config=None, tokenizer='/models/codegeex4-all-9b', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, rope_scaling=None, rope_theta=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=131072, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto, quantization_param_path=None, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='outlines'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None), seed=0, served_model_name=/models/codegeex4-all-9b, use_v2_block_manager=False, enable_prefix_caching=False)
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
WARNING 07-11 15:51:44 tokenizer.py:126] Using a slow tokenizer. This might cause a significant slowdown. Consider using a fast tokenizer instead.
WARNING 07-11 15:51:44 utils.py:562] Using 'pin_memory=False' as WSL is detected. This may slow down the performance.
INFO 07-11 15:55:24 model_runner.py:255] Loading model weights took 17.5635 GB

占用显存40GB，如果用xfomers的backend也一样

如题，可能是因为我的chromium浏览器的原因，所以我请求加入复制登陆链接的功能

代码补全出来的代码有时候是根本不存在的方法，如何关联现有项目已有的方法？

2024-07-05 18:43:44 - Invalid URL '': No scheme supplied. Perhaps you meant https://?
Traceback (most recent call last):
File "C:\Users\xxx\AppData\Local\Programs\Python\Python39\lib\site-packages\chainlit\utils.py", line 44, in wrapper
return await user_function(**params_values)
File "run.py", line 150, in main
for part in stream:
File "J:\Tools\CodeGeeX4\CodeGeeX4\repodemo\llm\api\codegeex4.py", line 21, in codegeex4
response = requests.post(url, json=data, headers=headers, verify=False, stream=True)
File "C:\Users\xxx\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\api.py", line 115, in post
return request("post", url, data=data, json=json, **kwargs)
File "C:\Users\xxx\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\xxx\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 575, in request
prep = self.prepare_request(req)
File "C:\Users\xxx\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 486, in prepare_request
p.prepare(
File "C:\Users\xxx\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\models.py", line 368, in prepare
self.prepare_url(url, params)
File "C:\Users\xxx\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\models.py", line 439, in prepare_url
raise MissingSchema(
requests.exceptions.MissingSchema: Invalid URL '': No scheme supplied. Perhaps you meant https://?

按照公众号视频号教程，下载项目装了库 打开了chainlit ，run了之后，就这样了，模型也执行对应文件下载完毕

我用ollama运行codegeex4运行一段时间后，会输出GGGGGGGGG，但刚加载后是正常的，GPU为双4090，ollama版本0.2.5。这个问题可以复现，重新加载后恢复正常，一端时间后又这样了。
我看ollama项目有个类似的issue，GLM4好像也有类似的问题。

训练模型中有这两大商业游戏引擎吗

我尝试将ollama启动codegeex作为后端，通过访问ollama api方式获得结果，使用了yield 将流式返回结果作为迭代器处理，在我调试中在迭代的第一次直接退出了程序，再次访问接口时（无论是流式还是非流式）模型生成结果总是不全且生成结果末尾包含GGGGGGGGG.......，"done"字段总是false，但是重启该模型后输出结果正常。

这是不正常的输出

结束该服务

重新访问正常

我不确定这是ollama的问题还是codegeex的问题。感觉似乎是流式返回仍在处理导致了这个BUG。^_^'

请问，我看function call例子中有一个tool的角色，但是我ollama部署的时候 只有system user assistant的选择，这个tool是怎么设置的呀:

我现在运行插件生成代码注释，看了下任务管理器gpu占用是0，是不是改成gpu生成会更快。

https://huggingface.co/THUDM/glm-4-9b/commit/5dd1ab74bb3717ceb6acc17fedbdcd17028ce841#d2h-054989

glm-4 系的各模型是否应该同步更新 model 类的实现？

如何微调

比如我想支持perl语言，我需要怎么做？

按照官网提示进行了本地配置，配置页面显示连接正确：

但是发送消息时提示后台错误：

但是相同的模型服务器配置同事使用vscode可以正常访问本地模型，所以我怀疑咱这个idea的插件我是不是配置的不对还是使用姿势不对？

I have tested this model with same system prompt of GLM4. This model only emits parameters, but not the function name. For example, here, get_weather is not generated:

You  > what's the weather like in beijng?
A.I. > 
{"city_name": "beijing"}