是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this? <ul clas

[BUG] <title>全参数微调qwen-14b-chat时卡住 about qwen HOT 1 CLOSED

PineappleWill commented on September 27, 2024

[BUG] 全参数微调qwen-14b-chat时卡住<p>from qwen.</p></section> </section> </article> <article> <h2 class="h2">Comments (1)</h2> <section class="issue-comment"> <section id="2144211364" class="issue-head"> <img class="issue-avatar" src="https://avatars.githubusercontent.com/u/17811943?v=4" alt="jklj077 avatar" /> <a class="issue-username" href="/jklj077">jklj077</a> <span class="issue-time"> commented on September 27, 2024 </span> </section> <section class="markdown markdown-js p-5"><p dir="auto">Hi, you could try upgrading to Qwen1.5 first and follow the instructions there. But based on the logs, your environment was simply broken (multiple system CUDA existed but appeared poorly configured and deepspeed complained NCCL not implemented, which should not happen). Honestly, it is not our place to debug your environment, but I would suggest a clean install or using the provided docker image.</p><p>from qwen.</p></section> </section> </article> <section> <h2 class="h2">Related Issues (20)</h2> <div class="issue"> <ul> <li> <a href="/qwenlm/qwen/issues/1289">[BUG] <title>Adding regular tokens is not supported</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 1</span> </li> <li> <a href="/qwenlm/qwen/issues/1290">如何修改模型的结构</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 1</span> </li> <li> <a href="/qwenlm/qwen/issues/1291">[BUG] <title> vLLM推理乱码</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 2</span> </li> <li> <a href="/qwenlm/qwen/issues/1294">Qwen 的开源模型能输出 logprobs吗？</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 3</span> </li> <li> <a href="/qwenlm/qwen/issues/1296">[BUG] docker_openai_api.sh 报can't open file 'openai_api.py'</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 1</span> </li> <li> <a href="/qwenlm/qwen/issues/1297">推理时的显存使用为啥这么少呢？</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 1</span> </li> <li> <a href="/qwenlm/qwen/issues/1299">[BUG] <title>Qwen2-7b-instruct使用SFT-FT，loss变为0，如何解决？</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 2</span> </li> <li> <a href="/qwenlm/qwen/issues/1300">大模型function call对比传统nlp方式有什么优势？</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 2</span> </li> <li> <a href="/qwenlm/qwen/issues/1301">[BUG] 百炼文档中function call 的示例有误</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 1</span> </li> <li> <a href="/qwenlm/qwen/issues/1302">请教下为什么Qwen/finetune.py和Qwen/eval/evaluate_ceval.py 的tokenizer的padding_side 不一样呢？</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 1</span> </li> <li> <a href="/qwenlm/qwen/issues/1303">[BUG] Qwen 1.8B 多线程推理时报错</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 2</span> </li> <li> <a href="/qwenlm/qwen/issues/1307">[BUG] <title> model_max_length 32768 not work</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 4</span> </li> <li> <a href="/qwenlm/qwen/issues/1308">[BUG] <title> 请问QWenLMHeadModel中的QWenModel模块是处理文本信息吗？</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 1</span> </li> <li> <a href="/qwenlm/qwen/issues/1309">官方推理脚本和模型文件中的pad_token不一致</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 1</span> </li> <li> <a href="/qwenlm/qwen/issues/1310">Qwen-Chat-RLHF和Qwen-Chat的区别</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 1</span> </li> <li> <a href="/qwenlm/qwen/issues/1311">[BUG] 增加上下文长度后输出乱码</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 1</span> </li> <li> <a href="/qwenlm/qwen/issues/1312">[BUG] <title>Nvidia Jetson Orin NX开发板上推理运行qlora微调之后的模型，报错：不支持QuantLinear()</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 1</span> </li> <li> <a href="/qwenlm/qwen/issues/1313">AWQ量化后，输出不能正常停止，不量化推理正常</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 1</span> </li> <li> <a href="/qwenlm/qwen/issues/1314">请问可以支持加入本地知识库进行微调大模型吗</a> <span class="text-red-600 text-xs font-normal py-0.5 px-1 border border-red-600 rounded-md">HOT 2</span> </li> <li> <a href="/qwenlm/qwen/issues/1315">qwen-7b-int4用vllm推理，为什么结果是乱码?</a> </li> </ul> </div> </section> </main> <section id="more" class="flex-none w-full md:w-60 text-gray-600 bg-gray-50 px-5 md:px-3 rounded-md dark-color"> <div class="w-full md:w-60 h-0.5"></div> <section>  <h2 class="h2 py-3.5">Recommend Projects</h2> <ul> <li class="mb-4"> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/facebook/react"> <img loading="lazy" class="inline-block w-6 h-6 rounded-md border border-white" width="24" height="24" src="https://raw.githubusercontent.com/facebook/create-react-app/master/packages/cra-template/template/public/logo192.png" alt="React photo" /> React </a> </h3> <p class="article-more pt-1">A declarative, efficient, and flexible JavaScript library for building user interfaces.</p> </article> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/vuejs/vue"> <img loading="lazy" class="inline-block w-6 h-6 rounded-md border border-white" width="24" height="24" src="https://camo.githubusercontent.com/c8f91d18976e27123643a926a2588b8d931a0292fd0b6532c3155379e8591629/68747470733a2f2f7675656a732e6f72672f696d616765732f6c6f676f2e706e67" alt="Vue.js photo" /> Vue.js </a> </h3> <p class="article-more pt-1">🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.</p> </article> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/microsoft/TypeScript"> <img loading="lazy" class="inline-block w-6 h-6 rounded-md border border-white" width="24" height="24" src="https://www.typescriptlang.org/favicon-32x32.png" alt="Typescript photo" /> Typescript </a> </h3> <p class="article-more pt-1">TypeScript is a superset of JavaScript that compiles to clean JavaScript output.</p> </article> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/tensorflow/tensorflow"> <img loading="lazy" class="inline-block w-6 h-6 rounded-md border border-white" width="24" height="24" src="https://camo.githubusercontent.com/c04e16c05de80dadbdc990884672fc941fdcbbfbb02b31dd48c248d010861426/68747470733a2f2f7777772e74656e736f72666c6f772e6f72672f696d616765732f74665f6c6f676f5f736f6369616c2e706e67" alt="TensorFlow photo" /> TensorFlow </a> </h3> <p class="article-more pt-1">An Open Source Machine Learning Framework for Everyone</p> </article> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/django/django"> <img loading="lazy" class="inline-block w-6 h-6 rounded-md border border-white" width="24" height="24" src="https://avatars2.githubusercontent.com/u/27804?s=200&v=4" alt="Django photo" /> Django </a> </h3> <p class="article-more pt-1">The Web framework for perfectionists with deadlines.</p> </article> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/laravel/laravel"> <img loading="lazy" class="inline-block w-6 h-6 rounded-md border border-white" width="24" height="24" src="https://laravel.com/img/logomark.min.svg" alt="Laravel photo" /> Laravel </a> </h3> <p class="article-more pt-1">A PHP framework for web artisans</p> </article> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/d3/d3"> <img loading="lazy" class="inline-block w-6 h-6 rounded-md border border-white" width="24" height="24" src="https://camo.githubusercontent.com/586ccf0aad9684edc821658cee04146cf36d1f1d5ec904bbefd72728909ccb2e/68747470733a2f2f64336a732e6f72672f6c6f676f2e737667" alt="D3 photo" /> D3 </a> </h3> <p class="article-more pt-1">Bring data to life with SVG, Canvas and HTML. 📊📈🎉</p> </article> </li> <li> <div> </div> </li> </ul> </section> <section>  <h2 class="h2 py-3.5">Recommend Topics</h2> <ul> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/topic/javascript"> javascript </a> </h3> <p class="article-more pt-1">JavaScript (JS) is a lightweight interpreted programming language with first-class functions.</p> </article> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/topic/web"> web </a> </h3> <p class="article-more pt-1">Some thing interesting about web. New door for the world.</p> </article> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/topic/server"> server </a> </h3> <p class="article-more pt-1">A server is a program made to process requests and deliver data to clients.</p> </article> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/topic/machine-learning"> Machine learning </a> </h3> <p class="article-more pt-1">Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.</p> </article> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/topic/visualization"> Visualization </a> </h3> <p class="article-more pt-1">Some thing interesting about visualization, use data art</p> </article> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/topic/game"> Game </a> </h3> <p class="article-more pt-1">Some thing interesting about game, make everyone happy.</p> </article> </li> <li> </li> </ul> </section> <section>  <h2 class="h2 py-3.5">Recommend Org</h2> <ul> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/facebook"> <img loading="lazy" class="inline-block w-6 h-6 rounded-md border border-white" width="24" height="24" src="https://avatars.githubusercontent.com/u/69631?v=4" alt="Facebook photo" /> Facebook </a> </h3> <p class="article-more pt-1">We are working to build community through open source technology. NB: members must have two-factor auth.</p> </article> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/microsoft"> <img loading="lazy" class="inline-block w-6 h-6 rounded-md border border-white" width="24" height="24" src="https://avatars.githubusercontent.com/u/6154722?v=4" alt="Microsoft photo" /> Microsoft </a> </h3> <p class="article-more pt-1">Open source projects and samples from Microsoft.</p> </article> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/google"> <img loading="lazy" class="inline-block w-6 h-6 rounded-md border border-white" width="24" height="24" src="https://avatars.githubusercontent.com/u/1342004?v=4" alt="Google photo" /> Google </a> </h3> <p class="article-more pt-1">Google ❤️ Open Source for everyone.</p> </article> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/alibaba"> <img loading="lazy" class="inline-block w-6 h-6 rounded-md border border-white" width="24" height="24" src="https://avatars.githubusercontent.com/u/1961952?v=4" alt="Alibaba photo" /> Alibaba </a> </h3> <p class="article-more pt-1">Alibaba Open Source for everyone</p> </article> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/d3"> <img loading="lazy" class="inline-block w-6 h-6 rounded-md border border-white" width="24" height="24" src="https://avatars.githubusercontent.com/u/1562726?v=4" alt="D3 photo" /> D3 </a> </h3> <p class="article-more pt-1">Data-Driven Documents codes.</p> </article> </li> <li> <article class="small-box"> <h3 class="article-title"> <a class="block break-all" href="/tencent"> <img loading="lazy" class="inline-block w-6 h-6 rounded-md border border-white" width="24" height="24" src="https://avatars.githubusercontent.com/u/18461506?v=4" alt="Tencent photo" /> Tencent </a> </h3> <p class="article-more pt-1">China tencent open source team.</p> </article> </li> <li> </li> </ul> </section> </section> </div> </div>  <footer class="sizeing text-xs text-center p-5"> <div>Friends: <a class="hover:underline" target="_blank" href="https://www.chanpinqingbaoju.com">ProductDiscover</a> </div> Copyright © 2024 Code Monkey  ❤️ <a class="hover:underline block md:inline" href="mailto:cs.victor.edison@gmail.com">Mail to me</a> </footer> </body> </html>