Comments (3)
When I was trying to reason the 13b model, I loaded the data set using only one card to run. My environment is 2*3090, and there are the following problems.torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 136.00 MiB. GPU 0 has a total capacity of 23.48 GiB of which 124.81 MiB is free. Including non-PyTorch memory, this process has 23.31 GiB memory in use. Of the allocated memory 23.06 GiB is allocated by PyTorch, and 2.76 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables).Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s]
Loading checkpoint shards: 33%|███▎ | 1/3 [00:04<00:09, 4.91s/it]
Loading checkpoint shards: 67%|██████▋ | 2/3 [00:09<00:04, 4.95s/it]
Loading checkpoint shards: 67%|██████▋ | 2/3 [00:11<00:05, 5.97s/it]
In fact, he only used the first card,
from opencompass.
[qwen-v1.5-14b-hf/LongBench_vcsum,qwen-v1.5-14b-hf/LongBench_narrativeqa,qwen-v1.5-14b-hf/LongBench_multifieldqa_zh,qwen-v1.5-14b-hf/LongBench_lsht,qwen-v1.5-14b-hf/LongBench_dureader,qwen-v1.5-14b-hf/LongBench_passage_retrieval_zh] For this tasks information, it seems that you use partitioner to allocate 4 tasks on 4 gpus, so if you want to use 4 gpus only for one task, just don't use partitioner will be ok. By the way, you can also use VLLM to do inference
from opencompass.
您好,我也遇到了相同的问题,窗口长度在32K时会OOM,请问您是怎么解决这个问题的?
from opencompass.
Related Issues (20)
- [Bug] CLUEWSC的测试结果全是 50%
- [Feature] Leval数据集少了两个config:codeU和sci_fi
- [Bug] 使用api测评时mode参数不起作用,超出max_seq_len并没有按mode切分输入
- [Feature] Falmes dataset evaluation seems to be missing configs and json file HOT 3
- [Bug] 评测lawbench数据集时偶现异常
- [Feature] 支持openai/GPT4-o的评测seting HOT 1
- GenInferencer PPLInferencer 不能集成到一起吗[Feature] HOT 2
- [Feature] 如何在needlebench 中使用api model? HOT 1
- [Feature] config的bug,提示下载configs,然后下载了又出现以下bug
- [Bug] unrecognized arguments: --no-batch-padding HOT 1
- opencompass榜单更新情况 HOT 2
- [Bug] hf_chatglm3_6b评测AFQMC数据集时,自测结果与官方不一致。且自测结果不稳定。 HOT 1
- No module named 'opencompass' HOT 9
- [Bug] Unable to use tutorial methods properly——KeyError: 'opt125m'or'opt350m' HOT 1
- [Bug] opencompass/cli和opencompass/datasets/IFEval下缺少__init__.py所以release版本是不能导入这两个包的 HOT 1
- [Bug] configs/datasets/agieval/agieval_mixed_713d14.py not found
- [Bug] llm-compression task faild at eval stage with latest version HOT 3
- [Bug] which version of the dataset should be selected When evaluating the Llama3 model,
- [Bug] run pytorch Qwen-7B-Chat with ARC-c ppl under CPU ,and result is not good HOT 1
- 大海捞针数据集初始化报错( Failed to get opencompass.datasets.needlebench.origin.NeedleBenchOriginDataset)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opencompass.