[Bug] failing everytime and getting CUDA out of memory about internlm HOT 6 CLOSED

same issue here! I cloned internlm2-chat-20b several days ago, but it turns out that I got OOM after around 100 inference. I even used 4 A800. No sure what issue happened.

from internlm.

Thanks for the feedback. We'll look into it ASAP. Could you please provide the config details for the failed run?

from internlm.

same issue here! I cloned internlm2-chat-1.8b several days ago, but it turns out that I got OOM. I used 1650 super and 16GB RAM. No sure what issue happened.

from internlm.

For InternLM2-20B, it might be the issue with TP, you may try to use TP=2. For InternLM2-1.8B, this is a weird issue, could you provide more information about the environment, such as the version of pytorch and transformers?

from internlm.

This issue is marked as stale because it has been marked as invalid or awaiting response for 7 days without any further response. It will be closed in 7 days if the stale label is not removed or if there is no further response.

from internlm.

This issue is closed because it has been stale for 7 days. Please open a new issue if you have similar issues or you have any new updates now.

from internlm.