GPT4ALL crashes when switching model about gpt4all HOT 7 OPEN

Have you tried waiting until model load is successful and then try to switch?

from gpt4all.

This should be fixed with new model loading revamp that just went in.

from gpt4all.

@YS92 Could you confirm that this was fixed in v2.7.1? I'm not 100% certain that the model loading changes fixed this apparent resource leak.

from gpt4all.

I have this issue in v2.7.3. At least, I suspect it's the same issue. When the client starts, I can pick a model and open a chat just fine. If I try to switch models, there's an extremely high probability that the client will simply crash. It will also reliably (but not always) crash if I reload the same model.

The issue only happens if I use my GPU. I'm not close to my memory limit, but I am above half (about 5GB used out of 8GB total VRAM). I'm fairly confident that the current model is not being unloaded from memory before the new one loads up.

from gpt4all.

@YS92 Could you confirm that this was fixed in v2.7.1? I'm not 100% certain that the model loading changes fixed this apparent resource leak.

In v2.7.1, v2.7.2 and v2.7.3, model loading does not lead to problems on my laptop.

from gpt4all.

I've done some further testing on my system. I have confirmed that the issue only exists when running in GPU mode. There are no issues if I force the program to use my CPU.

If you think it's appropriate, I could open a separate issue for this. Either way, here's my system info, if it's helpful:
GPT4All v2.7.3
OS: Windows 10 Pro 22H2
CPU: AMD Ryzen 5 1600
RAM: 16 GB
GPU: NVidia RTX 3070
GPU Driver Version: 551.61

I have tested this with the following models:
Orca 2 (Medium)
GPT4All Falcon
Nous Hermes 2 Mistral DPO
Mistral Instruct
MPT Chat

More info:
I haven't set a default model, so nothing loads on startup for me. I can reliably load any model twice. The third model I load always causes a crash. Any combination of different models, as well as reloading the same model, results in a crash. There is one exception to this: the MPT Chat model will not crash, unless it is the third model I load. This model seems to be an anomaly because (AFAIK) it is incapable of utilizing the GPU.

To add to this, ejecting the second model usually also causes a crash. This is not guaranteed, but it happens most of the time.

I'm not convinced the issue is a resource leak. The task manager shows that, for the first reload, GPT4All does release its allocated memory, both in RAM and VRAM. The second load also doesn't appear to take up any more memory (at least not noticeably) than the first.

from gpt4all.

Does this still happen for you with GPT4All 3.1?

from gpt4all.