v2.7.3 crashes when loading large models, where v2.5.1 did not about gpt4all HOT 4 OPEN

2.5.1 version can use various models, but the device can only use CPU and not GPU, with an error message indicating that GPU loading failed (out of VRAM). On the other hand, 2.7.3 version cannot use memory requirements of 16GB models, but it can use 8GB models and also use GPUs

from gpt4all.

How much RAM do you have? Do you think it is possible that GPT4All is running out of RAM (e.g. does it crash when you set the device to "CPU"), or is it really crashing when it runs out of VRAM? The latter is possible, but it would definitely be a bug and not an intentional occurrence.

from gpt4all.

I am having this issue as well, 4090 and 96gb of memory. Running on cpu fixes crash but runs slow af

from gpt4all.

I have the same problem, 80GB memory, NVIDIA RTX 3060.

QML debugging is enabled. Only use this in a safe environment.
[Debug] (Mon Apr 22 06:20:54 2024): deserializing chat "F:/AI/gpt4all/nomic.ai/GPT4All//gpt4all-3ca3afb4-8c17-4c97-8693-135477a84612.chat"
[Debug] (Mon Apr 22 06:20:54 2024): deserializing chats took: 4 ms
llama_new_context_with_model: max tensor size = 102.54 MB
llama.cpp: using Vulkan on NVIDIA GeForce RTX 3060
error loading model: Memory type index for buffer creation not found
llama_load_model_from_file_internal: failed to load model
LLAMA ERROR: failed to load model from F:/AI/gpt4all/nomic.ai/GPT4All/wizardcoder-python-34b-v1.0.Q4_0.gguf
GGML_ASSERT: C:\msys64\home\Jared\gpt4all-navarro\gpt4all-backend\llama.cpp-mainline\llama.cpp:552: data

from gpt4all.