Bug: gpu hang after bde7cd3cd949c1a85d3a199498ac98e78039d46f about llama.cpp HOT 4 CLOSED

i found that vram dont have significant increase while inferring. so maybe something else caused the issue

from llama.cpp.

bisected and found the commit caused the issus, so keep it open

from llama.cpp.

also #7769 #7640

from llama.cpp.

@0cc4m this is probably my bad, I made some changes to the way views are initialized in ggml-backend that may have created this issue. Views are now initialized in the buffer of their parent tensor, instead of on the compute buffer. The reason I made this change is because I came to the conclusion that allocating views on the compute buffer cannot work reliably because the compute buffer is not always of the same type as the buffer used to allocate the tensor originally, and backends should be able to use the same extra as their parent anyway. I thought it was safe to make this change because the CUDA backend no longer needs extras for normal buffers, but I didn't realize that the vulkan backend still does.

Looking at the ggml_tensor_extra_gpu of the vulkan backend I think it should be possible to do this, the only change is that you would have to calculate the offset as t->extra->offset + t->view_offs. Essentially, add the offset of the view to the offset of the extra. Does that sound right?

from llama.cpp.