Bug: Vulkan backend not detecting multiple GPUs anymore about llama.cpp HOT 8 OPEN

llama-cli.zip

Hi, can you test this build?

from llama.cpp.

I can't test it myself, but if this is the case, then it means I understand deviceID incorrectly - it is the same for different gpu of the same model.

Can you build llama.cpp with the LLAMA_VULKAN_DEBUG option (or define GGML_VULKAN_DEBUG) to see if there is something like ...have the same device id in the log?

from llama.cpp.

If that is the case then I think deviceUUID should be used instead. I will submit a fix later today.

from llama.cpp.

@Adriankhl was afraid you might ask that :) unfortunately I have like zero clue how to build things on Windows.

• I think you are exactly right though given what I see about that data structure
• I did test the build right before this commit and saw two entries in command line output, so confident it's not something like my GPU being offline.

it's strange there's not like a serial ID of a GPU on vulkan

Not sure if "pipelineCacheUUID" is enough :/

Seems like there's something

https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VkPhysicalDeviceIDProperties.html

but it looks harder to acquire.

from llama.cpp.

@richardanaya Thank you for reporting this, I don't have two GPUs of the same type so I didn't catch this bug. A workaround for you is setting the environment variable GGML_VK_VISIBLE_DEVICES=0,1.

from llama.cpp.

I can't test it myself, but if this is the case, then it means I understand deviceID incorrectly - it is the same for different gpu of the same model.

Can you build llama.cpp with the LLAMA_VULKAN_DEBUG option (or define GGML_VULKAN_DEBUG) to see if there is something like ...have the same device id in the log?

Alternatively, can you download vulkaninfo and upload the output on your system? That should show both deviceID and deviceUUID.

from llama.cpp.

thanks @0cc4m

It does look as suspected.

GPU0:
VkPhysicalDeviceProperties:

    apiVersion        = 1.3.277 (4206869)
    driverVersion     = 2.0.299 (8388907)
    vendorID          = 0x1002
    deviceID          = 0x744c
    deviceType        = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
    deviceName        = AMD Radeon RX 7900 XTX
    pipelineCacheUUID = 3396acc0-ea27-542c-926b-685c47c98626

VkPhysicalDeviceIDProperties:

    deviceUUID      = 00000000-4700-0000-0000-000000000000
    driverUUID      = 414d442d-5749-4e2d-4452-560000000000
    deviceLUID      = fa460100-00000000
    deviceNodeMask  = 1
    deviceLUIDValid = true

GPU2:
VkPhysicalDeviceProperties:

    apiVersion        = 1.3.277 (4206869)
    driverVersion     = 2.0.299 (8388907)
    vendorID          = 0x1002
    deviceID          = 0x744c
    deviceType        = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
    deviceName        = AMD Radeon RX 7900 XTX
    pipelineCacheUUID = 3396acc0-ea27-542c-926b-685c47c98626

VkPhysicalDeviceIDProperties:

    deviceUUID      = 00000000-0300-0000-0000-000000000000
    driverUUID      = 414d442d-5749-4e2d-4452-560000000000
    deviceLUID      = 1b700100-00000000
    deviceNodeMask  = 1
    deviceLUIDValid = true
    ```

from llama.cpp.

@Adriankhl works! I see two graphics cards

from llama.cpp.