what's the GPU memory utilization rate? Does it depend on read length? about dorado HOT 6 CLOSED

Hi @avilella - no GPU memory utilisation rate depends on Model, chunk size and batch size but not on read length.

The reason it does not depend on read length is that reads are "chunked" into fixed length segments before being processed by the GPU for performance reasons.

from dorado.

from dorado.

@avilella at the moment (until Multi GPU is fully working) for assigning tasks to GPUs I suggest you use --device "cuda:0" and export CUDA_VISIBLE_DEVICES=ID where ID is the GPU you want to use.

from dorado.

Closing this issue as I believe it's resolved. @avilella please reopen it if you have any more queries.

from dorado.

I kicked of dorado on a simulated [email protected] dataset using an NVIDIA RTX A4000 (16GB VRAM) GPU.

The memory usage with the different --batchsize N options is shown below:

1 = 1664MiB / 16117MiB
100 = 2466MiB / 16117MiB
1000 = 9615MiB / 16117MiB
1984 (2*(2^10)-(2^6)) = 15930MiB / 16117MiB

So --batchsize 1984 is the optimal for a 16Gb VRAM Nvidia GPU. Full command-line:

dorado basecaller --device cuda:0 --batchsize 2000 --remora-threads 1 /data/ONT/dorado/[email protected] . > calls.sam

On a smaller NVIDIA T1000 GPU with 4GB VRAM, the optimal batchsize was 384 (updated on @vellamike 's comments):

384 (2^9)/2+(2^7) = 3828MiB /  3911MiB

from dorado.

Maximising the utilised VRAM with the biggest possible batch size to increase performance is a good rule of thumb, but not necessarily something which always works. For example it is often the case that good batch sizes are powers of two or integer multiples of powers of two.

from dorado.