No input text for generation, why is the GPU occupancy 100%? about glm-130b HOT 4 CLOSED

Hi @Ant0082, the GPU always show 100% utilization while waiting for communication, but in fact it's idle (59W / 300W).

from glm-130b.

Well, thank you for your explanation.

from glm-130b.

After initialization, the power usage is around 10W per card. But after one request, the GPU utilization stays at 100% and the power consumption is around 50W per card. (I am using FasterTransformer for inference) How to get it back to 10 watts? Thanks.
@Sengxian

from glm-130b.

@frankang I think it's related to GPU's clock speed adjustment strategy. Our GPUs always have 50W idle power. 10W means your GPU is running in a relatively low clock speeds. After one request, GPU may boost the clock speed thus resulting in 50W idle power consumption.

from glm-130b.