selects too many cores by default on orange pi 5 (2x slower) about llama.cpp HOT 3 CLOSED

You might get better performance with all the 8 cores with #6915.

that's so cool! thanks for letting me know

i give my benchmarks with that branch (kunnis:MMThreadingPerfChange) :
-t 4
prompt: 11.22 tokens per second
eval: 7.58 tokens per second
(faster than before which was 7t/s, 7.2t/s at best)

-t 8
prompt: 12.65 tokens per second
eval: 8.64 tokens per second

resulting text is still the exact same

should i close the issue or wait until it gets merged before closing it?

from llama.cpp.

You might get better performance with all the 8 cores with #6915.

from llama.cpp.

#6915 merged

from llama.cpp.