Parallelization limit causes unintuitive timing benchmarks (was: Multi-thread support is unclear) about qvm HOT 5 OPEN

Hey @vincentelfving, thanks for the comment.

It is an open issue to define the proper "parallelization limit". This is statically defined at 19 qubits currently here, but it should be calibrated on a per-machine basis. The number 19 was chosen because that's what it was for one model of laptop I was using. This issue is described here. This would be a wonderful contribution to determine where the crossing point is.

As a side note, as you increase the number of cores, for some qubit numbers, adding a new core doesn't actually give you a speedup. For instance, on my machine for a 32q benchmark, 10 cores vs 20 doesn't provide anything. This is something I intend to investigate to see if we can eek out more parallelization speedup, and if not, determine why (e.g., memory bandwidth, blowing the cache, etc.).

from qvm.

@stylewarning alright, that makes sense! I realize that parallelization speedup is actually highly non-trivial.... I guess there is not 1 magic number.
I would, however, love to see if it is possible to gain more speedup for low qubit numbers like N=4-18, for testing in cases where a huge number of (variational) circuits and iterations are necessary. In that case even every microsecond per circuit adds up significantly to the total time.

from qvm.

@vincentelfving I just made a PR #31 to allow this parameter to at least be controllable by you, though it won't be calculated automagically. It should be merged by EOD, if you care to build from source. If not, it'll be in the next release of the QVM.

from qvm.

@stylewarning excellent, thanks a lot!

from qvm.

@vincentelfving Merged. Also, I notice you don't seem to be aware of the --compile or -c option. Try doing:

qvm --verbose --benchmark

and then

qvm --verbose --benchmark -c

It's not always the right thing to do, especially if you have a very small number of qubits, but it can bring great wins otherwise.

from qvm.