Initialize random seed at the beginning of an FBPIC simulation about fbpic HOT 12 OPEN

Hi @abonatto
I opened a corresponding PR here: #599
Would you be able to install this version of FBPIC and test whether this suits your need?
Note that, for execution on multi-threaded CPUs and on GPU, there could potentially still be some fluctuations, due to the fact the scheduling of computation on the different threads is not purely deterministic. Not sure how important this effect is, though.

from fbpic.

Note: we should probably at least indicate why the results of identical simulations can fluctuate, in the documentation.

from fbpic.

Would it make sense to use a fixed seed for the random number generator to have consistent results of two identical simulations?

from fbpic.

Yes, this would definitely make sense, but there are a few caveats:

• Setting the random seed with np.random.seed would fix the random angles at which particles are initialized. But it would not affect the random generator used for ionization calculations (which does not use numpy directly, but is instead automatically created by numba/numba.cuda).
• During a restart, it is not obvious how to retrieve the state in which a random generator was, at the time of the checkpoint. So a restarted simulation would probably still give different results than a full simulation. (Unless numpy provides an API for getting the current random state, and we then save it in the checkpoint.)

from fbpic.

Also, if we use np.random.seed, I am not sure when to call this:

• When FBPIC is imported ?
• When the Simulation object is created? (but then initializing several Simulation in the same script object would overwrite the seed.)

from fbpic.

Also, if we decide to set the seed when the Simulation object is created, we could add the seed as an argument of the Simulation object, which defaults to None (i.e. when this is None, the seed is not set and the results will fluctuate from one run to the next.)

from fbpic.

Hi @RemiLehe and @soerenjalas,

May I ask you if this enhancement is still on your radar?

The possibility of initializing the random seed (for both CPUs and GPUs) would be very useful in the context of applying algorithms such as, for example, Bayesian optimization. We are investigating a case (an LPA in the self-modulated regime) in which we defined an objective function to be sensitive to high-energy particles. For this case, the fluctuation of its values for repeated simulations with the same parameters are affecting the convergence.

By setting the numpy random seed and starting the simulation with an ionized plasma (rather than using the ionization), we eliminated the aforementioned fluctuation. However, the optimal parameters were affected by differences in the spectra with and without ionization (starting with an ionized plasma, we see lower maximum energies, and an increase of charge in certain energy ranges). Due to this, we would like keep using results from simulations in which the ionization process is adopted. In this context, having a way to set the random seeds for both CPUs and GPUs would be really great.

Regards,

Alex

from fbpic.

Hi @RemiLehe,

Thank you very much for your fast reply.

Sure thing. Let me install and test the version you pointed me out, and I will let you know the outcome. Hopefully the small fluctuations you mentioned will not be relevant for our case.

from fbpic.

Hi @RemiLehe,

I just installed it (pip install git+https://github.com/fbpic/fbpic.git@dev). In our current script, in order to set the CPU seed, we set the numpy seed (for example, np.random.seed(0)) within the simulation routine, after the sim = ... and before adding the particle species (elec = sim.add_new_species(...). Regarding the GPU seed, could you please point me out how can I set it? In addition, if I am using four GPUs, I suppose I should set a (distinct) seed for each of them, correct?

from fbpic.

Hi @abonatto
In order to install the code from the new PR (#599), you will need to do:

pip install git+https://github.com/RemiLehe/fbpic@fix_random_seed

Once you do this, you will not need to set np.random.seed in your input script anymore, but instead you would pass random_seed=0 directly to the Simulation object (since the code changes in the PR). This will also automatically set the seed for the GPU code.
Regarding setting a different seed for each MPI ranks: this would be a good idea, but I don't think it is crucial at this point. For now, if you just want to confirm that simulations are reproducible, you can simply set random_seed=0 and evaluate whether you get the same results when running the same simulation twice.

from fbpic.

Hi @RemiLehe,

Thanks for your detailed explanation. I will run the simulations using the new PR, and let you know the results.

from fbpic.

Hi @RemiLehe,

We performed four simulations, and the results now are reproducible. As expected, residual fluctuations are observed (0,7% in the total charge and 2% in the maximum energy observed in the final spectrum), but they indeed will not affect. By setting the seed, the fluctuations in our objective function (which we aim to maximize by using Bayesian optimization) decreased from ~10% to 0,2% :-).

Thank you very much for your support, Remi!

from fbpic.