thoughts on parameter formats about r3broot HOT 3 CLOSED

Hi Philipp,

Most of the concerns you described here are the pillars of what we have been working at the last months, the new parameter repository for R3B parameters. This new repository is already prepared for s455 and also as a template. so anyone can clone it and create new instances of it for every experiment. The idea is to massively use FairRoot instances of FairRunTimeDb so we can manage parameter versions with a timestamp granularity for every detector. Parameters are loaded in this way and the unpacking macros are prepared to read a metafile, load the versions and apply them when needed. Experimental dependant geometries are also included in each experiment repository and also simulation and other macros needed.

This framework structure and details have been discussed from time to time in analysis meetings. These repositories are private, but anyone belonging to the R3B collaboration can ask for it. Just ask Dmytro, Héctor or Valerii.

from r3broot.

@kresan: can I please have access to that repository?

Generally, I would prefer if the relevant repositories were public readable. Our calibration parameters and detector geometries should not be secret.

Without having seen the repository, I would prefer to use less FairRoot stuff, not more. I do not exactly see what FairRuntimeDb gives you that a filename (computed for each detector/parameter type based on the initial WRTS) would not also give you in a more transparent, flexible manner.

If we want the parameters under git control, human readable file formats (for e.g. calibrations) are also beneficial there. Compare the following diffs:
*.root:

Binary files cal_old.root and cal_new.root differ

*.par:

@@ -7708,7 +7708,7 @@
   -6.7447 1.24533 -4.79958 1.23281 -4.79958 1.23281 -4.79958 1.23281 -6.7447 1.24533  \
   -4.79958 1.23281 -6.7447 1.24533 -4.79958 1.23281 -4.79958 1.23281 -6.7447 1.24533  \
   -4.79958 1.23281 -6.7447 1.24533 -6.7447 1.24533 -6.7447 1.24533 -4.79958 1.23281  \
-  -6.7447 1.24533 -6.7447 1.24533 -2.86886 1.22048 -4.79958 1.23281 -4.79958 1.23281  \
+  -6.7447 1.24533 -6.7447 1.33542 -5.74362 1.22048 -4.79958 1.23281 -4.79958 1.23281  \
   -6.7447 1.24533 -4.79958 1.23281 -7.92316 1.28317 -4.79958 1.23281 -4.79958 1.23281  \
   -4.79958 1.23281 -4.79958 1.23281 -4.79958 1.23281 -4.79958 1.23281 -4.79958 1.23281  \
   -4.79958 1.23281 -4.79958 1.23281 -4.79958 1.23281 -4.79958 1.23281 -4.79958 1.23281  \

*.csv:

- 1236, 1.24533, -2.86886, 0
+ 1236, 1.33542, -5.74362, 0

*.json

- 1236: {"slope": 1.24533, "offset": -2.86886, "p_range":false},
+ 1236: {"slope": 1.33542, "offset": -5.74362, "p_range":false},

While I dislike the first two options, either of the last two would be fine with me. (csv is easily parse-able, but perhaps a bit more fragile. json is reasonably parsed only by a dedicated library, but a bit more verbose and can support more complex data structures. Not that we would want them in the first place: I would argue for one file per parameter set.)

from r3broot.

The R3B Technical Board decided that our parameters should be, at least for a time, private, with complete access for the R3B members. That is the reason why private repositories are going to be used for the parameters within each experiment.

In R3BRoot, we are going to define procedures using the FairRoot framework to obtain, store, merge and apply them on a WRTS basis. Corrections and improvements in the code and debugging are very welcome, and your comments above, pointing on real problems, deserve further investigation and amend. But a large modification of the procedures already existing and successfully working, as it was also suggested above, requires a large manpower involvement that we do not have, mainly when many people is already instructed and using the present scheme.

The procedures for the management of the parameters are being worked out in different analysis meetings in the last months. It would be very beneficial if any proposal for modification would take into account the work and ideas of others members of the collaboration. I would recommend the participation in these meetings for all the developers.

Of course, the users are free to select their own scheme for managing parameters and perform the analysis, but the reproducibility and traceability of any result produced (and to be scientifically approved by the collaboration) would be difficult if the R3BRoot standard procedures and parameter management is not observed.

from r3broot.