With gcc>9, the compiler complains about a missing dependecy on fairlogger (FairLogger::FairLogger) and boost headers such as serialization (Boost::headers). To be updated soon.

Need to check to make sure the map was set when initializing the run

Solved by adding FairLogger (DEPENDENCIES) in the AtData CMakeLists

AtTabInfo is only really used in AtTabMacro, and could be replaced with the subject/observer implementation. It would keep multiple instances of tracking information that may be updated asynchronously and shared across different classes from being in use simultaneously.

Some other class use the AtTabInfoFairRoot class, but not in a way that is required. By design that class is basically an observer in use, just the observing is left to the AtTab class and AtTabInfoFairRoot just acts as a wrapper for a DataHandling::AtBranch subject with easy access to the underlying data (with a similar interface to a smart pointer). This use does not require implementing the AtTabInfo base class. Instead you could have something like an DataHandling::AtFairRoot<T> that keeps the same interface as AtTabFairRoot, but is an DataHandling::AtSubject (which can let its observers know when the underlying object it is referring too has changed) and it is an DataHandling::AtObserver that gets notified if either the branch it is observing, or the event number in the event, has changed. It would also allow for a common interface for accessing entries in the underlying TClonesArray if there is not just one entry (like in the AtMCFitter implementation).

Moving unpackers so they are a FairSource, instead of a task will allow us access to the full capabilities of a FairRun, including the execution of certain base functions in a FairTask that are skipped when running without a proper FairSource.

Without a proper source, FairRun excecutes a DummyRun() instead of a normal run. A dummy run just runs the Exec() and FinishTask() methods, it skips the FinishEvent() method, and will not handle early interuptions to the event loop properly (and also does not respect calls to FairRootManager::Instance()->SetFinishRun().)

I have previously compiled ATTPCROOT on our cluster, but fail to compile a version I downloaded on July 20, 2022.

I am using GCC 10.2.0 and the output from cmake and make are attached.

cmake_output.txt

make_output.txt

CMakeCache.txt

If you see the error along the lines of G__AtToolsLinkDef dictionary payload:1797:2: error: #endif without #if it likely means on of your header files does not end with a newline.

In that case ROOT shoves the headers together in a way where the #ifndef in the header guard is commented out by the previous header file.

This line guarantees that the track with largest number of hits will be returned. Does not imply that this is actually the most convenient track.

https://github.com/ATTPC/ATTPCROOTv2/blob/develop/AtReconstruction/AtPatternRecognition/AtPRA.cxx#L58

It happens when digi and reco tasks are performed in different chained macros.

Revise destructors of base classes used polymorphically. They should be changed to virtual.

Check the performance of the base line correction due to the induced mesh signal for saturated pads.

For some reason, the update to Geant4 packaged in the most recent update to FairSoft breaks our simulations. The issue seems to be related to the G4eIonisation process where Geant4 throws an exception while trying to initialize that physics (I don't remember if it was an issue for positrons or electrons).

Until a fix is found, only use the nov22_patches branch of FairSoft.

When ATTPCROOT is installed in the system folders (i.e. /usr/local), the compiled version of the fitter analysis will find these first. Conflict appears when the ATTPCROOT system libraries are outdated with respect to the current version of the code.

Goal

To integrate the classification of events using PyTorch models. The training will be handled in python, it is just the classification that needs to be injected into the ATTPCROOT analysis stream. The generation of the tensor input should be done fully within the ATTPCROOT framework (because when classifying we are not calling any external python code)

Design Requirements

• Trained model from PyTorch, serialized into a file with TorchScript
• Ability to convert ATTPCROOT data (AtEvent/AtRawEvent) into a tensor input.
• Ability to save tensor input to an HDF5 file for training
• Ability to use trained model and tensor input to classify an event

Proposed design

There are two main additional tasks needed in the code.

1. Task to generate tensor from data and save to disk (in AtReconstruction?)
   • Proposed name: AtTensorTask
2. Task to classify events based on TouchScript model (in AtReconstruction?)
   • Proposed name: AtClassifierTask

In addition there will also be a namespace containing classes for generating input tensors from the ATTPCROOT data. These classes will be added to the LibTorch namespace torch. Our methods for generating tensors should follow the conventions established by LibTorch

AtTensorTask

This will be a wrapper task (similar to AtPSATask) where the bulk of the logic of generating the tensors lives in other classes that are more removed from ROOT.

• Input
  • std::unique_ptr<AtTensorConverter> (abstract base class) or maybe an enum indicating which tensor generation method to use.
  • Filename to save tensor to
• Output
  • HDF5 file filled with the tensors to use as an input for all events in the run (expect simulated)

AtClassifierTask

This task is responsible for classifying events based on a PyTorch model.

• Input
  • std::unique_ptr<AtTensorConverter> (abstract base class) or maybe an enum indicating which tensor generation method to use.
  • File path to TorchScript model
• Output
  • AtClassifiedEvent (Derived AtEvent with a field storing classification info)

Tensor generation

This section is by far the most nebulous. I haven't worked with PyTorch at all so am basing this mostly on a skim of the documentation. I think this will cohere into something reasonable as development continues. My initial idea was to have an abstract base class for the generation of tensors. Each experiment would then have their own, that made heavy use of private inheritance (or containment) for the creation of common components of a picture (pad plane for different detectors, image of the mesh signal, indication of total charge in an event etc). It is then simply the job of the experiment-specific class to combine all of these individual components into a single tensor. While testing the implementation of the code it would also be useful to have the ability to read a tensor from a h5 file (for example comparing with the full python implementation used by the GADGET group to make sure we can reproduce the same results).

It looks like the existing tensor generation methods of LibTorch follow a different convention using free functions grouped into a namespace rather than inheritance through classes. Following the conventions established by LibTorch we will also have a header file that defines a number of free functions in the torch namespace that operate as the factory for each way of generating tensors (think something similar to the use of AtEstimators in SampleConsensus where they exist as free functions in a namespace). It is likely that each of experiment or physics "problem" will end up implementing their own free function in this namespace. There should be a number of helper functions for generating these tensors for common tasks, which can then be spiced together. This maps almost directly onto my first idea, but you loose the ability to store a "state" which I don't think would be a huge deal. It depends on implementation choices.

Dependency changes

There are CPU-only and CUDA versions of LibTorch. I would like to support both, but I think the ability to do that well will depend heavily on how good there CMake is. Ideally, if we found both versions we would use CUDA-enabled if CUDA is available and fall back to CPU only if necessary.

CMakeLists.txt
ATTPC_geometry.zip
cmake_ATTPCROOT.txt

I get a segmentation error when trying to run the run_sim_attpc.C macro in the test directory. I have attached the cmake output and the geometry file.

Standard Catalina macos installation with PCL1.8 is not able to compile ATTPCRootv2 with the message

[100%] Linking CXX shared library ../lib/libATEventDisplay.dylib
ld: file not found: libpcl_io_ply.1.8.dylib for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [lib/libATEventDisplay.0.0.0.dylib] Error 1
make[1]: *** [eventdisplayAT/CMakeFiles/ATEventDisplay.dir/all] Error 2
make: *** [all] Error 2

despite a
PCL Support NOT Found: Classes and modules depending on PCL will not be built.
message appears when running cmake

The problem is easily solved in the develop version by removing the CMakeLists.txt line:

#find_package(PCL 1.8)

Currently the trajectories are just a vector of hit-indices.
In the near future this will be replaced by a new class ATTrajectory, which will contain the following information:

• indices of points belonging to that trajectory
• average curvature
• start index (index of the point at the very start of the trajectory)
• end index (because, why not?)
• length of the trajectory (following the curve; using a
  minimum-spanning-tree and shortest path from start to end)
• centerpoint of the trajectory (average position of all points belonging to that trajectory)
• main direction (basicly a PCA)

This function hard codes the branch for AtEvents, and also requires an event to be in the S800 gate by default. That's bad as most data is not S800 and that variable defaults to false it is constantly checking. Old behavior should be the default with an option to turn on gate checking that has to be specified by the user in the macro.

[FATAL ] [28.02.2016 18:54:59] [ATHoughSpaceCircle.cc::ATHoughSpaceCircle:44] No analysis run!
[FATAL ] [28.02.2016 18:54:59] We stop the execution of the process at this point.
[FATAL ] [28.02.2016 18:54:59] For later analysis we write a core dump to core_dump_6349

This can be solved by creating an instance of FairRootAna or just modifying the container for the Hough Space, this is to create a Hough container and not to save the Hough object in the TClonesArray.

The initial track cluster used for fitting has an undefined covariance matrix. This might be affecting the fitting perfomance.

The output (filtered) event of a AtPadFFT does not have the proper fourier components. They are too low (probably from over correcting for the sample size, possibly other issues?)

@giraudsimon @jczamorac I have modified the ATTPCIonDecay generator so it can work with any other generator. The connection with the d2He has to be redone or the code refactored in a way that it is treated universally.

AtSiPoint inherits from the TPC Mc point (not sure if it should be like this) and it is not linking properly (can be reproduced by creating a AtSiPoint in root command line)

Weighted Y sample will hang at certain events. Probably when all the hits in the event are vetoed.

I think clang-format changed how parameter packs were formatted by default in a new verison. I'll merge it and see if I can't set the setting on our clang-format parameter file to manually specify so it isn't clang-format version dependent.

For some reason it now wants AtPad *AddPad(Ts &&...params) instead of AtPad *AddPad(Ts &&... params)

In ATHoughSaceLine.cc:
HistHoughRZ (line 350)
HoughPar (line 612)
In VMEDecoder.cc:
CoinReg (line 343)

Is this intended?
If yes, I can fix it to get rid of the warning.

Some of the enumerators and free functions in the namespaces SampleConsensus and RandomSample are missing from their respective linkdefs, so the root interpreter cannot auto load the associated libraries.

Example error:

.

AtTPC20MgDecay current outputs ~1000 lines of clang-tidy messages.

Some of them look like they could be actual bugs, like if statements that always return true such as:

Add vertex position and energy as simulation output variable.

-- Configuring done
WARNING: Target "ATTPCReco" requests linking to directory "/lib". Targets may link only to libraries. CMake is dropping the item.
WARNING: Target "ATTPCReco" requests linking to directory "/lib". Targets may link only to libraries. CMake is dropping the item.
WARNING: Target "ATDigi" requests linking to directory "/lib". Targets may link only to libraries. CMake is dropping the item.
WARNING: Target "ATEventDisplay" requests linking to directory "/lib". Targets may link only to libraries. CMake is dropping the item.

This change will be reverted back. HoughSpaceLine and ProtoAnalysis have been modified to allow a recalibration of the Z position.

Some NOOPs I came across during my cleanup:

std::vector<std::pair<Double_t, Double_t>> ATHoughSpaceLine::GetMultiHoughParameters(TH2F *hist)
calculates maxBin, xpos1 and ypos1 but is never used. It always returns an empty std::vector<std::pair<Double_t, Double_t>>.

void ATHoughSpaceLine::CalcHoughSpace(ATEvent *event, TH2Poly *hPadPlane, const multiarray &PadCoord)
does nothing

For the physics generators, pointers should not be stored as member objects.

See for example ATTPCFissionGenerator class

Issue of file ordering when using a .txt file to merge graw data with several files per CoBo
Need to re-write the shell script (here or there) to correctly order the file names

Currently, the HDF unpacker does not exit when a file is not found.

This member function returns a pointer to an object allocated in the function but living in the TClonesArray of the same class. Needs proper documentation.