In terms of documentation, this only affects the README, the rest should be up-to-date already since it's already working in CMSSW_9_3_0_pre3.

This is to illustrate the issue of sporadically missing layer clusters.

It was observed that good multiclusters can sometimes miss one (or several) layer clusters even close to the shower maximum.

• This plot shows all rechits associated to any multicluster for a single 25 GeV photon. The color coding reflects the rechit_flag: 0 stands for core hits, while 2 are halo hits.
  It is clearly visible that in layer 9 most rechits are halo hits.
  

• This plot shows the same event as the previous but only for layer 9. There are only 2 core hits.
  

• In this plot the colour of the rechits corresponds to rechit energy. Most energy is deposited in the central hits.
  

• In this plot the colour of the rechits corresponds to the associated 2d cluster index. There are multiple 2d clusters, while only 1 has core hits.
  

• In this plot the colour of the rechits corresponds to the associated multi cluster index. The central hits (light blue colour) correspond to a multicluster with ~22 GeV, i.e. the one matching the photon. Since these hits are all halo hits, their energy (~6GeV) is also not taken into account in the total (~90GeV).
  

• In fact, the proper hits are assigned to two adjacent layer clusters. The color reflects the 2d cluster index.
  
  The boxes show the cluster seeds, and they appear to be adjacent cells. This would mean, that both hits pass the delta_c cut, but at least one of them should have a delta of 1cm – the cell distance.

I'll try to find this behaviour in the cmssw execution and post an update later.

Hello
the recipe from the ReadMe does not work.

The compilation fails with:
CMSSW_10_2_0_pre1/src/RecoNtuples/HGCalAnalysis/plugins/HGCalAnalysis.cc:1767:42: error: 'class hgcal::RecHitTools' has no member named 'getPositionLayer'; did you mean 'getPosition'?
const GlobalPoint pos = recHitTools_.getPositionLayer(ilayer);
^~~~~~~~~~~~~~~~
I can take a look at it unless someone already knows how to fix it.
F.

I'm not sure why we had to reuse FastSimulation tools inside the reco-ntuple.
Inspired by the problem Jeremy reported last Monday, I discovered that the FastSimEvent does implicitly make assumptions (hardcoded, of course) on the geometry that are indeed false and meaningless for the HGCAL case.
I'm, in particular, referring to these values that are required from here. This confuses the ntuplizer that will simply skip valid simTracks for no good reasons.
And no, of course they are not configurable.

There are a few handicaps with the class-based ntuple:

• cannot analyse ntuples without CMSSW/compiling and loading classes
• whenever the classes change, older ntuples cannot be read anymore

I would like to change the ntuple such that it contains only flat branches (and vectors, we probably need vector<vector>). This would mean that everyone needs to change their analysis scripts, but I hope it's not too much work.

Let me know what you think by replying here. We'll probably introduce a branch in parallel to master so that people can have a look before changing the default if people agree.

Sunanda suggested to define wafer/cell as a pair in the ntupliser, for the old geometry the second integer will be 0. This will change the ntuple format.

I'm not sure I can follow the logic of reachedEE in this code snippet. To my limited understanding and debugging, the reachedEE==1 case is never a valid one.
Also the hardcoded 160 and 25 are something we should avoid as much as possible.

We will descope the ntupliser such that it will only dump collections and the links between them. In particular, we'll remove:

• code that calculates additional variables such as the one for electrons
• remove MultiClusters since they are currently not maintained

Eventually, the ntupliser will be moved to CMSSW.

We have a lot of different ntuples available by now, but since things change quite frequently, we should add some provenance information such as

• CMSSW release used
• reco-ntuples and reco-prodtools git tags used
• what else?

An alternative could be a simple txt file that contains this information and maybe the config files used.

There is no longer 100 um thickness [1] (at least for V9 and V10). Same here [2]. .

[1] https://github.com/CMS-HGCAL/reco-ntuples/blob/master/HGCalAnalysis/plugins/HGCalAnalysis.cc#L1819

[2] https://github.com/CMS-HGCAL/reco-ntuples/blob/master/HGCalAnalysis/plugins/HGCalAnalysis.cc#L2047

The ntupliser curently assumes that sharing is not enabled and therefore a 1-to-1 association between rechits and layer clusters exists: https://github.com/CMS-HGCAL/reco-ntuples/blob/master/HGCalAnalysis/plugins/HGCalAnalysis.cc#L1831-L1851

With sharing enabled, a rechit can be associated to more than one cluster. If we want to preserve this information in the ntuple, we would need to change rechit_cluster2d_ (the variable that stores the index of the associated layer cluster, https://github.com/CMS-HGCAL/reco-ntuples/blob/master/HGCalAnalysis/plugins/HGCalAnalysis.cc#L1917) to a vector of integers.

Any comments or alternative suggestions?

see here: https://github.com/CMS-HGCAL/reco-ntuples/blame/master/HGCalAnalysis/plugins/HGCalAnalysis.cc#L1033

PFCluster, which are the realistic SimClusters for hadronic particles, need more information:

• correctedEnergy()
• hitsAndFractions()
  Anything else?

I am currently working on the following:
Additions:
A1) extrapolation of each generated particle to the first HGCal layer (whether it decayed or not). Corresponding 4 branches (exx,exy,exeta,exphi) are added to the ntuple. Needed for DNN studies.
A2) origin vertex information (x,y,z), there used to be only the decay vertex information

Bugfixes:
B1) change all extrapolations to FullSim rather than FastSim. FastSim does not do a good job in the inhomogeneous field in the endcaps

Changes:
C1) Right now, each electron or particle that fulfils "myTrack.genpartIndex()>=0" gets fully propagated to the HGCal and it's decay vertex is set to the first HGCal layer, even if it decays in the meantime. This overlaps with the additions in A1 and was always affected by bug B1. Please let me know if this feature is used by anybody. Since the same information is now given by addition A1, I would propose to remove this feature.
For the DNN studies, I need the information on the real decay vertex, that I would otherwise need to add for these particular particles in new branches.

Dear all,
instead of writing a longish email to explain something that should be already clear from the subject, I just paste here the link to the comparison between my branch and the current HEAD with all the changes I propose to include.

Besides adding electrons (and their associated multiclusters decomposed in PFClusters), I also added a very simple python prototype to be used for quick analysis, with a rudimentary Zee example. Maybe you can find it useful.

https://github.com/CMS-HGCAL/reco-ntuples/compare/master...rovere:devel?w=1

The variables added in #72 should be defined in https://github.com/CMS-HGCAL/reco-ntuples/blob/master/Definitions.md.

@riga

As mentioned on mattermost, the possibility of storing raw rechits was partially removed by this commit: 204d368, however, the flags still remain:

I'm having a look whether it's possible to keep the rawRechit flag working.

When matching hits via DetId between SimClusters (based on what's stored in hits and fractions) and RecHits, there are more hits associated to the SimCluster than there are in the RecHits collections. This is not the case for PFClusters. The ntupliser code should dump all RecHits there are, so there must be some thresholds applied to the RecHits before being saved to the respective subdetector collections. This has not been the case in the past, so this is not a bug in the ntupliser, but a possibly a (desired?) feature in CMSSW. Investigating in CMSSW...

Will implement 4-vectors and their ID.

52 is given hardcoded when using retrieveLayerPositions. There are no longer 52 layers in V10 (D41) but 50 and it would be good to read those values from CMSSW. This issue here is related [1].

[1] cms-sw/cmssw#26225

Add this information (4-vector , PDG ID + ?), but with a flag that is set to False (= do not write out) by default. What about filtering on status code?