cms-analysis / combineharvester Goto Github PK
View Code? Open in Web Editor NEWCMSSW package for the creation, editing and analysis of combine datacards and workspaces
Home Page: cms-analysis.github.io/CombineHarvester/
CMSSW package for the creation, editing and analysis of combine datacards and workspaces
Home Page: cms-analysis.github.io/CombineHarvester/
It appears full-documentation.md
and tutorials-part-2.md
are both outdated and redundant with other portions of the documentation. Should we remove them? Same for the tutorial2020
folder.
The lines 473, 474, 478 and 480
min = float(x)
max = float(y)
Should be changed to:
min = float(x.value)
max = float(x.value)
The way it's now it's not working, since x is not a number or a string, it is a ctypes.c_double object.
Dear all,
I've accidentally pushed some fork specific changes for CH to the official CH repo without making a proper pull request:
https://github.com/cms-analysis/CombineHarvester/compare/041188035d29b08580f5e11f94d79b529f7c640c..master
However, as you can see from the comparison, these are only some cosmetic changes to the (plotting), and extensions of the CMSHistFuncFactory, which are needed for MSSM application.
If you are OK with these changes, we could leave them as they are, overwise a git revert
would be needed.
I apologize for this inconvenience,
Cheers,
Artur
Currently sourcing files from the old repo
Hi!
The nominal mass point gets an unexpected value when calling BuildRooMorphing
with the file parameter, for example when creating the workspace and datacards including:
...
RooRealVar scannedParameter("scannedParameter", "scannedParameter", -4.0, 18.0);
RooWorkspace ws("htt", "htt");
ch::CombineHarvester cb;
...
string b = cb.cp().channel({channel}).bin_set().at(0);
string p = cb.cp().bin({b}).signals().process_set().at(0);
TFile debug('debug.root', "RECREATE");
ch::BuildRooMorphing(ws, cb, b, p, scannedParameter, "norm", true, true, false, debug);
...
and calling then FitDiagnostics
and PostFitShapesFromWorkspace
to have a look on the pre/postfits one gets as a prefit the plot that corresponds to scannedParameter=18 in this example, so to the last value set in RooRealVar. My colleague pointed me that the actual reason for this is the way the RooRealVar is handed along in the Morphing process (as a pointer) and the fact that its value gets modified when writing the debug file. One can come around it by not using the file parameter, not by using the --setParameters
option. I believe it's not a wanted behaviour?
Thank you for your assistence!
Best regards, Olena
When using -t 1 --toysFile <filename>.root
to load in a toy dataset, the Impacts algorithm will not account for the change in output file names that includes the seed.
For example, after --doInitialFit
, the output will be a file that looks like higgsCombine_initialFit_Test.MultiDimFit.mH{m}.123456.root
but the --doFits
step will look for higgsCombine_initialFit_Test.MultiDimFit.mH{m}.root
. I'm working around this by renaming the files between steps but I imagine this should be changed in lines like these [1].
[1] -
CombineHarvester/CombineTools/python/combine/Impacts.py
Lines 151 to 154 in 0675d6a
Should provide example scripts and documentation for the following:
Later:
Possible way to do it without modifying combine:
Now when we go outside the range the channel NLL no longer changes as a function of the parameters, just leaves a constant term which shouldn't affect the minimisation at all.
Dear CombineHarvester experts,
we noticed an unexpected behaviour of the PostFitShapesFromWorkspace
tool, when adding the datacard of a DNN score (0..1) for rebinning purposes:
The rebinned histogram had only as bin content the content of the first bin of the e.g. prefit_shapes
.
Is this behaviour known?
(omitting the rebinning option via -d
works fine and yields the expected output)
Best, Peter
Hi,
I was following the instruction in "Examples Part1" where doing the scram b, I am getting an error like
Compiling /uscms_data/d3/snandan/CMSSW_7_4_7/src/HiggsAnalysis/CombinedLimit/src/RooParametricShapeBinPdf.cc
/uscms_data/d3/snandan/CMSSW_7_4_7/src/HiggsAnalysis/CombinedLimit/src/RooParametricShapeBinPdf.cc: In member function 'virtual Double_t RooParametricShapeBinPdf::analyticalIntegral(Int_t, const char*) const':
/uscms_data/d3/snandan/CMSSW_7_4_7/src/HiggsAnalysis/CombinedLimit/src/RooParametricShapeBinPdf.cc:236:1: internal compiler error: in possible_polymorphic_call_targets, at ipa-devirt.c:1556
}
^
Please submit a full bug report,
I didn't modify anything inside. Could anyone please let me know what is wrong?
Thanks
Saswati
I’m trying to (gradually) switch to combineHarvester for the hh->bbtautau analyses, but it seems I’m not able to use it to parse a datacard/workspace with RooDataHist objects, do you know if this is a known feature, or if there’s something special to do to load RooDataHists in combineHarvester?
If you please, let me describe a bit in detail what is going on.
You can find here:
/afs/cern.ch/user/g/gortona/public/4Andrew/cards_MuTauprova/HHSM2b0jMcutBDTMT2/hh_2_C2_LHHSM_13TeV.txt
a simplified card with just few backgrounds and no shape systematics. It can be linked to either a file with one TH1F per process or to a file with a workspace where shapes are stored as RooDataHists (both files are in the same folder).
text2workspace works properly, so the card should be fine, and indeed when I parse the card linking it to the TH1Fs all goes through properly. The script I am using is “harvesta.py” in the same folder.
But when I link the card to the workspace, I hit "Problem: RooAbsData not found in workspace”.
I tried to dig a bit, and it seems that in loading the workspace into combineHarvester the datasets names are lost. Adding mapping.ws->Print() at line 319 of combineHarvester.cc prints out the message below, but unfortunately I couldn’t track the origin of this behaviour more than this.
Did this already happen to you? Do you know if I’m doing anything wrong or if is there any way to fix this behaviour?
Thank you a lot,
Giacomo
mapping.ws->Print()
RooWorkspace(w) w contents
variables
---------
(CMS_HHbbtt_scale_tau,MT2)
p.d.f.s
-------
RooHistPdf::ggHH_hbbhtt[ pdfObs=(MT2) ] = 0
datasets
--------
RooDataHist::(MT2)
RooDataHist::(MT2)
RooDataHist::(MT2)
RooDataHist::(MT2)
RooDataHist::(MT2)
RooDataHist::(MT2)
RooDataHist::(MT2)
RooDataHist::(MT2)
RooDataHist::(MT2)
RooDataHist::(MT2)
RooDataHist::(MT2)
RooDataHist::(MT2)
PS I’m using combineHarvester with combine v6.3.0 under CMSSW_7_4_7
In addition to the existing JSON formatted output it would be nice to introduce:
Future developments:
After seeing that this PR in Combine was merged in this branch, I started doing the same kind of work for CombineHarvester (here).
Adapting to ROOT 6.22 seems to be limited to just a couple of TPython
methods that have been moved in CPyCppyy
and changed names (TPython::ObjectProxy_FromVoidPtr
became CPyCppyy::Instance_FromVoidPtr
and TPython::ObjectProxy_AsVoidPtr
became CPyCppyy::Instance_AsVoidPtr
) in the transition from "old" PyROOT to new PyROOT.
However, probably due to my lack of experience with scram
, I haven't been able to succeed in correctly linking against libcppyy2_7.so
in CMSSW_11_2_0_pre10
.
To reproduce:
export SCRAM_ARCH=slc7_amd64_gcc900
cmsrel CMSSW_11_2_0_pre10
cd CMSSW_11_2_0_pre10/src
cmsenv
git clone -b 112x https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit.git HiggsAnalysis/CombinedLimit
git clone -b root622_comp https://github.com/maxgalli/CombineHarvester.git
scramv1 b
Running this returns a bunch of undefined references to the changed methods, like e.g.:
/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/gcc/9.3.0/bin/../lib/gcc/x86_64-unknown-linux-gnu/9.3.0/../../../../x86_64-unknown-linux-gnu/bin/ld: tmp/slc7_amd64_gcc900/src/CombineHarvester/CombineTools/src/Combin
eHarvesterCombineTools/CombineHarvester_Python.cc.o: in function `convert_py_root_to_cpp_root<TFile>::construct(_object*, boost::python::converter::rvalue_from_python_stage1_data*)':
CombineHarvester_Python.cc:(.text._ZN27convert_py_root_to_cpp_rootI5TFileE9constructEP7_objectPN5boost6python9converter30rvalue_from_python_stage1_dataE[_ZN27convert_py_root_to_cpp_rootI5TFileE9constructEP7_object
PN5boost6python9converter30rvalue_from_python_stage1_dataE]+0x5): undefined reference to `CPyCppyy::Instance_AsVoidPtr(_object*)'
What I tried:
root-config --libs
returns
-L/cvmfs/cms.cern.ch/slc7_amd64_gcc900/cms/cmssw/CMSSW_11_2_0_pre10/external/slc7_amd64_gcc900/bin/../../../../../../../slc7_amd64_gcc900/lcg/root/6.22.03-ghbfee2/lib -lCore -lImt -lRIO -lNet -lHist -lGraf -lGraf3d -lGpad -lROOTVecOps -lTree -lTreePlayer -lRint -lPostscript -lMatrix -lPhysics -lMathCore -lThread -lMultiProc -lROOTDataFrame -pthread -lm -ldl -rdynamic
The lib
directory printed, correctly contains libcppyy2_7.so
, but ld
does not seem to look there:
$ /cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/gcc/9.3.0/bin/../lib/gcc/x86_64-unknown-linux-gnu/9.3.0/../../../../x86_64-unknown-linux-gnu/bin/ld --verbose | grep SEARCH_DIR | tr -s ' ;' \\012
SEARCH_DIR("/data/cmsbld/jenkins/workspace/auto-builds/CMSSW_11_1_0_pre6-slc7_amd64_gcc900/build/CMSSW_11_1_0_pre6-build/tmp/BUILDROOT/610f04827ed0f827de9b8700ad3d9670/opt/cmssw/slc7_amd64_gcc900/external/gcc/9.3.0/x86_64-unknown-linux-gnu/lib64")
SEARCH_DIR("/data/cmsbld/jenkins/workspace/auto-builds/CMSSW_11_1_0_pre6-slc7_amd64_gcc900/build/CMSSW_11_1_0_pre6-build/tmp/BUILDROOT/610f04827ed0f827de9b8700ad3d9670/opt/cmssw/slc7_amd64_gcc900/external/gcc/9.3.0/lib64")
SEARCH_DIR("/usr/local/lib64")
SEARCH_DIR("/lib64")
SEARCH_DIR("/usr/lib64")
SEARCH_DIR("/data/cmsbld/jenkins/workspace/auto-builds/CMSSW_11_1_0_pre6-slc7_amd64_gcc900/build/CMSSW_11_1_0_pre6-build/tmp/BUILDROOT/610f04827ed0f827de9b8700ad3d9670/opt/cmssw/slc7_amd64_gcc900/external/gcc/9.3.0/x86_64-unknown-linux-gnu/lib")
SEARCH_DIR("/data/cmsbld/jenkins/workspace/auto-builds/CMSSW_11_1_0_pre6-slc7_amd64_gcc900/build/CMSSW_11_1_0_pre6-build/tmp/BUILDROOT/610f04827ed0f827de9b8700ad3d9670/opt/cmssw/slc7_amd64_gcc900/external/gcc/9.3.0/lib")
SEARCH_DIR("/usr/local/lib")
SEARCH_DIR("/lib")
SEARCH_DIR("/usr/lib")
Also appending the path to LD_LIBRARY_PATH
does not seem to work.
Do anyone have any suggestion concerning what to try and how to correctly implement it in a build process?
(ping also for @nsmith- and @andrzejnovak who worked on adapting Combine to ROOT 6.22).
Hi,
In many CMS papers, including the combine paper, the shown impact plots are nicer to look at. The colors are sharper. This style is not the one we get by running the default plotImpacts.py (as of v2.0.0).
It would be awesome if one of these nicer styles is included in the default script, lowering the barrier to publication quality impact plots.
Cheers,
Afiq
Discussed in cms talk[1]
The rounding changes if the uncertainty is large
https://github.com/cms-analysis/CombineHarvester/blob/main/CombineTools/scripts/plotImpacts.py#L28-L30
Current implementation in https://github.com/cms-analysis/CombineHarvester/blob/master/CombineTools/python/combine/LimitGrids.py builds grids of points from a syntax like 1:10:1
, which gives points 1,2,3,4,5...
. We should also support a binning-style syntax in the more familiar TH1 style. This should be clearly distinguishable from the current syntax. Something like 9;1;10
giving points 1.5,2.5,3.5,..
. It would also be nice to have a way of setting the grid automatically to the exact granularity and range of the theory input TH2s.
Makes sense to implement as a ch::Systematic
with type rateParam
, but a few details to be worked out, such as adapting AddSyst and WriteDatacards.
does not work when a nuisance has a same-sign impact for opposite-sign variations. This then ripples through to plotting oddities as it is used in
https://github.com/cms-analysis/CombineHarvester/blob/master/CombineTools/scripts/plotImpacts.py#L43
Perhaps instead of (paramScanRes[param][p][2] - paramScanRes[param][p][0])/2.
it is more sensible to use max(map(abs,(paramScanRes[param][p][2],paramScanRes[param][p][0])))
.
Hello,
I have installed the latest version of combine tool and CombineHarvester and I tried the "partialCorrelationEdit.py" script using one of the simple shape Datacard examples in the
CMSSW_10_2_13/src/HiggsAnalysis/CombinedLimit/data/benchmarks/shapes/simple-shapes-TH1.txt
with the following command
python ../../../../../CombineHarvester/CombineTools/scripts/partialCorrelationEdit.py simple-shapes-TH1.txt -m 125 --process lumi,0.6
but I always get the segmentation violation Error at the last stage of the script[1]. I also tried with different datacards but always get the same problem. could you please let me know if I am missing something or there is an issue with the script?
Thanks
Mohsen
[1][SetFlag] Changing value of flag "workspace-uuid-recycle" from 1 to 1
[SetFlag] Changing value of flag "workspaces-use-clone" from 0 to 1
[SetFlag] Changing value of flag "import-parameter-err" from 1 to 0
[SetFlag] Changing value of flag "zero-negative-bins-on-import" from 0 to 0
[SetFlag] Changing value of flag "check-negative-bins-on-import" from 1 to 0
Setting correlation coefficient of lumi to 0.600000
The following systematics will be cloned and adjusted:
125 1 0 signal 1 lumi lnN 1.1 0 0
125 1 0 background 0 lumi lnN 1 0 0
Writing new card and ROOT file: ('decorrelated_card.txt', 'decorrelated_card.shapes.root')
*** Break *** segmentation violation
For complicated models with many categories the post-fit plotting can be quite time consuming. The easiest way to speed up the process is to introduce parallelisation.
It would also be nice to have the following features:
One rather easy bug to encounter is mis-naming of parameters in the datacard for param
style nuisances. It would be good if the validation could spot this happening and warn the user.
The simplest way would be to scan over parameters (maybe just nuisance parameters) and check if they have more than just the Gaussian constraint in their Client list.
Note that its not enough to check if there's a param
which isn't encountered in the incoming workspaces since its feasible someone would create a param
which say gets fed into a rateParam
afterwards.
When trying to make "brazilian" limit plots with CombineTools,
python /my/eos/space/CMSSW_11_3_4/src/CombineHarvester/CombineTools/scripts/plotLimits.py limits_RSGluon.json --show exp --logy
I get an error where ROOT.Double is not an attribute that can be used. The traceback is included below for details:
Traceback (most recent call last): File "/my/eos/space/CMSSW_11_3_4/src/CombineHarvester/CombineTools/scripts/plotLimits.py", line 112, in <module> axis = plot.CreateAxisHists(len(pads), list(graph_sets[-1].values())[0], True) File "/my/eos/space/CMSSW_11_3_4/python/CombineHarvester/CombineTools/plotting.py", line 487, in CreateAxisHists h = CreateAxisHist(src, at_limits) File "/my/eos/space/CMSSW_11_3_4/python/CombineHarvester/CombineTools/plotting.py", line 469, in CreateAxisHist x = R.Double(0.) File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/lcg/root/6.22.08-ljfedo/lib/ROOT/_facade.py", line 163, in _fallback_getattr raise AttributeError("Failed to get attribute {} from ROOT".format(name)) AttributeError: Failed to get attribute Double from ROOT
For more details, I have installed the recommended version of Combine (v9), followed by version 2.0.0 of CombineHarvester, and a clone of auxiliaries
directory as recommended here
Following a forum discussion, it seems that this is because of the root version 6.22 (that I think comes with CMSSW 11-3-4).
Hello,
As the title says: I have a datacard with one root file per systematic (70 systematics total). This would take hours to hadd into one file and I would not like to do that.
text2workspace can read such datacards without any problem and create a workspace, but ValidateDatacards.py seems not be able to do that and prints the error below.
Is it possible to add this functionality?
Kind regards,
Andrej
[SetFlag] Changing value of flag "check-negative-bins-on-import" from 1 to 0
[SetFlag] Changing value of flag "workspaces-use-clone" from 0 to 1
Error in TSystem::ExpandFileName: input: ../../../miniTreeHistograms/MergedTrees/miniTree$SYSTEMATIC.root, output: ../../../miniTreeHistograms/MergedTrees/miniTree$SYSTEMATIC.root
Error in TFile::TFile: error expanding path ../../../miniTreeHistograms/MergedTrees/miniTree$SYSTEMATIC.root
Traceback (most recent call last):
File "/nfs/dust/cms/user/asaibel/PhDWorkFolder/HiggsCombineTool/SL7/CMSSW_10_2_13/bin/slc7_amd64_gcc700/ValidateDatacards.py", line 110, in
cb.ParseDatacard(args.cards,"","",mass=args.mass)
File "/nfs/dust/cms/user/asaibel/PhDWorkFolder/HiggsCombineTool/SL7/CMSSW_10_2_13/python/CombineHarvester/CombineTools/ch.py", line 21, in ParseDatacard
return self.ParseDatacard(card, analysis, era, channel, bin_id, mass)
RuntimeError:
Context: Function ch::GetClonedTH1 at
/nfs/dust/cms/user/asaibel/PhDWorkFolder/HiggsCombineTool/SL7/CMSSW_10_2_13/src/CombineHarvester/CombineTools/src/TFileIO.cc:21
Problem: TH1 Nominal/combined/GreaterThreeBTag/DeltaRbin1/GreaterThreeJet/data_obs/jet4_btag not found in ../../../miniTreeHistograms/MergedTrees/miniTree$SYSTEMATIC.root
Extend the CombineHarvester(or CH methods after the CH-combine merge functionality to include:
Up/Down variation smoothing with simple TH1 methods. Consider various methods, possibly with chi^2 test to make sure that the smoothed histogram is an accurate representation of initial histogram.
Template binning optimisation.
Post-fit uncertainty bands are not correctly evaluated from the cov. matrix sampling for bins with high statistics and large lnN uncertainties, reported at CMSTalk.
See cms-analysis/HiggsAnalysis-CombinedLimit#831 , and discussion here: https://root.cern/manual/python/
I encountered a bug in Python bindings. I'm trying to create a datacard for a shape-based analysis using the Python interface. With the current master
(6af279a), running
harvester = ch.CombineHarvester()
# ...
harvester.WriteDatacard('datacard.txt', 'shapes.root')
triggers a segmentation violation. The code works fine if I create the ROOT file beforehand:
shapes_file = ROOT.TFile('shapes.root', 'recreate')
harvester.WriteDatacard('datacard.txt', shapes_file)
The backtrace to the main WriteDatacard
method suggest that the problem is in the Python bindings:
#0 0x00007ffff675cb50 in ch::CombineHarvester::WriteDatacard(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, TFile&)@plt ()
from /user/aapopov/CMSSW/CMSSW_10_2_13__combine/lib/slc7_amd64_gcc700/libCombineHarvesterCombineTools.so
#1 0x00007ffff696dc7a in Overload3_WriteDatacard (cb=..., name="datacard.txt", file=...)
at /user/aapopov/CMSSW/CMSSW_10_2_13__combine/src/CombineHarvester/CombineTools/src/CombineHarvester_Python.cc:136
#2 0x00007ffff69cda79 in boost::python::detail::invoke<int, void (*)(ch::CombineHarvester&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::python::api::object&), boost::python::arg_from_python<ch::CombineHarvester&>, boost::python::arg_from_python<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>, boost::python::arg_from_python<boost::python::api::object&> > (
f=@0x68de18: 0x7ffff696dc1c <Overload3_WriteDatacard(ch::CombineHarvester&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::python::api::object&)>, ac0=..., ac1=..., ac2=...) at /cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/boost/1.63.0-gnimlf/include/boost/python/detail/invoke.hpp:81
#3 0x00007ffff69bf6a5 in boost::python::detail::caller_arity<3u>::impl<void (*)(ch::CombineHarvester&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::python::api::object&), boost::python::default_call_policies, boost::mpl::vector4<void, ch::CombineHarvester&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::python::api::object&> >::operator() (this=0x68de18, args_=0x7ffff6af2aa0)
at /cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/boost/1.63.0-gnimlf/include/boost/python/detail/caller.hpp:218
#4 0x00007ffff69b0343 in boost::python::objects::caller_py_function_impl<boost::python::detail::caller<void (*)(ch::CombineHarvester&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::python::api::object&), boost::python::default_call_policies, boost::mpl::vector4<void, ch::CombineHarvester&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::python::api::object&> > >::operator() (this=0x68de10, args=0x7ffff6af2aa0, kw=0x0)
at /cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/boost/1.63.0-gnimlf/include/boost/python/object/py_function.hpp:38
...
Note how the call is passed through Overload3_WriteDatacard
instead of Overload1_WriteDatacard
. Apparently Overload3_WriteDatacard
tries to convert a string into a TFile
. I also see that file_
inside it is null. Unfortunately, I don't know enough about the bindings to propose a fix.
Hi,
I noticed recently that negative bin are no longer put to zero. After looking at the code, the reason is that the line is commented? See
It's pointless to have a flag to control this feature is the code is commented :)
E.g. using MultiDimFit output with a loaded snapshot. The RooFitResult we construct from scratch doesn't have everything in it which the copy constructor expects.
Would be useful to have an option that copies input files to $TMPDIR at the beginning of the job, write output locally (not in the job submission $PWD), and then copies it back to the submission dir (or some other dir) at the end.
Because in principle we know the name of the output file at submission time could also think about tracking completed/lost jobs.
ValidateDatacards.py reports that shapeN is not supported:
Not sure if this is a real Harvester issue or just a lack of updating L453.
Hi,
It seems there are several plot1DScan.py
files in the script
directories of the various sub-packages... When running scram
these get copied to the CMSSW bin
folder in a random order and overwrite the "main" script, which is clearly undesirable. What's more, at least one is not an executable file, so it's not even found as a command.
It could be the case also for other scripts, I haven't checked. I can submit a PR to rename them but perhaps you have another preferred solution?
Hi,
I'm trying to figure out how to run GoodnessOfFit toys over CRAB. I understand that I can run e.g. 25 toys in a single crab job like this:
combineTool.py \
--job-mode crab3 \
-M GoodnessOfFit \
-d /path/to/card.root \
-t 25 \
# (other args)
This works fine. I would like to extend this though to be able to run many toys split over a number of jobs. I have tried two approaches to accomplish this:
using the --merge
argument. This does not seem to have any effect. I think this traces down to the fact that the combine tool thinks of my GoF command as "one entry in the job queue", rather than "25 independent entries".
using the --custom-crab
argument, and specifying config.Data.totalUnits = 50
for submission of e.g. 50 jobs. The submission works in this case, but the jobs fail because the script executed on the worker node tries to match the job ID to the job queue entries [1]. In the same vein as above, the jbo queue only has one entry here, so the script simply fails for all job IDs > 1.
Is there an existing well-defined way of doing this? If not, I can hotfix [1] for myself, but I'm not sure how to implement this in a sustainable way without creating spaghetti.
Any hints would be appreciated!
For some limit calculation methods, combine needs the signal process to have a mass value specified in the shape file. CH currently lets you make a datacard + shape file without mass value, if this is done it would be nice to assign the signal process a dummy mass value so that the datacard can be used with all combine methods.
Dear experts,
could you help with the ValidateDatacards.py problem, please?
Simple datacard is used (shown under [*]) and all other steps in the card validation and pull distributions were produced with combine tools without issues using this card. But ValidateDatacards.py datacard.txt fails with full error message printed below:
[SetFlag] Changing value of flag "check-negative-bins-on-import" from 1 to 0
[SetFlag] Changing value of flag "workspaces-use-clone" from 0 to 1
Traceback (most recent call last):
File "/cms/multilepton-3/olena/CMSSW_10_2_13/bin/slc7_amd64_gcc700/ValidateDatacards.py", line 110, in
cb.ParseDatacard(args.cards,"","",mass=args.mass)
File "/cms/multilepton-3/olena/CMSSW_10_2_13/python/CombineHarvester/CombineTools/ch.py", line 21, in ParseDatacard
return self.ParseDatacard(card, analysis, era, channel, bin_id, mass)
RuntimeError:
Context: Function ch::CombineHarvester::ParseDatacard at
/cms/multilepton-3/olena/CMSSW_10_2_13/src/CombineHarvester/CombineTools/src/CombineHarvester_Datacards.cc:454
Problem: Systematic type autoMCStats not supported
Please report issues at
https://github.com/cms-analysis/CombineHarvester/issues
[*] Datacard (simplified) :
imax 1 number of bins
jmax 2 number of processes minus 1
kmax 41 number of nuisance parameters
----------------------------------------------------------------------------------------------------------------------------------
shapes * ch1 shapes_WPhiPS50mu_3L_LS0_Run2.root WPhimmMinOSSF3Lbin5B0_$PROCESS WPhimmMinOSSF3Lbin5B0_$PROCESS_$SYSTEMATIC
----------------------------------------------------------------------------------------------------------------------------------
bin ch1
observation 1450
----------------------------------------------------------------------------------------------------------------------------------
bin ch1 ch1 ch1
process WPhiPS50mu Prompt MisID
process 0 1 2
rate 291.7487655130141 903.5581679344177 576.2285528182983
----------------------------------------------------------------------------------------------------------------------------------
BTagY16 shapeN2 - 1.0 -
BTagY17 shapeN2 - 1.0 -
BTagY18 shapeN2 - 1.0 -
LimiY16 shapeN2 1.0 1.0 -
LimiY16Y17Y18 shapeN2 1.0 1.0 -
LimiY17 shapeN2 1.0 1.0 -
LimiY17Y18 shapeN2 1.0 1.0 -
LimiY18 shapeN2 1.0 1.0 -
POGIDEleY16Y17Y18 shapeN2 1.0 1.0 -
POGIDMuY16Y17Y18 shapeN2 1.0 1.0 -
POGRecoEleY16Y17Y18 shapeN2 1.0 1.0 -
PU shapeN2 1.0 1.0 -
ch1 autoMCStats 0 0 1
If I change the last line to ch1 autoMCStats 001
(without spaces), there is this error message is does not appear. The output is:
[SetFlag] Changing value of flag "check-negative-bins-on-import" from 1 to 0
[SetFlag] Changing value of flag "workspaces-use-clone" from 0 to 1
================================
=======Validation results=======
================================
>>>There were no warnings
Could you advise if I'm using the wrong format for autoMCStats or what is the reason for ValidateDatacards.py not recognizing ch1 autoMCStats 0 0 1
(other combine tools did not have errors regarding this line), but recognizing ch1 autoMCStats 001
.
Thank you very much in advance,
Olena
Need a class with a configurable set of checks to perform on a CH instance. These checks would include:
Will add more things as they come up.
How should one cite the use of the CombineHarvester tool? Are there any publications about it specifically or is it enough to just use the citations listed on combine's FAQ?
CMSSW: CMSSW_10_2_13
Higgs combine tool env:
CombineHarvester branch: master (last commit: 128e41e)
Example data cards and data:
Issue:
While running ValidateDatacards.py datacard.txt
I receive this error:
[SetFlag] Changing value of flag "check-negative-bins-on-import" from 1 to 0
[SetFlag] Changing value of flag "workspaces-use-clone" from 0 to 1
Error in <TFile::Init>: 2017_shapes_df_input.csv not a ROOT file
Traceback (most recent call last):
File "/afs/cern.ch/work/r/rymuelle/public/nanoAODzPrime/higgscombine/CMSSW_10_2_13/bin/slc7_amd64_gcc700/ValidateDatacards.py", line 110, in <module>
cb.ParseDatacard(args.cards,"","",mass=args.mass)
File "/afs/cern.ch/work/r/rymuelle/public/nanoAODzPrime/higgscombine/CMSSW_10_2_13/python/CombineHarvester/CombineTools/ch.py", line 21, in ParseDatacard
return self.__ParseDatacard__(card, analysis, era, channel, bin_id, mass)
RuntimeError:
*******************************************************************************
Context: Function ch::CombineHarvester::ParseDatacard at
/afs/cern.ch/work/r/rymuelle/public/nanoAODzPrime/higgscombine/CMSSW_10_2_13/src/CombineHarvester/CombineTools/src/CombineHarvester_Datacards.cc:131
Problem: Workspace not found in file
*******************************************************************************
Please report issues at
https://github.com/cms-analysis/CombineHarvester/issues
*******************************************************************************
I appears like it expects a root file. A search through the documentation and github repo did not clearly show a solution.
When datacards include rateParam
lines featuring RooFormulaVar
s such as:
alphaMuon2FJ_2018_SRH rateParam Muon2FJ_2018_SRH DYJ @0 alphaMuon2FJ_2018_2Lep
the ValidateDatacards.py
tool seem not to able to compile the above expression and brakes:
Error in <RooFormula::Compile>: Bad numerical expression : "@0"
It seems that replacing the formula with "1.0*@0"
fixes the compilation.
Edit: It seems that in general, if the RooFormula is expressed as a string, this does not cause compilation issues. So even:
alphaMuon2FJ_2018_SRH rateParam Muon2FJ_2018_SRH DYJ "@0" alphaMuon2FJ_2018_2Lep
would work.
An example of datacard to reproduce the issue is this HIG-22-005 card.
Reporting an issue seen with CMSSW_10_2_13, combine v8.2.0
I am running an analysis with 2 POI (rggF and rVBF). I submitted 100 parallel jobs of 10 toys each, expecting 1000 toys as a result. Instead, I see exactly 4x the expected number of toys in the output tree. The output tree contains sets of 4 toys with identical POI fitted values and errors, but with different values in the "limit" branch. In one of the four toys in the set, the limit branch matches the fitted value of the first listed POI. I attach a few plots here for demonstration, and in case it is useful I have uploaded an example output file on lxplus here:
/afs/cern.ch/user/j/jdickins/public/combine-issue/bias20VBF.root
limit_rggF.pdf
limit_rVBF.pdf
limitErr_rggFErr.pdf
limitErr_rVBFErr.pdf
This is the command I used for toy submission:
combineTool.py -M FitDiagnostics --setParameters rVBF=$bias,rggF=1 --trackParameters rggF,rVBF --trackErrors rggF,rVBF -n bias${bias}VBF -d ${modelfile} --cminDefaultMinimizerStrategy 0 --robustFit=1 -t 10 -s 1:100:1 --job-mode condor --task-name VBF$bias
The issue is two-fold:
Thanks, and let me know if any additional information from me can be useful.
Would be useful to have plot1Dscan
(https://github.com/cms-analysis/CombineHarvester/blob/master/CombineTools/scripts/plot1DScan.py) as standard for CMS to report crossing values.
Currently the script doesn't allow choosing the values of the crossings and doesn't report all of the results to stdout, it would be helpful to have the results printed to screen and/or saved in a text file.
Hello,
I would like a package to be created for the ND-OSU Top EFT analysis. We would like to begin using CombineHarvester to create our datacards. The documentation instructed me to create an issue here. Let me know how I can help with this process.
Best,
Anthony Lefeld
Then do some extensive implementation/testing of the termination criteria for generating toys at a given point:
With the added complication that some will use "FAKE" shape entries and some won't.
When running ValidateDatacards.py datacard.txt
, I get:
*******************************************************************************
Context: Function ch::GetClonedTH1 at
/afs/cern.ch/work/r/rymuelle/public/nanoAODzPrime/higgscombine/CMSSW_10_2_13/src/CombineHarvester/CombineTools/src/TFileIO.cc:24
Problem: TH1 SR1-sys_0_nominal-0 not found in 2016/2016_shapes_df_input.root
*******************************************************************************
Please report issues at
https://github.com/cms-analysis/CombineHarvester/issues
*******************************************************************************
However, the datacard in question does not require "SR1-sys_0_nominal-0", and I am unsure what sort of pattern ValidateDatacard is looking for that would cause it to look for this TH1.
Datacard in question:
Combination of name0=2016/2016_SR1_BFFZprimeToMuMu_fit_M_125_dbs0p5.txt name1=2016/2016_SR2_BFFZprimeToMuMu_fit_M_125_dbs0p5.txt
imax 2 number of bins
jmax 1 number of processes minus 1
kmax 13 number of nuisance parameters
----------------------------------------------------------------------------------------------------------------------------------
shapes * name0 2016/2016_shapes_df_input.root SR1-sys_0_nominal-$PROCESS SR1-$SYSTEMATIC-$PROCESS
shapes background name0 2016/2016_shapes_df_input.root SR1-sys_0_nominal-background
shapes * name1 2016/2016_shapes_df_input.root SR2-sys_0_nominal-$PROCESS SR2-$SYSTEMATIC-$PROCESS
shapes background name1 2016/2016_shapes_df_input.root SR2-sys_0_nominal-background
----------------------------------------------------------------------------------------------------------------------------------
bin name0 name1
observation -1 -1
----------------------------------------------------------------------------------------------------------------------------------
bin name0 name0 name1 name1
process 125 background 125 background
process 0 1 0 1
rate -1 -1 -1 -1
----------------------------------------------------------------------------------------------------------------------------------
lumi lnN 1.025 - 1.025 -
sys_0.5_ISRFSR_2016_ shapeN2 1.0 - 1.0 -
sys_0.5_L1_2016_ shapeN2 1.0 - 1.0 -
sys_0.5_Muon_2016_ shapeN2 1.0 - 1.0 -
sys_0.5_btag_2016_ shapeN2 1.0 - 1.0 -
sys_0.5_elSF_2016_ shapeN2 1.0 - 1.0 -
sys_0.5_jer_2016_ shapeN2 1.0 - 1.0 -
sys_0.5_jes_2016_ shapeN2 1.0 - 1.0 -
sys_0.5_pdf_2016_ shapeN2 1.0 - 1.0 -
sys_0.5_pu_ shapeN2 1.0 - 1.0 -
sys_0.5_puid_2016_ shapeN2 1.0 - 1.0 -
sys_0.5_roch_2016_ shapeN2 1.0 - 1.0 -
sys_0.5_trigger_2016_ shapeN2 1.0 - 1.0 -
Hi there,
I have set up the CombineTool code on lxplus according to the instructions in the README for this repo (CMSSW_10_2_13 and Combine tag v8.2.0), but I see an error related to the numpy installation. Is this a known problem? And is there a workaround?
Thanks,
Jennet
Here is an example command and the accompanying error:
[jdickins@lxplus795 2016-prefit]$ combineTool.py -M Impacts -d $modelfile -m 125 --robustFit 1 --doInitialFit -t -1 --setParameters rggF=1,rVBF=1,rZbb=1
Traceback (most recent call last):
File "/afs/cern.ch/work/j/jdickins/hbb-prod-modes/test/CMSSW_10_2_13/bin/slc7_amd64_gcc700/combineTool.py", line 8, in
from CombineHarvester.CombineTools.combine.ImpactsFromScans import ImpactsFromScans
File "/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.12.07-gnimlf5/lib/ROOT.py", line 318, in _importhook
return _orig_ihook( name, *args, **kwds )
File "/afs/cern.ch/work/j/jdickins/hbb-prod-modes/test/CMSSW_10_2_13/python/CombineHarvester/CombineTools/combine/ImpactsFromScans.py", line 15, in
from numpy import matrix
File "/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.12.07-gnimlf5/lib/ROOT.py", line 318, in _importhook
return _orig_ihook( name, *args, **kwds )
File "/cvmfs/atlas.cern.ch/repo/sw/software/21.2/AnalysisBaseExternals/21.2.25/InstallArea/x86_64-slc6-gcc62-opt/lib/python2.7/site-packages/numpy-1.13.3-py2.7-linux-x86_64.egg/numpy/init.py", line 142, in
from . import add_newdocs
File "/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.12.07-gnimlf5/lib/ROOT.py", line 318, in _importhook
return _orig_ihook( name, *args, **kwds )
File "/cvmfs/atlas.cern.ch/repo/sw/software/21.2/AnalysisBaseExternals/21.2.25/InstallArea/x86_64-slc6-gcc62-opt/lib/python2.7/site-packages/numpy-1.13.3-py2.7-linux-x86_64.egg/numpy/add_newdocs.py", line 13, in
from numpy.lib import add_newdoc
File "/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.12.07-gnimlf5/lib/ROOT.py", line 318, in _importhook
return _orig_ihook( name, *args, **kwds )
File "/cvmfs/atlas.cern.ch/repo/sw/software/21.2/AnalysisBaseExternals/21.2.25/InstallArea/x86_64-slc6-gcc62-opt/lib/python2.7/site-packages/numpy-1.13.3-py2.7-linux-x86_64.egg/numpy/lib/init.py", line 8, in
from .type_check import *
File "/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.12.07-gnimlf5/lib/ROOT.py", line 318, in _importhook
return _orig_ihook( name, *args, **kwds )
File "/cvmfs/atlas.cern.ch/repo/sw/software/21.2/AnalysisBaseExternals/21.2.25/InstallArea/x86_64-slc6-gcc62-opt/lib/python2.7/site-packages/numpy-1.13.3-py2.7-linux-x86_64.egg/numpy/lib/type_check.py", line 11, in
import numpy.core.numeric as _nx
File "/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.12.07-gnimlf5/lib/ROOT.py", line 318, in _importhook
return _orig_ihook( name, *args, **kwds )
File "/cvmfs/atlas.cern.ch/repo/sw/software/21.2/AnalysisBaseExternals/21.2.25/InstallArea/x86_64-slc6-gcc62-opt/lib/python2.7/site-packages/numpy-1.13.3-py2.7-linux-x86_64.egg/numpy/core/init.py", line 26, in
raise ImportError(msg)
ImportError:
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try git clean -xdf
(removes all
files not under version control). Otherwise reinstall numpy.
Original error was: libptf77blas.so.3: cannot open shared object file: No such file or directory
I haven't yet investigated but I think this happens when using a physics model that scales some contributions by -1*POI
. When using the default physics model the distributions (for the specific example I was looking at) have the same number of bins as the input datacard.
General tasks that should probably be done before adding a bunch of new stuff:
Could have gotten sth wrong, but PostFitShapesFromWorkspace
doesn't seem take into account toys production. "data_obs" is always the original input from workspace, even when running the FitDiagnostics with -t -1
. Is this intended behaviour?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.