Comments (6)
I'm not sure to understand the issue here. What's the point to redo a production with the same code version? You'll just duplicate what you already done, no?
Currently, if you run the post-crab script for a sample which already is in the database (this mean that the sample name is already in the database), the sample is updated, otherwise a new one is added. By updated, I mean only the variables here [1] are updated, and the path is not one of them, since it was designed to handle update of an existing task, and not a completely new task with the same name. Moreover, the path is deprecated in favor for the list of files, which is correctly updated.
I'm also a bit surprised to see that the creation timestamp is updated, only the modified timestamp should change. I'll look at it when I have time. NVM it's normal
To summarize, if you relaunch a production with the exact same code, you really should delete the previous one from the database to avoid any issues. But indeed, we'll run into issues when launching the same code but with different python configuration (like systematics for example), so we'll need to make the name unique in this case. Maybe it's your case? Adding an hash of the python configuration to the sample name may be enough to solve the issue?
Also, just for information, it's not really an issue for the database to have samples with the same name, as samples are uniquely identified by their ids, and not by their name. It's however an issue for the post-crab script, since you only have access to the sample name. The timestamp is also already included in the database in the created and modified fields, so adding it into the name is not really needed.
[1] https://github.com/cp3-llbb/GridIn/blob/master/scripts/runPostCrab.py#L145-L167
from gridin.
Well in my case, this is something stupid: I did the changes to use the full lumi, and for some reason these changes were not ported to the crab config files (which I don't get, but perhaps a mistake from my side). Anyway, I wanted to run a second time with these changes taken into account, then I discovered the "issue" I mentioned.
But ok I understand that I didn't manage my entries in the best way, I'll try to simply delete the wrong entries and recreate them with the correct prod.
Thanks
S.
from gridin.
Btw, is there some indication how to delete entries in the db?
from gridin.
Ok I see.
Thinking a bit about this, I don't even think we have an easy way to remove sample for the database except doing raw SQL queries...
from gridin.
hmm, ok. Then perhaps for the time being I'll just hack the samples.json to indicate the correct addresses of the new prods, that's dirty but for now that is ok.
from gridin.
We solved the issue "offline" with Simon, but we should really provide a script to delete samples from the database. Closing this issue and opening a new one.
from gridin.
Related Issues (20)
- Spring cleaning
- Speed up runPostCrab.py HOT 6
- Add powheg ttbar sample
- Update JSON for Run2015B to use MiniAOD rereco HOT 1
- extra u in the python config file HOT 2
- /store cleaner script HOT 2
- Crash when running runPostCrab HOT 4
- Add wild card support in search_SAMADhi HOT 1
- Add analyzer name in the sample name string HOT 7
- Remove date from dataset name HOT 1
- Update datasets to MiniAOD v2 HOT 4
- Automatic analyzer configuration of data / mc HOT 1
- runPostCrab should not assume the user has forked
- DoubleEG Re-reco HOT 4
- Move datasets to a standalone repository HOT 5
- Some runPostCrab cleaning
- runPostCrab: key error: 'lfn' HOT 1
- Update readme
- Cleaning not attempted if there is no job number 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gridin.