Code Monkey home page Code Monkey logo

cernanalysispreservation / analysispreservation.cern.ch Goto Github PK

View Code? Open in Web Editor NEW
43.0 26.0 30.0 37.76 MB

Source code for the CERN Analysis Preservation portal

Home Page: https://analysispreservation.cern.ch/

License: GNU General Public License v3.0

Python 65.69% Shell 1.03% JavaScript 29.84% HTML 2.41% CSS 0.04% Dockerfile 0.48% SCSS 0.37% Less 0.14%
research-data research-data-management research-data-repository research-data-life-cycle-management reproducible-research reproducible-science reusable-artifacts python flask json-schema

analysispreservation.cern.ch's Introduction

CERN Analysis Preservation

image

image

image

image

image

image

What is this?

CERN Analysis Preservation is a data preservation service developed and run at CERN to provide a centralized platform for physicists to capture and preserve the knowledge behind their physics analyses so that it is understandable and reusable in the future. It is currently targeting analysis information produced by the LHC experiments.

CAP is designed to support physicists during their research, so we make sure to apply access control to all information added to the platform. Users retain full control of their information and data.

The source code behind this project is licensed under the GNU General Public License v3.0.

If you want to know more, please refer our official documentation.

Installation

If you want to try it out yourself or test some code you have written, you can install a demo site locally. Please check out the installation guide for more details.

Contributing

Do you want to help us grow? We are always happy to receive contributions:

Issues

If you find a bug or have a (feature) request, go ahead and add an issue for it or comment on an existing issue in case this has already been addressed!

Pull Requests

If you want to contribute directly to the project (e.g. code, JSON workflow files, documentation), you are welcome to create a pull request for your changes. If you want to know how to write good commit messages that will help us (now) and you (in a few weeks) to understand what you did, you may want to check out this.

Advanced Guide

If you feel like you want to know more details and are prepared to take some extra care to follow our contribution style, please go to the contributing guide for more information.

analysispreservation.cern.ch's People

Contributors

annatrz avatar anxheladani avatar artemislav avatar atrisovi avatar atrisovic avatar crepererum avatar daslerr avatar drjova avatar hjhsalo avatar ioannistsanaktsidis avatar javierdelgadofernandez avatar jirikuncar avatar jorgectf avatar kjili avatar lilykos avatar lukasheinrich avatar michamos avatar miguelgrc avatar pamfilos avatar papadopan avatar parths007 avatar suenjedt avatar tiborsimko avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

analysispreservation.cern.ch's Issues

groups: cannot list user groups

When trying to list all SSO groups one belongs to, the good old URL /yourgroups/display now leads to an Internal Server Error.

Investigate LHCb root metadata integration

Check with Ana if

  • the metadata integrated in root can extend the LHCb data model #45
  • parts from the LHCb data model can be auto-filled because information can be integrated in root n-tuples

deposit: icon for expand/collapse

When a user expands a deposition box, we may perhaps want to change icon from downward-pointing to upward-pointing, to better indicate expand/collapse status; or we may want to choose a more neutral static one.

deposit: create records

After JSON model is finalised in #9, the Deposit should be configured to call Upload as the last step, so that the data would be uploaded.

login: show only SSO method

Only the SSO login method should be shown in the page header and on the login page, just like we do for LABS.

LHCb: Final N-tuples description

As usrers are adding data files in "Final N-tuples" box, the "Data Files" item is well multiplied, but the "Description" item is not. So we end up with N-to-1 ration. What we need is a description field for each new data file added, so that every data file can be described; N-to-N ratio.

(Ditto for harvest-or-link decision radio, there has to be N of them, for each data file.)

fs: introduce storage abstraction

Some of the submitted files will be stored on Ceph, some on AFS. A filesystem abstraction should be introduced when it is ready in upstream invenio.

CMS deposit: new "Final state particles" input information

In the "Physics" information box, after "Trigger Selection", a new
facility to enter information about "Final state particles" is wanted.

basically this is a matrix of information looking like:

| particle | number | PT cut | eta cut |
|----------+--------+--------+---------|
| electron | 1      | 20 GeV | 0.4     |
| muon     | ...    | ...    | ...     |
| gamma    | ...    | ...    | ...     |
| b-jet    | ...    | ...    | ...     |
| tau-jet  | ...    | ...    | ...     |
| MET      | ...    | ...    | ...     |

The UI could work like this:

Final state particles:
----------------------
[--electron--] [number?] [PT cut?] [eta cut?]  [+ ADD MORE]

The user could choose particle from the drop-down selection menu
(electron, muon, etc) and enter PT cut and eta cut information
manually in input text areas.

The user could then click on "[+ ADD MORE]" to add more lines to fill
above matrix.

The tasks:

(1) Create "Final state particles" box either as a big static matrix
of the above kind, or better yet as a progressively deploying matrix
lines as people will be clicking on "[+ ADD MORE]".

(2) Interesting things happen if people choose more than one particle
(say 2 electrons) in the above "number?" box. A new selection box
should then pop up below saying something like:

combination?  (*) same sign ( ) opposite sign ( ) invariant mass [--free text--]

(3) Similarly to the above selection matrix, one can have have "Vetos"
matrix that will "unselect" events in similar way. (e.g. vetoing this and this
particle with that and that eta cut)

CMS deposit: auto-complete trigger information from primary datasets

Primary CMS datasets that users will enter in deposit forms look like
this:

/Electron/Run-2010B-Apr21ReReco-v1/AOD

or like this:

/ZeroBias/Run-2010B-Apr21ReReco-v1/AOD

If we represent this by /X/Run-YYYY* form, then for certain
combinations of X and YYYY the "Trigger Selection" box can be
auto-completed with suitable trigger values that people could simply
select from.

Here are the rules on how the trigger mappings look like for various
years:

For example, if a user submitted
/Electron/Run-2010B-Apr21ReReco-v1/AOD primary data set, then
X=Electron and Y=2010 and looking up the trigger values from the
first URL, one can get X->TRIGGER value mappings, leading to the
following trigger selection drop-down menu:

HLT_DoubleEle15_SW_L1R
HLT_DoubleEle17_SW_L1R
HLT_DoubleEle8_SW_HT70U_L1R
HLT_Ele10_MET45
HLT_Ele10_SW_EleId_HT70U_L1R
HLT_Ele10_SW_HT100U_L1R
HLT_Ele10_SW_HT70U_L1R
HLT_Ele12_SW_TightEleIdIsol_L1R
HLT_Ele12_SW_TightEleIdIsol_NoDEtaInEE_L1R
HLT_Ele12_SW_TighterEleIdIsol_L1R
HLT_Ele17_SW_CaloEleId_L1R
HLT_Ele17_SW_EleId_L1R
HLT_Ele17_SW_LooseEleId_L1R
HLT_Ele17_SW_TightCaloEleId_Ele8HE_L1R
HLT_Ele17_SW_TightCaloEleId_SC8HE_L1R
HLT_Ele17_SW_TightEleIdIsol_L1R
HLT_Ele17_SW_TightEleId_L1R
HLT_Ele17_SW_TighterEleIdIsol_L1R
HLT_Ele17_SW_TighterEleId_L1R
HLT_Ele22_SW_CaloEleId_L1R
HLT_Ele22_SW_TighterCaloIdIsol_L1R
HLT_Ele22_SW_TighterEleId_L1R
HLT_Ele27_SW_TightCaloEleIdTrack_L1R
HLT_Ele32_SW_TightCaloEleIdTrack_L1R
HLT_Ele32_SW_TighterEleId_L1R
HLT_Ele40_SW_L1R
HLT_IsoEle12_PFTau15

The tasks:

(1) We need to take X---YYYY---TRIGGER mappings from the above URLs
and create auto-completion facility that would analyse entered primary dataset
value and generate trigger selection drop-down box with appropriate values.

(2) Note that one can add more than one dataset, so more than one
trigger selection drop-down should be auto-completed in this way.

(3) Note that to be on the safe side, the deposit UI could offer users
a free text box to fill in case trigger selection auto-completion did
not work correctly. (Say X->Trigger mapping was not complete.)

deposit: Other ([please specify) does not open text input box

Box "User Code", option "How to reproduce", value "Other (please upload)" opens well a file upload possibility.

SImilar thing is probably needed for box "User Code", option "Code Type", value "Other (please specify)" that should open a text input box or a text area, but it does not seem to do anything now. So that users are forced to use common "Code Comment" text area.

deposit: forms editable by several persons

Since various steps of the same analysis may be performed by different persons, it would be profitable to allow different group of people to fill the same deposit.

(1) E.g. sharing could happen via "Save" facility before the record is uploaded, which is harder WRT locking and correct diff-propagation.

(2) E.g. sharing could happen simply via post-upload editing of the record, which is harder WRT DOI assignements etc.

The collaboration-wide visibility of temporarily co-edited & co-submitted material does not seem to be an issue.

Create json-LD schema

Create schema to translate json to json-ld for export based on the ontology work done with DASPOS

deposit: simplified deposition tree

Currently all three collaboration depositions are listed under the same generic "Data Analysis" type. This makes one extra user click. As we won't have many deposition types listed there, it would be more ergonomic to start with various collaboration deposition types directly, without single common heading.

(BTW note the name should be "ALICE", not "Alice".)

search: customise available fields

The site comes with default fields some of which are not applicable (reference) which should be taken out, while some of existing fields should be created based on data model (analysisname and friends).

deposit: error page for wrong depositions

When accessing a wrong deposition such as /deposit/foobarblah, the system replies with an Internal Server Error. An error handler is missing in the overlay, probably.

A nicer handling of missing doctype situation would be desirable, e.g. replying the user with "The deposition foobarblash is not available". The same may be needed for upstream Invenio.

Map LEGO json to CAP json

Map the ALICE metadata in json coming from the LEGO system to the CAP data model and extend/adjust this accordingly for the ALICE submission forms

CMS survey testing

  • check dependencies
  • check if we need specific testing conditions with users
  • reach out to CMS to plan test (participants)

Done by Anxhela, feedback by @pherterich and myself.

LHCb: complete fields from BK path

From LHCb data samples input path like:

sim://LHCb/Collision12/Beam4000GeV-VeloClosed-MagDo wn/RealData/Reco14/Stripping20/90000000 ( Full stream )/BHADR

as well as event samples input path:

sim://MC/MC11a/Beam3500GeV-2011-MagDown-Nu2-50ns-EmNoCuts/Sim05a/Trig0x40760037Flagged/Reco12a/Stripping17NoPrescalingFlagged/42

we should auto-complete several fields in concerned boxes on the form, such as:

  • Reco14
  • Stripping version 20

add atlas logo

We have the go-ahead.

(is this going to be the repository for future development, or will it go into the open data project or a new project?)

footer: s/Submit/Deposit

Page footer templates should be amended to use Deposit instead of Submit. (Ditto in invenio generic "pu" branch.)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.