The need

A script given an "ontology acronym" and a range of submission ids ( from "submission id" to "submission id" ), removes all the data (triple store) and files of the submissions inside that range.

Use case

From "stageportal" server, inside the "/srv/ontoportal/ncbo_cron" folder. The following command :

[ontoportal@stageportal ncbo_cron]$ bin/ncbo_ontology_submissions_eradicate -o MDRFRE --from 2 --to 5

Will :

1. show you the list of all submissions that will be affected(removed).
2. ask you to confirm (by typing "y'" for yes or "n" for no) if you want to remove (the data and files) of all the submission between 2 (included) and 5 (included).

Important: after the script a cache refresh is needed to see the result.

Details

• The arguments "from" and "to" are reversible, i.e. --from 2 --to 5 is equal to --from 5 --to 2
• The script delete the submission folder(files) then the data
• Only the archived "submissions" (not used in production) can be removed

At the end of the script, the ncbo_ontology_process script print in the console the content of the Annotator cache.
We should fix this as it prevents reading the script when run in the console.

Issue

There is only one where the testing if a new file exists, is in the pull_location CRON job . Which is done once a day.

The problem is that if we do an ontology reprocess with the ncbo_ontolgy_process script, it will not test the existence of a new version.

Solution

There are two ways to solve this

1. The first and simple one is to just add a script called ncbo_ontology_pull to do the pull on demand.
2. The second more complex, is to add in the submission process workflow a step that comes before the generate_rdf step called do_pull_location that will download and create a new submission if a new version is found.

version to merge: https://github.com/ncbo/ncbo_cron/releases/tag/v5.22.3
changes: gemfile updates
conflicts:

• none

Deploy:

• testportal
• stageportal
• agroportal
• bioportal

This is a sub task of ontoportal-lirmm/ncbo_annotator#4

This consist of modifying the process to generate the mgrep dictionary file in order to generate another file that would be normalized using TreeTager.

Assign to @vemonet and @amineabdaoui

in the config/config.rb add config.enable_pull_umls = true

 NcboCron.config do |config|
    config.redis_host           = Annotator.settings.annotator_redis_host
    config.redis_port           = Annotator.settings.annotator_redis_port
    config.enable_ontology_analytics = false
    config.search_index_all_url = 'http://localhost:8983/solr/term_search_core2'
    config.property_search_server_index_all_url = 'http://localhost:8983/solr/prop_search_core2'
    config.ontology_report_path = "#{$DATADIR}/reports/ontologies_report.json"
    config.enable_spam_deletion = false
    config.enable_pull_umls     = true
  end

Changes done in https://gite.lirmm.fr/bioportal/bioportal-configs/-/blob/master/agroportal/cron/config/config.rb for stage,agro and bioportal.

Requirement

bin/ncbo_ontology_process -o INRAETHES -t extract_metadata

Merge to the tag: v 5.22.1
Result : No conflicts

Changed files :

State:

• Deployed on stage
• Deployed on production