Code Monkey home page Code Monkey logo

modelstats's People

Contributors

dklein-pik avatar giannou avatar mikapfl avatar orichters avatar pfuehrlich-pik avatar pre-commit-ci[bot] avatar whitehacker avatar

Stargazers

avatar

Watchers

avatar avatar avatar avatar

Forkers

haxxenpirat orichters giannou chroetz pfuehrlich-pik mikapfl dklein-pik whitehacker

modelstats's Issues

Improve abort message

It would be nice to have a better abort message for rs2 -cr for when there are no current runs, than:
Global .Rprofile loaded!
Error in myruns[[i]] : subscript out of bounds
Calls: <Anonymous> -> grepl
Execution halted

Run `make test-full` during REMIND tests

I think it would be good if make test-full would be run during a REMIND modeltest. Could be implemented roughly like this:

logfile <- "make-test.txt" # best add some _2023-10-25 etc. here
system(paste0("sbatch --qos=priority --wrap='make test-full' --job-name=make-test --mail-type=END --output=", logfile, " --comment=REMIND_AMT_make_test-full"))

And then later add the outcome of this line to the gitlab commit:

paste0(logfile, ": ", tail(grep("[ FAIL", readLines(logfile), value = TRUE, fixed = TRUE)))

which returns something like

make-test.txt: [ FAIL 0 | WARN 0 | SKIP 6 | PASS 83 ]

AMT slurm priority wrong

Currently, AMTs are running in qos=priority, which is imho inappropriate for automated, periodic stuff.

modeltests() reports converged AMTs as "did not start at all"

https://gitlab.pik-potsdam.de/REMIND/testing_suite/-/blob/9ec9cfcb99005d10fc9150feb5af7ed84eac938d/README.md?blame=1#L32-50

These scenarios did not start at all:
SSP2EU-NDC-AMT
SSP2EU-PkBudg500-AMT
SSP2EU-PkBudg650-AMT
SSP2EU-PkBudg1050-AMT
SSP2EU-EU21-NDC-AMT
SSP2EU-EU21-PkBudg650-AMT
SSP2EU-EU21-PkBudg500-AMT
SSP2EU-EU21-PkBudg1050-AMT
SSP1-NDC-AMT
SSP1-PkBudg650-AMT
SSP1-PkBudg1050-AMT
SSP5-NDC-AMT
SSP5-PkBudg650-AMT
SSP5-PkBudg1050-AMT
SDP_MC-Base-AMT
SDP_MC-NDC-AMT
SDP_MC-NPi-AMT
SDP_MC-PkBudg650-AMT

vs.

/p/projects/remind/modeltests/remind/output $ ls *_2024-01-0{5,7}_*/log.txt | grep -f <( sed -n '/These scenarios did not start at all:/,$ { /^S[SD]/ p }' /p/projects/remind/modeltests/testing_suite/README.md ) | xargs grep "full.log states:" | sed 's/:/:@/' | column -ts '@'

SDP_MC-Base-AMT_2024-01-05_22.10.45/log.txt:          full.log states: *** Status: Normal completion
SDP_MC-NPi-AMT_2024-01-05_22.11.06/log.txt:           full.log states: *** Status: Normal completion
SDP_MC-PkBudg650-AMT_2024-01-07_04.11.05/log.txt:     full.log states: *** Status: Normal completion
SSP1-NDC-AMT_2024-01-07_06.46.39/log.txt:             full.log states: *** Status: Normal completion
SSP1-PkBudg1050-AMT_2024-01-07_04.10.42/log.txt:      full.log states: *** Status: Normal completion
SSP1-PkBudg650-AMT_2024-01-07_04.10.34/log.txt:       full.log states: *** Status: Normal completion
SSP2EU-NDC-AMT_2024-01-07_04.10.01/log.txt:           full.log states: *** Status: Normal completion
SSP2EU-PkBudg1050-AMT_2024-01-07_04.10.26/log.txt:    full.log states: *** Status: Normal completion
SSP2EU-PkBudg500-AMT_2024-01-07_04.10.10/log.txt:     full.log states: *** Status: Normal completion
SSP2EU-PkBudg650-AMT_2024-01-07_04.10.18/log.txt:     full.log states: *** Status: Normal completion
SSP5-PkBudg1050-AMT_2024-01-07_04.10.57/log.txt:      full.log states: *** Status: Normal completion
SSP5-PkBudg650-AMT_2024-01-07_04.10.49/log.txt:       full.log states: *** Status: Normal completion

wish list for the model tests …

From @0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q:

  • It would be really useful to have an overview table (say a .csv file) with the run name (e.g. SSP2EU-AMT-NDC), the date, and the RunStatus (maybe for good measure also the commit hash) to figure out since when certain runs fail in order to narrow down which changes might have caused this. Current example would be SSP2EU-AMT-NDC failing since some time. -> #59
  • (Bit of a pet peeve) The test run output directories are partly truncated in the README.md (e.g. SSP2EU-AMT-calibrate_2022-11-26_00 instead of SSP2EU-AMT-calibrate_2022-11-26_00.07.54) which is a nuisance when looking up results. -> #22

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.