Quality assurance of the MONAI/examples folder about tutorials HOT 9 CLOSED

sure. i will work on the shell script and then raise a pull request.

from tutorials.

@wyli Regarding the python files, we can setup running them during CI/CD right, but wont executing each file take time and increase build time?
Also, is there a way for running jupyter notebooks automatically, like through a command?

from tutorials.

yes, with this feature implemented, the CI/CD is likely to take significantly more time. but I think we should address it with a separate ticket (e.g. refactoring the CI job configures so that they could run in parallel and take less time).

for this ticket we need to implement some initial pipelines (welcome discussions) for:

• checking the coding style
• checking that the examples work fine:
  • no syntax/type hint error
  • can finish in expected execution time
  • can generate correct outcomes -- model files, model quality measurements, tensorboard files

the notebooks are trickier (see also Project-MONAI/MONAI#874), there are some discussions in https://github.com/Project-MONAI/MONAI/issues/296 of the relevant tools but we still need to check their feasibility for our use case

cc @IsaacYangSLA @Nic-Ma @ericspod @benjamin-gorman

from tutorials.

Thanks for the explanation. Regarding checking the syntax error, we can simply execute the python file during the CI/CD "python filename.py" and then give it some upper execution time limit? Also, if I can be of help here, I would like to contribute to some of the points mentioned above.

from tutorials.

thanks @arp95, would be great to have a simple run_example_tests.sh for those python filename.py commands with a timer (and perhaps some checks of the commandline output logs with grep?), then we take another iteration to integrate the commands into the ci configure for example:
https://github.com/Project-MONAI/MONAI/blob/8f1c290b58ddfb4f9a9f5810916c3a7d48e26643/.github/workflows/setupapp.yml#L171-L181

from tutorials.

@wyli i was running the file "densenet_training_array.py". It requires the data to be downloaded from this website, https://brain-development.org/ixi-dataset/. Should we save the data in some MONAI google drive for online running or is there another way around this? Because we cant expect to be downloading data everytime during the CI/CD build right?

from tutorials.

@wyli i was running the file "densenet_training_array.py". It requires the data to be downloaded from this website, https://brain-development.org/ixi-dataset/. Should we save the data in some MONAI google drive for online running or is there another way around this? Because we cant expect to be downloading data everytime during the CI/CD build right?

I put those demo data here: https://www.dropbox.com/s/y890gb6axzzqff5/testing_ixi_t1.tar.gz?dl=1 please let me know if there's further issue

from tutorials.

thanks for the help. will check it out and let you know if I face any issues.

from tutorials.

addressed by #107 , please file new issues for any follow-up

from tutorials.