microsoft / innereye-createdataset Goto Github PK

• Add more docs on how to use the key features from create dataset

This tool contains many useful and powerful features, but none of these are documented. Documentation on how to use this tool to its full potential needs to be created to make it easier for our clinical partners to work with their datasets.

Add dependencies and copyright notices

The pipeline triggers are currently not working and the build pipeline needs to be manually started for PRs

The following issues need correcting in the installation documentation:

• git lfs is now deprecated and should be updated to use modern git commands
• Emphasise more that VS 2017 is the version of VS that needs to be used.
• Correct package name VS++ to VC++

2>ConnectedComponentsClr.obj : /DEBUG:FASTLINK is not supported when managed code is present; restarting link with /DEBUG:FULL
2>ImageProcessing.lib(GaussianKernel1D.obj) : warning LNK4075: ignoring '/EDITANDCONTINUE' due to '/OPT:LBR' specification
2>LINK : fatal error LNK1104: cannot open file 'MSCOREE.lib'

Resolve by https://stackoverflow.com/questions/41030806/visual-studio-c-cli-mysterious-error-with-template#:~:text=Check%20in%20Visual%20Studio%20installer%20%27C%2B%2B%2FCLI%20support%27%20for,Add%20both%20entries%20from%20below%20separated%20by%20semi-colon, installing .net 4.6.1 SDK?

This tool does not add a "tags" column expected by InnerEye-DeepLearning and it generates this eerror:

File "InnerEyePrivate/ML/runner.py", line 50, in
main()
File "InnerEyePrivate/ML/runner.py", line 44, in main
runner.run(project_root=fixed_paths.repository_root_directory(),
File "/mnt/batch/tasks/shared/LS_root/jobs/radiomicsnn/azureml/master_1674467245_aec08e9c/wd/azureml/master_1674467245_aec08e9c/innereye-deeplearning/InnerEye/ML/runner.py", line 457, in run
return runner.run()
File "/mnt/batch/tasks/shared/LS_root/jobs/radiomicsnn/azureml/master_1674467245_aec08e9c/wd/azureml/master_1674467245_aec08e9c/innereye-deeplearning/InnerEye/ML/runner.py", line 220, in run
self.run_in_situ(azure_run_info)
File "/mnt/batch/tasks/shared/LS_root/jobs/radiomicsnn/azureml/master_1674467245_aec08e9c/wd/azureml/master_1674467245_aec08e9c/innereye-deeplearning/InnerEye/ML/runner.py", line 411, in run_in_situ
self.ml_runner.setup(azure_run_info)
File "/mnt/batch/tasks/shared/LS_root/jobs/radiomicsnn/azureml/master_1674467245_aec08e9c/wd/azureml/master_1674467245_aec08e9c/innereye-deeplearning/InnerEye/ML/run_ml.py", line 208, in setup
self.container.setup()
File "/mnt/batch/tasks/shared/LS_root/jobs/radiomicsnn/azureml/master_1674467245_aec08e9c/wd/azureml/master_1674467245_aec08e9c/innereye-deeplearning/InnerEye/ML/lightning_base.py", line 160, in setup
dataset_splits = self.config.get_dataset_splits()
File "/mnt/batch/tasks/shared/LS_root/jobs/radiomicsnn/azureml/master_1674467245_aec08e9c/wd/azureml/master_1674467245_aec08e9c/innereye-deeplearning/InnerEye/ML/model_config_base.py", line 214, in get_dataset_splits
splits = self.get_model_train_test_dataset_splits(dataset_df)
File "/mnt/batch/tasks/shared/LS_root/jobs/radiomicsnn/azureml/master_1674467245_aec08e9c/wd/azureml/master_1674467245_aec08e9c/InnerEyePrivate/ML/configs/segmentation/Prostate.py", line 23, in get_model_train_test_dataset_splits
test = list(dataset_df[dataset_df.tags.str.contains("ContinuousLearning")].subject.unique())
File "/azureml-envs/azureml_ee6e8ff99c839137094f58bdb42aca60/lib/python3.8/site-packages/pandas/core/generic.py", line 5130, in getattr
return object.getattribute(self, name)
AttributeError: 'DataFrame' object has no attribute 'tags'

For some reason new PRs don't automatically trigger the DevOps pipeline and require manual intervention. This is annoying and should be fixed.

The link in section https://github.com/microsoft/InnerEye-createdataset#run-analysis-on-a-converted-dataset is broken.

Hello,
Is there an option to call the create-dataset-utility on a linux environment (through the bash)? If yes can I find somewhere documentation (i.e. how to build the binary from source c code)?

Suggestion1: The utility gives the user the option to "restructure" ones nifti images+segmentations. The result will be a proper folder structure, nomenclature and modifications (normalization etc) of the nifti images+segmentations to adhere to the constraints of the innereye-deeplearning.
Example:

init_images/
├── HUSAH150_ANON0005_ax0.nii.gz (this is a head ct volume)
├── HUSAH150_ANON0005_ax0_seg.nii.gz (this is its corresponding lesion segmentation)
├── HUSAH150_ANON0008_ax0.nii.gz
├── HUSAH150_ANON0008_ax0_seg.nii.gz
├── HUSAH150_ANON0023_ax0.nii.gz
└── HUSAH150_ANON0023_ax0_seg.nii.gz

restruct_images/
├── subjectID1
│   ├── blood.nii.gz
│   └── head.nii.gz
├── subjectID2
│   ├── blood.nii.gz
│   └── head.nii.gz
└── subjectID3
    ├── blood.nii.gz
    └── head.nii.gz

Why: It is not necessary that the users initial image+segmentation sets will be in a DICOM format.

Suggestion2: For the case where the image+segmentation sets are in DICOM please consider the option of adding a proper DICOM IOD for the segmentations in your options (more details in http://dicom.nema.org/dicom/2013/output/chtml/part03/sect_A.51.html )
Why: The RTStruct is an old format that has been created for radiation therapy. It creates planar contours (surfaces) instead of volumes. When the geometry of the segmented structure is complex, converting the RTSTRUCT IOD to a binary labelmap (like in a typical nifti format segmentation with 0 values on all voxels that are of no interest and 1 for all the voxels of interest) will lead to mistakes and segmentation quality degradation. That's the reason modern segmentation software that can segment DICOM images will save the segmentations as SEG objects and not RTSTRUCT.
Since though a lot of segmentation material exist in the form of RTSTRUCT IODs, the converter of RTSTRUCT to NIFTI of this utility will certainly be beneficial.

The README currently recommends selecting the Windows SDK 10.0.17134.0 when installing Visual Studio components. This causes the build process to fail. The correct version is 10.0.19041.0