Cannot create dataset COVIDx8 about covid-net HOT 7 CLOSED

Hi Jessica,

Thanks for the detailed explanation of the issue! We'll have to take a closer look into what's going on here, but in the meantime I'd recommend downloading the prepared version of the dataset to save yourself some time and headache. The data preparation scripts can be a little finnicky, but the prepared version will exactly match the training and testing data used for CXR-2.

As a check, I re-downloaded the Kaggle data and ran eval.py with the CXR-2 model, yielding:

[[194.   6.]
 [  9. 191.]]
Sens Negative: 0.970, Positive: 0.955
PPV Negative: 0.956, Positive: 0.970

which matches the reported results.

from covid-net.

Hello @haydengunraj
Is there any prepared version of the dataset for the 3class version too? Because it seems like the dataset you shared above has some missing images when I tried to run train_tf.py for the 3class version.

from covid-net.

Hi @baranaldemir

On my end I seem to be able to run train_tf.py for 3-class prediction using the dataset from Kaggle. Note that you should use one of the 3-class models (e.g., CXR4-A) rather than CXR-2 (2-class). The command I used was:

python train_tf.py \
    --datadir /path/to/COVIDx8 \
    --weightspath /path/to/COVIDNet-CXR4-A \
    --ckptname model-18540 \
    --trainfile labels/train_COVIDx8A.txt \
    --testfile labels/test_COVIDx8A.txt \
    --n_classes 3 \
    --out_tensorname norm_dense_1/Softmax:0 \
    --logit_tensorname norm_dense_1/MatMul:0 \
    --training_tensorname batch_normalization_1/keras_learning_phase:0

The training gets through the first epoch without issue. For the baseline eval of CXR4-A I get:

[[ 94.   6.   0.]
 [  5.  95.   0.]
 [  4.   1. 195.]]
Sens Normal: 0.940, Pneumonia: 0.950, Covid-19: 0.975
PPV Normal: 0.913, Pneumonia: 0.931, Covid-19: 1.000

from covid-net.

Hi Jessica,

Thanks for the detailed explanation of the issue! We'll have to take a closer look into what's going on here, but in the meantime I'd recommend downloading the prepared version of the dataset to save yourself some time and headache. The data preparation scripts can be a little finnicky, but the prepared version will exactly match the training and testing data used for CXR-2.

As a check, I re-downloaded the Kaggle data and ran eval.py with the CXR-2 model, yielding:

[[194.   6.]
 [  9. 191.]]
Sens Negative: 0.970, Positive: 0.955
PPV Negative: 0.956, Positive: 0.970

which matches the reported results.

Thank you for your reply,

at the end I managed to create the COVIDx8 dataset using the provided scripts and to reproduce the results of the model COVIDNet-CXR-2. I think I had a slighly old version of the repository that hindered the creation of the dataset.
I wasn't aware about the availability of a prepared version of the dataset on Kaggle. I believe there is no mention of that in the documentation, ora at least I did not find it.
I would suggest to add the availability of this dataset in the documentation or to make it more evident :)

Is there also an available prepared version of older datasets?

All the best,
Jessica

from covid-net.

Hi @baranaldemir

On my end I seem to be able to run train_tf.py for 3-class prediction using the dataset from Kaggle. Note that you should use one of the 3-class models (e.g., CXR4-A) rather than CXR-2 (2-class). The command I used was:

python train_tf.py \
    --datadir /path/to/COVIDx8 \
    --weightspath /path/to/COVIDNet-CXR4-A \
    --ckptname model-18540 \
    --trainfile labels/train_COVIDx8A.txt \
    --testfile labels/test_COVIDx8A.txt \
    --n_classes 3 \
    --out_tensorname norm_dense_1/Softmax:0 \
    --logit_tensorname norm_dense_1/MatMul:0 \
    --training_tensorname batch_normalization_1/keras_learning_phase:0

The training gets through the first epoch without issue. For the baseline eval of CXR4-A I get:

[[ 94.   6.   0.]
 [  5.  95.   0.]
 [  4.   1. 195.]]
Sens Normal: 0.940, Pneumonia: 0.950, Covid-19: 0.975
PPV Normal: 0.913, Pneumonia: 0.931, Covid-19: 1.000

@haydengunraj Using the same command and Kaggle dataset but the output says there are some missing files. Plus, I used create_COVIDx.ipynb and created the dataset. Again, It gave me the same output.

from covid-net.

@GliozzoJ , we don't currently have prepared versions of the previous datasets, although we may have archived versions of them available internally. We may also be able to help with generating past versions from source if you have a particular version in mind.

@baranaldemir , are you using the most recent master branch of the repository? The results I mentioned above were obtained from a fresh pull of both the Kaggle data and codebase, so on our end everything appears to be working correctly. If the most recent master branch does not fix your issue (or if you're already using it), I'll try downloading everything again to see if I can reproduce the issue.

from covid-net.

Yeah, latest master branch did work. I'm so sorry for replying this late.

from covid-net.