Hello,

Could you please upload the trained models weights or state_dict()? Thank you.

Dear @JoHof,

first of all, thanks for sharing your code publicly and providing an easy way to run inference with your network.

I just wanted to report some strange behaviours running the LTRCLobes version of your network for some of the images in my dataset. Please look at the example below to have an idea:

Have you experienced similar behaviours using your network for lobe segmentation?

Hi there,
thank you so much for providing this tool! It is very easy and convenient to use.
I experimented with your model for the covid challenge (https://covid-segmentation.grand-challenge.org/COVID-19-20/). I used the R231CovidWeb model. Unfortunately it was producing some errors, especially in regions where covid-induced alterations were close to the border of the lungs. It appears to me that you are using a 2D U-Net to segment the lungs. This may be a suboptimal design choice for 3D medical image data.
Is the dataset you trained your model with publicly available? I would like to use it to train a 3D U-Net (with nnU-Net https://github.com/MIC-DKFZ/nnUNet) and see whether that improves the results on the CovidSeg challenge dataset.
Best,
Fabian

I have a nii file, shape is (41, 226, 295), when using the lungmask demo, it failed. (I have tried many other files and succeed. )

This is a great code, thanks for sharing!

I am trying to process a bunch of directories, but when I run the command

lungmask input/1864/19640829_CTCAP/NA_90_OMNI_350 output/1864_19640829_CTCAP_NA_90_OMNI_350,

I get this error:

INFO:root:Save result to: output/1864_19640829_CTCAP_NA_90_OMNI_350/
Traceback (most recent call last):
  File "/home/dss/.local/bin/lungmask", line 8, in <module>
    sys.exit(main())
  File "/home/dss/.local/lib/python3.8/site-packages/lungmask/__main__.py", line 60, in main
    sys.exit(sitk.WriteImage(result_out, args.output))
  File "/home/dss/.local/lib/python3.8/site-packages/SimpleITK/extra.py", line 394, in WriteImage
    return writer.Execute(image)
  File "/home/dss/.local/lib/python3.8/site-packages/SimpleITK/SimpleITK.py", line 5518, in Execute
    return _SimpleITK.ImageFileWriter_Execute(self, *args)
RuntimeError: Exception thrown in SimpleITK ImageFileWriter_Execute: /tmp/SimpleITK/Code/IO/src/sitkImageFileWriter.cxx:206:
sitk::ERROR: Unable to determine ImageIO writer for "output/1864_19640829_CTCAP_NA_90_OMNI_350/"

Any idea why this might be happening? I have verified that the code works perfectly on a single file, and the output dir exists and is writable.

The link to LTRC dataset on the README page is broken. Could to please share the updated LTRC dataset link?

Thanks.

Do you know of any use of your segmentation models in lung x-ray segmentation? If so, is it straightforward (minor code change)?

Hi,

Thank you so much for this amazing work.
I am experimenting with your model on our CT data and came across some data where it is failing.
I would like to finetune this model on our data.
It will be really helpful if you can provide the scripts to train your Network.

Thanks in Advance!

Hi @JoHof,

Just wondering if there is a simple way to hide the progress bars so I can avoid them showing up when I redirect STDOUT to a text file.

Thanks.

Hi @JoHof

I tried using the Unet R231 model on Dicom Images from the LIDC-IDRI Dataset
The model returned an empty mask.
I have attached a zip file with 1 dicom file and the code I am using to find the mask.
Segmentation_On_LIDC_IDRI_Dataset.zip

Thanks!

RuntimeError: Exception thrown in SimpleITK Image_CopyInformation: D:\a\1\sitk\Code\Common\src\sitkImage.cxx:227: sitk::ERROR: Source Image for information does not match this image's dimension.

While running the module on single slice of the lung

From the help page i see the resunet model, but it's not working at all.

The error is
model_url, n_classes = model_urls[(modeltype, modelname)] KeyError: ('resunet', 'LTRCLobes')

clearly because the array doesn't contain urls for resunet model.

Has not it been implemented yet?

Regards

GB

Hi, thank you for your work, and I follow the step by step in blog post and try to map images back to HU, but it seems something wrong with normalize_function, the value inside the function will be air = 1, ratio = 900, air_HU = -1000, with air = 1 and fat = 2 as the input , is that correct? But with this setting, the images can't map back to the HU i think, please help me with that.

Best

I have read your work, it's very nice. And i want to know how to train the model described in this repo and how many data have been used. Can you share some experiment data and training code?

Hello,

I have been using lungmask to segment CT images as a part of a radiogenomics project through my organization. We have a big data set with 130 patients where each patient has 256 DICOM images. We were really excited to use your tool because it will automate a lot of the segmentation we would have otherwise had to do. However, recently I stumbled on the following bug after many calls to lungmask.mask.apply.

SError: [Errno 24] Too many open files

I'm unsure of whether there are open file handles in the software or whether I'm doing something incorrectly. In order to reproduce this bug, you have to apply this to over 200 files consecutively. I realize this takes a while to do, so I was just wondering whether this error has ever come up on your side when developing this software.

If you need more information from me, please let me know.

It seems that the model hasn't been upload

so i am using a .nii file, and its giving me all black images. I have tried using the '--nopostprocess' flag and then not using it, but nothing changed. My .nii file has multiple slices.

Originally posted by @saranggoel in #7 (comment)

Hi, when I give the path to OUTPUT file, an error occurs:
sitk::ERROR: Unable to determine ImageIO writer for "/content/OUTPUT"
can you suggest something about it?

Dear all, the segmentation was working fine with my images but yesterday I installed a bunch of new packets and anaconda and now the segmentation is no more working.

I guess that it's somehow related with new version of the installed packages but I've not been able to solve the issue.

I tried to remove and reinstall, both in pip3 (/Library/Frameworks/Python.framework/Versions/3.7/bin/pip3) and pip (/Users/Giulio/opt/anaconda3/bin/pip) but the issue is permanent.

$ which lungmask
/Users/Giulio/opt/anaconda3/bin/lungmask

Using another computer and the same image the segmentation is correct.

Is there a way to check what's going on?

$ lungmask --modelname LTRCLobes /Users/Giulio/Desktop/test_seg/2_Std_25.nrrd /Users/Giulio/Desktop/test_seg/2_Std_25.seg.nrrd

Result in my pc (MacOS Mojave)

Result on my server (Ubuntu 19.04)

Thank you for the support.
If you want I can provide more details on installation/folders etc.

Regards.

Hi
When i give the input path to a dicom file which has whole lung surrounded by tissues, the segmentation mask is an all black image.
can you suggest something about it?

I ran the below command and getting the error.

!lungmask '/content/images_001/images/' '/content/images_001/mask/'

the dataset i am using is this https://www.kaggle.com/nih-chest-xrays/data

Error:

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

is there a problem with the type of input image

Dear @JoHof,

thanks for sharing this work. I am posting this issue just because it seems like a simple fix that can increase the robustness of the whole pipeline. While checking the performance on some scans I figured than in some cases, the model produced an additional segment outside of the body mass along with the left/right labels. I found cases with both 1 and 2 as label for this extra segment. Here is an example:

I would guess that a simple counting of the connected components in the final segmentation could fix this issue.

Hello again,

I had the idea to use your models as pre-trained models for segmenting
lungs and lesions in COVID, I tried to convert it to a tensorflow model with onnx
but I didn't succeed. Do you think the model and weights could be converted
and made accessible for tensorflow as well as .h5 files for example?
Apologies for asking such things ^^,

Thanks anyway,
Joris

Hi,

Thanks for sharing your work. I got the lung masks you kindly produced and are shared here. I see the masks have values ranging from 1 to 3. Could you please explain what each value correspond to? Something like 1 = right lung, 2= left lung etc. Thanks

Hey everyone,
thanks for the great package that you've developed.

I need to include mask making inside a wider pipeline. Could it be possible if you added an argument verbose that allows the user to avoid printing the tqdm progress bar and all other checkpoints/warnings?

If needed, I could give some help myself.

Thanks and take care
Andrea

Hello, i am very intestered by this one-liner that i found in your pre-processing of simpleitk images:

np.flip(data, np.where(directions[[0,4,8]][::-1]<0)[0])

Visualizing the data before and after this operation, it seems that it normalizes the direction of the CT volume.

Can you confirm this line alone is enough to normalize the 3D direction of data extracted using SimpleITK, from DCM and NIFT images?

Thank you!

Hello,
First of all thank you very much for this!
It's easy to use and working great.
I had a few question, I'm currently working on a model
to segment the COVID casis and plan to use your model
or a separate model to get the lung volumes or as a way
to initialize segmentations of lungs to be manually refined
in order to train a model of my own.
What do you think is the best solution for me ?
I have no idea what amount of data you used to have a robust model
and therefore I don't know how much data I need to label to have a
model working as good. Another solution would be to be able
to add the data I will produce and fine-tune/retrain your model as
more data get collected, would that be desirable/doable, in my case ?

Anyway be blessed for your contribution!
Joris

I use lungmask's apply_fused() in one of the functions of a class. I then call that function in a Celery task decorated function.
Using this command to collect celery tasks:

celery -A backend.celery worker -l info

and then calling the Celery task decorated function will result in the following error:

daemonic processes are not allowed to have children

The workaround I found is using a solo pool as:

celery -A backend.celery worker --pool=solo -l info

But I'm not sure if this workaround yields the best performance and run time.
Any suggestions?

By the way, here's the file where I use lungmask's apply_fused():
https://github.com/pesfahanian/dr_machine/blob/master/backend/CT_LIS/core/model.py
And here's where I define the celery task:
https://github.com/pesfahanian/dr_machine/blob/master/backend/CT_LIS/tasks.py

Is it possible to recursively produce a lungmask for each dicom in a folder and maintain the same file name?

For example with a -f flag?

lungmask -f ./dicom ./output

Thank you so much for this work! But I have problems applying the mask on png images.
This is the original image:

This is the code:
`
import SimpleITK as sitk
from lungmask import mask #lung segmentation model
model = mask.get_model('unet','R231CovidWeb')
i = sitk.ReadImage("../trainData/subject-level/Cap/cap003/CT0014.png")

i_np = sitk.GetArrayFromImage(i)
print(i_np.shape)
print(i_np.min())
print(i_np.max())

i_np2=i_np[None,::]
i_hu=sitk.GetImageFromArray(i_np2)

segmentation = mask.apply(i, model, noHU=True)
print(segmentation.shape)
print(segmentation.min())
print(segmentation.max())

s2 = mask.apply(i_hu, model)
print(s2.shape)
print(s2.min())
print(s2.max())
`
The output:
(512, 512)
38
255
/home/mist/.local/lib/python3.6/site-packages/lungmask/mask.py:51: FutureWarning: The behavior of rgb2gray will change in scikit-image 0.19. Currently, rgb2gray allows 2D grayscale image to be passed as inputs and leaves them unmodified as outputs. Starting from version 0.19, 2D arrays will be treated as 1D images with 3 channels.
tvolslices = skimage.color.rgb2gray(inimg_raw)

(1, 512, 512)
0
2

(1, 512, 512)
0
0

In the above code, I tried both: with the --noHU tag, and without the tag(adding a dimension by img=img[None,::]), But only got an almost all black mask using both ways. Could you please tell me why and how to modify it? Thanks so much!

To whom it may concern,

I've a problem with the LTRCLobes_R231 model. If I run it, I obtain the following error:

lungmask /Chest_CT/S700830/S30 /Chest_CT/out/output.nii.gz --modelname LTRCLobes_R231
INFO:root:Load model
INFO:root:Looking for dicoms in /Chest_CT/S700830/S30
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 830/830 [00:01<00:00, 700.76it/s]
INFO:root:There are 1 volumes in the study
INFO:root:Infer lungmask
Traceback (most recent call last):
File "/home/nico/.local/bin/lungmask", line 11, in
load_entry_point('lungmask==0.2.3', 'console_scripts', 'lungmask')()
File "/home/nico/.local/lib/python3.8/site-packages/lungmask/main.py", line 39, in main
result = mask.apply_fused('LTRCLobes','R231')
File "/home/nico/.local/lib/python3.8/site-packages/lungmask/mask.py", line 93, in apply_fused
res_l = apply(image, mdl_l, force_cpu=force_cpu, batch_size=batch_size, volume_postprocessing=volume_postprocessing)
File "/home/nico/.local/lib/python3.8/site-packages/lungmask/mask.py", line 26, in apply
voxvol = np.prod(image.GetSpacing())
AttributeError: 'str' object has no attribute 'GetSpacing'

If I use the default model, i have no errors.

What can I do ot resolve this issue?

Best Regards

P.s.: I changed the dataset, same error.

Hi all;

I have been reading the paper and I realised that the loss functions are not explicitly shown. Are you using the original loss functions described as in it architecture references?

Best!

Thank you for this repo. I was just curious about the meaning of the values in the array returned by the segmentation function mask.apply(input_image). I know 0 is the background, 1 and 2 are the lungs, but are other values possible and what do they mean? For example, I think the model detects air pockets?

Thanks for contributing such a good pathological lung dataset and segmentation model. I am now engaged in the segmentation task of pathological lung. May I ask if the R231 dataset can be made public after privacy is eliminated?
Thank you very much.

Hi,
Thanks for your excellent work, I used it to segment the lung, but the following error occurred:

Traceback (most recent call last):
  File "Lung_mask.py", line 133, in <module>
    save_lung_mask()
  File "Lung_mask.py", line 16, in save_lung_mask
    segmentation = mask.apply(input_image,model)
  File "/home/jhcheng/userfolder/Anaconda3/envs/py36/lib/python3.6/site-packages/lungmask/mask.py", line 62, in apply
    outmask = np.asarray([utils.reshape_mask(outmask[i], xnew_box[i], inimg_raw.shape[1:]) for i in range(outmask.shape[0])], dtype=np.uint8)
  File "/home/jhcheng/userfolder/Anaconda3/envs/py36/lib/python3.6/site-packages/lungmask/mask.py", line 62, in <listcomp>
    outmask = np.asarray([utils.reshape_mask(outmask[i], xnew_box[i], inimg_raw.shape[1:]) for i in range(outmask.shape[0])], dtype=np.uint8)
  File "/home/jhcheng/userfolder/Anaconda3/envs/py36/lib/python3.6/site-packages/lungmask/utils.py", line 79, in reshape_mask
    res[tbox[0]:tbox[2],tbox[1]:tbox[3]] = imgres
ValueError: could not broadcast input array from shape (26,871) into shape (26,803)

Hi, I have been trying to implement your model (I really like how the repo has been presented and packaged btw!). I follow the steps in the readme and when I input a dicom slice or 3D array I get an empty array (2D or 3D respectively) as the output. Has this been seen before? Any idea why this might be happening?

Hello,

I was trying to run some tif files, but it keeps giving me the error that I have to many indices for the array. This is probably due to my images being a 512x512 resolution, while the code accepts 256x256. Do you know what I may change in order for these tif files to work?

Great work! Using LTRCLobes_R231 I got 6 labels. Am I assuming correctly that the 6th label is where the LTRCLobes and R231 segmentations don't agree during the fusion step? If that assumption is correct LTRCLobes_R231 will output max 6 labels (right lung = [3,4,5], left lung = [1,2])

Dear Johannes,

When I tried to segment a test image, I was unable to proceed due to the absence of the fill_voids.py or even a function that does the same. I kindly request, if possible, that we include the file in the next commit.

`ImportError Traceback (most recent call last)
in ()
13
14
---> 15 from lungmask import mask
16 import SimpleITK as sitk

~/lungmask-git/lungmask/mask.py in ()
1 import numpy as np
2 import torch
----> 3 from . import utils
4 import SimpleITK as sitk
5 from .resunet import UNet

~/lungmask-git/lungmask/utils.py in ()
9 import logging
10 from tqdm import tqdm
---> 11 import fill_voids
12
13

ImportError: No module named 'fill_voids'`

Best regards

Hi,

first of all, thank you very much for making publicy available this amazing software.

I am interested in segmenting lots of lung in dicom format (I have 228 .dcm files for each individual) - I executed this for the first individual which is in the folder INPUT (in this folder I have 228 slices: 1.dcm, 2.dcm, ... 228.dcm)

input_image = sitk.ReadImage(INPUT)

and I get this error message - Any idea about how to fix this will be wellcome!

HDF5-DIAG: Error detected in HDF5 (1.10.6) thread 0:
#000: /tmp/SimpleITK-build/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5F.c line 370 in itk_H5Fis_hdf5(): unable open file
major: File accessibilty
minor: Not an HDF5 file
#1: /tmp/SimpleITK-build/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Fint.c line 830 in itk_H5F__is_hdf5(): unable to locate file signature
major: File accessibilty
minor: Not an HDF5 file
#2: /tmp/SimpleITK-build/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5FDint.c line 126 in itk_H5FD_locate_signature(): unable to read file signature
major: Low-level I/O
minor: Unable to initialize object
#3: /tmp/SimpleITK-build/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5FDint.c line 205 in itk_H5FD_read(): driver read request failed
major: Virtual File Layer
minor: Read failed
#4: /tmp/SimpleITK-build/ITK/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5FDsec2.c line 725 in H5FD_sec2_read(): file read failed: time = Wed Dec 15 20:45:17 2021
, filename = '/home/jr/radiomics/eclipse/', file descriptor = 17, errno = 21, error message = 'Is a directory', buf = 0x7ffed2a058e8, total read size = 8, bytes this sub-read = 8, bytes actually read = 18446744073709551615, offset = 0
major: Low-level I/O
minor: Read failed
RuntimeError: Exception thrown in SimpleITK ImageFileReader_Execute: /tmp/SimpleITK/Code/IO/src/sitkImageReaderBase.cxx:105:
sitk::ERROR: Unable to determine ImageIO reader for "/home/jr/radiomics/eclipse/"

I used a piece of PNG data and also used noHU tag, but it did not segmet the lung mask. Could you help give a more specific example?
Thank you

Dear @JoHof
Thank you so much for this amazing library, but for some reason it is not working out for me
I hope you could be of help

When I read the image using SimpleITK and then try to apply a mask on that image, I get an error:
IndexError: index 4 is out of bounds for axis 0 with size 4

The error happens in the mask function on line 29:
inimg_raw = np.flip(inimg_raw, np.where(directions[[0,4,8]][::-1]<0)[0]

I am trying to work on CT-images downloaded online, their format is either jpg or png

What can I do to fix this issue?

Is it possible to get the segmented image (c) in the image as opposed to the current result of (b)?

Maybe another library?

Would it be possible to share your trained weights from the network?
thanks in advance

Hello,
I installed lungmask and try to run it but got the error below.
Could you help me to solve this problem?
I am using WIndows without GPU.

INFO:root:No GPU support available, will use CPU. Note, that this is significantly slower!
0%| | 0/102 [00:00<?, ?it/s]
Traceback (most recent call last):

File "C:\Users.spyder-py3\COVID.py", line 16, in
segmentation = mask.apply(input_image) # default model is U-net(R231)

File "C:\Users\Anaconda3\lib\site-packages\lungmask\mask.py", line 64, in apply
for X in tqdm(dataloader_val):

File "C:\Users\Anaconda3\lib\site-packages\tqdm\std.py", line 1129, in iter
for obj in iterable:

File "C:\Users\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 352, in iter
return self._get_iterator()

File "C:\Users\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 294, in _get_iterator
return _MultiProcessingDataLoaderIter(self)

File "C:\Users\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 801, in init
w.start()

File "C:\Users\Anaconda3\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)

File "C:\Users\Anaconda3\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)

File "C:\Users\Anaconda3\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)

File "C:\Users\Anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 89, in init
reduction.dump(process_obj, to_child)

File "C:\Users\Anaconda3\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)

BrokenPipeError: [Errno 32] Broken pipe

Hi JoHof,
I try your mode on COVID data as you mentioned in readme, the performance is really amazing.

And i read you code mask.py, and don't understand the threshold: 600, 1024, 1624. Could you give some explanation.

Thanks in advance.

tvolslices, xnew_box = utils.preprocess(inimg_raw, resolution=[256, 256])
tvolslices[tvolslices > 600] = 600
tvolslices = np.divide((tvolslices + 1024), 1624)

Hi,
could you please publish the package on Pypi?

Hi JoHof! I was just googling around to see how people were using my libraries and I saw that the postrocessing function could be made a lot faster. I saw many people have used this work as a template (congrats!), so hopefully many different projects can benefit from a speedup. It's possible this is not a bottleneck in your code, in which case please ignore this issue, as I don't want to waste your time.

https://github.com/JoHof/lungmask/blob/master/lungmask/utils.py#L194-L250

Here's some example improvements using the fastremap, and cc3d libraries. I am not certain I understood every aspect of your algorithm and it hasn't been tested, so view this as just a guide.

I'm a little confused by why you need to extract the largest component after fusing neighboring components by size, but I'm probably missing something. However, if you can skip that step, this code will be blazingly fast. I added a renumbering step at the end so the labels that come out will be numbered from 1.

def postrocessing(label_image, spare=[]):
    '''some post-processing mapping small label patches to the neighbout whith which they share the
        largest border. All connected components smaller than min_area will be removed
    '''

    # merge small components to neighbours
    regionmask, N = cc3d.connected_components(label_image, return_N=True)
    stats = cc3d.statistics(regionmask)
    volumes = stats['voxel_counts']
    edges = cc3d.region_graph(regionmask, connectivity=connectivity) 
    
    remap = {}
    for label in range(N+1):
        neighbours = set([ label ])
        for edge in edges:
            if label not in edge:
                continue
            neighbours.add(edge[0])
            neighbours.add(edge[1])

        neighbours = list(neighbours)
        biggest_label = neighbors[ 
            np.argmax([ volumes[lbl] for lbl in neighbours ])
        ]
        remap[label] = biggest_label

    for val in spare:
        remap[val] = val

    fastremap.remap(regionmask, remap, in_place=True)

    if regionmask.shape[0] == 1:
        # holefiller = lambda x: ndimage.morphology.binary_fill_holes(x[0])[None, :, :] # This is bad for slices that show the liver
        holefiller = lambda x: skimage.morphology.area_closing(x[0].astype(int), area_threshold=64)[None, :, :] == 1
    else:
        holefiller = fill_voids.fill

    outmask = np.zeros(regionmask.shape, dtype=np.uint8)
    for label, mask in cc3d.each(regionmask):
        mask = cc3d.largest_k(mask, k=1)
        outmask[holefiller(mask)] = label

    fastremap.renumber(outmask, in_place=True)
    return outmask

I hope this is helpful. Thanks for reading!

As I'm deploying your model quite often in a subprocess, with a high-level tqdm for number of samples/CTs to process, it would be convenient if it was possible to silence the tqdm bar from running inference. Ideally as a flag.