Code Monkey home page Code Monkey logo

aics-ml-segmentation's Introduction

NOTE: This repository has a new home. New development and release will be available via https://github.com/AllenCell/aics-ml-segmentation

Overview

The Allen Cell Structure Segmenter is a Python-based open source toolkit developed for 3D segmentation of intracellular structures in fluorescence microscope images, developed at the Allen Institute for Cell Science. This toolkit consists of two complementary elements, a classic image segmentation workflow with a restricted set of algorithms and parameters and an iterative deep learning segmentation workflow. We created a collection of 20 classic image segmentation workflows based on 20 distinct and representative intracellular structure localization patterns as a lookup table reference and starting point for users. The iterative deep learning workflow can take over when the classic segmentation workflow is insufficient. Two straightforward human-in-the-loop curation strategies convert a set of classic image segmentation workflow results into a set of 3D ground truth images for iterative model training without the need for manual painting in 3D. The Allen Cell Structure Segmenter thus leverages state of the art computer vision algorithms in an accessible way to facilitate their application by the experimental biology researcher. More details including algorithms, validations, examples, and video tutorials can be found at allencell.org/segmenter or in our bioRxiv paper.

Note: This repository has only the code for the "Iterative Deep Learning Workflow". The classic part can be found at https://github.com/AllenInstitute/aics-segmentation

Installation:

  1. prerequisite:

To perform training/prediction of the deep learning models in this package, we assume an NVIDIA GPU has been set up properly on a Linux operating system, either on a local machine or on a remote computation cluster. Make sure to check if your GPU supports at least CUDA 8.0 (CUDA 9.0 and up is preferred): NVIDIA Driver check.

The GPUs we used to develop and test our package are two types: (1) GeForce GTX 1080 Ti GPU (about 11GB GPU memory), (2) Titan Xp GPU (about 12GB GPU memory), (3) Tesla V100 for PCIe (with about 33GB memory). These cover common chips for personal workstations and data centers.

Note 1: As remote GPU clusters could be set up differently from institute to institute, we will assume a local machine use case through out the installation and demos.

Note 2: We are investigating alternative cloud computing service to deploy our package and will have updates in the next few months. Stay tuned :)

  1. create a conda environment:
conda create --name mlsegmenter python=3.6

(For how to install conda, see here)

  1. activate your environment and do the installation within the environment:
conda activate mlsegmenter 

(Note: always check out conda documentation for updates. If you are using an older version of conda, you may need to activate the environment by source activate mlsegmenter.)

  1. Install Pytorch

Go to PyTorch website, and find the right installation command for you.

  • we use version 1.0 (which is the stable version at the time of our development)
  • we use Linux (OS), Conda (package), python 3.6 (Language), CUDA=10.0 (Question about CUDA? see setup CUDA).

Make sure you use either the automatically generated command on PyTorch website, or the command recommended on PyTorch website for installing older version

  1. Install Allen Cell Segmenter (deep learning part)
git clone https://github.com/AllenInstitute/aics-ml-segmentation.git
cd ./aics-ml-segmentation
pip install -e .

The -e flag when doing pip install will allow users to modify any the source code without the need of re-installing the package afterward. You may do the installation without -e, if you don't want any change on the code.

Level of Support

We are offering it to the community AS IS; we have used the toolkit within our organization. We are not able to provide guarantees of support. However, we welcome feedback and submission of issues. Users are encouraged to sign up on our Allen Cell Discussion Forum for community quesitons and comments.

aics-ml-segmentation's People

Contributors

aetherunbound avatar basuc avatar jxchen01 avatar ruiany avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aics-ml-segmentation's Issues

ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (4,2) and requested shape (5,2)

I was able to train with cpu locally (using a smaller size_in/out), but got the following errors when using GPU nodes to train. I'm using the same trainging set. What goes wrong here?

2019-09-16 07:39:28,593 [MainThread] INFO ModelTrainer - {'model': {'name': 'unet_xy'}, 'nchannel': 1, 'nclass': [2, 2, 2], 'size_in': [50, 156, 156], 'size_out': [22, 68, 68], 'checkpoint_dir': '/home/usr1/Images/laminin_staining/checkpoint/', 'resume': None, 'learning_rate': 1e-05, 'weight_decay': 0.005, 'epochs': 400, 'save_every_n_epoch': 50, 'loss': {'name': 'Aux', 'loss_weight': [1, 1, 1], 'ignore_index': None}, 'loader': {'name': 'default', 'datafolder': '/home/usr1/Images/laminin_staining/trainset/', 'batch_size': 8, 'PatchPerBuffer': 200, 'epoch_shuffle': 5, 'NumWorkers': 1}, 'validation': {'metric': 'default', 'leaveout': [0], 'OutputCh': [0, 1, 1, 1, 2, 1], 'validate_every_n_epoch': 25}, 'device': device(type='cuda', index=0)}
model initialization succeeds !
2019-09-16 07:39:34,965 [MainThread] INFO ModelTrainer - Number of learnable params 17729902
start a new training
2019-09-16 07:39:34,966 [MainThread] INFO ModelTrainer - Sending the model to 'cuda:0'
prepare the data ... ...
Traceback (most recent call last):
  File "/usr/local/bin/dl_train", line 11, in <module>
    load_entry_point('aicsmlsegment', 'console_scripts', 'dl_train')()
  File "/aics-ml-segmentation/aicsmlsegment/bin/train.py", line 41, in main
    trainer.train()
  File "/aics-ml-segmentation/aicsmlsegment/training_utils.py", line 231, in train
    train_set_loader = DataLoader(train_loader(train_filenames, loader_config['PatchPerBuffer'], config['size_in'], config['size_out']), num_workers=loader_config['NumWorkers'], batch_size=loader_config['batch_size'], shuffle=True)
  File "/aics-ml-segmentation/aicsmlsegment/DataLoader3D/Universal_Loader.py", line 83, in __init__
    img_pad0 = np.pad(input_img, ((0,0),(0,0),(padding[1],padding[1]),(padding[2],padding[2])), 'constant')
  File "<__array_function__ internals>", line 6, in pad
  File "/usr/local/lib/python3.7/dist-packages/numpy/lib/arraypad.py", line 741, in pad
    pad_width = _as_pairs(pad_width, array.ndim, as_index=True)
  File "/usr/local/lib/python3.7/dist-packages/numpy/lib/arraypad.py", line 516, in _as_pairs
    return np.broadcast_to(x, (ndim, 2)).tolist()
  File "<__array_function__ internals>", line 6, in broadcast_to
  File "/usr/local/lib/python3.7/dist-packages/numpy/lib/stride_tricks.py", line 182, in broadcast_to
    return _broadcast_to(array, shape, subok=subok, readonly=True)
  File "/usr/local/lib/python3.7/dist-packages/numpy/lib/stride_tricks.py", line 127, in _broadcast_to
    op_flags=['readonly'], itershape=shape, order='C')
ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (4,2) and requested shape (5,2)

Unable to train on CPU

Added a device item in the config file to train on cpu, and it looks ok in the initialization message, but an error occur when start training. I was able to train on the GPU with the same config (except device).
Any idea how to make it work on CPU? I got a very weak GPU but powerful CPUs.

2019-09-06 15:55:04,481 [MainThread] INFO ModelTrainer - Number of learnable params 17729902
Loading checkpoint '/mnt/home/jingliulj/Images/laminin_staining/checkpoint/checkpoint_epoch_0.pytorch'...
2019-09-06 15:55:06,394 [MainThread] INFO ModelTrainer - Sending the model to 'cpu'
prepare the data ... ...
/mnt/home/jingliulj/Images/aics-ml-segmentation/aicsmlsegment/utils.py:24: YAMLLoadWarning: calling yaml.load() without Loader=... is depre$
  return yaml.load(open(config_file, 'r'))
^M0it [00:00, ?it/s]^M0it [00:07, ?it/s]
Traceback (most recent call last):
  File "/mnt/home/jingliulj/miniconda3/envs/img/bin/dl_train", line 11, in <module>
    load_entry_point('aicsmlsegment', 'console_scripts', 'dl_train')()
  File "/mnt/home/jingliulj/Images/aics-ml-segmentation/aicsmlsegment/bin/train.py", line 41, in main
    trainer.train()
  File "/mnt/home/jingliulj/Images/aics-ml-segmentation/aicsmlsegment/training_utils.py", line 264, in train
    outputs = model(inputs)
  File "/mnt/home/jingliulj/miniconda3/envs/img/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/mnt/home/jingliulj/Images/aics-ml-segmentation/aicsmlsegment/Net3D/unet_xy.py", line 90, in forward
    down1 = self.ec1(x)
  File "/mnt/home/jingliulj/miniconda3/envs/img/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/mnt/home/jingliulj/miniconda3/envs/img/lib/python3.7/site-packages/torch/nn/modules/container.py", line 92, in forward
    input = module(input)
  File "/mnt/home/jingliulj/miniconda3/envs/img/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/mnt/home/jingliulj/miniconda3/envs/img/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 478, in forward
    self.padding, self.dilation, self.groups)
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same

Error "name 'quit_curation' is not defined" using curator_sorting

`[2019-09-03 18:12:05,765 - root - 206][DEBUG] --------------------------------------------------------------------------------
[2019-09-03 18:12:05,765 - root - 241][DEBUG] Working Dir:
[2019-09-03 18:12:05,765 - root - 242][DEBUG] /home/small/Images
[2019-09-03 18:12:05,765 - root - 243][DEBUG] Command Line:
[2019-09-03 18:12:05,765 - root - 244][DEBUG] /home/small/miniconda3/envs/img/bin/curator_sorting --d --raw_path /home/small/Images/laminin_staining/data/ --data_type .tiff --input_ch 3 --seg_path /home/small/Images/laminin_staining/seg_res/ --mask_path /home/small/Images/laminin_staining/mask/ --csv_name /home/small/Images/laminin_staining/curator_sorting_tracker.csv --train_path /home/small/Images/laminin_staining/trainset/ --Normalization 1
[2019-09-03 18:12:05,765 - root - 245][DEBUG] Args:
[2019-09-03 18:12:05,765 - root - 247][DEBUG] debug: True
[2019-09-03 18:12:05,765 - root - 247][DEBUG] output_dir: ./
[2019-09-03 18:12:05,765 - root - 247][DEBUG] struct_ch: 0
[2019-09-03 18:12:05,765 - root - 247][DEBUG] xy: 0.108
[2019-09-03 18:12:05,766 - root - 247][DEBUG] raw_path: /home/small/Images/laminin_staining/data/
[2019-09-03 18:12:05,766 - root - 247][DEBUG] data_type: .tiff
[2019-09-03 18:12:05,766 - root - 247][DEBUG] input_channel: 3
[2019-09-03 18:12:05,766 - root - 247][DEBUG] seg_path: /home/small/Images/laminin_staining/seg_res/
[2019-09-03 18:12:05,766 - root - 247][DEBUG] train_path: /home/small/Images/laminin_staining/trainset/
[2019-09-03 18:12:05,766 - root - 247][DEBUG] mask_path: /home/small/Images/laminin_staining/mask/
[2019-09-03 18:12:05,766 - root - 247][DEBUG] csv_name: /home/small/Images/laminin_staining/curator_sorting_tracker.csv
[2019-09-03 18:12:05,766 - root - 247][DEBUG] Normalization: 1
[2019-09-03 18:12:05,766 - root - 208][DEBUG] --------------------------------------------------------------------------------
the csv file for saving sorting results exists, sorting will be resumed
[2019-09-03 18:12:16,782 - root - 382][ERROR] =============================================
[2019-09-03 18:12:16,783 - root - 384][ERROR]

Traceback (most recent call last):
File "/home/small/Images/aics-ml-segmentation/aicsmlsegment/bin/curator/curator_sorting.py", line 379, in main
exe.execute(args)
File "/home/small/Images/aics-ml-segmentation/aicsmlsegment/bin/curator/curator_sorting.py", line 297, in execute
score = gt_sorting(raw_img, seg)
File "/home/small/Images/aics-ml-segmentation/aicsmlsegment/bin/curator/curator_sorting.py", line 132, in gt_sorting
cid2 = fig.canvas.mpl_connect('key_press_event', quit_curation)
NameError: name 'quit_curation' is not defined

[2019-09-03 18:12:16,783 - root - 385][ERROR] =============================================
[2019-09-03 18:12:16,783 - root - 386][ERROR]

name 'quit_curation' is not defined

[2019-09-03 18:12:16,783 - root - 387][ERROR] =============================================
`
It looks like an event callback is missing from your code.

Training data folder contents

Hi!

I am trying to use this repo and couldn't find enough documentation about how the datafolder, a key in the config file should be organized.

  1. What images should it have and how should they be organized, can you please point me to some documentation regarding that
  2. After going around the repo, I have found that the ground truth masks are saved in the same folder as the images as GT_ome.tiff files, and there is a cost mapper image that should be saved. Can you please explain to me the contents of the cost mapper file and its data type. Is it the probability map of the classes in the mask?
  3. Also, can you please explain how the sizeIn, sizeOut, and outputCh were set in your config file ? I have looked at this documentation https://github.com/AllenInstitute/aics-ml-segmentation/blob/master/docs/doc_train_yaml.md but I couldn't figure out what p and kp are in the equation

Thanks a lot!

No validation config for train.yaml

It's better to provide a sample config file for both the training and prediction. Otherwise, users have to look into the source code to figure out what is missing from the configuration.
Here is what I added in the train.yaml to make it work:

validation:
  leaveout: [1]
  metric: 'MeanIoU'
  OutputCh: [1]
  validate_every_n_epoch: 40

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.