cytomining / deepprofiler Goto Github PK

They are too slow and limited by Python multi-threading. It would be great if we use instead a Keras data generator or something similar.

The script 05LearnCNN.py can be factorized and reorganized in the package learning. A configuration file should be expected with information of the compressed data and indices.

Divide pixel intensity values by the robust max of the plate. Currently, the robust max is used only to trim tails, it should be used for normalization.

Profiling is using an old definition of the input graph. Needs to be refactorized and fixed.

The input metadata for the compression routine takes a list of TIFF files to generate the PNG outputs. However, the locations of PNGs are different to the directory structure of TIFFs, and the extensions are also different. Right after compression, the command should create a copy of the metadata file, with the same fields, but with updated paths to the new files.

https://github.com/jccaicedo/DeepProfiler/blob/be34255534c5bd2583a20a0dfbaaeeff5798aac8/dataset/pixels.py#L15

@massachusett added the .pytest_cache/v/cache directory to the master. Please remove it.

Right now this is a function that returns a dictionary with tensorflow placeholders. In order to extend the input graph with different targets (labels, tasks, etc.), this function can be transformed into a class that handles events.

@massachusett please add your name and github handle at the end of the AUTHORS.md file

The label structure is assumed to be a single column in the metadata. Different tasks may require more than one column to define targets.

Target image size is used for compression, and should also be used for creating cell location files. Currently this is ignored, and the parameter should be shared between scripts. e.g. luad/03 and luad/04

Currently, the scripts for metadata preparation are dataset specific. There are some invariants when the input comes from Cytominer. We can parameterize the common variables and refactor the script, assuming that labels come from treatments and data splits come from replicates. In addition, the script can receive data splits and labels separately from other inputs.

Dataset now needs a keyGen function in the constructor. Some scripts may be broken because of this.

This is the error:

target = <deepprofiler.dataset.target.MetadataColumnTarget object at 0x7fa5f7114ef0>, values = [5, 81, 83, 17, 71, 12, ...]

    def test_init(target, values):
        field_name = 'test'
        shuffle(values)
        assert target.field_name == field_name
>       assert len(target.index) == len(values)
E       assert 9 == 10
E        +  where 9 = len({5: 1, 8: 2, 12: 3, 17: 4, ...})
E        +    where {5: 1, 8: 2, 12: 3, 17: 4, ...} = <deepprofiler.dataset.target.MetadataColumnTarget object at 0x7fa5f7114ef0>.index
E        +  and   10 = len([5, 81, 83, 17, 71, 12, ...])

tests/dataset/test_target.py:24: AssertionError

This distribution should be used to normalize pixel values for learning (not for compression). The distribution can be computed along with illumination correction and other pixel statistics.

Validation routines work with the previous single prediction model as well as the old input graph definition. Some factorization will be needed to make it work again. Validation is currently broken.

https://github.com/travis-ci/travis-ci