cytomining / deepprofiler-handbook Goto Github PK

Merge chapters 5 and 6 into a single chapter "Downstream Analysis".
Add information about the notebooks for aggregation and evaluation (with links).

We need to put the Profiling section before the Training section. This is going to be the trend for documentation in general.

We also need to reflect this in the actual config.json files. @Arkkienkeli can you put the Profiling section third, and the Training section fourth? Also, can you change the order in the example configuration file, and in all of our other configuration files that we plan to release?

Hi,
Thanks for your great tool for cell painting, it helped me a lot. Now, I have a problem with combinedset_cellsout_e30.hdf5 file, could you provide this file for data processing(as you mentioned in your paper work)?

Best

• Update pyenv instructions
• Add conda example
• Remove the docker

This page is looking good! We need to expand its content in the following way:

• Show an example image (from the example data) with the 5 channels separated and named. The dimensions should also be illustrated. Create this image in the Google Slides and export it as a PNG. @yhan8 Can you create this example figure?
• Add an image of how the outlines look like, and more instructions on how to obtain them. This is part of the CellProfiler segmentation pipeline, we need to clarify these instructions. I believe @rsenft1 knows more details about this?

Add links to example configuration files

We need to review pycytominer section if it still valid and complete enough to be useful.

It'd be nice to give examples of the outputs we get when training a model on the example data. Here is a couple of items we need:

• A screenshot of the console output after a few epochs.
• Plots of the data stored in the log file (training vs validation loss, accuracy, etc).

Adding that and describing / commenting on those outputs will be useful for people to understand what is going on.

Code to single-cell loading (training):

• Link the function of DeepProfiler.
• Point to the PyTorch code (single-cells repository we had for other experiments).

The metadata columns are explained in this chapter, and each column may have examples coming from the example data.

@Arkkienkeli Is it correct that the Treatment and Replicate are named pert_name and pert_name_replicate respectively?

If that is the case, please make a comment here and close the issue.

Thank you!

Hello @rsenft1 I have tried installing DeepProfiler (both the docker version and installing from source) by following these instructions. I also download the example data as specified in 2.2, and set up an example project folder. In both the docker instance as well as from source, I am unable to get a working version of DeepProfiler. In particular, when I run the command:

python3 deepprofiler --root=/home/wongd26/JUMP1DeepProfiler/ --config config.json train

I get the following error:

AttributeError: module 'plugins.models.resnet' has no attribute 'ModelClass'

Looking into plugins/models/resnet.py, I see that ModelClass is not explicitly defined as a class.

Is this a bug? Or am installing incorrectly? Thank you for any help you can provide!

• This chapter should come before training!
• The order of profiling options should be:
  • The Cell Painting CNN first
  • ImageNet pre-trained second
  • Your own trained model third

Also, is it true that we strongly recommend to prepare the dataset (compression) before profiling? I thought DeepProfiler was able to compute features for images without compression (which is desirable if the dataset is too large). Our model is robust to illumination changes, so this should not affect downstream performance. I recommend to remove this and instead clarify that DeepProfiler does not need compression to profile (I added a Note with this comment in Chapter 3).

Remove chapter and move any useful information to chapter 2

The first paragraph mentions Figure 3:

• Can we add a link to the figure?
• Are we sure the text refers to Figure 3 or Figure 2.
• This chapter shows Figure 2 and another figure without caption. Can we add a caption and assign a number to the second figure?

In the paragraph before Figure 2, there is a link to Section 3. This link is incorrect and should now point to Section 5.

• Separate training from profiling (different chapters).
• Expand the training chapter with single cell exports, example outputs (what is expected).
• The profiling chapter needs to expand the options (Cell Painting CNN, ImageNet pretrained and your own models).

Include general instructions, a pointer to the configuration file and a pointer to the dataset.

Update the intro with the Cell Painting CNN overview

• Update the example data section (2.2) @Arkkienkeli
• Bring metadata to a separate chapter and expand that chapter with locations (Metadata and single-cell locations) @rsenft1
• Bring the cell masks to a separate chapter and expand with images information (Images and cell masks) @yhan8
• Figure 2 needs to be updated (pretrained goes away and we need to bring outlines). @rsenft1

From slack thread:
Question:

How should the inputs/locations/ directory be organized? I couldn’t find any documentation on what the naming conventions should be for the CSV files (in order to link them with the right corresponding images), if CSVs should be separated by field or well, etc. Is there documentation somewhere? Thanks!

Solution:

locations path for an image is supposed to be locations/plate/well-site-Nuclei.csv

Make this more clear in documentation.