cytomining / cytotools Goto Github PK

• Specify option to filter out images that fail QC
• Specify which image features to aggregate

The argument univariate is missing in the aggregate function.

Most scripts in https://github.com/broadinstitute/cytominer_scripts assume a specific folder structure, which is great for keeping the options compact (only need to specify batchname and plate_id for most cases). But this makes it inflexible. Keep the current options, but also have the option to explicitly specify paths.

See the http://docopt.org docs to make sure we do it the right way.

These are the scripts that need to be updated:

• select.R
• sample.R: output and workspace_dir can be specified, but expects a predefined folder structure for this to work. It is not worth the effort to make this more flexible – the code gets far too complex.
• preselect.R
• normalize.R: all paths can be specified explicitly
• compare_plates.R
• collapse.R
• audit.R
• annotate.R: same as sample.R

#8 appended an aggregation tag. But this should be handled in cytominer, not here.

Convert the collapse script to an R function.

Update these files to be consistent with new behavior of cytominer::aggregate

https://github.com/cytomining/cytotools/blob/master/inst/extdata/backend/batch0/SQ00015116/SQ00015116_augmented.csv
https://github.com/cytomining/cytotools/blob/master/inst/extdata/backend/batch0/SQ00015116/SQ00015116.csv
https://github.com/cytomining/cytotools/blob/master/tests/testthat/test-aggregate.R

See cytomining/cytominer#121

Convert the select script to an R function.

cytominer-database drops the Image_ prefix from image table. Let's update cytotools such that the Image_ prefix is assumed to not be present:

• Remove

  cytotools/R/annotate.R

  Lines 53 to 55 in 16fb2b7

  	profiles %<>%
  	setNames(names(profiles) %>%
  	stringr::str_replace_all("^Image_Metadata", "Metadata"))

• Update

  cytotools/R/aggregate.R

  Line 18 in 16fb2b7

  strata = c("Image_Metadata_Plate", "Image_Metadata_Well"),

• Update test fixtures

/cc @shntnu @cells2numbers

Convert the annotate script to an R function.

This is intended to be used right after the SQLite file is produced. It strips Image_ from column names with prefix Image_Metadata_ in the image table.

I'm not thrilled about the fact that this is a hack, but it helps remove code downstream where this is done repeatedly

Here's a stub

db <- DBI::dbConnect(RSQLite::SQLite(), sqlite_file)

image <- dplyr::tbl(src = db, "image") %>%

columns_names <- colnames(image) %>% stringr::str_sub("^Image_Metadata_")

purrr::walk(columns_names, function(column_name) DBI::dbSendQuery(db, sprintf("'UPDATE image SET %s = %s;'", column_name, stringr::str_replace(column_name, "^Image_", ""))))

DBI::dbDisconnect(db)

It would be great to have one test data set that can be used to test all functions implemented in cytotools.

Some parts should be available as csv and sql.

• level 2 - sql . single cell data (four wells, two control wells, two sites per well)
• level 3 - csv. aggregated version (aggregated to wells or collapsing the wells, results has four rows )
• level 4 - csv. normalized and aggregated data, same format as above (four rows).
• level 4 - sql. normalized single cell data.
• level 4 - csv. sample returns a subset of the aggregated data. possible test: sample returns one of the replicates (= 1 well) and we want to check if this is equal to one of the aggregated wells (one is chosen randomly). No new data needed.
• level 4 - csv. a feature list is needed to test feature selection (preselect).
• level 4 - csv. select returns a csv file with the same number of rows as the aggregated data but with fewer columns.
• level 5 - csv. collapsed on replicate level (from level 4 to level 5). normalized data with 2 rows.
• profiling stuff (not part of cytotools)

Level 1: Raw unprocessed images from microscopes
Level 2: Per-cell level measurements
Level 3: Per-well level aggregated measurements
Level 4: Morphological profiles computed using z-scores relative to the plate population as control or relative to the plate vehicle control.
Level 5: Replicate-collapsed perturbation signatures

• add option to add prefix to feature name to indicate aggregation approach
• specify compartment

Convert the sample script to an R function.

The cytominerScript function normalize.R (https://github.com/broadinstitute/cytominer_scripts/blob/master/normalize.R) is missing and needs to be implemented in cytotools.

Convert the preselect script to an R function.