Description of the bug

The modules.config entry for Mesmer includes --image-mpp=0.215 in ext.args which is not appropriate. We do need a smart way to pass the pixel size to Mesmer (and other tools that need it), but hard coding it here is definitely wrong. Should this be perhaps extracted from the Ashlar output image and included in the metadata map or in its own separate channel? While we're at it we could extract other useful image metadata too. We could also extract metadata from the raw image data -- this would be particularly useful to cross-check against the samplesheet and markersheet content in the input validation step.

Description of feature

I realized that --illumination manual may be semi-redundant with including the ffp and/or dfp columns in the sample sheet. We should either add a check in PIPELINE_INITIALISATION that these two things are aligned, or rethink the interface to this feature (perhaps the presence of user-provided illumination profiles would imply illumination="manual" and the illumination param would only serve to invoke basicpy or another future tool).

Description of feature

The test profile should run our small tonsil test image data through as much of the pipeline as we have working.

Description of the bug

Allow any non-empty string for marker_name and filter in schema_marker

I'll add a test for this too

Command used and terminal output

No response

Relevant files

No response

System information

No response

Description of the bug

nf-validaton is not catching If a file given in the samplesheets does not exist

Command used and terminal output

No response

Relevant files

No response

System information

No response

Description of the bug

Subdirectories should be named after the corresponding module name, with possible parallel outputs from different tools under different names or sub-sub-directories therein. E.g. instead of ashlar/ we should have registration/.

Command used and terminal output

No response

Relevant files

No response

System information

No response

Description of the bug

Running nfcore/mcmicro test locally on a macOS (m2) runs into an issue with Mesmer.
Probably due to Docker container incompatibility with amd64 processing.

Command used and terminal output

nextflow run main.nf --input ./assets/samplesheet-test.csv --outdir ./output/ -profile docker

N E X T F L O W  ~  version 23.10.1
Launching `main.nf` [nauseous_cantor] DSL2 - revision: 68b73cf768
WARN: Access to undefined parameter `monochromeLogs` -- Initialise it to a default value eg. `params.monochromeLogs = some_value`


------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/mcmicro v2.0.0
------------------------------------------------------
Core Nextflow options
  runName        : nauseous_cantor
  containerEngine: docker
  launchDir      : /Users/jnimoca/Jose_BI/MCMICRO/mcmicro
  workDir        : /Users/jnimoca/Jose_BI/MCMICRO/mcmicro/work
  projectDir     : /Users/jnimoca/Jose_BI/MCMICRO/mcmicro
  userName       : jnimoca
  profile        : docker
  configFiles    : 

Input/output options
  input          : ./assets/samplesheet-test.csv
  outdir         : ./output/

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use nf-core/mcmicro for your analysis please cite:

* The pipeline
  

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/nf-core/mcmicro/blob/master/CITATIONS.md
------------------------------------------------------
executor >  local (2)
executor >  local (2)
executor >  local (2)
[56/3e9311] process > NFCORE_MCMICRO:MCMICRO:ASHLAR (TEST1)          [100%] 1 of 1 ✔
[23/f99a61] process > NFCORE_MCMICRO:MCMICRO:DEEPCELL_MESMER (TEST1) [100%] 1 of 1, failed: 1 ✘
[-        ] process > NFCORE_MCMICRO:MCMICRO:MCQUANT                 -
[-        ] process > NFCORE_MCMICRO:MCMICRO:MULTIQC                 -
Execution cancelled -- Finishing pending tasks before exit
-[nf-core/mcmicro] Pipeline completed with errors-
ERROR ~ Error executing process > 'NFCORE_MCMICRO:MCMICRO:DEEPCELL_MESMER (TEST1)'

Caused by:
  Process `NFCORE_MCMICRO:MCMICRO:DEEPCELL_MESMER (TEST1)` terminated with an error exit status (125)

Command executed:

  python /usr/src/app/run_app.py mesmer         --squeeze         --nuclear-image TEST1.ome.tif         --output-directory .         --output-name mask.tif                  --image-mpp=0.215 --nuclear-channel 0 --compartment nuclear
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_MCMICRO:MCMICRO:DEEPCELL_MESMER":
      deepcell_mesmer:: 0.4.0
  END_VERSIONS

Command exit status:
  125

Command output:
  (empty)

Command error:
  Unable to find image 'vanvalenlab/deepcell-applications:0.4.1' locally
  0.4.1: Pulling from vanvalenlab/deepcell-applications
  docker: no matching manifest for linux/arm64/v8 in the manifest list entries.
  See 'docker run --help'.

Work dir:
  /Users/jnimoca/Jose_BI/MCMICRO/mcmicro/work/23/f99a614e8a4663dda3090b8e713ba2

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

 -- Check '.nextflow.log' file for details

Relevant files

nextflow.log

System information

Nextflow version 23.10.1
Hardware: MacBook Pro Apple M2 Pro 16Gb Memory
Executor: Local
Container Engine: Docker
OS: Sonoma 14.0 (23A344)
nfcore/mcmicro version: https://github.com/jmuhlich/mcmicro/tree/sample_sheet_mod

Description of feature

create a local module to stage the macsima .tif files for usage with ashlar. Macsima produces .tif files per channel, per tile, per exposure time, etc. The staging module should stack all this .tif files into an ome.tif per cycle as ashlar requires.

Description of the bug

Specifying a param value like --input_cycle as a bare filename does not work -- nextflow doesn't seem to "see" the file.
Prefixing the filename with its full absolute path works around this. This is probably something about nf-validation handling of the params.

Command used and terminal output

# samplesheet.csv DOES exist here!
$ nextflow run nf-core/mcmicro --input_cycle samplesheet.csv ...

...
ERROR ~ Error executing process > 'NFCORE_MCMICRO:MCMICRO:INPUT_CHECK:SAMPLE_AND_MARKER_SHEET_CHECK'

Caused by:
  Not a valid path value: 'samplesheet.csv'
...

Relevant files

No response

System information

No response

Description of feature

Hey all,

I will create a local module to aggregate all single csv files from mcquant, to a single csv file.
An extra column will be created with "sample_id" as the labelling column.

Why? In the hackathon we want to create some simple modules that produce simple spatial analysis outputs.
Phenotyping with gates, and spatialLDA outputs.
@migueLib @chiarasch <-- working on SpatialLDA

I am aware that there is a step somewhere in vanilla mcmicro that does this, but we wanted a single module as a intermediate step.

Description of feature
Adding a phenotyping module for automated phenotyping using MAPS. In the first version, this module takes a user-pretrained model and the quantification table as input. The output will be the modified quantification table with an additional column for cell phenotypes.

An associated issue already exists in nf-core/modules to create it not only as a local module

Description of feature

We should decide whether markers.csv should have a per-cycle channel number column in addition to the running global channel number.

Description of feature

Currently MCQuant has the option to pass additional skimage regionprops using the CLI, which is very nice. We would like to extend that by including specific glcm features in the MCQuant script.
Further, we would like McQuant to accept all available bioformats (e.g. also qptiff)
Other adjustments might have to be implemented if SpatialData is included (again depending on where - from start, after ASHLAR or after quant?)

Description of the bug

basicpy output file paths/names are based only on the input filenames. If a run has multiple samples with the same filenames then their basicpy outputs will clash and overwrite each other.

Command used and terminal output

No response

Relevant files

No response

System information

No response

Description of the bug

I am running Exemplar001.
I created a samplesheet (1 row per cycle)
Running defaults

It gets stuck at ASHLAR creating a too large image.. is it just an ashlar bug? or something else going wrong?
Will try to change some parameters and report back

Command used and terminal output

nextflow run nf-core/mcmicro -r dev 
--outdir /Users/jnimoca/Jose_BI/P12E19_nfcore_mcmicro/output/ 
--marker_sheet 
/Users/jnimoca/Jose_BI/P12E19_nfcore_mcmicro/markers.csv 
-profile 
docker 
--input_cycle 
/Users/jnimoca/Jose_BI/P12E19_nfcore_mcmicro/samplesheet.csv

Output:

N E X T F L O W  ~  version 23.10.1
Launching `https://github.com/nf-core/mcmicro` [silly_newton] DSL2 - revision: b8d46de48e [dev]
WARN: Access to undefined parameter `monochromeLogs` -- Initialise it to a default value eg. `params.monochromeLogs = some_value`


------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/mcmicro v2.0.0-gb8d46de
------------------------------------------------------
Core Nextflow options
  revision       : dev
  runName        : silly_newton
  containerEngine: docker
  launchDir      : /Users/jnimoca/Jose_BI/P12E19_nfcore_mcmicro
  workDir        : /Users/jnimoca/Jose_BI/P12E19_nfcore_mcmicro/work
  projectDir     : /Users/jnimoca/.nextflow/assets/nf-core/mcmicro
  userName       : jnimoca
  profile        : docker
  configFiles    :

Input/output options
  input_cycle    : /Users/jnimoca/Jose_BI/P12E19_nfcore_mcmicro/samplesheet.csv
  outdir         : /Users/jnimoca/Jose_BI/P12E19_nfcore_mcmicro/output/
  marker_sheet   : /Users/jnimoca/Jose_BI/P12E19_nfcore_mcmicro/markers.csv

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use nf-core/mcmicro for your analysis please cite:

* The pipeline


* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/nf-core/mcmicro/blob/master/CITATIONS.md
------------------------------------------------------
WARN: Samplesheet warnings:
	The samplesheet contains following unchecked field(s): [Filter]
executor >  local (3)
[ca/23253c] process > NFCORE_MCMICRO:MCMICRO:INPUT_CHECK:SAMPLE_AND_MARKER_SHEET_CHECK (markers.csv) [100%] 1 of 1 ✔
[a3/b43fba] process > NFCORE_MCMICRO:MCMICRO:INPUT_CHECK:MARKER_SHEET_CHECK (markers.csv)            [100%] 1 of 1 ✔
executor >  local (3)
[ca/23253c] process > NFCORE_MCMICRO:MCMICRO:INPUT_CHECK:SAMPLE_AND_MARKER_SHEET_CHECK (markers.csv) [100%] 1 of 1 ✔
[a3/b43fba] process > NFCORE_MCMICRO:MCMICRO:INPUT_CHECK:MARKER_SHEET_CHECK (markers.csv)            [100%] 1 of 1 ✔
[62/73bd10] process > NFCORE_MCMICRO:MCMICRO:ASHLAR (Exemplar001)                                    [  0%] 0 of 1
[-        ] process > NFCORE_MCMICRO:MCMICRO:DEEPCELL_MESMER                                         -
[-        ] process > NFCORE_MCMICRO:MCMICRO:MCQUANT                                                 -
[-        ] process > NFCORE_MCMICRO:MCMICRO:MULTIQC                                                 -

Relevant files

ASHLAR output from work folder:

WorkFolder.zip

System information

Nextflow: version 23.10.1
M2 Macbook pro
executor: local
Container engine: Docker
OS: MacOS
nf-core/mcmicro v2.0.0-gb8d46de

Description of feature

Sarek has some nice logic to restart a previous run from a specific step in the pipeline. At each restart point in the pipeline, they emit a samplesheet into outdir that represents the state of the pipeline, potentially referencing other files in outdir. When a later run specifies the step param, the relevant samplesheet is automatically pulled out of outdir. This seems like a good way to implement the old mcmicro start-at feature while still retaining the separation of inputs and outdir.

Description of feature

For a simple way to specify module parameters, for example change ashlar's max_shift to 20.. or change sigma?
I believe they should be added to nextflow.config.

sample sheet format:
sample, cycle_number, channel_count, image_tiles

the marker sheet (markesrs.csv) will be input via a pipeline parameter

To Do:

• change input_check.nf:create_fastq_channel():meta.id to sample_name + cycle
• modify sample sheet schema for the new format (validation handled by nf-validation)
• image_tiles file(s) currently hard-coded in workflow. Need to use the file(s) in the sample sheet.
  • working with one image, but will require change for multiple images
• add marker sheet pipeline parameter
• in validation function check:
  • simple validation that required entries exist and that files have required suffix and that integers are actually integers
  • that channel_count in sample sheet matches the number of channels in the marker sheet
  • uniqueness of marker name in marker sheet
  • uniqueness of (cycle, channel) tuple in marker sheet
• add test(s)

Add Palom as a registration option for users that have prestitched whole-slide data with a disclaimer that for ensuring proper result quality, tiles with overlaps would be preferred.

TODO:

• Docker image
• CLI
• initially a local module
• parameter for registration option
• Documentation

Description of feature

Create a module to stage the PhenoImager .tif files for usage with Ashlar. PhenoImager produces float32 .tif files per tile per cycle. The staging module would transform float32 to uint16, extract metadata information and stack all .tif files into an ome.tif per cycle as Ashlar requires.

Description of the bug

When using input_sample, the image directory files are collected via Groovy's File.eachFileRecurse which does not yield the results in any sorted order. The results must be collected and then sorted explicitly by filename. In addition it's probably better not to recurse and require all files are provided at the top level, as user expectations will vary on how files in subdirectories should be sorted relative to each other. This is as simple as replacing eachFileRecurse with eachFile. Or eachFileMatch could be used to perform the .ome.tif filename match as well.

Command used and terminal output

No response

Relevant files

No response

System information

No response

Description of feature

Merge this into modules.config:

process {
    withName: BASICPY {
        ext.args = "--darkfield"
    }
}

Description of feature

move bin/check_marker_sheet.py and bin/check_sample_and_marker_sheet.py to groovy

Description of the bug

Even though the outdir parameter is defined as required in the schema file the workflow runs when it is missing.

Command used and terminal output

'''
nextflow run ~/github/mcmicro-nf-core/main.nf --input_cycle ~/github/mcmicro-nf-core/assets/samplesheet_1_row_sample_cycle_multi_2.csv --marker_sheet ~/github/mcmicro-nf-core/assets/markers_multi_8.csv -profile test_work -params-file ~/github/mcmicro-nf-core/conf/workflow.json
'''
It runs to completion, but output is stored in a directory named 'null'.

Relevant files

No response

System information

No response