This pipeline has been adapted from https://github.com/gustaveroussy/EaCoN. It leverages the EaCoN R package to conduct preprocessing and normalization, segmentation and copy number estimation from specific microarray .CEL files. The EaCoN package supports copy number estimation from Cytoscan, Oncoscan, SNP6 arrays as well as WES data, however, the current pipeline has only implemented support for microarray data.

This pipeline leverages the snakemake Python package for workflow management. As a result the pipeline and its dependencies are easily installed from this repository, allowing quick setup, configuration and deployment.

For more information on Snakemake, please see: https://snakemake.readthedocs.io/en/stable/.

Dependency management for this pipeline is handled via conda for Python and renv for R. To get started with setup you can install miniconda3 using the instructions available here: https://docs.conda.io/en/latest/miniconda.html.

Alternatively you can install it directly from CRAN as described here: https://cran.r-project.org/.

The first step to deploying an analysis pipeline is to install the various software packages it depends on. We have included the env/eacon.yml and renv.lock files here to easily accomplish this.

All commands should be executed from the top level directory of this repository unless otherwise indicated.

The first step to deploying an analysis pipeline is to install Python, Snakemake and Singularity via conda. We have included the envs/eacon.yml which specifies all the requisite dependencies to use snakemake for this pipeline, including R.

You can use conda to install all Python and OS system dependencies using:

conda env create --file env/eacon.yml

This will take some time to run as it gathers and installs the correct package versions. The environment it creates should be called eacon.

If it is not automatically activated after installation please run conda activate eacon before proceeding to the next step.

R dependencies are handled via renv and all rules in this pipeline will use the local R package cache stored in the renv directory.

To install all R dependencies run:

Rscript -e 'library(renv); renv::init();'

If you wish to isolate the R dependencies from your Conda environment R libraries, you can use this command instead:

Rscript -e 'library(renv); renv::isolate(); renv::init(bare=TRUE)'

If intialization doesn't trigger dependency installation, you can do so manually using:

Rscript -e 'renv::restore()'

For more information on renv and how it can be used to manage dependencies in your project, please see: https://rstudio.github.io/renv/articles/renv.html.

This pipeline assumes the following directory structure:

.
├── env
├── metadata
├── procdata
├── rawdata
├── renv
├── results
└── scripts

Please at minimum create the rawdata and metadata directories, as they are assumed to hold the raw microarray plate data (.CEL) and the pairs file, respectively. For more information on the correct formatting for your pairs file, please see https://github.com/gustaveroussy/EaCoN. The remaining missing directories will be created automatically as the pipeline runs.

This file hold the relevant pipeline documentation. Here you can specify the paths to all the parameters for your current pipeline use case. Documentation is provided in the config.yaml file on what each field should contain.

To run the pipeline end-to-end:

snakemake --cores <n_cores> --use-conda

Where <n_cores> is the number of cores to parallelize over.

For details on deploying this pipleine via your local HPC cluster or in the cloud, please consult the Snakemake documentation. Deployment to these platforms requires minimal additional configuration.

The pipeline can also be run rule by rule using the rule names.

snakemake --cores 2 --use-conda batch_process_rawdata

snakemake --cores 2 --use-conda segment_processed_data

snakemake --cores 2 --use-conda estimate_copy_number

snakemake --cores 2 --use-conda select_optimal_gamma

snakemake --cores 2 --use-conda build_ragged_experiment

snakemake --cores 2 --use-conda sample_quality_control