combiz / scflow Goto Github PK

What cell types? What's percentage mito in different cell types? How many reads in each celltype?

Something similar to the individual file reports, but merged across the study

There does need to be a subpipeline for just "QC / Clustering / Annotation steps"

Amy sent me the files through sharepoint. I need to upload to project space.

1. Create a doc with side by side figures showing cells from individuals + batches
2. Use kBet metric for calculating how well mixed samples are

Any cutoffs, i.e. percentage mitochondrial should be set to 'optimal' values by algorithms. What other values might be worth setting this way?

It looks really bad that we don't know where the files are. We need to get the data organised.

The sample annotation file is here: https://docs.google.com/spreadsheets/d/11CC8q84hBrk1A3_SsbVkBz_TsPB6WF3H5V1LuYUPaWk/edit#gid=0

We need to add the IGF ID and a file path on the server for the FASTQ file and Cell*Gene matrix (with introns as exons) for each file.

@MahdiMoradiMarjaneh can you get the documentation written up so that it explains how to install the R package from scratch? Easiest to do it first on a computer you've got full admin rights for.

Originally posted by @NathanSkene in #13 (comment)

Reports to include dynamic bibliography based on used packages.

Combiz is doing this currently

Should be easier to find the major values, i.e. average read depth. Perhaps a chunk of text at top of the document?

1. Check with this command dpkg -l | grep gdal | cut -c-72. If you have libgdal-dev then go this link
2. Download the latest rgdal_1.4-7.tar.gz.
   3.Install using R CMD INSTALL rgdal_1.4-7.tar.gz.

solution was taken from this link

After the main differential expression analysis is done, the values of the main value tested (i.e. Braak stage) should be permuted, and the DE repeated. Are a similar number of genes found to be differentially expressed with the permuted labels?

Tsai requires an MTA

We could just use one of the Karolinska mouse datasets and randomly assign cells to 'individuals'

As in "./nextflow run hello"

Should a pipeline for running just the clustering be maintained as seperate 'pipeline' from the differential expression one? @combiz says NFcore does not like 'subpipelines'. How best to deal with cases where there are genuine use cases for subfunctions?

Presumably many people would try doing this the first time they try using it.

Mahdi reported: "A few packages need to be installed separately. Still struggling with “units”. It seems the only way is to first installing libudunits2-dev which needs a sudo access. Any clue? Or I will check with the RCS team."

If it cannot be done, the requirement should be clearly stated alongside installation instructions

Without this implemented users have to rely 100% on the automated cell name assignments

Another monocle related error during scFlow installation that should be mentioned in installation documentation

The first gives this error:

configure: error: in `/tmp/RtmpORsFJ4/R.INSTALL3175f370031/units':
configure: error: 
--------------------------------------------------------------------------------
  Configuration failed because libudunits2.so was not found. Try installing:
    * deb: libudunits2-dev (Debian, Ubuntu, ...)
    * rpm: udunits2-devel (Fedora, EPEL, ...)
    * brew: udunits (OSX)
  If udunits2 is already installed in a non-standard location, use:
    --configure-args='--with-udunits2-lib=/usr/local/lib'
  if the library was not found, and/or:
    --configure-args='--with-udunits2-include=/usr/include/udunits2'
  if the header was not found, replacing paths with appropriate values.
  You can alternatively set UDUNITS2_INCLUDE and UDUNITS2_LIBS manually.
--------------------------------------------------------------------------------

See `config.log' for more details
ERROR: configuration failed for package ‘units’
* removing ‘/home/nskene/Projects/scFlowExamples/renv/library/R-3.6/x86_64-pc-linux-gnu/units’
Error: Failed to install 'scflow' from GitHub:
  Failed to install 'monocle3' from GitHub:
  (converted from warning) installation of package ‘units’ had non-zero exit status

I've made a (wiki page)[https://github.com/combiz/scflow/wiki/%23Error:-libudunits2.so-was-not-found] about that error.

The second error is:

* installing *source* package ‘sf’ ...
** package ‘sf’ successfully unpacked and MD5 sums checked
** using staged installation
configure: CC: gcc -std=gnu99
configure: CXX: g++ -std=gnu++11
checking for gdal-config... no
no
configure: error: gdal-config not found or not executable.
ERROR: configuration failed for package ‘sf’
* removing ‘/home/nskene/Projects/scFlowExamples/renv/library/R-3.6/x86_64-pc-linux-gnu/sf’
Error: Failed to install 'scflow' from GitHub:
  Failed to install 'monocle3' from GitHub:
  (converted from warning) installation of package ‘sf’ had non-zero exit status

Got the following error while installing:

Installing 26 packages: batchelor, DelayedMatrixStats, grr, Matrix.utils, pbmcapply, pheatmap, proxy, pscl, rsample, RhpcBLASctl, slam, spdep, speedglm, viridis, furrr, sp, spData, sf, deldir, LearnBayes, coda, expm, gmodels, classInt, units, e1071
Installing packages into ‘/home/nskene/Projects/scFlowExamples/renv/library/R-3.6/x86_64-pc-linux-gnu’
(as ‘lib’ is unspecified)
Error: Failed to install 'scflow' from GitHub:
Failed to install 'monocle3' from GitHub:
(converted from warning) packages ‘batchelor’, ‘DelayedMatrixStats’ are not available (for R version 3.6.1)

Would this be the best way to structure it? Combiz suggested it might be better to have two repositories for the environment? One without scFlow installed and one with it installed?

On what data would this be run?

Sample annotation sheet is here: https://docs.google.com/spreadsheets/d/11CC8q84hBrk1A3_SsbVkBz_TsPB6WF3H5V1LuYUPaWk/edit#gid=0

It needs two column's added to store locations in the project folder of the FASTQ and CellRanger files

This report would be produced post-merge, post-reddim/cluster/celltype annotation and would report:

• read depth per cell type
• total cells per cell type
• mitochondrial percentage per cell type
• distribution across individuals per cell type
• ...

Possibly integrate as an option for the 2d plot_feature_umap function.

We need to have a dataset for a tutorial. The Tsai dataset cannot be shared. So which published case/control datasets can be shared?

Code for nbTSNE is in Ken Harris's github repo

This would show the before/after LIGER alignment plots (or before/after regression of vars)

In this file https://github.com/combiz/scflow/wiki/Generating-a-Manifest-File-and-SampleSheet-for-scflow it is not stated what the input files are required to be. Should be clarified that the manifest file should point to folders containing matrix.mtx.gz / features.tsv.gz / barcodes.tsv.gz

Ideally the pipeline should be able to handle data from any species. Currently it doesn't recognise any genes that have MGI IDs.

At what stage would this be done?

What type of data would this be run on? Liger factors?

How would it be integrated with the DE analysis / case-control analysis?

The LDSC github repo is a good example of how to do this: https://github.com/bulik/ldsc

They have the following sections:
Getting Started
Updating LDSC
Where Can I Get LD Scores? (for us this will be, 'where can i get an example dataset?'
Support (this section points to relevant parts of the wiki)
Citation
License
Authors

We should mirror this setup

I am getting the following error when generating QC report. scflow is installed on Ubuntu R version 3.6.1.

Error: pandoc version 1.12.3 or higher is required and was not found (see the help page ?rmarkdown::pandoc_available).

The error may generate during installing R package rgl.

Install the following packaged using the command sudo apt install libftgl2 libcgal-dev libglu1-mesa-dev libglu1-mesa-dev .

solution was taken from this link

Currently these are just set to defaults. LIGER has functions for scanning for optimal values but expects them to then be chosen visually. Should be possible to just write a script to find the elbow.

Sample annotation sheet is here: https://docs.google.com/spreadsheets/d/11CC8q84hBrk1A3_SsbVkBz_TsPB6WF3H5V1LuYUPaWk/edit#gid=0

It needs two column's added to store locations in the project folder of the FASTQ and CellRanger files

I'm using PuTTY to SSH to a Linux system (GNU/Linux 4.15.0-74-generic x86_64).
After merging and dimensionality reduction, I get the following error when running sce@metadata:

> sce@metadata
$reduced_dim_plots
$reduced_dim_plots$umap3d_plot_ly
$reduced_dim_plots$umap3d
Couldn't find a suitable web browser!
Set the BROWSER environment variable to your desired browser.
Warning: program returned non-zero exit code #1
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: www-browser: not found
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: links2: not found
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: elinks: not found
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: links: not found
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: lynx: not found
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: w3m: not found
xdg-open: no method available for opening '/tmp/RtmpCOkJfQ/viewhtml7b8f2acc62e6/index.html'
> Couldn't find a suitable web browser!
Set the BROWSER environment variable to your desired browser.
Warning: program returned non-zero exit code #1
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: www-browser: not found
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: links2: not found
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: elinks: not found
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: links: not found
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: lynx: not found
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: w3m: not found
xdg-open: no method available for opening '/tmp/RtmpCOkJfQ/viewhtml7b8f7ff070b3/index.html'

sce@metadata tries to open all reduced dimension plots of which umap3d needs a web browser.