10xgenomics / cellranger Goto Github PK

It seems that STAR will always run on 4 cores, also when --localcores is set at two, which should be the maximal number of cores used according to the parameter description.

Linked to #25 but then from the perspective of making number of cores listen to --localcores instead of using all available cores

How can i make csv from data of cellranger?

I am now handling the single cell expression data of 2000 to 3000 cells from cell line

I tried cellranger mat2csv order, but it did not work may be due to memory problem

Otherwise, if i could modify mtx data such as removing null data and normalizing, i think that csv converting might be possible.

Is there any suggest for me?

I was running cell ranger pipeline trying to align reads from single-cell data to viral genome
the viral genome is pretty small, so instead of making mixture genome of mm10 and viral sequence I decided to align separately to mm10 and viral genome

I created an index genome from FASTA file and made GTF by my own,
this GTF file contained only one exon and one CDS region.

After that, I got an error executing line 356 from file /lib/python/cellranger/transcriptome.py
chrom_bins = [bisect.bisect_left(tx_starts, cs) for cs in chrom_starts]

TypeError: len() of unsized object from line

For me, it looks like when you read tables "transcriptInfo.tab" and "exonInfo.tab" before

    tx_info = np.loadtxt(tx_path, tx_dtype, skiprows=1, unpack=True)
    ex_info = np.loadtxt(ex_path, ex_dtype, skiprows=1, unpack=True)

if every file had only one line, then np.loadtxt will read a 1d array instead of 2d
to prevent this you can force np.loadtxt to read 2d array like

    tx_info = np.loadtxt(tx_path, tx_dtype, skiprows=1, unpack=True, ndmin=2)
    ex_info = np.loadtxt(ex_path, ex_dtype, skiprows=1, unpack=True, ndmin=2)

fix shown above seems to solve the problem

Hi,
this is more a information request than a software issue.

I'd like to use cellranger for a custom genome reference. I know that my data will not map uniquely on that genome, I expect multiple hits of many reads. The standard cellranger count workflow will discard such reads. That would leave me without data at the end ;-)

• I could not find a way to alter the STAR alignment parameters, e.g. outFilterMultimapNmax.
  Can you provide some info where I find the parameter setting for STAR in the package?

• Is there a way to get "intermediate" data? An alternative would be to use the umi-collapsed reads from cellranger and feed these into some standard aligner like bwa with subsequent "manual" analysis (I don't need spliced alignment).

I'd appreciate any hints ..

best,
Sven

Hi,

I have installed bcl2fastq and cell ranger, for using the mkfastq command. It was working fine, but it has just stopped working. To verify whether I messed up with the PATH, I installed cell ranger and bcl2fastq on a different system, and run the commands from there. I am still getting errors.
Please find below the commands
##Checking the version of bcl2fastq
[sbhattach2@CRI-13-130 cellranger-2.1.0]$ bcl2fastq --version
BCL to FASTQ file converter
bcl2fastq v2.20.0.422
Copyright (c) 2007-2017 Illumina, Inc.
##running mkfastq
[sbhattach2@CRI-13-130 cellranger-2.1.0]$ sudo ./cellranger mkfastq --id=single_cell_zhe --run=/data/Suro/180112_NS500434_0271_AHFCW5BGX5/ --samplesheet=/home/sbhattach2/Sample_Zhe.csv
/home/sbhattach2/cellranger-2.1.0/cellranger-cs/2.1.0/bin
cellranger mkfastq (2.1.0)
Copyright (c) 2017 10x Genomics, Inc. All rights reserved.

Martian Runtime - '2.1.0-2.3.0'
Serving UI at http://CRI-13-130:34100?auth=HvO_950umeofy7Jo1aYtXyKrK3lEiBUDzbXi_pjVOZ0

Running preflight checks (please wait)...
Checking run folder...
Checking RunInfo.xml...
Checking system environment...
Checking barcode whitelist...
Checking read specification...
Checking samplesheet specs...

[error] No bcl2fastq found on path. demux requires bcl2fastq v2.17 or greater for RTA version: 2.4.11

Can you please guide me regarding the changes I need to make?

Thanks in advance,
Surajit

I run cellranger count and got this error:
[error] Pipestance failed. Error log at:
Aid/SC_RNA_COUNTER_CS/SC_RNA_COUNTER/_BASIC_SC_RNA_COUNTER/EXTRACT_READS/fork0/chnk087-ud4680f64f3/_errors

Log message:
exit status 2

How to solve it? All files' MD5 are checked ok.

The current version of Cell Ranger available from the 10x website is 2.1.0. But, I believe this repo is 2.0.2. Do you plan to update this repo with the latest version of Cell Ranger?

Hey were you planning on updating the repo to include the code for cellranger 3?

Hello,

Given the MIT license, is there any chance that you might release python modules within CellRanger such as tenkit, cellranger, striped_smith_waterman, etc as individual packages to pypi or as separate repos? If not, would you permit other people doing so? I think this can be very useful to those wanting to use/develop based on those code but reluctant to clone/update the big repo. Many thanks!

this is true even if the path is quoted:

cellranger mkfastq --id="cellranger-tiny-bcl-1_2_0-2" --run="/work/urology/ghenry/Grimoire/ASTROCYTE/cellranger_mkfastq/work/b6/ee0d425b17b1836f73121834665782/cellranger-tiny- bcl-1_2_0-2" --csv=design.checked.csv -r $SLURM_CPUS_ON_NODE  -p $SLURM_CPUS_ON_NODE  -w $SLURM_CPUS_ON_NODE

causes the error:

[error] Specified run folder does not exist: /work/urology/ghenry/Grimoire/ASTROCYTE/cellranger_mkfastq/work/b6/ee0d425b17b1836f73121834665782/cellranger-tiny-

hi,
Does any meet the same problem ?
It looks like some issue in the subprocess
I have checked md5, no problem founded, any advise is ok

[error] Pipestance failed. Error log at:
SRR8315772/SC_VDJ_ASSEMBLER_CS/SC_VDJ_ASSEMBLER/ASSEMBLE_VDJ/fork0/chnk29-u01581b59ef/_errors

Log message:
Traceback (most recent call last):
  File "/project/chensy/salyut/Tool/cellranger-3.0.2/martian-cs/v3.2.0/adapters/python/martian_shell.py", line 590, in _main
    stage.main()
  File "/project/chensy/salyut/Tool/cellranger-3.0.2/martian-cs/v3.2.0/adapters/python/martian_shell.py", line 555, in main
    self._run(lambda: self._module.main(args, outs))
  File "/project/chensy/salyut/Tool/cellranger-3.0.2/martian-cs/v3.2.0/adapters/python/martian_shell.py", line 524, in _run
    cmd()
  File "/project/chensy/salyut/Tool/cellranger-3.0.2/martian-cs/v3.2.0/adapters/python/martian_shell.py", line 555, in <lambda>
    self._run(lambda: self._module.main(args, outs))
  File "/project/chensy/salyut/Tool/cellranger-3.0.2/cellranger-cs/3.0.2/mro/stages/vdj/assemble_vdj/__init__.py", line 128, in main
    run_assembly(args.chunked_bam, martian.make_path(''), args)
  File "/project/chensy/salyut/Tool/cellranger-3.0.2/cellranger-cs/3.0.2/mro/stages/vdj/assemble_vdj/__init__.py", line 87, in run_assembly
    tk_subproc.check_call(cmd, cwd=os.getcwd())
  File "/project/chensy/salyut/Tool/cellranger-3.0.2/cellranger-cs/3.0.2/tenkit/lib/python/tenkit/log_subprocess.py", line 37, in check_call
    return subprocess.check_call(*args, **kwargs)
  File "/project/chensy/salyut/Tool/cellranger-3.0.2/miniconda-cr-cs/4.3.21-miniconda-cr-cs-c10/lib/python2.7/subprocess.py", line 186, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['vdj_asm', 'asm', u'/project/chensy/benchmark/results/10X-scTCR/SRR8315772/SC_VDJ_ASSEMBLER_CS/SC_VDJ_ASSEMBLER/FILTER_VDJ_READS/fork0/chnk29-u01581b58cf/files/barcode_chunked_bams.bam', '/project/chensy/be
nchmark/results/10X-scTCR/SRR8315772/SC_VDJ_ASSEMBLER_CS/SC_VDJ_ASSEMBLER/ASSEMBLE_VDJ/fork0/chnk29-u01581b59ef/files/', '--kmers=2', '--min-contig=250', '--min-qual=2', '--score-factor=0.6', '--qual-factor=0.9', '--min-sw-score=50', '-
-rt-error=0.0001', '--min-umi-reads=556']' returned non-zero exit status 101

Dear all,

I have recently installed cellranger on my Mac to create a .gtf file for Velocyto analysis.
I have installed cellranger -3.0.2 following the instruction on 10x website.
However I couldn't use any function from cellranger. Below is the screenshot of my error.
Any help would be appreciated.
(https://user-images.githubusercontent.com/35354607/57561036-6d464880-733e-11e9-84c7-2ce0e0891bd7.png)

It would be better if instead of Jet, which is a problematic colormap, a perceptually uniform colormap was used. Included in Plotly is Viridis but really any sequential colormap is better than Jet. By choosing a better default, you can hopefully help people move away from using Jet.

Dear,
I have a problem when used the cellranger count.
This is my command:

cellranger count --id=sample --transcriptome=genome_index --fastqs=PATH --sample=J1

But I get a stderr with cann't find mrp.

cellranger/bin/../tenkit/bin/common/_mrp: line 26: mrp: command not found

Any help is much appreciated.
Thanks.

The LICENSE file mentions that the license for the louvain code is under LGPLv3 and GPLv3. The directory lib/louvain indeed has a file for both licenses but all individual files reference only the LGPLv3. Can you clarify the license of those files?

The LICENSE file also mentions that the rest of the code is under AGPLv2 but the individual files have the much less clear "all rights reserved" with no mention of the license. Making things a bit more confusing there's also lib/python/cellranger/analysis/irlb.py which claims to be under Apache 2 and lib/python/cellranger/webshim/lz_string.py being based on http://pieroxy.net/blog/pages/lz-string/index.html which is under WTFPL. The FSF how to use GNU licenses mentions that each file should have the license notice to make this clearer, which would be nice. Could all files files under AGPLv2 and owned by 10x genomics, Inc have the license notice to make things clearer?

The LICENSE file also mentions Laurens van der Maaten from Delft University reserving all rights of tsne for himself. I couldn't find the tsne code on the repository. Laurens van der Maaten has a python tsne module online but in there, the copyright is claimed by Tilburg University not Delft. The binary releases of cell ranger include a tsne module by Daniel Rodriguez. The PKG_INFO of that package claims to be under Apache 2. However, the LICENSE file on his repository have 4-clause BSD license (which by the way, is GPL and AGPL incompatible).

Would it be possible to issue warnings when unknown/mistyped command line parameters are used? v2.1.1 just silently does nothing in such a case (typed --samples instead of --sample, wasted a lot of time, could be handled better by an argument handler checking the arguments)

I found the build instructions (in README.md) less complete - needed more dependencies and had to workaround few issues when I tried out on a bare AWS Ubuntu 16.04 instance.

Here's a more complete version:
https://github.com/dpeerlab/cellranger#building-on-aws-ubuntu-1604

Hi :
I'd like to use cellranger to analysis my 10X single cell transcript data of a sample infected by a virus. I want to quantify each of the virus genes expression level within each of cells. I have two strategies below :
(1) First map to Human Reference Genome and then the reads unmapped were mapped to the virus reference genome. The most difficult things is how to get umapped paired raw reads fastq files after running cellranger. (Even The second straight is better and easy to do. I still want to know how to get unmapped paired reads by cellranger :) )
(2) Add the virus genome to Human reference genome and add the annotation of virus GTF files to human annotation files too. Then analysis together with cellranger.

Which one is bettter for my questions , and do you have some advices and more better straight to do that ???

Hope for reply
Best
Seadon

I am currently looking into the variation that can be expected between profiles of the same cell type, produced at different labs, and I believe that PCR amplification biases come into play here, even though heavily reduced by using UMIs the way they are used now. I get the feeling that UMI duplicate reads are discarded and not used further. I imagine that if you have fewer copies of the molecules for a gene, that gene is likely less amplified, and such counts should be scaled up since the probability of finding the molecules of that gene is lower, at least if the number of reads are below UMI saturation. I’m just curious if this has been tried, and if not, if you are willing to? I could contribute with some calculations if needed.

Hi everyone,

I'm trying to run 'cellranger count' pipline to get the counts output data using the following code:

cellranger count --id=pbmc
--fastqs=/mnt/c/Users/dell/scRNA/run_cellranger_count/pbmc_1k_v3_fastqs
--sample=pbmc
--transcriptome=/mnt/c/Users/dell/scRNA/run_cellranger_count/refdata-cellranger-GRCh38-3.0.0

But sadly got this error: "Error fetching io for 1923: no such file or directoryChecking sample info..."

It seems like unable to find the fastq files in the given directory, but it certainly does have the files, as the ls commend return the files name and it dose have all the fastq files needed:

DESKTOP-1T8GAGA:/mnt/c/Users/dell/scRNA/run_cellranger_count$ ls
pbmc pbmc_1k_v3 pbmc_1k_v3_fastqs refdata-cellranger-GRCh38-3.0.0

So I'm wondering what's wrong with the scripts or any tiny details that I missed.
Thank you !

Hi,
There are some errors about that returned non-zero exit status1. I think this is a problem in python. But i do not how to fix it? I do not think I could change the python script.
Any advices is OK. Thank you!

2018-10-11 21:06:25 [runtime] (update)          ID.normal2.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.CHUNK_READS.fork0 chunks_running
2018-10-11 21:12:26 [runtime] (update)          ID.normal2.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.CHUNK_READS.fork0 chunks_running
2018-10-11 21:15:08 [runtime] (failed)          ID.normal2.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.CHUNK_READS

[error] Pipestance failed. Error log at:
normal2/SC_RNA_COUNTER_CS/SC_RNA_COUNTER/_BASIC_SC_RNA_COUNTER/CHUNK_READS/fork0/chnk0-u55bdbf4968/_errors

Log message:
Traceback (most recent call last):
  File "/test/software/cellranger-2.1.1/martian-cs/2.3.2/adapters/python/martian_shell.py", line 529, in _main
    stage.main()
  File "/test/software/cellranger-2.1.1/martian-cs/2.3.2/adapters/python/martian_shell.py", line 495, in main
    self._run(lambda: self._module.main(args, outs))
  File "/test/software/cellranger-2.1.1/martian-cs/2.3.2/adapters/python/martian_shell.py", line 464, in _run
    cmd()
  File "/test/software/cellranger-2.1.1/martian-cs/2.3.2/adapters/python/martian_shell.py", line 495, in <lambda>
    self._run(lambda: self._module.main(args, outs))
  File "/test/software/cellranger-2.1.1/cellranger-cs/2.1.1/mro/stages/common/chunk_reads/__init__.py", line 53, in main
    tk_subproc.check_call(chunk_reads_args)
  File "/test/software/cellranger-2.1.1/cellranger-cs/2.1.1/tenkit/lib/python/tenkit/log_subprocess.py", line 37, in check_call
    return subprocess.check_call(*args, **kwargs)
  File "/test/software/cellranger-2.1.1/miniconda-cr-cs/4.3.21-miniconda-cr-cs-c9/lib/python2.7/subprocess.py", line 186, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['chunk_reads', '--reads-per-fastq', '5000000', '/test/XX/RNAseq/Batch2_normal/normal2/SC_RNA_COUNTER_CS/SC_RNA_COUNTER/_BASIC_SC_RNA_COUNTER/CHUNK_READS/fork0/chnk0-u55bdbf4968/files/', 'fastq_chunk', '--martian-args', 'chunk_args.json', '--compress', 'lz4']' returned non-zero exit status 1

I have noticed what I believe to be an edge case bug when running the aggr program. In order for the bug to be observed, cellranger count has to be run on a transcriptome that has only one gene (one row). In this situation, cellranger aggr will raise the error listed below on molecular info files generated in cellranger count. I believe this is because, when performing the numpy operation on a dataset with a single column (only one gene), numpy returns an number rather than an array of shape (,1). Reshaping the array might fix this error. I confirmed that this error does not occur when passing in two genes to the transcriptome.

Log message:
Traceback (most recent call last):
  File "/opt/cellranger-3.0.2/martian-cs/v3.2.0/adapters/python/martian_shell.py", line 590, in _main
    stage.main()
  File "/opt/cellranger-3.0.2/martian-cs/v3.2.0/adapters/python/martian_shell.py", line 561, in main
    self._run(lambda: self._module.join(
  File "/opt/cellranger-3.0.2/martian-cs/v3.2.0/adapters/python/martian_shell.py", line 524, in _run
    cmd()
  File "/opt/cellranger-3.0.2/martian-cs/v3.2.0/adapters/python/martian_shell.py", line 562, in <lambda>
    args, outs, chunk_defs, chunk_outs))
  File "/opt/cellranger-3.0.2/cellranger-cs/3.0.2/mro/stages/aggregator/check_invariants/__init__.py", line 103, in join
    if len(input_feature_counts[lib_gg]) != len(output_feature_counts[lib_gg]):
TypeError: len() of unsized object

Hello,

I have single cell sequencing data from bovine endometrium and I am following 10x genomics instructions (https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/advanced/references) to build a reference to use cellranger on bovine cells.

To do so, I’ve downloaded the GCF_002263795.1_ASR-UCD1.2_genomic.gff file from NCBI and converted it into a .gtf file using gffread.

Now, I tried to use cellranger mkgtf to filter the gtf file and got error below. Can somebody help with that?

[jmoraes@lewis4-r630-hpc4-node302 reference_genome]$ cellranger mkgtf /home/jmoraes/data/Illumina_example/Lucy_Moraes/outs/fastq_path/HY7FHBGX9/reference_genome/ASR_Converted.gtf output_GTF_TESTING.gtf --attribute=key:allowable_value
/storage/hpc/data/jmoraes/cellranger/cellranger-3.0.2/cellranger-cs/3.0.2/bin
cellranger mkgtf (3.0.2)
Copyright (c) 2019 10x Genomics, Inc. All rights reserved.

Writing new genes GTF file (may take 10 minutes for a 1GB input GTF file)...
Property 'gene_id' not found in GTF line 1791443: NC_006853.1 RefSeq exon 364 430 . + . transcript_id "rna78868";

Please fix your GTF and start again.

Thank you!!

Hello, does anyone know how to download older version of cellranger-3.0.2? Why isn't it in the Software Downloads page?Because I have differents batches of data to deal with,and they belong to one experiment, I think change cellranger version among batches is bad.
Unluckily, our linux server went wrong and I lost cellranger-3.0.2 package. Before I can download it in the Software Downloads page(as the old screenshot shows), but now it's replaced by latest version.I can find 3.0.0,but not 3.0.2.
Can anyone tell me the right page or share me with this package?By the way, does different versions of CellRanger 3.0 affect analysis result?

@nlhepler

I've been running cellranger-ATAC.
It always generates a gigantic SC_ATAC_COUNTER_CS folder that's 200GB + until it runs out of space.
Do you know why?

I'm running into an issue where some genes that appear in the gtf I feed to cellranger mkref do not appear in the finished reference (ref/star/geneInfo.tab). Something must have happened in the STAR run, although output looks normal:
Generating STAR genome index (may take over 8 core hours for a 3Gb genome)...
Jul 20 13:58:16 ..... Started STAR run
Jul 20 13:58:16 ... Starting to generate Genome files
Jul 20 13:58:21 ... starting to sort Suffix Array. This may take a long time...
Jul 20 13:58:22 ... sorting Suffix Array chunks and saving them to disk...
Jul 20 14:02:04 ... loading chunks from disk, packing SA...
Jul 20 14:02:08 ... Finished generating suffix array
Jul 20 14:02:08 ... Generating Suffix Array index
Jul 20 14:02:21 ... Completed Suffix Array index
Jul 20 14:02:21 ..... Processing annotations GTF
Jul 20 14:02:22 ..... Inserting junctions into the genome indices
Jul 20 14:03:24 ... writing Genome to disk ...
Jul 20 14:03:24 ... writing Suffix Array to disk ...
Jul 20 14:03:25 ... writing SAindex to disk
Jul 20 14:03:25 ..... Finished successfully
...done.

There are 25000 genes in my original gtf, but only 19000 entries in my geneInfo.tab file.
Is there a workaround for this?

I ran R2 of my FASTQ 10X sequencing data through FastQC and I found several over represented short sequences (up to 2% of sequences) at the start/end of reads, apparently deriving from Evrogen Mint PlugOligo-1 adapters. I'm not sure how these adapters were introduced? Will CellRanger handle them appropriately or should they be trimmed before running cellranger count? Thanks

hi, there, thank you in advance, I want ot know how to downlaod fastqs from https://support.10xgenomics.com/single-cell-gene-expression/datasets, thanks

Thanks for this excellent work here.
I have another requirement, is there a way to get the pre-matrix which is also the same form like the one we got but remains duplicates inside. In other words, is there a way that I can recover the gene expression matrix before discarding the duplicated UMI?
For Cellranger, I looked at the code then simply remove this line "read.is_duplicate = True" in script "./mro/stages/counter/mark_duplicates/init.py", but I got key error.
Any suggestion on that?
I really appreciate any information you might provide.

Hi,
I am new in single-cell RNA-seq. I got a library with a mixture of human and mouse cells. The mouse cells ratio is only 2.5%. I used cellranger count to get the matrix. But the output of cellranger count is a mix of human and mouse. How I can filter out mouse cells and only get a matrix of human cells?
Thanks!

hi eveyone,
anyone has used the --libraries as described in https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/feature-bc-analysis ?

I cannot use cellranger count as it keeps asking me path for --fastqs

thanks
ib

Hi,

I am using cellranger vdj to analyze the single cell vdj libraries.

But I am confused about the clonotype definition by the tool.

From the documentation, it says

Cell barcodes are grouped together into clonotypes if they share the same set of productive CDR3 nucleotide sequences by exact match.

Does cellranger require that the CDR3 sequence of all the chains of IG or TR (e.g. IGH/IGK/IGL, TRA/TRB, TRD/TRG) are the samme or any one of these chains? If only the CDR3 match is only required by one of these chains, one cell may be assigned to different clonotypes.

I am new to this. Sorry if the question is too simple.

Thank you for your time.

Bests,
Yiwei

Hello,

I'm trying to append the sequence for GFP onto the mm10 genome so I can check if my cells express GFP.

I concatenated the GFP fasta sequence to the mm10 fasta file and the corresponding GTF with the mm10 GTF file. However, I keep getting the below error when making the reference.

cellranger mkref --genome=gftgenome --fasta=newgenome.fa --genes=newgtf.gtf
/gpfs/ycga/apps/hpc/software/cellranger/2.1.1/cellranger-cs/2.1.1/bin
cellranger mkref (2.1.1)
Copyright (c) 2017 10x Genomics, Inc. All rights reserved.

Creating new reference folder at /gpfs/ycga/scratch60/ysm/cafferty/ng384/gftgenome
...done

Writing genome FASTA file into reference folder...
...done

Computing hash of genome FASTA file...
...done

Writing genes GTF file into reference folder...
WARNING: The following transcripts appear on multiple chromosomes in the GTF:

This can indicate a problem with the reference or annotations. Only the first chromosome will be counted.
...done

Computing hash of genes GTF file...
...done

Writing genes index file into reference folder (may take over 10 minutes for a 3Gb genome)...
Traceback (most recent call last):
File "/gpfs/ycga/apps/hpc/software/cellranger/2.1.1/cellranger-cs/2.1.1/bin/mkref", line 100, in
main()
File "/gpfs/ycga/apps/hpc/software/cellranger/2.1.1/cellranger-cs/2.1.1/bin/mkref", line 97, in main
referenceBuilder.build_reference()
File "/gpfs/ycga/apps/hpc/software/cellranger/2.1.1/cellranger-cs/2.1.1/lib/python/cellranger/reference.py", line 213, in build_reference
self.write_genome_gene_index(new_gene_index, new_gene_gtf, new_genome_fasta)
File "/gpfs/ycga/apps/hpc/software/cellranger/2.1.1/cellranger-cs/2.1.1/lib/python/cellranger/reference.py", line 311, in write_genome_gene_index
gene_index = GeneIndex(in_gtf_fn, in_fasta_fn)
File "/gpfs/ycga/apps/hpc/software/cellranger/2.1.1/cellranger-cs/2.1.1/lib/python/cellranger/reference.py", line 386, in init
fasta_parser = FastaParser(self.in_fasta_fn)
File "/gpfs/ycga/apps/hpc/software/cellranger/2.1.1/cellranger-cs/2.1.1/lib/python/cellranger/reference.py", line 316, in init
self.chroms = self.load_fasta(in_fasta_fn)
File "/gpfs/ycga/apps/hpc/software/cellranger/2.1.1/cellranger-cs/2.1.1/lib/python/cellranger/reference.py", line 333, in load_fasta
current_chrom = FastaParser._get_chrom_name(line)
File "/gpfs/ycga/apps/hpc/software/cellranger/2.1.1/cellranger-cs/2.1.1/lib/python/cellranger/reference.py", line 321, in _get_chrom_name
return match.groups(1)[0]
AttributeError: 'NoneType' object has no attribute 'groups'

Thanks!!

I was running cellranger-atac.
The error message I got was always:

cellranger-atac count (1.1.0)
Copyright (c) 2019 10x Genomics, Inc.  All rights reserved.
-------------------------------------------------------------------------------

Usage:
    count
        --id=ID
        --fastqs=PATH
        [--sample=PREFIX]
        [options]
    count <run_id> <mro> [options]
    count -h | --help | --version

It turned out that I mistakenly used --transcriptome
Could you update the error message with something more informing?

It's my first time to use the version 2.2.0 of cellranger and I did try running cellranger testrun. But I got some error message.

cellranger testrun --id=tiny

"errors": [
"lstat */MERGE_CLUSTERS/fork0/join-u211eda6f1f/files/clusters_csv: no such file or directory",
"lstat */MERGE_CLUSTERS/fork0/join-u211eda6f1f/files/clusters_h5.h5: no such file or directory",
"lstat */PREPROCESS_MATRIX/fork0/join-u211eda6ed0/files/preprocessed_matrix_h5.h5: no such file or directory",
"lstat */RUN_DIFFERENTIAL_EXPRESSION/fork0/join-u211eda6f3e/files/diffexp_csv: no such file or directory",
........

When running cellranger-atac mkref, running into an error during the check_reference_format step does not properly print the error due to a bug in they way marian-cs attempts to output the error message. In this particular case, the problem was that there were more than 100 PRIMARY_CONTIGS.

cellranger-atac mkref (1.2.0)                                                                                                                                                                                                                                                      
Copyright (c) 2019 10x Genomics, Inc.  All rights reserved.                                                                                                                                                                                                                        
-------------------------------------------------------------------------------                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                                   
Non-standard genome name detected, building custom reference...                                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                                   
>>> Creating reference for ci_kh_ghostdb_kh2012_ensembl_mt <<<                                                                                                                                                                                                                     
                                                                                                                                                                                                                                                                                   
Creating new reference folder at /reference-genomes/ciona_intestinalis/ciona_intestinalis_kh_ghostdb_ensembl_mt/cell_ranger_atac_mkref/ci_kh_ghostdb_kh2012_ensembl_mt                                                                                                             
Copying original fasta file into reference folder... done                                                                                                                                                                                                                          
                                                                                                                                                                                                                                                                                   
Generating samtools index...                                                                                                                                                                                                                                                       
done                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                   
Generating pyfasta indexes...                                                                                                                                                                                                                                                      
    Number of contigs: 1272                                                                                                                                                                                                                                                        
    Total genome size: 115227500                                                                                                                                                                                                                                                   
done                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                   
Copying original gene annotation file into reference folder... done                                                                                                                                                                                                                
                                                                                                                                                                                                                                                                                   
Writing TSS and transcripts bed file...                                                                                                                                                                                                                                            
    Parsed 44757 unique TSS and 48038 unique transcripts.                                                                                                                                                                                                                          
done                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                   
Generating bwa index (may take over an hour for a 3Gb genome)...                                                                                                                                                                                                                   
[bwa_index] Pack FASTA... 0.74 sec                                                                                                                                                                                                                                                 
[bwa_index] Construct BWT for the packed sequence...                                                                                                                                                                                                                               
[BWTIncCreate] textLength=230455000, availableWord=28215648                                                                                                                                                                                                                        
[BWTIncConstructFromPacked] 10 iterations done. 46542904 characters processed.                                                                                                                                                                                                     
[BWTIncConstructFromPacked] 20 iterations done. 85983976 characters processed.                                                                                                                                                                                                     
[BWTIncConstructFromPacked] 30 iterations done. 121035288 characters processed.                                                                                                                                                                                                    
[BWTIncConstructFromPacked] 40 iterations done. 152184952 characters processed.                                                                                                                                                                                                    
[BWTIncConstructFromPacked] 50 iterations done. 179866808 characters processed.                                                                                                                                                                                                    
[BWTIncConstructFromPacked] 60 iterations done. 204466488 characters processed.                                                                                                                                                                                                    
[BWTIncConstructFromPacked] 70 iterations done. 226326712 characters processed.                                                                                                                                                                                                    
[bwt_gen] Finished constructing BWT in 73 iterations.                                                                                                                                                                                                                              
[bwa_index] 42.33 seconds elapse.                                                                                                                                                                                                                                                  
[bwa_index] Update BWT... 0.55 sec                                                                                                                                                                                                                                                 
[bwa_index] Pack forward-only FASTA... 0.41 sec                                                                                                                                                                                                                                    
[bwa_index] Construct SA from BWT and Occ... 20.21 sec                                                                                                                                                                                                                             
[main] Version: 0.7.17-r1188                                                                                                                                                                                                                                                       
[main] CMD: bwa index /reference-genomes/ciona_intestinalis/ciona_intestinalis_kh_ghostdb_ensembl_mt/cell_ranger_atac_mkref/ci_kh_ghostdb_kh2012_ensembl_mt/fasta/genome.fa                                                                                                        
[main] Real time: 64.742 sec; CPU: 64.243 sec                                                                                                                                                                                                                                      
done                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                   
Downloading pfm files from source...                                                                                                                                                                                                                                               
done                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                   
Finishing up...                                                                                                                                                                                                                                                                    
Traceback (most recent call last):                                                                                                                                                                                                                                                 
  File "/opt/cellranger-atac-1.2.0/cellranger-atac-cs/1.2.0/bin/mkref", line 68, in <module>                                                                                                                                                                                       
    main()                                                                                                                                                                                                                                                                         
  File "/opt/cellranger-atac-1.2.0/cellranger-atac-cs/1.2.0/bin/mkref", line 62, in main                                                                                                                                                                                           
    reference_builder.build_reference()                                                                                                                                                                                                                                            
  File "/opt/cellranger-atac-1.2.0/cellranger-atac-cs/1.2.0/lib/python/reference.py", line 548, in build_reference                                                                                                                                                                 
    check_reference_format(self.ref_path)                                                                                                                                                                                                                                          
  File "/opt/cellranger-atac-1.2.0/cellranger-atac-cs/1.2.0/lib/python/preflights.py", line 200, in check_reference_format                                                                                                                                                         
    martian.exit(error_msg)                                                                                                                                                                                                                                                        
  File "/opt/cellranger-atac-1.2.0/martian-cs/v3.2.4/adapters/python/martian.py", line 265, in exit                                                                                                                                                                                
    _INSTANCE.metadata.write_assert(message)                                                                                                                                                                                                                                       
AttributeError: 'NoneType' object has no attribute 'metadata'  

Hi all :
I want to compare between cellranger 2.0 and 3.0 but I think the results are quite different .
First : I know that version 3.0 use new cell calling method. I got 6688 cells by using version 3.0 while I only got 812 cells by using version 2.0 . My expect cells is about 4000 cells. I think v 2.0 call cells are too little while v 3.0 are too many cells ? How to handle and choose ?
Then : The sum UMIs in each cells is different also. I cant find the mapping step change info of v 3.0 in release notes. What changed for mapping and counts reads changed in v 3.0 compare to 2.0 ? If I use v 2.0 for some samples and v 3.0 for other samples, can I still aggr them together to analysis ?

Best
Seadon

When I try running cellranger version 2.1.0, I get the following error:

ls: cannot access /fh/fast/bloom_j/software/cellranger/martian-cs/2.3.0/jobmanagers/*.template: No such file or directory

The problem appears to be that the relevant files are named *.template.example, not *.template.

For instance, ls martian-cs/2.3.0/jobmanagers/ yields:

config.json              retry.json            slurm.template.example
lsf.template.example     sge_queue.py          torque.template.example
pbspro.template.example  sge.template.example

Hi,

I've generated libraries of TCR gamma and delta chains using the 5' chemistry and custom reverse primers. Cellranger vdj returns contigs with gamma/delta chains in the all_contig_annotations.csv output, however, the consensus_annotations.csv, clonotypes.csv, and vloupe files are not generated. I've noticed in the all_contig_annotations.csv file that all the TCRgd chains are called as "FALSE" under "is_cell" so I suspect these are being filtered during the annotation. Is there a potential workaround for gamma delta chains?

Hello @pryvkin10x
I have put together a scRNAseq pipeline that is compatible with 10x 3' chemistry and outputs comparable results to cellranger.

I have tried to run it with 10x 5' data, but I get way less genes/cell. I have looked at the chemistry of the 5' and the source code of cellranger but haven't been able to determine what the problem seems to be.

Would you have some insights as to what part I should look into?

Dear Paul Ryvkin :
I want to get the unmapped reads when after mapping to the genomeA by using cellragner. And then use the unmapped paired reads(with UMI) to map to genomeB. For full length without UNI based protocol this process is easy to handle. But for UMI based protocol I have no ideal ideas. What I can do I think is :
(1) samtools view -f 4 possorted_genome_bam.bam > unmapped.bam
(2) bam2fastq tools to convert unmapped.bam to unmapped_R2.fastq (V2 3` chemistry)
(3) select the reads in raw_R1.fastq file(with UMI) to paired with the reads in unmapped_R2.fastq so I can get the new unmapped pair-end fastq files named unmapped_R1.fastq and unmapped_R2.fastq (this step I don't know which tools can used ???)
(4) finally , rerun cellranger with the unmapped data to the genomeB

So any hints for me to achieved my aims. Can cellranger output the unmapped pired fastq files straightly ?
Thank you for replay.

Best
Seadon

Hi,

I am trying to build the reference genome for drosophila by using cellranger mkref command. Please find below the commands used.
[sbhattach2@CRI-13-127 cellranger-2.1.0]$ sudo ./cellranger mkgtf /data/Suro/Drosophila_melanogaster.BDGP6.91.gtf /data/Suro/Drosophila_melanogaster.BDGP6.91_filtered.gtf --attribute=gene_biotype:protein_coding --attribute=gene_biotype:lincRNA --attribute=gene_biotype:antisense

[sbhattach2@CRI-13-127 cellranger-2.1.0]$ sudo ./cellranger mkref --genome=dros6 --fasta=../Drosophila_melanogaster.BDGP6.cds.all.fa.gz --genes=/data/Suro/Drosophila_melanogaster.BDGP6.91_filtered.gtf
/home/sbhattach2/cellranger-2.1.0/cellranger-cs/2.1.0/bin
cellranger mkref (2.1.0)
Copyright (c) 2017 10x Genomics, Inc. All rights reserved.

Creating new reference folder at /home/sbhattach2/cellranger-2.1.0/dros6
...done

Writing genome FASTA file into reference folder...
...done

Computing hash of genome FASTA file...
...done

Writing genes GTF file into reference folder...
WARNING: The following transcripts appear on multiple chromosomes in the GTF:

This can indicate a problem with the reference or annotations. Only the first chromosome will be counted.
...done

Computing hash of genes GTF file...
...done

Writing genes index file into reference folder (may take over 10 minutes for a 3Gb genome)...
Traceback (most recent call last):
File "/home/sbhattach2/cellranger-2.1.0/cellranger-cs/2.1.0/bin/mkref", line 100, in
main()
File "/home/sbhattach2/cellranger-2.1.0/cellranger-cs/2.1.0/bin/mkref", line 97, in main
referenceBuilder.build_reference()
File "/home/sbhattach2/cellranger-2.1.0/cellranger-cs/2.1.0/lib/python/cellranger/reference.py", line 213, in build_reference
self.write_genome_gene_index(new_gene_index, new_gene_gtf, new_genome_fasta)
File "/home/sbhattach2/cellranger-2.1.0/cellranger-cs/2.1.0/lib/python/cellranger/reference.py", line 311, in write_genome_gene_index
gene_index = GeneIndex(in_gtf_fn, in_fasta_fn)
File "/home/sbhattach2/cellranger-2.1.0/cellranger-cs/2.1.0/lib/python/cellranger/reference.py", line 386, in init
fasta_parser = FastaParser(self.in_fasta_fn)
File "/home/sbhattach2/cellranger-2.1.0/cellranger-cs/2.1.0/lib/python/cellranger/reference.py", line 316, in init
self.chroms = self.load_fasta(in_fasta_fn)
File "/home/sbhattach2/cellranger-2.1.0/cellranger-cs/2.1.0/lib/python/cellranger/reference.py", line 336, in load_fasta
current_seq.append(line)
AttributeError: 'NoneType' object has no attribute 'append'

I have put the error and the warning in bold. Can you please suggest me a workaround?

Thanks,
Surajit

In Cell Ranger 3.x, we've run into an issue using reference genomes that work fine in 2.x. The issue occurs when there are two lines for a given gene_id and one line has a gene_name attribute and the other does not. During analysis, Cell Ranger 3.x ends up creating two records for these two lines since each have a different name, but later throwing an error since they are two records with the same id.

Example gtf lines

NC_004570   StringTie   transcript  4542    5039    1000    +   .   gene_id "MSTRG.12"; transcript_id "Cisavi.CG.ENS81.NC_004570.4542-5039.21034.t"; ref_gene_id "Cisavi.CG.ENS81.NC_004570.4542-5039"
NC_004570   StringTie   exon    4542    5039    1000    +   .   gene_id "MSTRG.12"; transcript_id "Cisavi.CG.ENS81.NC_004570.4542-5039.21034.t"; exon_number "1"; ref_gene_id "Cisavi.CG.ENS81.NC_004570.4542-5039"
NC_004570   StringTie   transcript  5019    5086    1000    +   .   gene_id "MSTRG.12"; transcript_id "Cisavi.mt-tRNA.ENS81.NC_004570.5019-5086.21035.t"; gene_name "Cisavi.mt-tRNA.ENS81.NC_004570.5019-5086"; ref_gene_id "Cisavi.mt-tRNA.ENS81.NC_004570.5019-5086"
NC_004570   StringTie   exon    5019    5086    1000    +   .   gene_id "MSTRG.12"; transcript_id "Cisavi.mt-tRNA.ENS81.NC_004570.5019-5086.21035.t"; exon_number "1"; gene_name "Cisavi.mt-tRNA.ENS81.NC_004570.5019-5086"; ref_gene_id "Cisavi.mt-tRNA.ENS81.NC_004570.5019-5086"

Example error

[error] Pipestance failed. Error log at:
chencao_cellranger_ensembl_stringtie_fix_10hpf_Cs/SC_RNA_COUNTER_CS/SC_RNA_COUNTER/_BASIC_SC_RNA_COUNTER/EXTRACT_READS/fork0/chnk00-u09384073d4/_errors

Log message:
Traceback (most recent call last):
  File "/Genomics/levineshare/People/Chen/tools/cellranger-3.1.0/martian-cs/v3.2.3/adapters/python/martian_shell.py", line 589, in _main
    stage.main()
  File "/Genomics/levineshare/People/Chen/tools/cellranger-3.1.0/martian-cs/v3.2.3/adapters/python/martian_shell.py", line 554, in main
    self._run(lambda: self._module.main(args, outs))
  File "/Genomics/levineshare/People/Chen/tools/cellranger-3.1.0/martian-cs/v3.2.3/adapters/python/martian_shell.py", line 523, in _run
    cmd()
  File "/Genomics/levineshare/People/Chen/tools/cellranger-3.1.0/martian-cs/v3.2.3/adapters/python/martian_shell.py", line 554, in <lambda>
    self._run(lambda: self._module.main(args, outs))
  File "/Genomics/levineshare/People/Chen/tools/cellranger-3.1.0/cellranger-cs/3.1.0/mro/stages/common/extract_reads/__init__.py", line 167, in main
    args.feature_reference)
  File "/Genomics/levineshare/People/Chen/tools/cellranger-3.1.0/cellranger-cs/3.1.0/lib/python/cellranger/rna/feature_ref.py", line 280, in from_transcriptome_and_csv
    return FeatureReference(feature_defs, all_tag_keys)
  File "/Genomics/levineshare/People/Chen/tools/cellranger-3.1.0/cellranger-cs/3.1.0/lib/python/cellranger/feature_ref.py", line 43, in __init__
    (this_fd_str, seen_fd_str))
FeatureDefException: Found two feature definitions with the same ID: (ID: MSTRG.12; name: Cisavi.mt-tRNA.ENS81.NC_004570.5019-5086; type: Gene Expression) and (ID: MSTRG.12; name: MSTRG.12; type: Gene Expression). All feature IDs must be distinct.

While I realize this is somewhat strange data, it does seem there is an assuption being made that all gene_id records share the same gene_name, but this is not explicitly checked during mkref or later, resulting in a somewhat confusing error.

Hello team,

Wondering why strand information is missing from the bams aligned using Cellranger. I guess
--outSAMstrandfield flag was missed in the alignment command, strand of each read pair is stored with XS attribute of the bam line. STAR does not include this in their alignments by default.

Any thoughts?

Thank you.

Right now, we're seeing that cellranger count jobs (on 5' scRNAseq data) can use up to a TB of disk during runtime. That drops down to a few dozen GB after completion - mostly the bam file.

As we're using a cluster with lots of temp space on blades, it would be great to be able to specify a TEMPDIR as an option, so that those temp files can be written to local disk, without impacting more limited networked disk. (It would also probably provide a significant speedup on many cluster architectures).

I know cellranger is quite sensitive to directory structure during runtime, but could this even be as simple as just dropping a softlink into place? We're also open to alternative ideas if you have any. Thanks!

I'm trying to run cell ranger aggr with two datasets, one with the single cell 3' v1 chemistry and the other with the single cell 3' v2 chemistry. Looking through the documentation there appears to be a batch option that can correct for chemistry batch effects, but I'm unclear how to implement this in the CSV file. I've tried adding a third column to the CSV file called batch and used SC3Pv1/SC3Pv2 for the corresponding data row, however I still get the following output/error:

Started on '/bin/hostname'
/share/apps/cellranger-2.2.0/cellranger-cs/2.2.0/bin
cellranger aggr (2.2.0)
Copyright (c) 2018 10x Genomics, Inc.  All rights reserved.
-------------------------------------------------------------------------------

Martian Runtime - '2.2.0-v2.3.3'
Serving UI at http://compute-3-5.local:38201?auth=QNS5ea2n6eJ6r6dfqhCNtblyGitP_w4DvHDo4HAWRgg

Running preflight checks (please wait)...
2018-12-17 07:56:51 [runtime] (ready)           ID.healthyCMMLCompare.SC_RNA_AGGREGATOR_CS.PARSE_CSV
2018-12-17 07:56:54 [runtime] (split_complete)  ID.healthyCMMLCompare.SC_RNA_AGGREGATOR_CS.PARSE_CSV
2018-12-17 07:56:54 [runtime] (run:local)	ID.healthyCMMLCompare.SC_RNA_AGGREGATOR_CS.PARSE_CSV.fork0.chnk0.main
2018-12-17 07:57:00 [runtime] (chunks_complete) ID.healthyCMMLCompare.SC_RNA_AGGREGATOR_CS.PARSE_CSV
2018-12-17 07:57:03 [runtime] (join_complete)   ID.healthyCMMLCompare.SC_RNA_AGGREGATOR_CS.PARSE_CSV

[error] Molecules were produced with chemistries with incompatible barcode whitelists: SC3Pv1 is incompatible with SC3Pv2

Any insight you can give me would be greatly appreciated. Thanks.

I have customized the length of the barcode and the length of the UMI. How do I edit the script of cellranger count program to get the correct UMI count?

The _BASIC_SC_RNA_COUNTER.ALIGN_READS (STAR alignment) step in cellranger count pipeline is ultra slow. The default setting of cellranger split the file int 96 chunks and alignment run one by one in a sequential order. Meanwhile each STAR alignment run use only 4 CPU core. Seems this is the bottleneck of the whole pipeline.

Some of other tools in cellranger can detect the core number in host machine and utilize the CPU usage.

Increase the threads number in cellranger-cs/3.0.2/mro/sc_rna_counter_cs.mro will doube the performance, but there is still other constraints in the settings.

Hi all,
in your notes of bcl2fastq2 you are mentioning that the default options fro the following valueas are modified. Could you please care to wrote the changes from the default values:

These bcl2fastq arguments are overridden by this pipeline:
--fastq-cluster-count
--minimum-trimmed-read-length
--mask-short-adapter-reads

thank you all

We've been having some trouble with running cellranger mkfastq for our data, although we can run it successfully with the sample data. It isn't able to find the files it is looking for. For example, one of the error messages said it was unable to find the BCL file for a tile (like 's_1_11205') in the path to the cycle folders (/Data/Intensities/BaseCalls/L00*/C*.1)

We believe that the issue may stem from the fact that the data we received was set up differently than the tiny-bcl folders. In essence, while there were multiple cycle folders and one file in each folder for the tile, we only had one cycle folder (despite having > 100 cycles) and the same number of files as cycles (instead of the number of tiles). It is also worth mentioning that the name of the files did not match the file names for the tiny-bcl (they would have names ranging from to 0001.bcl.bgzf to 0127.bcl.bgzf). We suspect that all of the files for each tile were condensed into one file, but we are unsure of this.

Is there a way for us to circumvent this issue?