Parallel "batch" sometimes fails to generate output files (.cnn/.cns) about cnvkit HOT 12 CLOSED

When batch -p 12 stopped with sample.cnr file, I ran below and got errors:

./python2.7 cnvkit.py segment sample.cnr -o sample.cns

Dropped 21 outlier bins:
chromosome start end gene log2 weight
0 chr1 91460164 91460431 MIR3675 -6.24330 0
1 chr1 202172548 202172815 RNVU1-17 -3.66398 0
2 chr2 122144558 122144825 COX5B -3.69414 0
3 chr2 136347229 136347496 COX5B -5.11515 0
4 chr2 169466973 169467240 COX5B -25.15570 0
5 chr3 1707598 1707865 ITPR1-AS1 -8.33714 0
6 chr3 169770460 169770726 MIR8060 -3.66135 0
7 chr6 49096817 49097084 HUS1B -4.86744 0
8 chr6 54252019 54252286 HUS1B -5.99444 0
9 chr7 7679073 7679340 LOC100507642 -25.16300 0
10 chr8 33318966 33319233 DEFB130 -7.19119 0
11 chr8 35248295 35248562 DEFB130 -6.22393 0
12 chr10 65087720 65087987 FAM25C -3.93370 0
13 chr11 22222406 22222673 SCGB1C1 2.25662 0
14 chr11 37743744 37744010 SCGB1C1 -25.02420 0
15 chr12 48056310 48056576 LOC101927038 -25.28670 0
16 chr15 74977246 74977513 MIR4508 2.89748 0
17 chr15 89508615 89508882 LOC103171574 -5.46744 0
18 chrX 25823571 25823837 MIR6089 -24.54650 0
19 chrX 95139335 95139601 SPIN4 -25.23700 0
...
Traceback (most recent call last):
File "cnvkit.py", line 11, in
args.func(args)
File "/cnvkit/cnvlib/commands.py", line 647, in _cmd_segment
rlibpath=args.rlibpath)
File "/cnvkit/cnvlib/segmentation/init.py", line 55, in do_segmentation
seg_out = ngfrills.call_quiet('Rscript', script_fname)
File "/cnvkit/cnvlib/ngfrills/init.py", line 42, in call_quiet
% (' '.join(args), err))
RuntimeError: Subprocess command failed:
$ Rscript /var/tmp/pbs.494475.mgmt1/tmpf4tHy6

PSCBS v0.61.0 (2016-02-03) successfully loaded. See ?PSCBS for help.

Attaching package: ‘PSCBS’

The following objects are masked from ‘package:base’:

append, load

Loading probe coverages into a data frame
Pre-processing the probe data for segmentation
Segmenting the probe data
Error in segment(list(chrom = c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, :
all weights should be positive
Calls: segmentByCBS ... eval -> eval -> eval -> eval -> value -> value.Future
Execution halted

James

from cnvkit.

Which version of CNVkit are you running? The current version is 0.7.10; there was a bug in an earlier release (maybe 0.7.8 a few versions before it) where bin weights could be zero, crashing PSCBS. This would happen when the wrong exome target BED file was used as input, or possibly when antitarget coverages were taken in a targeted amplicon capture where no off-target reads are present.

from cnvkit.

THX etal,

Now I used 0.7.5, but change to 0.7.10 now. I generated access file of hg38 using 'cnvkit access' and used it as a target file (see below):

chr1 10000 207666 MIR6859-3
chr1 257666 297968
chr1 347968 535988 OR4F3
chr1 585988 2702781 LOC101928626
chr1 2746290 12954384 ACTRT2
chr1 13004384 16799163 PRAMEF27
chr1 16849163 121976459 MIR3675
chr1 122026459 125184587
chr1 143184587 223558935 RNVU1-17
chr1 223608935 228558364 DEGS1
chr1 228608364 248946422 RNA5S2
chr2 10000 89330679 FAM110C
chr2 89530679 89685992
chr2 89753992 90402511
chr2 91402511 92138145
chr2 92188145 94090557
chr2 94140557 94293015
chr2 94496015 97439618 TEKT4
chr2 97489618 242183529 COX5B
chr3 10000 90722458 ITPR1-AS1
chr3 90772458 91249905
chr3 91256421 91265381
chr3 91276994 93655574
chr3 93705574 198235559 MIR8060

I also annotated 'access-5k-mappable.hg38.bed' with RefFlat.txt of UCSC hg38, as you can see above. Some intervals are outside genes, so no IDs. Is this a wrong target bed. Note I ran WGS, so the target bed is the same as the access file.

Below is my antitarget.bed
chr1 10500 108833 Background
chr1 108833 207166 Background

Thank you for your time.

James

from cnvkit.

Why did you choose those two intervals for your antitarget BED file? For WGS, I think it's best to use an empty antitargets file unless you have a specific reason to think those regions are amplified or sequenced differently.

For WGS, your commands should look something like:

# Make the access file
cnvkit.py access hg38.fasta -s 5000 -x exclusions.bed -o access-5k.hg38.bed

# Make an empty file for antitargets
touch empty.bed

# Run the WGS pipeline
cnvkit.py batch *_Tumor.bam -n *_Normal.bam --target access-5k.hg38.bed \
    --antitarget empty.bed --access access-5k.bed \
    --split --annotate refFlat.txt --target-avg-size 1000

from cnvkit.

I will try above. One last question?

My refFlat.txt from UCSC hg38 has different format from your refFlat.txt (hg19).

Your refFlat.txt:

head refFlat.txt
DRD4 NM_000797 chr11 + 637304 640705 637304 640603 4 637304,639432,639647,640400, 637589,639545,640306,640705,

My refFlat.txt for hg38:
chr1 34610 36081 FAM138A 0 - 36081 36081 0 3 564,205,361, 0,666,1110,
chr1 34610 36081 FAM138C 0 - 36081 36081 0 3 564,205,361, 0,666,1110,
chr1 34610 36081 FAM138F 0 - 36081 36081 0 3 564,205,361, 0,666,1110,
chr1 69090 70008 OR4F5 0 + 69090 70008 0 1 918, 0,

Is this the format suitable for '--annotate refFlat.txt'? Thankyou

James

from cnvkit.

No, the file you have is in BED12 format. The file format CNVkit expects is in the "annotation database" section of UCSC Genome Browser, here for hg38. (Download refFlat.txt.gz and decompress it.)

from cnvkit.

Beautifully! I ran commands above successfully.

Now I am running with an extra flag '--fasta' to get my data corrected. I keep you updated.

James

from cnvkit.

fasta option - Fine too!

James

from cnvkit.

This issue went in a completely different direction

from cnvkit.

It did. Anyway, I think the underlying issue is that when a function being run in parallel by multiprocessing crashes (for some reason that would usually also crash in serial, e.g. bad input), it simply halts and returns rather than raising an exception at the top level, and batch continues until it needs the output of that function, or just finishes without any indication of an error. So the solution is to figure out how to tell multiprocessing to (re-)raise exceptions encountered in worker processes.

from cnvkit.

This thread suggests some solutions to allow exceptions raised in multiprocessing worker threads to propagate back to the top level with an appropriate error message:
http://stackoverflow.com/questions/6728236/exception-thrown-in-multiprocessing-pool-not-detected

from cnvkit.

Fixed by #124.

from cnvkit.