submitSomaticSniper: unable to open file /mnt/<path-to-RESULTS>/1/1.bed about somaticseq HOT 7 CLOSED

Can you show me the full submit_callers_multiThreads.sh command (redact the full paths if you'd like to)?

from somaticseq.

Sure. It's basically the same as the example. I just changed threads and action (to bash, since we don't run SGE).

$SOMATICSEQ_HOME/utilities/singularities/submit_callers_multiThreads.sh \
--normal-bam      /fdb/DREAM/SMC/synthetic.challenge.set1.normal.v2.bam \
--tumor-bam       /fdb/DREAM/SMC/synthetic.challenge.set1.tumor.v2.bam \
--human-reference /fdb/igenomes/Homo_sapiens/Ensembl/GRCh37/Sequence/WholeGenomeFasta/genome.fa \
--output-dir      /abspath/to//RESULTS \
--dbsnp           /fdb/dbSNP/organisms/human_9606_b150_GRCh37p13/00-All.vcf.gz \
--threads         36 \
--action          bash \
--mutect2 --varscan2 --somaticsniper --vardict --muse --lofreq --scalpel --strelka --somaticseq

from somaticseq.

I think I know what the problem is.
What the submit_callers_multiThreads.sh script intends to do, is to

1. Create the SomaticSniper run script that takes the whole bam files, since SomaticSniper does not take partial bam input. The run script has a final step that splits its own VCF into partial VCF files into the directories named 1, 2, 3....
   The problem is, those directories are not created until the SomaticSniper script is created. It didn't expect SomaticSniper to finish running before going to the subsequent code.....

2. After SomaticSniper script is created, then directories 1, 2, 3, ..., will be created and BED files created in them, then all the other mutation calling scripts are also created.

I think you can try "--action echo," and then run those .cmd files as bash scripts after they're created.

Alternatively, I can put the SomaticSniper script after, instead of before, all those region-splitting steps. I'll probably do that now and make a mini-release. Having "--action bash" is not something I foresaw.

from somaticseq.

I made the latest commit where I put SomaticSniper after all the other callers, so it should avoid your problem.
Still, waiting for each mutation callers to finish one-by-one is going to take a while. It won't be parallelized if you do "--action bash," since it'll just bash each separate script one by one.

from somaticseq.

Ah, ok. That kind of brings me to a related question I was having. --action bash was not something ideal anyway, because it would run each caller in serial, but I was testing to see if I could get the whole pipeline to run within a single job allocation.

We could do an --action sbatch, but then we'd need to add analogous comments to the created job scripts to define the sbatch parameters like how you do for qsub.

from somaticseq.

I made the latest commit where I put SomaticSniper after all the other callers, so it should avoid your problem.

Ok, thanks. For some reason, I thought echo would just do a dry run. I didn't realize it still made the scripts. I'll probably go that route and use a workflow manager to run the pipeline on the cluster.

from somaticseq.

Yeah, that should work.
My scripts in this repo don't do dependencies.
All the individual mutation callers can be submitted in parallel as soon as they're created (that's what "--action qsub" would do), but the SomaticSeq scripts in RESULTS/[1,2,3...]/SomaticSeq/logs/sseq_*.cmd should only be submitted after each mutation caller job is complete. Without a workflow engine, those scripts are just created there, but need to be submitted manually. So "--action qsub" do not submit those scripts.

from somaticseq.