BAM sorting error about rasflow HOT 3 CLOSED

Hi Pablo,

The error 137 seems to happen because of the lack of memory assigned to samtools.

As the manual of samtools says:

sort      samtools sort [-no] [-m maxMem] <in.bam> <out.prefix>

                 Sort  alignments  by  leftmost  coordinates.  File  <out.prefix>.bam  will be created. This command may also create temporary files <out.pre-
                 fix>.%d.bam when the whole alignment cannot be fitted into memory (controlled by option -m).

                 OPTIONS:

                 -o      Output the final alignment to the standard output.

                 -n      Sort by read names rather than by chromosomal coordinates

                 -m INT  Approximately the maximum required memory. [500000000]

You can try increasing the maximum memory for samtools using the parameter -m: modify the command in the file /home/sam/Downloads/pablo2021/RASflow/workflow/align_count_genome.rules at line 97.

Let me know if this solves your problem.

XK

from rasflow.

Hi @zhxiaokang

I have tried changing the -m parameter from 500000000 to 5000000000, however I am still getting the same error with samtools sort.

`Rules claiming more threads will be scaled down.
Job counts:
count jobs
1 alignment
2 alignmentQC
1 end
2 featureCount
1 getReads
2 sortBAM
1 summaryReport
10

[Wed Jun 16 11:19:11 2021]
rule sortBAM:
input: data/output/test/genome/bamFile/200357.bam
output: data/output/test/genome/bamFileSort/200357.sort.bam
jobid: 7
wildcards: sample=200357

samtools sort: couldn't allocate memory for bam_mem
[Wed Jun 16 11:19:11 2021]
Error in rule sortBAM:
jobid: 7
output: data/output/test/genome/bamFileSort/200357.sort.bam

RuleException:
CalledProcessError in line 97 of /home/sam/Downloads/pablo2021/RASflow/workflow/align_count_genome.rules:
Command ' set -euo pipefail; samtools sort -@ 8 data/output/test/genome/bamFile/200357.bam -o data/output/test/genome/bamFileSort/200357.sort.bam -m 5000000000 ' returned non-zero exit status 1.
File "/home/sam/Downloads/pablo2021/RASflow/workflow/align_count_genome.rules", line 97, in __rule_sortBAM
File "/home/sam/anaconda3/envs/rasflow/lib/python3.6/concurrent/futures/thread.py", line 56, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /home/sam/Downloads/pablo2021/RASflow/.snakemake/log/2021-06-16T111911.184687.snakemake.log
Start doing DEA!
Building DAG of jobs...
MissingInputException in line 18 of /home/sam/Downloads/pablo2021/RASflow/workflow/dea_genome.rules:
Missing input files for rule combineSamples:
output/test/genome/countFile/200358_count.tsv
output/test/genome/countFile/200357_count.tsv
DEA is done!
Visualization is not required and RASflow is done!
`
I am also including the config file so that you have an idea of what I am trying to do.
config_main (copy).txt

Any idea on how to fix this issue? Perhaps there is a command to tell samtools to sort the sequences in smaller pieces?

Cheers,
Pablo

from rasflow.

Hi Pablo,

Apparently, samtools is complaining about the memory. It seems that you don't have enough space on your disk? But as I can see from your config file, you're using the example dataset, which is very small. So I'm really confused about the error.

Could you play around with the command in question, trying out different settings of the parameters? Don't worry about re-running RASflow many times, since it will always start from where it stopped, so the previous successfully-run steps will automatically be skipped.

XK

from rasflow.