Comments (5)
@mahesh-panchal Hi Mahesh, ntCard does not generate intermediate files. The output histogram files from ntCard are about few hundred bytes.
From your script I see you're using .gz files as inputs. So, the issue may be related to OS TEMP space related to gzip processes. Can you change your TMPDIR
to somewhere with enough space such as /var/tmp
?
export TMPDIR=/var/tmp
Another solution could be reducing the number of processes in your script, i.e. $SLURM_NPROCS
.
from ntcard.
Hmm. Interesting. The problem cannot be the TMPDIR since this is set to use node scratch disk (/scratch/<job_id>) and that has quite a bit of space.
What is the reasoning behind using reducing the number of cores used?
from ntcard.
I've tried some other things too now (including setting TMPDIR), like cd'ing to the output directory, symlinking the files to the output directory, but I'm still not getting output. The cluster hasn't reported an error this time either, but the weird thing is the absence of output now. All the input files definitely exist, and are not broken symlinks.
Is there supposed to be more written to the screen than:
Runtime(sec): 4638.3723
real 77m18.667s
user 262m28.935s
sys 14m13.065s
?
from ntcard.
What is the reasoning behind using reducing the number of cores used?
Every thread will be working on separate .gz
file in parallel. Each thread will fork a gzip process to read the fq.gz
files and the higher the number of threads, the more TMP space required for gzip processes in total.
Is there supposed to be more written to the screen than:
I just realized you haven't specified the value(s) of k
in your script. Please include it by adding -k
option. For example for k=64
use:
time ntcard -k 64 -t $SLURM_NPROCS -p spruce_freq $FQDIR/*.fq.gz
By default the outputs will be written on freq_k$k.hist
in the CWD
. In your script you have specified spruce_freq
as prefix for outputs, so you should see spruce_freq_k64.hist
in the current or specified working directory.
from ntcard.
Thank you Hamid,
After including the -k option, it now works. The output is now there.
Thanks again for puzzling this through with me.
Regards,
Mahesh.
from ntcard.
Related Issues (20)
- ntcard file.gbk fails, but file.gff works? HOT 6
- K-mers with odd frequency having 0 counts HOT 9
- Support Uracil nucleotides in input files HOT 5
- Reg: kmer dump generation HOT 1
- Support for U bases not working properly HOT 2
- ntcard run error: Abort trap 6 HOT 3
- ntCard report interpretation HOT 3
- v1.2.0 does not work with FASTA files HOT 4
- Compilation blocked because of Werror HOT 1
- Jagged kmer coverage profiles with gzipped FASTA HOT 3
- Setting the verbosity arguments HOT 3
- Integer overflow on calculating kmer distribution HOT 5
- ntcard with lrz compressed files HOT 1
- MACOS problem HOT 2
- Problems w.r.t speed of ntCard and accuracy in genome size estimation HOT 8
- integer underflow bug? HOT 6
- Use of read names to determine input file type
- Do you plan to upgrade to latest nthash
- dyld: cannot load Abort trap: 6
- Can't build from source HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ntcard.