rkajitani / metaplatanus Goto Github PK

View Code? Open in Web Editor NEW

12.0 12.0 1.0 18.58 MB

De novo metagenome assembler

License: GNU General Public License v3.0

Makefile 0.12% Perl 2.00% Shell 1.32% C++ 89.13% Python 4.47% C 2.86% Dockerfile 0.01% Vim Script 0.09%

metaplatanus's People

Contributors

Stargazers

Watchers

Forkers

sarangian

metaplatanus's Issues

metaplatanus runs forever

Hi,

I'm trying metaplatanus (version 1.3.0) on an AWS instance with 4 cores for now 32 hours and I'm still stuck at step 1. Despite providing 4 cores, I observed that most of the time only one core is used. I use 11*2 Gb Illumina reads and 2.2 Gb Nanopore reads.

Do you have an idea of how long the run could take? Would it be possible to have a verbose option to have a better idea of where the current software is and what it's doing?

Regards,
JS

How to set memory

My data illumina 2X250bp 930M; pacbio 695M.

When I use the default value of memory, metaplatanus were crashed.

out.assembleLog:

MetaPlatanus version: 1.3.0
metaplatanus_core assemble -tmp . -c 2 -C 4 -t 30 -m 756 -f /prj/hybrid-assembly/kang/process/hybrid/real_data/nwcs/metaplatanus/r1.fastq /prj/hybrid-assembly/kang/pr>

AVE_READ_LEN=171.854
K = 25, saving kmers from reads...

PROCESS INFORMATION

VmPeak: 199.949 GByte
VmHWM: 3.771 GByte
26.54user 162.73system 3:19.48elapsed 94%CPU (0avgtext+0avgdata 203953792maxresident)k
0inputs+1839896outputs (0major+51022101minor)pagefaults 0swaps

When I use 80G memory, metaplatanus were crashed at the second step iterative assembly.

out.iteLog:

MetaPlatanus version: 1.3.0
metaplatanus_core iterate -tmp . -c initial_merged.fa initial_mergedJunctionKmer.fa -k out_kmer_occ.bin -t 30 -m 80 -IP1 /prj/hybrid-assembly/kang/process/hybrid/real>

Error(16): Divide exception!!
meta_platanus divide cannot finished correctly
Command exited with non-zero status 16
0.58user 0.10system 0:00.54elapsed 127%CPU (0avgtext+0avgdata 21652maxresident)k
0inputs+17776outputs (0major+40912minor)pagefaults 0swaps

ioctl error with conda version

Dear Kajitani,

After reading the paper and the great performance of this software I am trying to use MetaPlatanus for an hybrid assembly of human gut metagenomes. I installed the conda version in a newly created environment following the instructions in a CentOS machine. Then I tried a run with the sample data...

metaplatanus -IP1 ./UPLOAD/PE_1.fq ./UPLOAD/PE_2.fq -ont ./UPLOAD/ONT.fq --o out > log 2>&1

etaplatanus version v1.3.0
Error: contig_assembly error!
Inappropriate ioctl for device

ls out_intermediate/1_contig_asm/
cmd.bash  cmd.stderr  cmd.stdout  out.assembleLog

The out.assembleLog says something like no folder here in Spanish due to my installation.
cmd.bash: línea 10: /usr/bin/time: No existe el fichero o el directorio
Is there a way to fix this? Any help would be appreciated.

Thanks for your help

Andrés

Scaffold exception

Dear authors,

Thanks for your great work! I am trying this method for hybrid assembling our paired-end reads and Pacbio long reads. However, I encountered the error after having run for 3 hours:

$ cat out_intermediate/3_iterative_asm/out.iteLog 
MetaPlatanus version: 1.3.1
metaplatanus_core iterate -tmp . -c initial_merged.fa initial_mergedJunctionKmer.fa -k out_kmer_occ.bin -t 64 -m 500 -IP1 /Data/5G_1.fq.gz /Data/5G_2.fq.gz -p /Data/S1_1.2G.fastq -o out 

Error(13): Error, Scaffold exception!!
meta_platanus scaffold command failed.

Do you have any idea how to solve this error?
Thank you!

Read file exception!!

Dear Rkajitani,
I'm interested in your MetaPlatanus. Recently I run it and encouter a issue.

Command: metaplatanus -IP1 r1.fq r2.fq -ont long_10x_2.fq > log.txt 2>&1
MetaPlatanus version: 1.3.0
Error(2): Error, Read file exception!!
Read file exception!!
Read file is not FASTA/FASTQ format.
Command exited with non-zero status 2
1.63user 0.43system 0:02.07elapsed 99%CPU (0avgtext+0avgdata 19988maxresident)k
0inputs+927960outputs (0major+28471minor)pagefaults 0swaps

My paired-end reads:
r1.fq:

r2.fq:

Do you know how to address the problem? Thank you very much!

either compiling OR using a conda recipe I get the same error: Inappropriate ioctl for device error in the end

I summarized part of my attempts to install and use MetaPlatanus here but in order to be more concrete here are two ways I tried to solve this.

ATTEMPTING TO COMPILE ON MY OWN

I used mamba and installed all the dependencies in a new environment.
I manually installed NextPolish, I added the path to .bashrc file and I checked that it worked.

mamba create --name platanus
conda activate platanus
mamba install -c bioconda samtools tgsgapcloser bwa metabat2 seqkit minimap2 racon
pip install paralleltask
unizp master.zip
cd Metaplatanus
make
cp ./sub_bin/* ~/.local/miniconda/envs/platanus/bin/
cp metaplatanus  ~/.local/miniconda/envs/platanus/bin/

Here is the environment

_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
bwa                       0.7.17               h7132678_9    bioconda
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.18.1               h7f98852_0    conda-forge
ca-certificates           2021.10.8            ha878542_0    conda-forge
htslib                    1.15                 h9753748_0    bioconda
k8                        0.2.5                hd03093a_2    bioconda
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.19.2               h3790be6_4    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
libcurl                   7.81.0               h2574ce0_0    conda-forge
libdeflate                1.10                 h7f98852_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 11.2.0              h1d223b6_13    conda-forge
libgomp                   11.2.0              h1d223b6_13    conda-forge
libnghttp2                1.47.0               h727a467_0    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libssh2                   1.10.0               ha56f1ee_2    conda-forge
libstdcxx-ng              11.2.0              he4da1e4_13    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libzlib                   1.2.11            h36c2ea0_1013    conda-forge
metabat2                  2.12.1                        1    bioconda
minimap2                  2.24                 h7132678_1    bioconda
ncurses                   6.3                  h9c3ff4c_0    conda-forge
openssl                   1.1.1l               h7f98852_0    conda-forge
paralleltask              0.1.1                    pypi_0    pypi
perl                      5.32.1          2_h7f98852_perl5    conda-forge
pip                       22.0.3             pyhd8ed1ab_0    conda-forge
psutil                    5.9.0                    pypi_0    pypi
python                    3.10.2          h85951f9_3_cpython    conda-forge
python_abi                3.10                    2_cp310    conda-forge
racon                     1.4.20               hd03093a_2    bioconda
readline                  8.1                  h46c0cb4_0    conda-forge
samtools                  1.15                 h1170115_1    bioconda
seqkit                    2.1.0                h9ee0642_0    bioconda
setuptools                60.9.3          py310hff52083_0    conda-forge
sqlite                    3.37.0               h9cd32fc_0    conda-forge
tgsgapcloser              1.0.3                h5b5514e_2    bioconda
tk                        8.6.12               h27826a3_0    conda-forge
tzdata                    2021e                he74cb21_0    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
zlib                      1.2.11            h36c2ea0_1013    conda-forge

That didn't work I got an error with tgsgapcloser_mod but even when I used metaplatanus -IP1 PE_1.fq PE_2.fq -ont ONT.fq -no_tgsgapcloser -o ZERO

ATTEMPTING TO CREATE A CONDA ENVIRONMENT AND USING A CONDA RECIPE

mamba create --name oldplatanus
conda activate oldplatanus
mamba install -c conda-forge -c bioconda metaplatanus

Here is the conda list

_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
bwa                       0.7.17               h7132678_9    bioconda
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.18.1               h7f98852_0    conda-forge
ca-certificates           2021.10.8            ha878542_0    conda-forge
gzip                      1.11                 h7f98852_0    conda-forge
htslib                    1.10.2               hd3b49d5_1    bioconda
k8                        0.2.5                hd03093a_2    bioconda
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.19.2               h3790be6_4    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
libcurl                   7.81.0               h2574ce0_0    conda-forge
libdeflate                1.6                  h516909a_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 11.2.0              h1d223b6_13    conda-forge
libgomp                   11.2.0              h1d223b6_13    conda-forge
libnghttp2                1.47.0               h727a467_0    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libssh2                   1.10.0               ha56f1ee_2    conda-forge
libstdcxx-ng              11.2.0              he4da1e4_13    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libzlib                   1.2.11            h36c2ea0_1013    conda-forge
llvm-openmp               8.0.1                hc9558a2_0    conda-forge
metabat2                  2.15                 h986a166_1    bioconda
metaplatanus              1.3.0                hf1761c0_1    bioconda
minimap2                  2.24                 h7132678_1    bioconda
ncurses                   6.2                  h58526e2_4    conda-forge
openmp                    8.0.1                         0    conda-forge
openssl                   1.1.1l               h7f98852_0    conda-forge
perl                      5.32.1          2_h7f98852_perl5    conda-forge
pip                       22.0.3             pyhd8ed1ab_0    conda-forge
psutil                    5.9.0           py310h6acc77f_0    conda-forge
python                    3.10.2          h62f1059_0_cpython    conda-forge
python_abi                3.10                    2_cp310    conda-forge
racon                     1.4.20               hd03093a_2    bioconda
readline                  8.1                  h46c0cb4_0    conda-forge
samtools                  1.10                 h2e538c0_3    bioconda
seqkit                    2.1.0                h9ee0642_0    bioconda
setuptools                60.9.3          py310hff52083_0    conda-forge
sqlite                    3.37.0               h9cd32fc_0    conda-forge
tgsgapcloser              1.0.3                h5b5514e_2    bioconda
tk                        8.6.12               h27826a3_0    conda-forge
tzdata                    2021e                he74cb21_0    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
zlib                      1.2.11            h36c2ea0_1013    conda-forge

I get the same error

Error: contig_assembly error!
Inappropriate ioctl for device

I am trying to solve this in order to assemble several metagenomes. Could you help me to resolve this? I can provide more logs or attempt more things just let me know. Thanks for your help.

High memory usage

when I only used a 15G*2 PE data to run MetaPlatanus without any naonpore or PacBio data, a high High memory usage (nearly 500 G) occurred, than MetaPlatanus crashed (cpu=0, memory=99%). I have no naonpore or PacBio data, I want to try whether MetaPlatanus provide more nice assembly result than Metaspades. This is my log
out.assembleLog.txt

Inappropriate ioctl for device

Hi,

I would like to use MetaPlatanus to assemble my metagenome sample. However, when I run MetaPlatanus using the perl wrapper, I get the following error.
How can I run correctly?
My os is ubuntu18.04 with AMD threadripper 3990x installed.
MetaPlatanus Linux binary was downloaded from HP (http://platanus.bio.titech.ac.jp/metaplatanus)

$ perl meta_platanus.pl cons_asm --IP1 pair_R1.fq pair_R2.fq --o outprefix --t 20
Inappropriate ioctl for device

Readme typo

Hi,

in the conda instruction:

conda install -c conda-forge -c bioconda mataplatanus

should be metaplatanus.

Regards,
JSG

Some clarification about MetaPlatanus pipeline

Dear Devs,

Thank you for MetaPlatanus, it is a nice addition to the few pipeline out there that use both Illumina and long-reads for metagenome assembly. I have a question regarding the use of NextPolish after the binning step with MetaBAT2.

Given the high error rate of long-reads, especially ONT reads, this could, in theory, affect the TNF of some scaffolds and therefore, MetaBAT binning. For this reason, wouldn't be more advisable running the polishing step after the scaffolding step?

Thank you!

metaplatanus fails in 3_binning (Error: 256 binning (metabat2) error!)

Hello!

I hope you are well. I am trying to do a hybrid metagenome assembly using an Illumina 2x150bp library and an ONT library. I am unsure why, but the assembly consistently fails on step 3, throwing an error to the log file that reads:

`metaplatanus version v1.3.0

Error: 256 binning (metabat2) error!
File exists`

My installation is directly from bioconda (following instructions from the readme) in a clean environment. It seems others have not had this issue with the latest version of metaplatanus so I am not quite sure what is causing this. I am running this on my university's PBS Torque system and am running exclusively on the open allocation. Therefore, I run into walltime restraints and resume the job whenever walltime limits are reached and I am wondering if perhaps resuming the assembly is causing issues with files already existing. Do you have any recommendations on things to try to resolve this?

Any help would be much appreciated!

-Nick

Error: contig_assembly error! Function not implemented

Hi!

MetaPlatanus was installed with conda as described in manual without any issues.

I used PE 2x200bp Illumina reads and run ended with error.
Here is log file content:

Error: contig_assembly error!
Function not implemented

used command:

metaplatanus -IP1 BFR_R1_paired.fq BFR_R2_paired.fq -o metaplat_out -t 10 >log.txt 2>&1

also here is log file from folder metaplat_out_intermediate
metaplat_out.assembleLog.txt

How to resolve it?
Thanks!

Inappropriate ioctl for device on docker

Hi,

I use metaplatanus v1.3.0 by docker and got the same issue of #1 (comment)

docker run -v $PWD:/data --rm rkajitani/metaplatanus metaplatanus -IP1 R1.fastq.gz R2.fastq.gz -p pacbio.fasta --o out -t 50 > log 2>&1

Could you solve the problem?

Yuya