diltheylab / metamaps Goto Github PK

View Code? Open in Web Editor NEW

96.0 96.0 23.0 778 KB

Long-read metagenomic analysis

License: Other

Perl 56.53% Makefile 0.17% Shell 0.01% M4 0.15% R 9.49% C++ 31.74% C 1.91%

metamaps's People

Contributors

Stargazers

Watchers

metamaps's Issues

Typo in help for command "index"

There is a typo for metamaps index -h

It says:

--maxmemory <value>, --mm <value>
    maximum memory, in GB [default e-03]

The [default e-03] is from the p-value parameter.

Error when creating database

Hi. I run the downloadRefSeq.pl command -- downloadRefSeq.pl --seqencesOutDirectory data/metamaps-db/refseq --taxonomyOutDirectory data/metamaps-db/taxonomy, and after about two days of churning data and printing progress output, it just failed with "Cannot change working directory into assembly path na na: No such file or directory" and no other explanation. It had successfully processed all bacterial genomes but only got through 5 out of 323 fungal genomes. Looking into the data/metamaps-db/refseq/fungi dir, I actually see only six subdirectories for six species. assembly_summary.txt lists a lot more. I have about 20TB free disk space left, so it can't be that.

Does it mean that some previous data retrieval steps failed? Is there a way to safeguard against this? Or fix it and resume from where it left off?

Dropbox download issue

Hello,

thank you for the tool - seems very useful. Dropbox must have reached bandwidth limit since download keeps failing. Thought I'd give you a heads-up.

Cheers

metamaps: src/meta/taxonomy.h:229: meta::taxonomy::taxonomy(std::__cxx11::string): Assertion `T.count(id) == 0' failed

After mapping the reads to database, I used command "metamaps classify --mappings classification_results --DB databases/miniSeq+H "to classify them . While there is an error . How should I do to solve the problem?

meta::getTaxonIDsFromMappingsFile(std::string): Assertion `mappingsStream.is_open()' failed

Hello,

I ran metamaps mapDirectly, and get the output.x files. I am not sure what to expect from this first step, but the .output file (without .1 or .x) is sometimes missing, and this seems a first issue discussed here: #7 (comment)

In the second step, i am getting the following error. Commands:

conda activate metamap-0.1
metamaps classify --threads 12 --DB /path/to/miniSeq+H --mappings /path/to/file/nanopore_file.metamaps_standard_output

Error:

metamaps: src/meta/fEM.h:1362: std::set<std::basic_string<char> > meta::getTaxonIDsFromMappingsFile(std::string): Assertion `mappingsStream.is_open()' failed.
/.../source_activate.sh: line 36: 213339 Aborted                 (core dumped) "$@"

Could you help on this issue ?
Thank you in advance,
Sylvain

Installation/Command Lines

Just to let you know, running make metamaps does not work. make install alone does, but the executable isn't ./metamaps, it makes the executable of mashmap

Can the software help me construct my own databases only with my own genome data?

Hello,
I am very excited that you have made a great software with great algorithm. While, may I build the database with my data only? if can do this ? What file should I input ?

Thanks
tenlives

Error during second step

Hi, I got the following error during the second step. I'm using the downloaded miniSeq+H database

Outputting mappings with adjusted alignment qualities. mashmap: src/meta/taxonomy.h:64: std::string meta::taxonomy::getFirstNonXNode(std::string): Assertion nodeID.find("x") == std::string::npos' failed. Aborted (core dumped)

Command Lines:
mashmap mapDirectly --all -r miniSeq+H/DB.fa -q sample1.fastq -o sample1_metamaps_classification mashmap classify --mappings sample1_metamaps_classification --DB miniSeq+H

Update License

The license file of the project needs minor updating :)

Problems integrating MetaMaps with Boost

There is a problem with where MetaMaps expects the Boost components to be. If Boost is installed according to the steps given in https://www.boost.org/doc/libs/1_72_0/more/getting_started/unix-variants.html (and don’t forget step 5.1, Easy Build and Install), the resulting folder structure is different from what MetaMaps expects.

Suppose Boost is installed in /usr/local/boost_1_72_0. The include folder appears in /usr/local/boost_1_72_0/boost/include, but MetaMaps expects it in /usr/local/boost_1_72_0/include. If the include folder is copied to where MetaMaps expects it, this resolves that issue.

However, MetaMaps expects the lib folder to appear in /usr/local/boost_1_72_0, but it appears in /usr/local/boost_1_72_0/stage. After the lib folder is copied to where MetaMaps expects it, then ‘make metamaps’ finds it. However, when you now try to run MetaMaps, the following error occurs:

./metamaps: error while loading shared libraries: libboost_filesystem.so.1.72.0: cannot open shared object file: No such file or directory

The libboost files are in the /usr/local/boost_1_72_0/lib folder, but MetaMaps does not find it there. Is there a way to install and run MetaMaps to avoid these contortions?
Where does MetaMaps expect to find the libboost files?

Feature Request: Compiled executable and Conda package

Feature Request
It would be really nice to have a compiled executable for linux as well as a conda package for MetaMaps. MetaMaps is challenging to install on a managed server without su privilege. I think a side-effect to easier installation would be a larger user base. Great work and thanks!

Informations on output needed

Hello !

Is it possible to have somewhere an example of an output file produced by metamaps ? Or does it already exist somwhere and I couldn't reach it ? I would like to know what does the output look like and how I could visualize it.

Thanks a lot !

Roxane

Mapping step spent very long time is normal?

Hi, everyone
I used MetaMaps to mapping, and it spent a large time. My dataset is below:
reference: databases/miniSeq+H/DB.fa
query: two_reads.fasta

maxmemory: 50
threads: 20

My Query is only two reads, but mapping time needs ten hours. Is it correct and normal?

Thanks for any advice on this!

Installation failed

I just tried to install MetaMaps on my Ubuntu 18.04 computer but it failed, the last installation message I can get from the console is:

g++ -O2 -ggdb -fopenmp -std=c++11 -Isrc -L /opt/boost_1_68/lib -I /opt/boost_1_68/include  -include src/common/memcpyLink.h -Wl,--wrap=memcpy -DUSE_BOOST src/map/mash_map.o   -o metamaps /opt/boost_1_68/lib/libboost_math_c99.a -lstdc++ -fopenmp -lz -lm -lpthread -lboost_system -lboost_filesystem -lboost_serialization -lboost_regex
src/map/mash_map.o: In function `boost::re_detail_106501::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::unwind_extra_block(bool)':
/usr/include/boost/regex/v4/perl_matcher_non_recursive.hpp:1352: undefined reference to `boost::re_detail_106501::put_mem_block(void*)'
src/map/mash_map.o: In function `boost::system::error_category::std_category::equivalent(std::error_code const&, int) const':
/usr/include/boost/system/error_code.hpp:686: undefined reference to `boost::system::generic_category()'
/usr/include/boost/system/error_code.hpp:689: undefined reference to `boost::system::generic_category()'
/usr/include/boost/system/error_code.hpp:701: undefined reference to `boost::system::generic_category()'
src/map/mash_map.o: In function `boost::system::error_category::std_category::equivalent(int, std::error_condition const&) const':
/usr/include/boost/system/error_code.hpp:656: undefined reference to `boost::system::generic_category()'
/usr/include/boost/system/error_code.hpp:659: undefined reference to `boost::system::generic_category()'
src/map/mash_map.o:/usr/include/boost/system/error_code.hpp:206: more undefined references to `boost::system::generic_category()' follow
src/map/mash_map.o: In function `_GLOBAL__sub_I__Z5errExRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE':
/usr/include/boost/system/error_code.hpp:210: undefined reference to `boost::system::system_category()'
src/map/mash_map.o: In function `void boost::re_detail_106501::raise_error<boost::regex_traits_wrapper<boost::regex_traits<char, boost::cpp_regex_traits<char> > > >(boost::regex_traits_wrapper<boost::regex_traits<char, boost::cpp_regex_traits<char> > > const&, boost::regex_constants::error_type)':
/usr/include/boost/regex/v4/cpp_regex_traits.hpp:449: undefined reference to `boost::re_detail_106501::get_default_error_string(boost::regex_constants::error_type)'
src/map/mash_map.o: In function `void boost::re_detail_106501::raise_error<boost::regex_traits_wrapper<boost::regex_traits<char, boost::cpp_regex_traits<char> > > >(boost::regex_traits_wrapper<boost::regex_traits<char, boost::cpp_regex_traits<char> > > const&, boost::regex_constants::error_type)':
/usr/include/boost/regex/pattern_except.hpp:75: undefined reference to `boost::re_detail_106501::raise_runtime_error(std::runtime_error const&)'
src/map/mash_map.o: In function `__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > boost::re_detail_106501::re_is_set_member<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, char, boost::regex_traits<char, boost::cpp_regex_traits<char> >, unsigned int>(__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::re_detail_106501::re_set_long<unsigned int> const*, boost::re_detail_106501::regex_data<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > > const&, bool)':
/usr/include/boost/regex/v4/cpp_regex_traits.hpp:966: undefined reference to `boost::re_detail_106501::cpp_regex_traits_implementation<char>::transform_primary[abi:cxx11](char const*, char const*) const'
/usr/include/boost/regex/v4/cpp_regex_traits.hpp:962: undefined reference to `boost::re_detail_106501::cpp_regex_traits_implementation<char>::transform[abi:cxx11](char const*, char const*) const'
src/map/mash_map.o: In function `boost::re_detail_106501::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::extend_stack()':
/usr/include/boost/regex/v4/perl_matcher_non_recursive.hpp:222: undefined reference to `boost::re_detail_106501::get_mem_block()'
src/map/mash_map.o: In function `boost::re_detail_106501::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::find_imp()':
/usr/include/boost/regex/v4/perl_matcher_non_recursive.hpp:107: undefined reference to `boost::re_detail_106501::get_mem_block()'
src/map/mash_map.o: In function `boost::re_detail_106501::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::find_imp()':
/usr/include/boost/regex/v4/perl_matcher_common.hpp:300: undefined reference to `boost::re_detail_106501::verify_options(unsigned int, boost::regex_constants::_match_flags)'
src/map/mash_map.o: In function `boost::re_detail_106501::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::find_imp()':
/usr/include/boost/regex/v4/perl_matcher_non_recursive.hpp:115: undefined reference to `boost::re_detail_106501::put_mem_block(void*)'
/usr/include/boost/regex/v4/perl_matcher_non_recursive.hpp:115: undefined reference to `boost::re_detail_106501::put_mem_block(void*)'
src/map/mash_map.o: In function `bool boost::regex_search<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >, char, boost::regex_traits<char, boost::cpp_regex_traits<char> > >(__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::match_results<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > >&, boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > > const&, boost::regex_constants::_match_flags, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >)':
/usr/include/boost/regex/v4/perl_matcher.hpp:383: undefined reference to `boost::re_detail_106501::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::construct_init(boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > > const&, boost::regex_constants::_match_flags)'
collect2: error: ld returned 1 exit status
Makefile:20: recipe for target 'metamaps' failed
make: *** [metamaps] Error 1

Can someone help me? Thanks!

Problem at the classify step

Hello there,

I have been having some issues running the second step.

here is the command I enter:
/home/gurzil/MetaMaps/metamaps classify\ --mappings mapDirectly/classification_results\ --DB /home/gurzil/MetaMaps/databases/myDB/DB.fa\ -t 5

and here are the 2 last lines of the output:
metamaps: src/meta/fEM.h:402: std::map<std::__cxx11::basic_string<char>, long unsigned int> meta::getMappingStats(std::__cxx11::string): Assertion statsStream.is_open()' failed.
/var/spool/slurmd/job11323153/slurm_script: line 12: 107872 Aborted (core dumped) /home/...`

I realize the issue is similar to the one in https://github.com/DiltheyLab/MetaMaps/issues/16 but from what i gather the first step seems to be running smoothly and i have plenty of memory.

Thanks!

mapAgainstIndex frozen

I indexed miniseq and am now trying to map against my index. I am aware this is not the recommended approach, however I am trying to save some time (#36). MetaMaps is unfortunately getting frozen at the mapping stage:

It hangs at this screen for hours at 0% CPU, 144GB RES ram. The mapDirectly function works no problem.
My index command is:
~/Programs/MetaMaps/metamaps index -r databases/miniSeq+H/DB.fa -i metamaps_mini --mm 140 -t 30
and it appears to succeed too.

downloadRefSeq.pl fails with error message

Hi,

I am running the following command from the README to build a database with bacteria from RefSeq:

perl downloadRefSeq.pl --DB refseq --seqencesOutDirectory download/refseq --taxonomyOutDirectory download/taxonomy --targetBranches bacteria --skipIncompleteGenomes 1

It downloads thousands of genomes and then suddenly exits with Can't use an undefined value as a symbol reference at /usr/share/perl5/Net/FTP/dataconn.pm line 54.

I have tried it multiple times but if always fails at some point.

Classify is killed during EM round 0

I don't know if MetaMaps is still being maintained, but I'm currently trying out different Metagenomic analysis software.

Anyway, I'm getting following error without any further explanation during the "Classify" step (the "mapDirectly" step is working flawless):

>>>>>>>>>>>>>>>>>>
DB = database_plastid/MetaMaps_db/database
Mappings = TRNLa_M000197_metamaps_result
Min. reads for 'U' = 10000
Threads = 10
>>>>>>>>>>>>>>>>>>
Read taxonomy from database_plastid/MetaMaps_db/database/taxonomy -- have 9503 nodes.
Starting EM...
EM round 0
Killed

How can I find more information on what went wrong? It might have to do something with the size of the dataset I'm using (memory consumption?). When I'm using only 10% of the reads (and the same database, same settings), it works fine.

Any ideas?

MetaMaps configure issue

Hi,

I am trying to compile MetaMaps on a linux server (CentOS Linux). I use miniconda to manage packages, therefore, installed boost library via conda (https://anaconda.org/anaconda/boost). I could locate the path to my boost library and provided when configure the program:

./configure --with-boost=/path/to/boost/ -- --prefix=/path/to/bin/

However, it still complains with error messages:

checking boost/math/distributions/binomial.hpp usability... no
checking boost/math/distributions/binomial.hpp presence... no
checking for boost/math/distributions/binomial.hpp... no
configure: error: Boost Library headers not found.

Could you help me to resolve the issue?

Many thanks!
Ding

Assertion error in EM

I've encountered the following error while running metamaps classify for the very first time:

Starting EM...
metamaps: src/meta/fEM.h:500: void meta::doEM(const skch::Parameters&, const string&): Assertion `abs(1 - f_sum) <= 1e-3' failed.
Aborted (core dumped)

Not sure what to make of that.

Error in classify step

Hello,
I am testing your tool and I get this error when I run:
./metamaps classify --mappings classification_results.1 --DB databases/miniSeq+H

The error is as follows:
Min. reads for 'U' = 10000

metamaps: src/meta/fEM.h:350: std::map<std::__cxx11::basic_string, long unsigned int> meta::getMappingStats(std::__cxx11::string): Assertion `statsStream.is_open()' failed.

Thank you in advance!

Nathaly

downloadRefSeq.pl fails

Hi,

I'd like to make my database, but downloadRefSeq.pl does not go to the end. I used the following commands to download my purpose species from NCBI refseq database.

downloadRefSeq.pl --targetBranches archaea,bacteria,fungi,protozoa,viral,plasmid --seqencesOutDirectory ./refseq --taxonomyOutDirectory ./taxonomy

I get an error while dealing with that of bacteria.
Copy from just before the error output.

Net::FTP=GLOB(0x4875ea0)<<< 226 Transfer complete
Genome 893 / 204017 ; species 189 / 53849 bacteria (Streptomyces_sampsonii) -- version 2 / 2: GET GCF_001865315.1_ASM186531v1_assembly_report.txt Net::FTP=GLOB(0x4875ea0)>>> PASV
Net::FTP=GLOB(0x4875ea0)<<< 227 Entering Passive Mode (130,14,250,10,195,140).
Net::FTP=GLOB(0x4875ea0)>>> RETR GCF_001865315.1_ASM186531v1_assembly_report.txt
Net::FTP=GLOB(0x4875ea0)<<< 150 Opening BINARY mode data connection for GCF_001865315.1_ASM186531v1_assembly_report.txt (2057 bytes)
Net::FTP=GLOB(0x4875ea0)<<< 226 Transfer complete
Net::FTP=GLOB(0x4875ea0)>>> CWD /genomes/all/GCF/000/623/015/GCF_000623015.1_Acin_baum_LAC-4_V1
Net::FTP=GLOB(0x4875ea0)<<< 250 CWD command successful
Net::FTP=GLOB(0x4875ea0)>>> PASV
Net::FTP=GLOB(0x4875ea0)<<< 227 Entering Passive Mode (130,14,250,10,196,68).
Net::FTP=GLOB(0x4875ea0)>>> NLST
Net::FTP=GLOB(0x4875ea0)<<< 150 Opening BINARY mode data connection for file list
Net::FTP=GLOB(0x4875ea0)<<< 226 Transfer complete
Genome 894 / 204017 ; species 190 / 53849 bacteria (Acinetobacter_baumannii_LAC_4) -- version 1 / 2: GET GCF_000623015.1_Acin_baum_LAC-4_V1_genomic.fna.gz Net::FTP=GLOB(0x4875ea0)>>> PASV
Net::FTP=GLOB(0x4875ea0)<<< 227 Entering Passive Mode (130,14,250,10,197,40).
Net::FTP=GLOB(0x4875ea0)>>> RETR GCF_000623015.1_Acin_baum_LAC-4_V1_genomic.fna.gz
Net::FTP=GLOB(0x4875ea0)<<< 150 Opening BINARY mode data connection for GCF_000623015.1_Acin_baum_LAC-4_V1_genomic.fna.gz (1169068 bytes)
Net::FTP=GLOB(0x4875ea0)<<< 226 Transfer complete
Genome 894 / 204017 ; species 190 / 53849 bacteria (Acinetobacter_baumannii_LAC_4) -- version 1 / 2: GET GCF_000623015.1_Acin_baum_LAC-4_V1_genomic.gff.gz Net::FTP=GLOB(0x4875ea0)>>> PASV
Net::FTP=GLOB(0x4875ea0)<<< 227 Entering Passive Mode (130,14,250,10,196,20).
Net::FTP=GLOB(0x4875ea0)>>> RETR GCF_000623015.1_Acin_baum_LAC-4_V1_genomic.gff.gz
Net::FTP=GLOB(0x4875ea0)<<< 150 Opening BINARY mode data connection for GCF_000623015.1_Acin_baum_LAC-4_V1_genomic.gff.gz (269654 bytes)
Net::FTP=GLOB(0x4875ea0)<<< 226 Transfer complete
Genome 894 / 204017 ; species 190 / 53849 bacteria (Acinetobacter_baumannii_LAC_4) -- version 1 / 2: GET GCF_000623015.1_Acin_baum_LAC-4_V1_protein.faa.gz Net::FTP=GLOB(0x4875ea0)>>> PASV
Net::FTP=GLOB(0x4875ea0)<<< 227 Entering Passive Mode (130,14,250,10,196,62).
Net::FTP=GLOB(0x4875ea0)>>> RETR GCF_000623015.1_Acin_baum_LAC-4_V1_protein.faa.gz
Net::FTP=GLOB(0x4875ea0)<<< 150 Opening BINARY mode data connection for GCF_000623015.1_Acin_baum_LAC-4_V1_protein.faa.gz (735401 bytes)
Net::FTP=GLOB(0x4875ea0)<<< 226 Transfer complete
Genome 894 / 204017 ; species 190 / 53849 bacteria (Acinetobacter_baumannii_LAC_4) -- version 1 / 2: GET GCF_000623015.1_Acin_baum_LAC-4_V1_assembly_report.txt Net::FTP=GLOB(0x4875ea0)>>> PASV
Net::FTP=GLOB(0x4875ea0)<<< 227 Entering Passive Mode (130,14,250,10,196,130).
Net::FTP=GLOB(0x4875ea0)>>> RETR GCF_000623015.1_Acin_baum_LAC-4_V1_assembly_report.txt
Net::FTP=GLOB(0x4875ea0)<<< 150 Opening BINARY mode data connection for GCF_000623015.1_Acin_baum_LAC-4_V1_assembly_report.txt (2461 bytes)
Net::FTP=GLOB(0x4875ea0)<<< 226 Transfer complete
Net::FTP=GLOB(0x4875ea0)>>> CWD /genomes/all/GCF/000/786/735/GCF_000786735.1_ASM78673v1
Net::FTP=GLOB(0x4875ea0)<<< 250 CWD command successful
Net::FTP=GLOB(0x4875ea0)>>> PASV
Net::FTP=GLOB(0x4875ea0)<<< 227 Entering Passive Mode (130,14,250,10,197,65).
Net::FTP=GLOB(0x4875ea0)>>> NLST
Net::FTP=GLOB(0x4875ea0)<<< 150 Opening BINARY mode data connection for file list
Net::FTP=GLOB(0x4875ea0)<<< 226 Transfer complete
Genome 895 / 204017 ; species 190 / 53849 bacteria (Acinetobacter_baumannii_LAC_4) -- version 2 / 2: GET GCF_000786735.1_ASM78673v1_genomic.fna.gz Net::FTP=GLOB(0x4875ea0)>>> PASV
Net::FTP=GLOB(0x4875ea0)<<< 227 Entering Passive Mode (130,14,250,10,196,160).
Net::FTP=GLOB(0x4875ea0)>>> RETR GCF_000786735.1_ASM78673v1_genomic.fna.gz
Net::FTP=GLOB(0x4875ea0)<<< 150 Opening BINARY mode data connection for GCF_000786735.1_ASM78673v1_genomic.fna.gz (1175088 bytes)
Net::FTP=GLOB(0x4875ea0)<<< 226 Transfer complete
Genome 895 / 204017 ; species 190 / 53849 bacteria (Acinetobacter_baumannii_LAC_4) -- version 2 / 2: GET GCF_000786735.1_ASM78673v1_genomic.gff.gz Net::FTP=GLOB(0x4875ea0)>>> PASV
Net::FTP=GLOB(0x4875ea0)<<< 227 Entering Passive Mode (130,14,250,10,196,180).
Net::FTP=GLOB(0x4875ea0)>>> RETR GCF_000786735.1_ASM78673v1_genomic.gff.gz
Net::FTP=GLOB(0x4875ea0)<<< 150 Opening BINARY mode data connection for GCF_000786735.1_ASM78673v1_genomic.gff.gz (268737 bytes)
Net::FTP=GLOB(0x4875ea0)<<< 226 Transfer complete
Genome 895 / 204017 ; species 190 / 53849 bacteria (Acinetobacter_baumannii_LAC_4) -- version 2 / 2: GET GCF_000786735.1_ASM78673v1_protein.faa.gz Net::FTP=GLOB(0x4875ea0)>>> PASV
Net::FTP=GLOB(0x4875ea0)<<< 227 Entering Passive Mode (130,14,250,10,196,71).
Net::FTP=GLOB(0x4875ea0)>>> RETR GCF_000786735.1_ASM78673v1_protein.faa.gz
Net::FTP=GLOB(0x4875ea0)<<< 150 Opening BINARY mode data connection for GCF_000786735.1_ASM78673v1_protein.faa.gz (732025 bytes)
Net::FTP=GLOB(0x4875ea0)<<< 226 Transfer complete
Genome 895 / 204017 ; species 190 / 53849 bacteria (Acinetobacter_baumannii_LAC_4) -- version 2 / 2: GET GCF_000786735.1_ASM78673v1_assembly_report.txt Net::FTP=GLOB(0x4875ea0)>>> PASV
Net::FTP=GLOB(0x4875ea0)<<< 227 Entering Passive Mode (130,14,250,10,196,137).
Net::FTP=GLOB(0x4875ea0)>>> RETR GCF_000786735.1_ASM78673v1_assembly_report.txt
Net::FTP=GLOB(0x4875ea0)<<< 150 Opening BINARY mode data connection for GCF_000786735.1_ASM78673v1_assembly_report.txt (1550 bytes)
Net::FTP=GLOB(0x4875ea0)<<< 226 Transfer complete
Net::FTP=GLOB(0x4875ea0)>>> CWD /genomes/all/GCF/000/623/335/GCF_000623335.1_ASM62333v1
Net::FTP=GLOB(0x4875ea0)<<< 250 CWD command successful
Net::FTP=GLOB(0x4875ea0)>>> PASV
Net::FTP=GLOB(0x4875ea0)<<< 227 Entering Passive Mode (130,14,250,10,196,118).
Net::FTP=GLOB(0x4875ea0)>>> NLST
Net::FTP: Net::Cmd::getline(): unexpected EOF on command channel: at ../../src/MetaMaps/downloadRefSeq.pl line 258.
Net::FTP: Net::Cmd::_is_closed(): unexpected EOF on command channel: at ../../src/MetaMaps/downloadRefSeq.pl line 252.
Net::FTP: Net::Cmd::_is_closed(): unexpected EOF on command channel: at ../../src/MetaMaps/downloadRefSeq.pl line 252.
Cannot change working directory into assembly path /genomes/all/GCF/000/451/685/GCF_000451685.2_ASM45168v2 [Net::FTP] Connection closed at ../../src/MetaMaps/downloadRefSeq.pl line 255.

Error with classify command (boost-related?)

Hello,

I've compiled MetaMaps using boost installed via conda (since I don't have admin rights on the cluster). First step (mapping) worked without any issues; however, classify command fails with the following message:

DB = /pub37/alexp/MetaMaps/databases/miniSeq+H
Mappings = A.class
Min. reads for 'U' = 10000
Threads = 1

terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injectorstd::logic_error >'
what(): Attempt to access an uninitialzed boost::match_results<> class.
Aborted

Not sure how to debug it - probably related to issues mentioned in another issue, with boost directory structure not being as expected.

Installation problem: get "fatal error: boost/serialization/unordered_map.hpp: No such file or directory" after executing `make metamaps`

I was trying to install the metaMaps in my centos7 by following the instruction . Everything is fined until executing make metamaps, I got this error message:

'''g++ -c -O2 -ggdb -fopenmp -std=c++11 -Isrc -L /public2/software/boost_1_71_0/lib -I /public2/software/boost_1_71_0/include -include src/common/memcpyLink.h -Wl,--wrap=memcpy -DUSE_BOOST -o src/map/mash_map.o src/map/mash_map.cpp
In file included from src/map/mapWrap.h:28:0,
from src/map/mash_map.cpp:19:
src/map/include/winSketch.hpp:33:49: fatal error: boost/serialization/unordered_map.hpp: No such file or directory
#include <boost/serialization/unordered_map.hpp>
^
compilation terminated.
make: *** [src/map/mash_map.o] Error 1
'''

I got the same error even after installing the Autoconf 2.69.

Do you have any idea to fix this? Thanks in advance.

Error in classify step

Hi Alexander,
I got the following error in the second step of MetaMaps. The command I ran was
metamaps classify --mappings test10000.classification --DB databases/miniSeq+H

>>>>>>>>>>>>>>>>>>
DB = databases/miniSeq+H
Mappings = test10000.classification
Min. reads for 'U' = 10000
>>>>>>>>>>>>>>>>>>
Read taxonomy from databases/miniSeq+H/taxonomy -- have 17771 nodes.
Starting EM...
EM round 0
terminate called after throwing an instance of 'std::out_of_range'
  what():  stod

Thanks for any advice on this!

Symbol lookup error

I am getting this error when running metamaps and can't figure out why.

convert metamaps results to kraken style report - question

Hi, any chance can we convert metamaps results to kraken style report file?
thanks

How to download others branches using downloadRefSeq.pl

Hello there,

I would like to know if there is anyway to download data from other species than this list : "archae bacteria fungi protozoa viral" ?

For specific purpose, I would like to construct a wider database, with less species from each branche (like 1 or 2 fungi, 1 or 2 archae etc) including vertebrate species. I see in the perl code that it was somewhat planned to do so.

Line 90 :

#my @target_subdirs = qw/bacteria fungi protozoa viral invertebrate plant unknown vertebrate_other vertebrate_mammalian/;

I'm really tempted to uncomment it, but I guess it is commented in purpose. Any idea of how I could achieve that ? Is it something you are working on ?

Cheers,

Roxane

Building database and adding new genomes.

I want to add a new genome to the existing database. So far I had been using the default miniSeq database provided along the software. I have executed step 3 from the steps of construction. But I think it is required for the database to be constructed from scratch for the following step to work.
Therefore, I started from the top with step 1 and got the following output. [screenshot included] It says that 0 genomes were downloaded for the bacteria branch. And I wonder why?

Thanks in advance for the help.
Best regards
Parul

no file taxdump.tar.gz with downloadRefSeq.pl

When trying to download a reference database with the downloadRefSeq.pl script I get the following error:
No file taxdump.tar.gz in /pub/taxonomy/ on ftp.ncbi.nlm.nih.gov? at /opt/MetaMaps-master/downloadRefSeq.pl line 70.

Parameters used:
perl /opt/MetaMaps-master/downloadRefSeq.pl --seqencesOutDirectory /datadrivehdd/MetaMaps_DB/refseq --taxonomyOutDirectory /datadrivehdd/MetaMaps_DB/taxonomy --targetBranches viral

The taxdump.tar.gz file is still available on ftp.ncbi.nlm.nih.gov/pub/taxonomy/, any idea what is going wrong here?

Running Classify module then throw 'boost::wrapexcept<std::logic_error>' and 'core dump'

After finishing mapDirectly, then I try to continue classify step with the code:

metamaps classify --mappings ./metamap_classification_results --DB ./database/metamap_ecoli_homos/ecoli_homosp

And get error as showing below:

terminate called after throwing an instance of 'boost::wrapexcept<std::logic_error>'
what():  Attempt to access an uninitialized boost::match_results<> class.
[1]    26696 abort (core dumped)

the DB I used is Human+ecoli genome constructed as manual page.

In addition, during the mapDirectly step, the DB loading time is very long(about 30min) which main are human chr loading to mem, I do not know if it is fessiable to build DB with the original HG38 reference.

Extracting all representative and reference genomes from GenBank and put it into DB.

I would like to make a very extensive ref. DB for MetaMaps.

Ideally I take all genomes with RefSeq category reference genome and representative genome so that I have one genome per Species.

What are your thoughts? Is this pure madness or do you think this should work on a decent compute infrastructure?

Any thoughts on how to optimize this?

filterLowIdentityEntities.pl

I cannot find the script filterLowIdentityEntities.pl, can you tell me where to download or find the script?

Unable to buildDB

I am unable to buildDB, initially I suspected it was because the fna files were gunzipped but after un-gzipping them, a new issue arose.

I used the following commands to download a smaller version of the NCBI refseq database, as the command recommended took over 1 week without completion:

perl downloadRefSeq.pl --DB refseq --seqencesOutDirectory download/refseq --taxonomyOutDirectory download/taxonomy --targetBranches bacteria --skipIncompleteGenomes 1
perl annotateRefSeqSequencesWithUniqueTaxonIDs.pl --refSeqDirectory download/refseq --taxonomyInDirectory download/taxonomy --taxonomyOutDirectory download/taxonomy_uniqueIDs
gunzip -r download/refseq/bacteria/
perl buildDB.pl --DB databases/bacteria --FASTAs download/refseq --taxonomy download/taxonomy_uniqueIDs

And I got the following error message after adding the gunzip line:

~/MetaMaps$ perl buildDB.pl --DB databases/bacteria --FASTAs download/refseq --taxonomy download/taxonomy_uniqueIDs

Number of found FASTA input files: 2968

Reading taxonomy from download/taxonomy_uniqueIDs ..
        done.

Expect taxon ID in contig identifier - file download/refseq/bacteria/Nitratireductor_sp__SY7/GCF_007922615.2_ASM792261v2/GCF_007922615.2_ASM792261v2_genomic.fna - line 1 at /home/x/MetaMaps/perlLib/Util.pm line 53, <F> line 1.

As a side note, without the gunzip command step, I get:

Number of found FASTA input files: 0

And it fails differently.

Didn`t find th script named "filterLowIdentityEntities.pl"

Hello !
Could you provide me with the script ? I couldn`t find it in you provided files ,Thank you very much!

Best wishes !

Need to add mkdir databases to readme

Minor for sure, but the user needs to create the databases dir before executing step 4. of the Databases section in the readme.

Question: will MetaMaps work to map 18S MinION reads from a pcr product?

If I use, say, SILVA 18S database as the reference?

If yes, what would be good parameters for such mapping?
Thanks!

why is the taxonID negative?

hmp7_2_short_miniSeq+H.EM.WIMP, one of MetaMaps_example_output，I don't know why some 'taxonID' is negative and the same. some values of the 'Absolute' are also the same. Thank you!

metamaps: error while loading shared libraries: libboost_filesystem.so.1.71.0: cannot open shared object file: No such file or directory

metamaps mapDirectly --all -r databases/miniSeq+H/DB.fa -q input.fastq -o classification_results

@：metamaps: error while loading shared libraries: libboost_filesystem.so.1.71.0: cannot open shared object file: No such file or directory

I cant open metamaps（it is a application/x-sharedlib）so I dont know how to put this (libboost_filesystem.so.1.71.0)

EM not converging with default database

I am using the default minSeq+H database. I have run the following commands to classify a FASTQ file:

metamaps mapDirectly --all -r /metamaps_db/default/databases/miniSeq+H -q input.fastq -o /output_dir
metamaps classify --mappings /output_dir --DB /metamaps_db/default/databases/miniSeq+H

However, the standard output shows the following endlessly:

EM round 29017654                                                                              

        Log likelihood: 0
        Improvement: 0
        Relative   : -nan
EM round 29017655

        Log likelihood: 0
        Improvement: 0
        Relative   : -nan
EM round 29017656

        Log likelihood: 0
        Improvement: 0
        Relative   : -nan
EM round 29017657

        Log likelihood: 0
        Improvement: 0
        Relative   : -nan

It seems that the EM algorithm is not converging and/or updating.

I running this using the Metamaps BioContainer using the Docker imagequay.io/biocontainers/metamaps:0.1.98102e9--h176a8bc_0.

Make error

Hi @AlexanderDilthey,

I'm running into some installation problems that were discussed in other issues (#20), but the solutions mentioned before don't seem to be working for me.

I used a conda environment with all dependencies and I get the make error
collect2: error: ld returned 1 exit status
Here are the steps I followed:

conda create -n metamaps -c conda-forge autoconf boost zlib icu==64.2
conda activate metamaps
conda install -c anaconda gcc
git clone https://github.com/DiltheyLab/MetaMaps
cd MetaMaps
./bootstrap.sh
./configure --with-boost=$CONDA_PREFIX --prefix=$CONDA_PREFIX
LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH
make

And here is the make error message:

(metamaps2) gd98309@sapelo2-sub1 MetaMaps$ make
g++ -c -O2 -ggdb -fopenmp -std=c++11 -Isrc -L /home/gd98309/miniconda/envs/metamaps2/lib -I /home/gd98309/miniconda/envs/metamaps2/include  -include src/common/memcpyLink.h -Wl,--wrap=memcpy -DUSE_BOOST -o src/map/mash_map.o src/map/mash_map.cpp
g++ -O2 -ggdb -fopenmp -std=c++11 -Isrc -L /home/gd98309/miniconda/envs/metamaps2/lib -I /home/gd98309/miniconda/envs/metamaps2/include  -include src/common/memcpyLink.h -Wl,--wrap=memcpy -DUSE_BOOST src/map/mash_map.o   -o metamaps /home/gd98309/miniconda/envs/metamaps2/lib/libboost_math_c99.a -lstdc++ -fopenmp -lz -lm -lpthread -lboost_system -lboost_filesystem -lboost_serialization -lboost_regex
/usr/bin/ld: warning: libicudata.so.64, needed by /home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: libicui18n.so.64, needed by /home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: libicuuc.so.64, needed by /home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so, not found (try using -rpath or -rpath-link)
src/map/mash_map.o: In function `save_primitive<boost::archive::text_oarchive, std::basic_string<char> >':
/home/gd98309/miniconda/envs/metamaps2/include/boost/archive/detail/oserializer.hpp:93: undefined reference to `boost::archive::text_oarchive_impl<boost::archive::text_oarchive>::save(std::string const&)'
src/map/mash_map.o: In function `save_primitive<boost::archive::binary_oarchive, std::basic_string<char> >':
/home/gd98309/miniconda/envs/metamaps2/include/boost/archive/detail/oserializer.hpp:93: undefined reference to `boost::archive::basic_binary_oprimitive<boost::archive::binary_oarchive, char, std::char_traits<char> >::save(std::string const&)'
src/map/mash_map.o: In function `perl_matcher':
/home/gd98309/miniconda/envs/metamaps2/include/boost/regex/v4/perl_matcher.hpp:391: undefined reference to `boost::re_detail_107000::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::string>, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::string> > >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::construct_init(boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > > const&, boost::regex_constants::_match_flags)'
src/map/mash_map.o: In function `boost::cpp_regex_traits<char>::transform_primary(char const*, char const*) const':
/home/gd98309/miniconda/envs/metamaps2/include/boost/regex/v4/cpp_regex_traits.hpp:966: undefined reference to `boost::re_detail_107000::cpp_regex_traits_implementation<char>::transform_primary(char const*, char const*) const'
src/map/mash_map.o: In function `boost::cpp_regex_traits<char>::transform(char const*, char const*) const':
/home/gd98309/miniconda/envs/metamaps2/include/boost/regex/v4/cpp_regex_traits.hpp:962: undefined reference to `boost::re_detail_107000::cpp_regex_traits_implementation<char>::transform(char const*, char const*) const'
src/map/mash_map.o: In function `boost::re_detail_107000::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::string>, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::string> > >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::match_prefix()':
/home/gd98309/miniconda/envs/metamaps2/include/boost/regex/v4/perl_matcher_common.hpp:346: undefined reference to `boost::match_results<__gnu_cxx::__normal_iterator<char const*, std::string>, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::string> > > >::maybe_assign(boost::match_results<__gnu_cxx::__normal_iterator<char const*, std::string>, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::string> > > > const&)'
src/map/mash_map.o: In function `boost::re_detail_107000::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::string>, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::string> > >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::match_match()':
/home/gd98309/miniconda/envs/metamaps2/include/boost/regex/v4/perl_matcher_non_recursive.hpp:1108: undefined reference to `boost::match_results<__gnu_cxx::__normal_iterator<char const*, std::string>, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::string> > > >::maybe_assign(boost::match_results<__gnu_cxx::__normal_iterator<char const*, std::string>, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::string> > > > const&)'
src/map/mash_map.o: In function `save_primitive<boost::archive::text_oarchive, std::basic_string<char> >':
/home/gd98309/miniconda/envs/metamaps2/include/boost/archive/detail/oserializer.hpp:93: undefined reference to `boost::archive::text_oarchive_impl<boost::archive::text_oarchive>::save(std::string const&)'
src/map/mash_map.o: In function `load_primitive<boost::archive::text_iarchive, std::basic_string<char> >':
/home/gd98309/miniconda/envs/metamaps2/include/boost/archive/detail/iserializer.hpp:108: undefined reference to `boost::archive::text_iarchive_impl<boost::archive::text_iarchive>::load(std::string&)'
src/map/mash_map.o: In function `save_primitive<boost::archive::text_oarchive, std::basic_string<char> >':
/home/gd98309/miniconda/envs/metamaps2/include/boost/archive/detail/oserializer.hpp:93: undefined reference to `boost::archive::text_oarchive_impl<boost::archive::text_oarchive>::save(std::string const&)'
/home/gd98309/miniconda/envs/metamaps2/include/boost/archive/detail/oserializer.hpp:93: undefined reference to `boost::archive::text_oarchive_impl<boost::archive::text_oarchive>::save(std::string const&)'
src/map/mash_map.o: In function `save_primitive<boost::archive::binary_oarchive, std::basic_string<char> >':
/home/gd98309/miniconda/envs/metamaps2/include/boost/archive/detail/oserializer.hpp:93: undefined reference to `boost::archive::basic_binary_oprimitive<boost::archive::binary_oarchive, char, std::char_traits<char> >::save(std::string const&)'
src/map/mash_map.o: In function `load_primitive<boost::archive::binary_iarchive, std::basic_string<char> >':
/home/gd98309/miniconda/envs/metamaps2/include/boost/archive/detail/iserializer.hpp:108: undefined reference to `boost::archive::basic_binary_iprimitive<boost::archive::binary_iarchive, char, std::char_traits<char> >::load(std::string&)'
src/map/mash_map.o: In function `load_primitive<boost::archive::text_iarchive, std::basic_string<char> >':
/home/gd98309/miniconda/envs/metamaps2/include/boost/archive/detail/iserializer.hpp:108: undefined reference to `boost::archive::text_iarchive_impl<boost::archive::text_iarchive>::load(std::string&)'
/home/gd98309/miniconda/envs/metamaps2/include/boost/archive/detail/iserializer.hpp:108: undefined reference to `boost::archive::text_iarchive_impl<boost::archive::text_iarchive>::load(std::string&)'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_filesystem.so: undefined reference to `__cxa_throw_bad_array_new_length@CXXABI_1.3.8'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::__cxx11::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >::str() const@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::basic_stringstream(std::_Ios_Openmode)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::runtime_error::runtime_error(std::runtime_error const&)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::__cxx11::numpunct<wchar_t> const& std::use_facet<std::__cxx11::numpunct<wchar_t> >(std::locale const&)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::__cxx11::collate<wchar_t> const& std::use_facet<std::__cxx11::collate<wchar_t> >(std::locale const&)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `bool std::has_facet<std::__cxx11::messages<wchar_t> >(std::locale const&)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `icu_64::Locale::~Locale()'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::runtime_error::runtime_error(char const*)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `u_charType_64'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::overflow_error::overflow_error(char const*)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `icu_64::Locale::Locale(icu_64::Locale const&)'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_filesystem.so: undefined reference to `std::_V2::error_category::~error_category()@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `bool std::has_facet<std::__cxx11::messages<char> >(std::locale const&)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_filesystem.so: undefined reference to `std::_V2::error_category::_M_message[abi:cxx11](int) const@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::__cxx11::numpunct<char> const& std::use_facet<std::__cxx11::numpunct<char> >(std::locale const&)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `u_isblank_64'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::logic_error::logic_error(char const*)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_filesystem.so: undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)@GLIBCXX_3.4.20'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_filesystem.so: undefined reference to `std::runtime_error::runtime_error(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::invalid_argument::invalid_argument(char const*)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::__cxx11::messages<char> const& std::use_facet<std::__cxx11::messages<char> >(std::locale const&)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `icu_64::Locale::Locale()'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_filesystem.so: undefined reference to `std::runtime_error::runtime_error(char const*)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `u_isspace_64'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_filesystem.so: undefined reference to `std::_V2::generic_category()@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `u_tolower_64'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_filesystem.so: undefined reference to `typeinfo for std::_V2::error_category@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_filesystem.so: undefined reference to `operator delete(void*, unsigned long)@CXXABI_1.3.9'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::runtime_error::runtime_error(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::logic_error::logic_error(std::logic_error const&)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `icu_64::Collator::createInstance(icu_64::Locale const&, UErrorCode&)'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::out_of_range::out_of_range(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::~basic_stringstream()@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `u_digit_64'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `u_charFromName_64'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::__cxx11::collate<char> const& std::use_facet<std::__cxx11::collate<char> >(std::locale const&)@GLIBCXX_3.4.21'
/home/gd98309/miniconda/envs/metamaps2/lib/libboost_regex.so: undefined reference to `std::__cxx11::messages<wchar_t> const& std::use_facet<std::__cxx11::messages<wchar_t> >(std::locale const&)@GLIBCXX_3.4.21'
collect2: error: ld returned 1 exit status
make: *** [metamaps] Error 1

Configure issue (usability)

I have the same symptom as #20, except I don't use conda. I am running Ubuntu 18.04. The configure command I use is:

./configure --with-boost=/media/analynano/Data500GB/ArnieFiles/boost_1_71_0/boost --prefix=/media/analynano/Data500GB/ArnieFiles/MetaMaps

The error lines are:

checking boost/math/distributions/binomial.hpp usability... no
checking boost/math/distributions/binomial.hpp presence... no
checking for boost/math/distributions/binomial.hpp... no

What does this error indicate?

Assertion 'combinedOutput.is_open()' failed

Hi Alexander,

I just ran into an issue when using mapDirectly with my custom database.

The last lines of output are:

Added C3600|kraken:taxid|185218|NC_012519.1 with length 7781; est. memory ~24.2109 GB
Added C5076|kraken:taxid|389469|NC_008717.1 with length 36415; est. memory ~24.2112 GB
Added C4950|kraken:taxid|673515|NC_021069.1 with length 10865; est. memory ~24.2113 GB
Added C959|kraken:taxid|170617|NC_005038.1 with length 99657; est. memory ~24.2123 GB
Added C4768|kraken:taxid|1746063|NC_028391.1 with length 1940; est. memory ~24.2123 GB
Added C5424|kraken:taxid|1688637|NC_027706.1 with length 9829; est. memory ~24.2124 GB
Added C2200|kraken:taxid|10815|NC_001928.2 with length 2632; est. memory ~24.2125 GB
Added kraken:taxid|4232|C1004|NW_019010654.1 with length 9845; est. memory ~24.2125 GB
Added C2630|kraken:taxid|223307|NC_002049.1 with length 2542; est. memory ~24.2126 GB
Added C5772|kraken:taxid|71186|NC_009605.1 with length 2562; est. memory ~24.2126 GB
Added C640|kraken:taxid|201862|NC_007216.1 with length 9695; est. memory ~24.2127 GB

Call storeCurrentState with 7338

INFO, skch::Sketch::computeFreqHist, Frequency histogram of minimizers = (1, 70758415) ... (242454, 1)
INFO, skch::Sketch::computeFreqHist, With threshold 0.001%, ignore minimizers occurring >= 7400 times during lookup.
INFO, skch::Map::mapQuery, [count of mapped reads, reads qualified for mapping, total input reads] = [160689, 401370, 1705578]
INFO, skch::Map::Map, Time spent mapping the query : 492.806 sec
INFO, skch::Sketch::build, minimizers picked from reference = 685249698

Index construction DONE, wrote 1 files.

metamaps: src/map/mapWrap.h:40: void mapWrap::unifyFiles(std::__cxx11::string, const skch::Parameters&, std::vector<std::__cxx11::basic_string<char> >, std::vector<std::__cxx11::basic_string<char> >): Assertion `combinedOutput.is_open()' failed.

After running the following command:

/usr/bin/time -v /opt/MetaMaps-master/metamaps mapDirectly -t 20 --all -r ./myDB/DB.fa -q /datadrivehdd1/nanopore/03_projects/campanula/reads/campanula_PCS108_a2.3.0.fastq -o ./classification_results --maxmemory 100 &> output.txt

I have added the corresponding output.txt file if you would like to see the full output.
output.txt

Only a 'classification_results.1' file is created, and as I read earlier this means that classify didn't finish completely.

Error: cannot open file 'plotIdentities_EM.R': No such file or directory

Hi,
I would like to plot the classification results. However, the below command results in error.
Rscript plotIdentities_EM.R classification_results
Fatal error: cannot open file 'plotIdentities_EM.R': No such file or directory

I checked in the repo and couldn't find the script plotIdentities_EM.R. Could you kindly guide me how to resolve the issue?

Also, would like to ask if MetaMaps has compatibility with Pavian(like Kraken, Centrifuge reports) for visualization of the classification.

Thank you in advance!!

downloadRefseq.pl timeout

Running from the latest commit (881373c). Tried downloading the database. Unfortunately timing out as per below.

perl ~/Programs/MetaMaps/downloadRefSeq.pl --seqencesOutDirectory download/refseq --taxonomyOutDirectory download/taxonomy --targetBranches archaea --skipIncompleteGenomes 1

citations.dmp
delnodes.dmp
division.dmp
gencode.dmp
merged.dmp
names.dmp
nodes.dmp
gc.prt
readme.txt

Taxonomy downloaded and extracted into download/taxonomy

Now download genomes for 299 archaea species (328 genomes - refseq - skip incomplete genomes: 1).
         Genome 4 / 328 ; species 4 / 299 archaea (Candidatus_Nitrosocaldus_cavascurensis) -- version 1 / 1: GET GCF_900248165.1_Nitrosocaldus_cavascurensis_SCU2_chromosome_assembly_report.txt                    
         Genome 50 / 328 ; species 49 / 299 archaea (Methanosarcina_vacuolata_Z_761) -- version 1 / 1: GET GCF_000969905.1_ASM96990v1_protein.faa.gz

Timeout at /home/sam/.conda/envs/metamaps/lib/5.26.2/Net/FTP.pm line 583.```

ERROR, skch::validateInputFiles, Could not open test.fastq

Hi,
I am quite new to this. I face this issue while trying to use metamaps for my long read metagenomics data. Could you please help me in solving this?

Metamaps cant find boost files

I am having some trouble building metamaps. It seems to be struggling to find some files that definitely exist. I will provide the full list of commands I ran (I am doing this inside a container).

    #============================================
    # INSTALL ZLIB
    #============================================
    VERSION="1.2.11"
    wget http://www.zlib.net/zlib-"$VERSION".tar.gz -O - | tar xzf -
    cd zlib-"$VERSION"
    ./configure --prefix=/usr/local
    make
    make install
    cd ..

    #============================================
    # INSTALL BOOST 
    #============================================
    wget https://sourceforge.net/projects/boost/files/boost/1.62.0/boost_1_62_0.tar.gz -O - | tar xzf -
    cd boost_1_62_0
    ./bootstrap.sh --prefix=/usr/local 
    ./b2 install
    cd ..

    #============================================
    # INSTALL METAMAPS 
    #============================================
    apt install -y autoconf
    URL=https://github.com/DiltheyLab/MetaMaps.git
    COMMIT="8eb0ddbeb630c01f808f6ca79c1a28a6c39df40c"
    git clone "$URL"
    cd MetaMaps || exit 1
    git checkout "$COMMIT"
    ./bootstrap.sh
    ./configure --with-boost=/usr/local
    make metamaps

Everything runs fine but when I got to execute the binary with ./metamaps I get the following error

./metamaps: error while loading shared libraries: libboost_system.so.1.62.0: cannot open shared object file: No such file or directory

Running find / -name libboost_system.so.1.62.0 gives

/usr/local/lib/libboost_system.so.1.62.0

which is where I told metamaps boost was located....

Result in file 'classification_results.EM.contigCoverage' with an error

taxonID equalCoverageUnitLabel contigID start stop nBases readCoverage
5334 Escherichia 1169349.3.patric C732271|kraken:taxid|5334|accn|ASTZ01000025 150000 150999 1754 9.50845e-17

Hello,
I find that the "readCoverage" in the end of every contig in a genome is not correct, 1754/1000=1.754. While the result is 9.50845e-17. It occurs in in the end of every contig in a genome. Could you help me solve the problem ? Thank you very much!
Best wishes!

mapAgainstIndex

Hi.
Just wondering how and when to use the mapAgainstIndex and index commands?
Thanks,

Memory usage???

Hi, I am trying to run MetaMaps with the small database ( miniSeq+H) that you provided.

I downloaded the database and unpacked it.

MetaMaps was installed using Conda on our HP cluster and I am using slurm to run an analysis of a dataset.

Now I see that the memory usage is pretty big (> 140Gb), and I am not really understanding why that is. The result is that I need to up my memory, since the first command I run get's killed as some point.

This is the command:
metamaps mapDirectly -t 12 --all -r $MM_DB/DB.fa -q ${DATAFILE} -o ${DATAFILE}.classification_results

The last lines of the output before it gets killed:

Added C854|kraken:taxid|x709|NZ_CP016627.1 with length 2173408; est. memory ~150.235 GB
Added C10455|kraken:taxid|1266844|NC_021993.1 with length 9914; est. memory ~150.235 GB
Added C5413|kraken:taxid|x1885|NZ_CP009616.1 with length 4687494; est. memory ~150.264 GB
/var/spool/slurmd/job785761/slurm_script: line 56: 54322 Killed                  metamaps mapDirectly -t 12 --all -r $MM
_DB/DB.fa -q ${DATAFILE} -o ${DATAFILE}.classification_results

Is this just loading the database into memory?

I know I can limit the memory usage, but what is the effect of that on the analysis run?

diltheylab / metamaps Goto Github PK

metamaps's People

Contributors

Stargazers

Watchers

Forkers

metamaps's Issues

Recommend Projects

Recommend Topics

Recommend Org