pbmm2 using too much memory about deepconsensus HOT 5 CLOSED

You can't use a full chip worth of Hifi reads as references. DeepConsensus is in the proof of concept stage, not meant for production yet. You can try this new tool, but there's no official support for it yet: https://github.com/PacificBiosciences/align-clr-to-ccs

from deepconsensus.

Hi @zhoudreames

As @armintoepfer indicates, DeepConsensus isn't yet scalable to run a full SMRT cell on an external machine. The v0.1 release still uses many systems that are efficient within Google's distributed systems, but are very inefficient when run on a single machine. We plan to improve this in future releases, but for now, it will be too slow for a full SMRT cell. If there is something you can use a targeted part of a sequence run, that might be better.

For the error you are encountering, I assume that this is in the pre-processing step? Thank you for pointing this out. This is one of the areas we will improve in the next release (potentially by using the tool that Armin linked). For now, to map the reads, you would need to use fewer reads for your input.

Thank you,
Andrew

from deepconsensus.

You can't use a full chip worth of Hifi reads as references. DeepConsensus is in the proof of concept stage, not meant for production yet. You can try this new tool, but there's no official support for it yet: https://github.com/PacificBiosciences/align-clr-to-ccs

thanks for your help

from deepconsensus.

Hi @zhoudreames

As @armintoepfer indicates, DeepConsensus isn't yet scalable to run a full SMRT cell on an external machine. The v0.1 release still uses many systems that are efficient within Google's distributed systems, but are very inefficient when run on a single machine. We plan to improve this in future releases, but for now, it will be too slow for a full SMRT cell. If there is something you can use a targeted part of a sequence run, that might be better.

For the error you are encountering, I assume that this is in the pre-processing step? Thank you for pointing this out. This is one of the areas we will improve in the next release (potentially by using the tool that Armin linked). For now, to map the reads, you would need to use fewer reads for your input.

Thank you,
Andrew
when will the next version releases ? How much do I need to split reads for the 500G menmory ？ thanks~

from deepconsensus.

Hi @zhoudreames

I cannot give you an estimate for when the next version will be ready. With a fair amount of certainty, I will say likely more than 1 month from now and less than 6 months.

The changes in the next release will probably change the memory use, and I cannot now give you an estimate of memory at that time.

from deepconsensus.