mgx-metagenomics / pipelines Goto Github PK

https://github.com/steineggerlab/Metabuli

https://www.biorxiv.org/content/10.1101/2023.05.31.543018v1

In gitlab by @sjaenick on Feb 28, 2017, 18:14

http://www.dantaslab.org/resfams/

In gitlab by @sjaenick on Dec 21, 2016, 20:03

• increase job array size
• adapt flowthrottle node

ARG-Annot still uses blastx, we can improve throughput by switching to Diamond

MvirDB annotation pipelines still use BLAST; as the database is sufficiently small,
GHOSTZ should be used instead.

Affected files:

mgxgene_mvirdb.xml
mvirdb.xml

MetaPhyler hasn't been updated since 2012; remove tool/workflow

In gitlab by @sjaenick on Feb 28, 2017, 22:31

data stalling at input of GetSubject node

id: 147, type GetSubject`2, name: , state: CONNECTED, time: 2/28/2017 10:24:48 PM
        name: input, state: FINISHED, passed: 1567718, pending: 1713608
        name: output, state: CONNECTED, passed: 1567717, pending: 0
--
id: 147, type GetSubject`2, name: , state: CONNECTED, time: 2/28/2017 10:26:48 PM
        name: input, state: FINISHED, passed: 1569221, pending: 1712105
        name: output, state: CONNECTED, passed: 1569220, pending: 0
--
id: 147, type GetSubject`2, name: , state: CONNECTED, time: 2/28/2017 10:28:48 PM
        name: input, state: FINISHED, passed: 1570891, pending: 1710435
        name: output, state: CONNECTED, passed: 1570890, pending: 0
--
id: 147, type GetSubject`2, name: , state: CONNECTED, time: 2/28/2017 10:30:48 PM
        name: input, state: FINISHED, passed: 1573242, pending: 1708084
        name: output, state: CONNECTED, passed: 1573241, pending: 0

In gitlab by @sjaenick on Jul 18, 2015, 17:53

https://github.com/jkawulok/cometa

http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0121453

In gitlab by @burkhard.linke on Mar 12, 2015, 14:35

Performance with amplicon data

Amplicon reads mostly consists of 16S fragments. The blast filter in the default 16S pipeline is not necessary for this kind of data and consume too much resources (time, compute).

Proposed solution

Since the blast filter acts on the steam of reads and filters out reads, it should be possible to use a Conditional node to enable/disable the filter itself. For amplicon data the filter should be disable and all
reads be passed to the rdp classifier itself

Compatibility

The workflow has to be modified in a way that keeps the standard behaviour (active blast filter) as default. A workflow configuration options should be implemented and documented.

Open questions

• Automatically deactivate the filter if an amplicon dataset is analysed? Is the sample / extraction information available in Conveyor?

In gitlab by @sjaenick on Nov 26, 2015, 17:57

phiX residue still present in many datasets provided by commercial companies; need to implement dedicated pipeline that will set the discard flag for phiX matches and remove all existing annotations (including attrcount)

In gitlab by @sjaenick on Nov 26, 2015, 17:54

got exception during validation in node Id:16 () : Conveyor.Core.MissingLinkException: link endpoint jobIn not connected in node Id:16 () (type Conveyor.MGX.MarkDiscard)
at Conveyor.Core.Node.Verify () [0x00027] in /homes/sjaenick/TEST/conveyor/Conveyor.Core/Conveyor.Core/Node.cs:202

In gitlab by @sjaenick on Mar 8, 2015, 19:02

Make sure all pipelines use 0-based coordinates for start/stop.

Checked and fixed:

• GC
• readlength
• besthit-AA
• MvirDB
• MetaCV
• Pfam/TIGRFAMS

Currently, Kraken2 is invoked with default parameters, but a recent
publication suggests setting a confidence cutoff.

https://www.microbiologyresearch.org/content/journal/mgen/10.1099/mgen.0.000949