f210c6c

Proposed change

Brief tutorial should be added to project's page

f210c6c

Description

1. mothulity_fc does not fire up since argparse import is missing.
2. mothulity_fc does not fire up since names_sanitizer is not properly referenced with mut

Steps to reproduce

1. Install mothulity v1.6.0 with pip.
2. Run analysis.

Expected behaviour

Should run the analysis

Actual behaviour

Throws the import error.

Software supports only paired-end reads analysis.

b1a8511

Proposed change

There's an ugly repetition of the menu at the project's page. It should be done once, using jekyll's _layouts

mothulity still tries to find beta-diversity-specific HTML files despite there's only one sample.
Run mothulity.py /path/to/files/ --render-html to reproduce.

f4d4887

Proposed change

The threshold for defining what is a junk group and what is not should be made upon SD rather than plain mean division by an arbitrary number.

f4d4887

Proposed feature

mothulity know each step of analysis from each other. This might be achieved with few steps:

• Enclosing particular mothur commands, instead of keeping them in a plaing text in the template. The question is: to what extent the logic from templates should be removed?
• Once the command is passed to mothulity, it should know how the files would be named after each step. If it's done - resolving #9 is straightforward.
• mothulity already creates the hidden logfile. It can be used as the cache.

It would be convenient to enable email notification using mothulity.py arguments and not to bother with headnode_notifier itself.

b12439f

Proposed change

Mothulity should be written in python3

b12439f

Proposed change

Allow setting path to config file with CLI.

4328477

Proposed change

mothulity should be added to pypi. The deployment should be managed by travis-ci.

ee44239

Proposed change

The output example is now reference at the end of the brief tutorial. It should be also accessible from the top menu

Venn diagrams for species overlap between samples feature

4328477

Proposed change

mothulity should be added to bioconda. The deployment should be managed by travis-CI

commit

8d27eb9

Proposed change

There is a terrible mess introduced into INSTALL.sh when travis CI was attached to the repo.

ab2a2de

Proposed change

PathTests must include if mothur and blast executables can be found after installation.

Description

Software fails to deliver proper final results/visualization of samples with high biodiversity. During converting samples taxonomy (tax.summary obtained from mothur) to xml (needed for Kroma) part of the data is lost.

Steps to reproduce

1. Run analysis of samples with high biodiversity.

Expected behaviour

Software performs analysis and produces a full report including proper krona charts for taxonomical representation of the observed OTUs.

Actual behaviour

Software fails to convert tax.summary to xml, providing a wrong taxonomical representation of the observed OTUs (many taxa are missing, part of the pie chart is described as "other bacteria").

mpi exec when --nodes opt > 1 and mem-per-cpu and processors and partition

Software does not support checkpointing and does not allow user to continue analysis from a last/given step.

f4d4887

Proposed feature

The heatmap should be sortable and interactive in a fancy way.

commit

3bc6bd9

Proposed change

• doctests and unittests are not enough. mothulity actual actions should be checked during each build.
• As it not feasible with restricted resources of travis, it should be done at least as a comparison with the script generated with --dry-run arg.
• Once this is accomplished, run on MiSeq dataset requirement can be removed from the pull request template.

missing apostrophe at the end, unwanted slash before make.contigs command, messed up slurm flags newlines, pairwise.seqs command typo

4f5823a

Proposed change

If there no fastq files or for other reason the input directory is obviously wrong, everything tries and fails. It should be detected before mothur is called.

ab2a2de on branch UPD#64

Description

doctests do not work after py extension is removed from the python scripts. Also, these are the basic level math functions that would be suitable for that kind of tests.

Steps to reproduce

1. Run python -m doctest on any of the mothulity scripts

Expected behaviour

Passing doctests

Actual behaviour

Failing doctest mostly due to undefined variables since the tested functions self-reference fails without py extension.

Hey,
Would you like to provide users example data for testing your software possibilities? It could be also used to validate installation & usage processes.

Cheers.

4328477

Proposed change

mothulity help page needs some review and refurb. Some args' help describe action:

while other describe objects:

b12439f

Description

Some of the mothur's commands are unable to read path with a dash. Mothulity should warn the user.

Steps to reproduce

1. Create a project with a dash in its input path.
2. Run mothulity.py on it.

Expected behaviour

This is one of the commands unable to read the path with the dash.

filter.seqs(fasta=current, vertical=T, trump=.)
Using /home/dizak/Pulpit/MiSeq_SOP/temp/temp-temp/mothur.job.trim.contigs.good.unique.good.fasta as input file for the fasta parameter.
Unable to open /home/dizak/Pulpit/MiSeq_SOP/temp/temp. Trying default /home/dizak/anaconda2/envs/mothulity/bin/temp
Unable to open /home/dizak/anaconda2/envs/mothulity/bin/temp. Trying output directory /home/dizak/Pulpit/MiSeq_SOP/temp/temp-temp/temp
Unable to open /home/dizak/Pulpit/MiSeq_SOP/temp/temp-temp/temp. It will be disregarded.
Unable to open temp/mothur.job.trim.contigs.good.unique.good.fasta. Trying default /home/dizak/anaconda2/envs/mothulity/bin/mothur.job.trim.contigs.good.unique.good.fasta
Unable to open /home/dizak/anaconda2/envs/mothulity/bin/mothur.job.trim.contigs.good.unique.good.fasta. Trying output directory /home/dizak/Pulpit/MiSeq_SOP/temp/temp-temp/mothur.job.trim.contigs.good.unique.good.fasta

Actual behaviour

The analysis should have run normally with full output.

Description

Mothulity_dbaser calls function download (for silva_102) with wrong argument names (download_path instead of download_directory).

Steps to reproduce

1. Download silva_102 database i.e. during installation.

Expected behaviour

Downloads silva_102 database.

Actual behaviour

Execution fails with nasty python error.

0cc7c8e

mothulity.py should not require the positional argument in some cases, for instance, when setting the default database path.

There is no need for database path search when --analysis-only is invoked. It should be omitted.

3381998

Proposed change

The descriptions provided should be a little bit updated since adding mothulity to pypi

Adding path autocompletion for guided installation.

Proposed change

Create bash wraper for main mothulity python script, remove shebang edition from installation.

Would be really nice if readme could:

• briefly introduce mothulity basic execution options,
• provide example of usage i.e. based on MiSeqSOP dataset mentioned in "example data section".

Databases (provided by mothulity_dbaser and used by mothulity in analysis) could be by default stored in mothulity directory.

Due to the limitations of acg & dcg methods with default parameters (in complex analysis many bacteria are assigned to "other bacteria" category i.e. proteobacteria), one must consider tuning the setup if the acg will remain the main clustering method.

The issue needs to be further discussed.

Evaluate whether CLI arguments are not impossible or out of logical range, eg max-length < min-length or cluster-method out of allowed values.

Description

Software does not download database unite-ITS-02.

Download path: ./Unite_ITS_02.zip
Connecting...
Failed to establish connection. Response code 404

Steps to reproduce

Execute the following command:
$ ./mothulity_dbaser.py --unite-ITS-02

ee44239

Proposed change

google analytics should be added to the project's page

Metadata should be handled for complete beta-diversity analysis.

Following sources should be supported:

• CSV
• Excel
• Google Sheets

17bb386

Steps to reproduce

1. Pass relative path to mothulity.py --set-align-database-path or mothulity.py --set-taxonomy-database-path.

Expected behavior

The path supplied to mothulity.py --set-align-database-path or mothulity.py --set-taxonomy-database-path should be expanded to an absolute path.

Actual behavior

The path supplied to mothulty.py --set-align-database-path or mothulity.py --set-taxonomy-database-path is passed literally to the config file.

When mothulity_dbaser.py fails to download one of the multiple databases, it gives an unwrapped error and quits prematurely without going to the next in the queue.

0dc2ce9

Proposed change

There's a cool website pepy that show the number of downloads from pypi and even gives a badge! It should be added to the project's page.

7441da4

Proposed change

There is vsearch option available for chimeric reads search.

Proposed change

The dereplicate param in the chimeric reads search might need investigation whether it is truly needed.

As the OTU-clustering part is the most computing power and time-consuming part of the analysis, there is an urgent need for using some alternative way of computing it.

b12439f

Proposed change

INSTALL.sh should provide a clean way of omitting the database download step. Right now it is possible using the -t 6 opt but it is not clear for the user. Also, there should be no break just semicolons here:

no path is passed for unpacking so it works only for current directory

f4d4887

Description

If the precision parameter in the OTU-clustering command is changed, the preprocessing part runs just fine, but the analysis part does not.

Steps to reproduce

1. Dry-run preprocessing part with agc clustering method.
2. Modify the precision parameter in the cluster command to cluster.split(fasta=current, count=current, taxonomy=current, cutoff=0.03, large=T, method=agc, precision=1000);
3. Run the modified script.

Expected behaviour

Once the preprocessing step is done, mothulity should run the analysis steps and output the final report in the HTML file.

Actual behaviour

The final report HTML is not generated. The shared (both full and subsampled) and tax.summary files are in the analysis/OTU/ directory. The alpha biodiversity part runs as expected, though the beta directory contains only the mothur's log files.