qiime2 / q2galaxy Goto Github PK

Improvement Description
This will require a new QIIME 2 metadata dataset format in Galaxy. We should be able to map the column types to Galaxy metadata, but I'm not sure how much work moving that logic from the framework will be.

Addition Description
Docker-compose is one way to orchestrate docker services. It allows users or devs to define infrastructure-as-code and could help users deploy this plugin more easily.

Current Behavior
Docker services are built and launched one at a time

Proposed Behavior
Provide infrastructure to deploy q2galaxy along with the rest of galaxy, perhaps like this:
https://github.com/bgruening/docker-galaxy-stable

Questions
Project scope: Given that Qiime2 is focused on end users, deployment and orchestration could be out of scope for this plugin
Where's the best place to put this code? Within this repo, a separate repo, or as part of an existing docker infrastructure project?
What orchestration engine to use? Why not use Kubernetes instead of docker-compose?

Addition Description
A clear and concise description of what the addition is.

Current Behavior
Please provide a brief description of the current behavior, if applicable.

Proposed Behavior
Please provide a brief description of the proposed behavior.

Questions

1. An enumerated list of questions related to the proposal.
2. If not applicable, please delete this section.

Comments

1. An enumerated list of comments related to the proposal that don't fit anywhere else.
2. If not applicable, please delete this section.

References

1. An enumerated list of links to relevant references, including forum posts, stack overflow, etc.
2. If not applicable, please delete this section.

Currently, all citations are rendered with type="bibtex" (because that was easy).
When a DOI is available, we should use type="doi"

See qiime2/galaxy-tools#11 for prior discussion.

Seems that the tool version command outputs some extra text:

Matplotlib created a temporary config/cache directory at /tmp/matplotlib-kaxxsc2g because the default path (/home/qiime2/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
diversity version 2022.11.1

I guess MPLCONFIGDIR should be set for the version command (and maybe even for command).

Alternatively one could use some redirection / grep / tail to get only the actual version.

Addition Description
There needs to be a way to view .qzv files inside Galaxy. It isn't yet clear how to dynamically serve files as each .qzv file is essentially a mini website with static assets and an index.html file.

Questions

1. Should a visualization plugin be used for this? Or should it be a display application to q2view? The latter won't work with non-TLS endpoints.

References

1. https://galaxyproject.org/develop/visualizations/

cc @blankenberg, any ideas here?

Given this xml:

   <command>q2galaxy run diversity pcoa '$inputs'</command>
   <configfiles>
      <inputs name="inputs"/>
   </configfiles>
   <inputs>
      <param name="example" type="integer" value="0" />
      <param format="qza" help="The distance matrix on which PCoA should be computed." name="distance_matrix" type="data">
         <options options_filter_attribute="metadata.semantic_type">
            <filter type="add_value" value="DistanceMatrix"/>
         </options>
      </param>
   </inputs>

The $inputs file is not including a key/value pair for distance_matrix. This could be related to my patch galaxyproject/galaxy#6443, or perhaps this has never quite worked right with options_filter_attribute/filter on data-params.

In any case, we have two options:

1. fix the config file in galaxy (probably the right way to do this)
2. actually make a command line interface for galaxy with key/value pairs

I'm definitely leaning towards option 1 and I'll see what I can dig up inside galaxy for why this might be happening.

cc @bgruening

Hi，@ebolyen I used docker to pull and run q2galaxy. When building my workflow, I found that there are no basic tools, such as unzip_collection, Flatten collection and Sort. How can I install these tools in q2galaxy?

@bernt-matthias and @bgruening I'm splitting this off from conversation on #48 because this issue isn't directly related to that PR at this point.

Addition Description
We need a built-in tool for importing data into a QZA file and exporting as well. Ideally we let Galaxy's auto-detect figure out the export situation. Perhaps in the future we could use EDAM to do some of the translation...

Improvement Description
Typically we use hidden actions for the inner part of a parallel pipeline loop, e.g.

def some_parallel_pipeline():
  ...
  collection = partition_data()
  for key, val in collection.items():
    result[key] =  _hidden_method(val)
  viz = collate_data(result)
  ...

where partition_data, _hidden_method, and collate_data are all qiime2 actions.

With Galaxy, it is better to use the built-in for-each on a collection which will play nicely with whatever scheduler is present and will accomplish the same goal as the pipeline. So the user should be using the underlying actions directly instead of the pipeline.

Hi,

we got this error report, let me know if you need more information.

Thanks for this great tool suite!

Tool ID: toolshed.g2.bx.psu.edu/repos/q2d2/qiime2_core__tools__import/qiime2_core__tools__import/2022.11.1+dist.h2bda5906.2

Error:

Traceback (most recent call last):
  File "/opt/galaxy/server/lib/galaxy/jobs/runners/__init__.py", line 291, in prepare_job
    job_wrapper.prepare()
  File "/opt/galaxy/server/lib/galaxy/jobs/__init__.py", line 1254, in prepare
    ) = tool_evaluator.build()
        ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/galaxy/server/lib/galaxy/tools/evaluation.py", line 566, in build
    global_tool_logs(self._build_config_files, config_file, "Building Config Files")
  File "/opt/galaxy/server/lib/galaxy/tools/evaluation.py", line 96, in global_tool_logs
    raise e
  File "/opt/galaxy/server/lib/galaxy/tools/evaluation.py", line 92, in global_tool_logs
    return func()
           ^^^^^^
  File "/opt/galaxy/server/lib/galaxy/tools/evaluation.py", line 633, in _build_config_files
    self.__write_workdir_file(config_filename, config_text, param_dict, is_template=is_template)
  File "/opt/galaxy/server/lib/galaxy/tools/evaluation.py", line 756, in __write_workdir_file
    value = fill_template(content, context=context, python_template_version=self.tool.python_template_version)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/galaxy/server/lib/galaxy/util/template.py", line 143, in fill_template
    raise first_exception or e
  File "/opt/galaxy/server/lib/galaxy/util/template.py", line 87, in fill_template
    return unicodify(t, log_exception=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/galaxy/server/lib/galaxy/util/__init__.py", line 1153, in unicodify
    value = str(value)
            ^^^^^^^^^^
  File "/opt/galaxy/venv/lib/python3.11/site-packages/Cheetah/Template.py", line 1053, in __unicode__
    return getattr(self, mainMethName)()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "cheetah_DynamicallyCompiledCheetahTemplate_1699011302_5547879_70105.py", line 119, in respond
  File "cheetah_DynamicallyCompiledCheetahTemplate_1699011302_5547879_70105.py", line 102, in stringify
  File "cheetah_DynamicallyCompiledCheetahTemplate_1699011302_5547879_70105.py", line 113, in stringify
AttributeError: 'TreeDict' object has no attribute 'is_collection'

Current Behavior

Proposed Behavior
Remove the info-box as it is lying about what files have been added.

Comments

1. This is a Galaxy dataset thing, so this repo isn't actually implicated.

Hi,
I've seen several people asking to install Qiime2 on different Galaxy usegalaxy.* servers.
What I see on this repo is interesting but it seems like it's not possible to install the tools the "standard way" from the ToolShed. Is this planned at some point on your roadmap?

References
Using this: https://docs.galaxyproject.org/en/latest/dev/schema.html#tool-inputs-section

Bug Description
Using the recommended command line input to start an image
docker run -d -p 8080:80 -p 8021:21 -p 8022:22 -v $HOME/q2galaxy_data/:/export/ quay.io/qiime2/q2galaxy
results in a warning
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
furthermore it seems to prevent exposure of port 8080 on the localhost.

Steps to reproduce the behavior

1. Follow the docker steps in the README for Docker setup up to the last terminal command.
2. While running macOS on a M1 chip go to terminal.
3. Run docker run -d -p 8080:80 -p 8021:21 -p 8022:22 -v $HOME/q2galaxy_data/:/export/ quay.io/qiime2/q2galaxy
4. See error WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

Expected behavior
I expected the docker image to be created successfully.

Computation Environment

• OS: [macOS Sonoma 14.1.1]
• QIIME 2 Release [2024.2.1]

Fix

1. Run this command to start the container instead (added --platform linux/amd64)
   docker run --platform linux/amd64 -d -p 8080:80 -p 8021:21 -p 8022:22 -v $HOME/q2galaxy_data/:/export/ quay.io/qiime2/q2galaxy

References

1. Where I found the fix

This is a small list of things that can make the wrappers a bit better. All of those are not important for the first version I think.

    <param name="seqs" type="data" format="qza" label="seqs: MultiplexedPairedEndBarcodeInSequence" help="[required]  The paired-end sequences to be demultiplexed.">

The [required] string can go away. If you do not use optional="true" everything is required and Galaxy will make sure the user knows about it.

    <section name="__q2galaxy__GUI__section__extra_opts__" title="Click here for additional options">

The __ might be confusing down the line. E.g. this will appear in Galaxy workflows etc and might be confusing for users that deal with it.

        <repeat name="front_f" help="[optional]  Sequence of an adapter ligated to the 5' end. The adapter and any preceding bases are trimmed. Partial matches at the 5' end are allowed. If a `^` character is prepended, the adapter is only found if it is at the beginning of the read. Search in forward read." title="front_f: List[Str]">

The title does not look nice. [optional] should be removed

            <param name="column" type="text" label="Column Name">

Should this not be a column parameter?

        <param name="minimum_length" type="integer" min="1" value="1" label="minimum_length: Int % Range(1, None)" help="[default: 1]  Discard reads shorter than specified value. Note, the cutadapt default of 0 has been overridden, because that value produces empty sequence records."/>

The [default: 1] can be removed, it is already encoded in the UI.