Description
Replace the basic table of jobs by a clickable graph display.

Proposal
Job dependencies are stored in the Job object, so from the list of Jobs, we could display the workflow as a graph instead of plain table.
First step is to research available libraries for DAG visualization and compatibility with our current data model.
A few options to explore: d3 or d3-based angular libraries, ngx-graph

Description
Right now, to register a new plugin, the user/developer has to manually copy and paste the JSON content of the plugin manifest file. It would be better to be able to upload the file, or specify a link to the file.

Proposal
The user/developer should be able to:

• register a new plugin by selecting a file on his machine
• register a new plugin by specifying a link to a JSON file

Ideally, the JSON would then be displayed for the user to double check before confirming that the plugin should be registered.

Additional context
Layout of the options in the UI TBD.

Description
We need Jobs in several pages (workflows, stitching vectors).

Proposal
We can create a pop up / create a page for each step of a workflow

Additional context
The image below shows how it looked in WIPP 2.3.0

Description
Currently all available plugins are displayed in a drop down list when creating a new task in the workflow detail page, which will not be convenient when tens or hundreds of plugins will be available. User should be able to at least filter plugins by category (segmentation, feature extraction, ...)

Proposal
User can select from a list of plugin categories before seeing the list of available plugins.
Category names TBD.

Summary

findByName methods in data services (images-collection.service and stitching-vector.service) fail if they are given null http params, such as when they are called from the search widget.
Params should be tested for null value before constructing the HttpParams.

Description
We want to proxy /tensorboard and /argo-ui requests so we can add links to Tensorboard and Argo UI in the WIPP frontend.

Proposal
Add new locations with proxy_pass in nginx conf.

Lot of pieces of style here and there in the HTML files.
They should be moved to the CSS files and duplications should be avoided when same style is reused in different places.

Description
User should be able to quickly access the workflow monitoring page in the Argo dashboard from the WIPP frontend workflow detail page.

Proposal
A new button "Monitor in Argo" in the workflow detail page.
This button should only appear if workflow.generatedName is not null (this property is filled with the unique name generated by Argo after submission of the workflow).
A click on the button will open a new tab with the workflow view on Argo, URL will be as follow:
environment.argoUiBaseUrl/{workflow.generatedName}

• In environment.ts, argoUiBaseUrl will be set to 'http://localhost:30501/workflows/default' (dev setup, Argo UI service is expected to be patched, see patch in WIPP-deploy)
• In environment.prod.ts, argoUiBaseUrl will be set to 'ARGOUIBASE_URL' (prod setup, using seds during Docker build)

Additional context
In terms of icon to add to the button, we could use:

• the sitemap font-awesome icon (Argo also uses this one for the DAG view)
• the Argo logo (there is one with transparent background in their repo)
  The tooltip should include the external-linkl-alt at the end of the message (see example if csv detail page for link to Plots Ui) to make the user aware that it will open a new tab

Description
To use outputs from a job in a workflow, the data type of the ouput must be handled by the workflow component.

Proposal
Add tensorflow models and tensorboard logs data types to the workflow component.

Description
When you user goes to Data -> Images Collections -> New Images collection and fills in the form to create a new collection, if the API returns an error (for example, collection name is already taken), nothing happens, and the user does not get any feedback.

Proposal
An error should pop up displaying the error message coming from the API, so the user knows why the creation of the new collection failed.

Description
Now we use the @qontu/ngx-inline-editor library to modify an image collection name.
This library requires rxjs-compat which is not required anymore in Angular 7
Proposal
Use another method to change an image collection name

Additional context

Summary
Task creation UI does not show all available plugins.

What is the current bug behavior?
In a Workflow task creation plugin selection drop-down menu only shows maximum of 10 plugins, even though more than 10 plugins are registered. It looks like it is showing only plugins from the page selected in the Plugins tab.

What is the expected correct behavior?
It should show all available plugins in the drop-down menu. Or sort them by categories (#48)

Steps to reproduce

1. Register more than 10 plugins
2. Create new Workflow
3. Star creating new Task
4. When choosing Plugin, verify only maximum of 10 plugins are shown

Relevant screenshots and/or logs

Environment info

WIPP 3.0.0 (commit fa19b3b)

Possible fixes

Summary

When viewing a file collection, file numbering on increments/decrements by 10 regardless of how many files per page are being viewed.

What is the current bug behavior?

When viewing 25, 50, or 100 files per page in collections, turning pages only increments/decrements the numbering by 10.

What is the expected correct behavior?

The file numbering should increment/decrement by the number of files viewed per page.

Steps to reproduce

Upload 26 files.
View 25 files per page.
Turn the page once to view the 26th file (but will be assigned number 11).

Relevant screenshots and/or logs

Environment info

WIPP 3.0.0
Chrome Version 75.0.3770.142

Possible fixes

Description
From the csvCollection detail view, we want to have a button that would take us to the Polus Plots app so we can visualize the data.

Proposal
A click on the button would redirect to the following URL: {plotsUiUrl}/plots/{id}, where:

• {id} is the csvCollectionId
• {plotsUiUrl} is already defined in environment as the actual Plots App URL

Description
Add the "Pyramid" data type to the WIPP-frontend, with list page and detail page.

Proposal
List view, detail view (similar to what was in WIPP 2.xx, except for the actual deep zoom view that will be treated in a separate issue)

Additional context
Will have to wait for the backend integration of pyramids to be done before starting working on this issue.

Summary

Sort by image size not working in images collection detail page.

What is the current bug behavior?

Clicking on the "Size" header to sort by image size does not work.

What is the expected correct behavior?

Clicking on the "Size" header to sort by image size triggers the refresh of the list according to Size asc or desc.

Steps to reproduce

Go to Data -> Images collections -> click on a non-empty images collection -> click on the "Size" column header in the images list table.

Possible fixes

The sort param being passed to the request to the API is size, whereas the expected param for sorting by image size is fileSize.
images-collection-detail component should be updated to replace column headers and sort params size by fileSize for both images and metadataFiles.

Several things could be improved in the job detail view/popup:

• input that are WIPP data types should be displayed as links to the detail view displaying the name of the object (currently, this is only happening for images collections, this should happen for all WIPP data types)
• the InputType row should be removed
• the columns are not the same size across the three tables, even though all of them are 2-columns tables
• sometimes the tables are not fitting in the modal

• to make the view less busy, the inputs and outputs sections could be collapsed at first
• the modal could be a bit larger in order to better accommodate longer input names to display

Description
When viewing large numbers of files in a collection (>10,000), it is cumbersome to click through every page to get to the end of the collection (even at 100 files per page). It would be beneficial to have a box to type in the page number and navigate to that page of files.

Proposal
An input box to type in a collection file page number and a submit button that takes the user to that page.

Additional context

Description
Problem: It is impossible to delete a plugin or modify parameters of a plugin after closing a new task dialog although a workflow has not been launched

Proposal
(1) Enable deleting a plugin (i.e., click to select a task/plugin and then provide an option to delete the plugin.
(2) Enable opening a dialog with parameters and editing parameters before a workflow is launched

Additional context
This feature will avoid costly mistakes during workflow construction and reduce the number of unused workflows.

Summary

Rapid panning can cause WDZT to crash when filters are enabled.

What is the current bug behavior?

The web page starts lagging and eventually crashes (Chrome will display an "Aw snap" error page)

What is the expected correct behavior?

Web page does not crash, user can pan around smoothly while filters are enabled

Steps to reproduce

Go to a pyramid view, zoom in, enable filters (tested with Brightness and Contrast both enabled), pan around a lot trying to go toward the edges really fast, until WDZT starts lagging and web page crashes.
Not always easy to reproduce.

Relevant screenshots and/or logs
Chrome dev console logs:

ERROR RangeError: Failed to execute 'getImageData' on 
'CanvasRenderingContext2D' : OutOfMemory at ImageData creation

ERROR RangeError: Array buffer allocation failed
    at new ArrayBuffer (<anonymous>)
    at new Uint8Array (<anonymous>)
    at Function.e.dataArray (scripts.bbe2fff076b6aac980c8.js:1)
    at n.finishInit (scripts.bbe2fff076b6aac980c8.js:1)
    at n.initCanvas (scripts.bbe2fff076b6aac980c8.js:1)
    at n.setup (scripts.bbe2fff076b6aac980c8.js:1)
    at n.<anonymous> (scripts.bbe2fff076b6aac980c8.js:1)
    at scripts.bbe2fff076b6aac980c8.js:1
    at e.invokeTask (polyfills.e3633a42d1123afc5fc9.js:1)
    at Object.onInvokeTask (main.5e27f782d3f1dd388555.js:1)

No significant logs seen in backend and frontend containers for now.

Environment info

I was able to reproduce a couple of times on the AWS instance using Chrome from a Ubuntu 18.04 machine.
@Nicholas-Schaub is also reporting being able to reproduce on his local instance.

Possible fixes

Possible explanation: from the console logs, it may be that the browser is overwhelmed and the web page crashes when trying to keep up with the computations due to the filters, but needs to be investigated more.

cc @pbajcsy @Nicholas-Schaub

Description
If the name of an argo workflow contains a space or underscore, the job will not run.

Proposal
It would be useful to have some simple checking in the browser to inform the user if their workflow name is valid. Maybe prevent a user from submitting a job with an invalid name.

Also, when a job is submitted with an invalid name it shows submitted even though the wipp-backend logs shows the error. It would be useful to show the job has failed if the cause is an invalid workflow name.

Description
User may want to run several jobs in the same workflow that are really similar, for example with the same inputs but different values for a specific parameter. I would be easier and less error-prone for the user to be able to copy on the jobs already configured in the workflow rather that reconfiguring a new job from scratch.

Proposal
User has an option to "copy" an existing job in the workflow, which will open the "New task" form with pre-populated inputs and params, and a task name sourceJobName-copy.
This capability will only be available within the same workflow, and only while the workflow is in the CREATED state.

Additional context
(Add any other context or screenshots about the feature request here)

Description
In order to support a variety of data types, we cannot just keep on adding items side by side to the navigation bar.
Images Collections should be replaced by Data with a drop-down menu for selecting a specific data type.

Summary

List of outputs from other tasks in workflow is stored in an array instead of retrieved from API (when a new task is added to the workflow, the outputs of the task are added to an array in the component, instead of being retrieved from the WIPP API).

What is the current bug behavior?

If the user refreshes the page while configuring his workflow, then the array of "available outputs" from the tasks that are already configured will become empty, and the user will not be able to use these outputs as inputs for other tasks in the workflow.

What is the expected correct behavior?

When configuring a workflow, user should always be able to use outputs from already configured tasks as inputs for other tasks (within the same data type).
These means that the list of available outputs has to be retrieved from the WIPP REST API.

Description
Some of the image metadata are not displayed in the deep zoom right now because OME metadata are not handled yet in the C++ accelerated pyramid building plugin.
This affects the scale bar (not displayed) and cursor info module on the left (acquired intensity info not displayed).

Proposal
As soon as metadata are handled by the pyramid building plugin, add the metadata handling in the getPyramidManifest function in pyramid.service.ts.

• AngularJS snippet from WIPP 2.xx for populating the manifest layer with metadata info:

.then(getTimeSliceMetadata)
.then(function (metadata) {
    if (metadata.acquiredIntensity) {
        layer.acquiredIntensity = metadata.acquiredIntensity;
    }
    if (metadata.pixelsPerMeter) {
        layer.scalebar = {
            pixelsPerMeter: metadata.pixelsPerMeter,
            unitType: "length"
        };
    }
    return manifest;
});

• AngularJS snippet from WIPP 2.xx for getting the metadata from the pyramid:

function getPyramidMetadata(pyramid) {
    return getPyramidTimeSlices(pyramid, {
        size: 1
    }).then(function (resource) {
        return resource.$get('pyramidTimeSlices');
    }).then(function (timeSlices) {
        return getTimeSliceMetadata(timeSlices[0]);
    });
}

function getTimeSliceMetadata(timeSlice) {
    return getTimeSliceOme(timeSlice).then(function (omeXml) {
        return {
            pixelsPerMeter: getPixelsPerMeter(omeXml),
            acquiredIntensity: getAcquiredIntensity(omeXml)
        };
    });
}

function getTimeSliceOme(timeSlice) {
    return $http({
        method: 'GET',
        url: timeSlice.$href('ome'),
        cache: true
    }).then(function (data) {
        var parser = new DOMParser();
        return parser.parseFromString(data.data, "application/xml");
    });
}

function getPixelsPerMeter(omeXml) {
    var $omeXml = jQuery(omeXml);
    var pixels = $omeXml.find(':root>Image>Pixels');
    var physicalSizeX = pixels.attr('PhysicalSizeX');
    var physicalSizeXUnit = pixels.attr('PhysicalSizeXUnit');

    if (!physicalSizeX || !physicalSizeXUnit) {
        return undefined;
    }
    var mpp = physicalSizeX * getFactor(physicalSizeXUnit);

    return 1 / mpp;
}

function getFactor(unit) {
    if (unit.length === 1) {
        return 1;
    }
    var factors = {
        'Y': 24,
        'Z': 21,
        'E': 18,
        'P': 15,
        'T': 12,
        'G': 9,
        'M': 6,
        'k': 3,
        'h': 2,
        'da': 1,
        'd': -1,
        'c': -2,
        'm': -3,
        'µ': -6,
        'n': -9,
        'p': -12,
        'f': -15,
        'a': -18,
        'z': -21,
        'y': -24
    };
    var prefix = unit.substring(0, unit.length - 1);
    var factor = factors[prefix];
    if (factor === undefined) {
        throw "Unknown unit " + unit;
    }
    return Math.pow(10, factor);
}

function getAcquiredIntensity(omeXml) {
    var $omeXml = jQuery(omeXml);
    var $step = $omeXml.find(
        ':root>StructuredAnnotations' +
        '>XMLAnnotation[Namespace="nist.gov/ProcessingSteps"]' +
        '>Value>ProcessingSteps>Step[type="IntensityScaling"]' +
        '>Parameters');

    var intensityScalerType = $step.find('IntensityScalerType')
        .text();
    if (['TRUNCATION', 'GAMMA'].indexOf(intensityScalerType) < 0) {
        return undefined;
    }

    return {
        type: intensityScalerType === 'TRUNCATION' ?
            'threshold' : 'gamma',
        min: parseInt($step.find('RangeStart').text()),
        max: parseInt($step.find('RangeEnd').text())
    };
}

Additional context
(Add any other context or screenshots about the feature request here)

Description
User should be able to easily search for specific data by sorting data tables by column and filter by name (similar to WIPP 2.xx).

Proposal

• add sorting on table columns
• add filter (input field where user can type in a filter string) to 'name' columns
  A good starting point will be to mimic behavior that was present in WIPP 2.xx for images collections and jobs/workflows.
• options to explore:
  • use Angular Material, see tutorial on how to use data tables with server-side pagination for example (as highlighted in the tutorial, MatTableDataSource should not be used here because it is designed for client-side pagination)
  • use ng-bootstrap: https://ng-bootstrap.github.io/#/components/table/overview

Additional context
Reminder: the WIPP REST API is paginated, so any sorting or filtering happens server-side. Asc/desc sorting params can be passed to the GET request, and search methods for filtering are already available for most data REST repositories (such as findByNameContainingIgnoreCase...). Any missing filtering capability will have to be added by PR on WIPP-backend.

Summary

Sort by image name in image collection detail view not working.

What is the current bug behavior?

Clicking on the arrow in the "Name" column header to trigger sort of images by name in Image Collection detail view does not trigger sort of images.

What is the expected correct behavior?

Sort is triggered, images are sorted by name in ascending or descending order.

Possible fixes

This is probably caused by using name instead of fileName for the matColumn "Name", causing the request to be sent to the API with the sort param sort=name,asc|desc instead of sort=fileName,asc|desc

Description
Currently, certain file types like tiff are automatically converted to tiled tiff. In some cases it may be necessary to keep the original image files for preprocessing metadata that Bioformats may not handle. In these cases, there should be an option to upload files without converting to tiled tiff. However, this should not be the default function, and the user should be given warnings that native formats are generally unsupported by almost all WIPP plugins.

Proposal
Provide an option when creating a WIPP image collection to keep the native file format when uploading files. When the option is selected, warn the user that native file formats are incompatible with most WIPP plugins, and make sure they have a confirmation box that this is something they want to do.

Description
Currently, only jobs are shown in the workflow visualization. It would be helpful to show input/output collections (or vectors, etc) as separate nodes in the workflow graph.

Proposal
Add data nodes to the workflow graph that show data as inputs or outputs to job nodes.

Description
Add views for the new data type csvCollection (see usnistgov/WIPP-backend#15)

Proposal
Add usual set of components/changes for new data type: new Module, list view, detail view, new entry In "Data" menu, handling in workflow details and search widget, handling in job details

Description
When a user is creating a new task in a workflow, there should be some form validation happening based on the plugin manifest so that the user can not submit the form if it is not valid. User should also get information about what is invalid in the form.

Proposal
Leverage validation done by ngx-schema-form and create custom rules if needed for WIPP data types and conditional validation.

Additional context
By default, inputs specified in WIPP plugin manifest are considered as required if the required property is not present.

Looks like an nginx issue, probably introduced by new proxy pass for pyramid tiles fetching usnistgov/WIPP@ff5e401

How to reproduce
Go to Data->Pyramids-> click on a pyramid, then reload page

(this issue is copied from usnistgov/WIPP#2 as Docker/nginx conf is now is this repo)

Description
Add Visualizations to Data drop-down menu, create list view and detail view similar to the ones in WIPP 2.x.
Visualization detail view uses WDZT directive.

Description
Currently CSV collections can only be generated as the output of a job. Due to the power of the Plots application, some users may want to upload a CSV file to view data in Plots.

Proposal
Add a method to create a new CSV collection from the WIPP frontend.

Additional context
Addition of this method can also aid in debugging plugins and functions that use CSV collections as input, since a pipeline will not need to be run in order to create a csv collection.

Description
Now that a user can create an images collection from the Catalog app (see usnistgov/WIPP-backend#111), user may also want to go back to the original view of Catalog from which he created the new collection.

Proposal
Add management of Catalog App Url in config and links to specific pages of Catalog from images collections created from Catalog:

• add catalogUiUrl entry in src/assets/config.json and sample-config.json
• update ImagesCollection class in images-collection module
• modify images collection detail page to accommodate the fact that we now have multiple sources possible for collections (uploaded, job ou catalog), one way to do that could be to change the presentation to:
  • Source: Uploaded if importMethod is UPLOADED or importMethod is null and sourceJob is null
  • Source (job): link if sourceJob is not empty or null
  • Source (catalog): link if importMethod is CATALOG
    Catalog link will be built from catalogUiUrl and imagesCollection.sourceCatalog.

Additional context
(Add any other context or screenshots about the feature request here)

Summary

Reported by the NCATS team, WDZT in pyramid detail view is only working in Chrome.
Reproduced in both AWS and testing server instances. Reproduced in dev environment as well.

What is the current bug behavior?

WDZT section is empty.

What is the expected correct behavior?

WDZT should appear, and image is displayed.

Steps to reproduce

From WIPP UI, go to Data -> Pyramids -> click on one using Safari or Firefox (not tested on IE/Edge)

Relevant screenshots and/or logs

No errors in the console.

Environment info

Happening in WIPP 3.0 only (no issues with WIPP 2.3 or ISG website), so it may be due to the Angular 6 integration.

Possible fixes

(If you can, link to the line of code that might be responsible for the problem - optional)

The plugin schema is expecting a "format type" (leftover from WIPP-plugins prototype) that we no longer need (see this line).
It needs to be removed as it throws an error and prevents the form to be generated when the input description does not have a format type.

Latest version is 2.4.1, but depending on compatibility, we may need to use 2.3.5, or even 2.3.3 (see changelog)

Description
Add stitching vector views, similar to what was in WIPP 2.xx

Proposal
List view, detail view, upload stitching vector view

Summary

Image upload has a bug. The throttle refresh doesn't work on the last image uploaded. The browser must be refreshed.

What is the current bug behavior?

• When an image is uploaded, the browser must be refreshed or else it won't appear as converted to OME TIFF. The field "Images being converted" stays indefinitely set to "1".

• When multiple images are uploaded, The last image uploaded won't appear as converted to OME TIFF. The field "Images being converted" stays indefinitely set to "1". The browser must be refreshed to fix the issue.

What is the expected correct behavior?

When an image or multiple images are uploaded, the image(s) must appear as "ome.tiff" after the conversion is done on the backend side.

Steps to reproduce

On branch issue3-convert-image-tiled-tiff-upload of the backend, upload an image to any image collection.

Relevant screenshots and/or logs

Environment info

(WIPP version, web browser, deployment info)

Possible fixes

(If you can, link to the line of code that might be responsible for the problem - optional)

Description
Integrate WDZT in the WIPP-frontend for pyramid visualization

Proposal
Add the wdzt as a npm package and integrate in the pyramid detail view (with necessary changes to the pyramid data service for manifest management) for pyramid visualization.

Additional context
Wait for #13 to be done before starting this one.

Description
Add new fields described in usnistgov/WIPP-backend#96

Proposal
When fields are empty, N/A should be displayed. 'Website' should be displayed as link (clickable).

Description
Add the "TensorflowModel" data type to the WIPP-frontend, with list page and detail page.

Proposal
List view and detail view

Description
New data type notebook in being added to WIPP (see usnistgov/WIPP-backend#76) to support Jupyter Notebook usage as inputs of jobs in WIPP.

Proposal
New views to be created:

• list view
• detail view (see additional context below)
• template
  Custom handling for notebook data type should also be added in the SearchWidget component and the workflow detail component.

Additional context
For the detail view, we would like to render a view of the notebook. So far, we looked at these two options:

• nbviewer.js (nice visualization, but does not seem to be on npm)
• notebookjs (available on npm, rendering is a bit less fancy, but we could add additional styling)

@ktaletsk please feel free to chime in

Summary

In Workflow details page, workflow DAG visualization is cut if job names are too long.

What is the current bug behavior?

We don't see some of the DAG nodes.

What is the expected correct behavior?

We can view the full DAG.

Relevant screenshots and/or logs

Summary

When creating new Task in the Workflow, if one of the parameters is "number", then it is impossible to type fractional number in the corresponding UI field, as it only accepts integers. However, it is possible to copy-and-paste such a fractional number.

What is the current bug behavior?

• Impossible to type fractional number in the "number" field

What is the expected correct behavior?

Either

• Should be able to type fractional number in "number" field
  OR
• Introduce "float" type or similar to hold float numbers in parallel to "number" type which will only hold integers

Steps to reproduce

1. Create new Workflow
2. Click to add new Task
3. Choose plugin that has "number" input parameter
4. Try to type fractional number (i.e. 0.1) in the corresponding UI field

Relevant screenshots and/or logs

(Insert screenshots or paste any relevant logs - please use code blocks (```) to format console output,
logs, and code as it's very hard to read otherwise.)

Environment info

• Latest WIPP version from CI
• Safari 13.0.4, MacOS Mojave 10.14.6

Description
Search widget component (search for an images collection to use as input when configuring a task in the workflow) only supports images collections for now, but it should be modified so that it can work with or can be extended to multiple data types.

Summary

No error is displayed when Stitching vector creation (upload) fails

What is the current bug behavior?

When creating a new stitching vector, if an error happens on the backend while uploading the file, nothing is displayed in the frontend, so the user only knows that something went wrong because the stitching vector does not appear in the list.

What is the expected correct behavior?

A popup or modal should appear, informing the user that something went wrong, and displaying the error message sent by the backend API.

Steps to reproduce

See for example usnistgov/WIPP-backend#45

Summary

Task table in workflow detail page displays threshold type instead of plugin type

What is the current bug behavior?

Column "Type" will display the threshold type if thresholding plugin was used, nothing otherwise.

What is the expected correct behavior?

Column "Type" should display plugin name.

Steps to reproduce

Create a new workflow, create a new task for the workflow.

Relevant screenshots and/or logs

Possible fixes

Change value to name of the plugin