invictus-integration / docs-ifa Goto Github PK

Add generic build and deploy YAML pipelines in the documentation.

Related to #165, the same TLS 1.2 update should also happen for Storage Accounts.

• Bump to a minimum "apiVersion": "2021-01-01"
• Add "minimumTlsVersion": "TLS1_2"

Currently we're using Shared Access Keys for access to the Storage Account from all the applications needing it. All of these applications are running in Azure, so we should investigate the work to switching this over to Managed Identities. (This is coming from a customer that also requires Managed Identity instead of Shared Access Keys that can be "leaked")

Is your feature related to a problem? Please describe:
Currently, we 'assume' that people use regular command shell when installing the prerequisites for Windows installations. This leads to commands like cd %USERPROFILE% which only works on a command line; not PowerShell.

What solution do you propose?
Should we mention a small reminder to use cd ~ to navigate to the user home directory for PowerShell users?

@pim-simons
Coincidentally, today a question came from a customer whether we can also set up permissions in the dashboard on subfolder, that is not yet possible in v2 but perhaps something for the roadmap?

Historical Admin Users are not automatically given folder permission.

If a new folder is created all admins should be given this by default.

Currently the SQL server that is provisioned for Invictus is created without any Minimum TLS version. For security reasons some customers require a minimum of TLS 1.2.

I don't expect any problems adding this, since the applications connecting to the SQL Database are TLS 1.2 compliant.

• Bump apiVersion for SQL Server to at least 2019-06-01-preview
• Add "minimalTlsVersion": "1.2" to the properties

[Anton for client Vervaeke]

Improvement for visibility: In the current dashboard the flow runs have alternating background colors. Would it be possible to add that as well to the new dashboard? That helps a lot to visually separate the runs from each other.

[Anton for client Vervaeke]
Question: Is there a way to show all flow columns by default for all users? Right now, every user must manually select the columns that aren’t shown automatically, e.g. when there are 8 columns, it only shows 6 by default, so they have to know that they can select the other 2 to also see that information. It would be easier if there was a setting or something like that to show all columns by default.

[Pim Simons]
Recently Microsoft published an update on how they emit telemetry for Application Insights in Azure Logic Apps (Standard). See https://techcommunity.microsoft.com/t5/integrations-on-azure-blog/application-insights-enhancements-for-azure-logic-apps-standard/ba-p/3758909.

This is in public-preview and opt-in, you need to edit your Logic App's host.json file and add:

However, after trying this I found out that after enabling this option the information from my Logic App does not appear in the Invictus for Azure dashboard anymore. It appears not only is the telemetry to Application Insights changed, but also the telemetry outputted to the EventHub.

While this is currently public-preview and opt-in, we need to make sure we support this new way of outputting telemetry as well.

https://teams.microsoft.com/l/message/19:[email protected]/1687338902751?tenantId=7517bc42-bcf8-4916-a677-b5753051f846&groupId=f2ebbeb0-4e8b-4764-9835-98011ae154e9&parentMessageId=1687338902751

We're currently using "Vault access policy" for the internal Invictus Keyvault, when switching to MI (#168) we can also change to using RBAC for this keyvault.

I have had multiple requests from my customer for the following feature in Invictus Dashboard. Let me context the situation:

In my current setup (mainly integration) I am logging errors to the dashboard using the ‘Event Text’ property baked in on the Invictus Dashboard:

When the customer has 100+ errors (mostly functional errors so beyond my control) they have to open 100+ entries on the dashboard one by one to check what the error message is. Customer is wondering whether an export feature is in the pipeline/can be added to the pipeline that can export either all properties from detailed info on the dashboard or a feature where you can select what fields to add in the export.

Right now, there is an export functionality, but that does only take an export of the main entry (head level on the dashboard). It does not add the sublevel details.

Is your feature related to a problem? Please describe.
The hybrid Invictus framework installation guide only provides the Helm command to deploy a Helm chart but I'm guessing there is some missing information or there is some obscure assumptions being made.

What solution do your propose?
I'm guessing there needs to be an ARM/Bicep template deployed before we can run this Helm command? So you can place the necessary connection strings/names as arguments for the command parameters?
We should provide maybe more info on each argument, as right now, it only describes the name of the parameter, not the function, the reason why it is there, or where you should find it.

Configure Netlify for documentation previews so that you can check the rendered version of the PR before merging.

This is what it looks like:

You can preview the docs but also blocks PRs where packaging fails.

Configuration

You can easily configure this by:

1. Installing the Netlify GitHub app (https://github.com/apps/netlify)
2. Create a Netlify account
3. Add site for this GitHub repo

[Annelotte Mons]
I've noticed that the dashboard doesn't have a cancelled state. Is this something that could be added? Right now cancelled runs in Azure are showing up in the dashboard as "Succeeded", making it harder to track them.

We cancel runs when we detect a malformed message for example. In order to not overload the dashboard with false negatives we opt to cancel the runs instead.

[Keith Grima]
Hi Annelotte, we will check what the diagnostic data is sending and see how this can be implemented

Some customers don't allow public access to storage accounts and are enforcing this with policies.
Since all our blob storages are private, we could disable public access on the Storage Account level

• Bump apiVersion to at least "apiVersion": "2021-01-01"
• Add "allowBlobPublicAccess": false to properties

Is your feature related to a problem? Please describe.
Currently, the hybrid Invictus on-premise installation guide does not have any end-result verification to show if all the previous steps have done correctly.

What solution do you propose?
Provide a way to status-check the installation/configuration so that users know that the installation process was successful on their machine.

Is your feature request related to a problem? Please describe.
Whenever a flow has a lot of errors (for example several thousand), we want to be able to bulk resume / resubmit these flows from the IfA dashboard. At the moment we can do several resume / resubmit actions at once but only by manually expanding the table (Load More Results) in the flow view and than clicking Resume / Resubmit. If we want to resume / resubmit several thousands of failed flows, we now need to scroll to the bottom of the page and click the Load More Results button to get 50 more results. This is very time consuming when you need to resume / resubmit several thousands of failed flows.

Describe the solution you'd like
We would like to be able to bulk resume and resubmit failed flows in the dashboard.
For example an option to resume / resubmit all failed flows within a timeperiod for a single flow.

[Ruud Wichers Schreur]
Dank voor de presentatie zojuist, ziet er goed uit! Een handige feature zou zijn dat als er een flow ge-ignored wordt, er een comment toegevoegd kan worden.

Concreet voorbeeld waar dit nuttig kan zijn: De klant waar ik o.a. voor werk (HFG) draaien wij mee in de DevOps daystart. Hier moeten we bij sommige failed flows een ticket aanmaken. Afhankelijk van of er voor ons nog vervolgstappen zijn ignoren we hem of laten we hem even op failed staan. Als we de link naar een ticket bijvoorbeeld als comment toe kunnen voegen weten andere mensen die het dashboard gebruiken dat deze in behandeling is.

Is your feature related to a problem? Please describe.
When installing the hybrid Invictus framework components on Windows, the Microk8s may not be able to activate Hyper-V (not enough additional info on reason, may be because it is already active).

Describe what solution you propose:
Mention the possibility that activating Hyper-V through Microk8s installation may fail. Maybe there is a FAQ or Troubleshooting guide for Microk8s that explains more why it may fail to activate Hyper-V during installation.

[Martin Peters]
In the Invictus dashboard, the Logic App name is not visible for Logic Apps Standard. Only the workflow name. Is it possible to add the Logic App name, for example as LogicAppName/WorkflowName in the "Workflow Name" column? In our case the workflow name is the same for different Logic Apps, so we cannot see which workflow the line belongs.

[Pim Simons]
At several of our customers I have some flows that either don't require the Resubmit and Resume buttons or the flow implementation does not support it (no Logic Apps). Archiveis still used to archive manually handled error instances.

I usually just explain that for these flows the Resubmit and Resume cannot be used and the users should ignore them. But I think it would be better if we could hide these buttons for these flows. For example by introducing an extra setting under the Advanced Settings of a flow like this:

Not sure if already included into the code, but either way the parameter name for the deploy.ps1 script is missing in the installations docs

@pim-simons There is no entry in the documentation that the Invictus and Infra resourcegroups should be linked to the yaml release pipeline

Some paths need updating when running on linux build agents

Framework

$ArtifactsPath$archive to $ArtifactsPath/$archive

Dashboard

$ArtifactsPath$archive to $ArtifactsPath/$archive

$ArtifactsPath\dashboard.zip to $ArtifactsPath/dashboard.zip

Publish docs on *.invictus-integration.com to make it more consumer friendly, for example azure.invictus-integration.com.

This can be easily done with CloudFlare similar to Arcus (http://eventgrid.arcus-azure.net/, http://webapi.arcus-azure.net/, ...)

Unless I am missing something, this requires some attention. Especially setting authentication against pre-configured AAD roles needs both explanation and examples.

Hi,

At a customer we are using groups to grant access to the dashboard flows and that works.
But I have noticed that when someone is part of a group that has access to a flow and also a dashboard system admin, then those flows are appearing two times for them.
Is this normal behavior and is there a way to fix this?

Thank you.

Kind regards,
Anton

[Anton for client Vervaeke]
Small improvement: Is it possible to be able to click on a folder name to open it as well as the dropdown arrow? Intuitively I always click on the name (as we do now in the current dashboard) and then have to change to click on the arrow.
It would be logical if clicking on the name worked as well.

The Pub/Sub connector has some issues

• using an (very) old version of Web API development
• using deprecated Azure packages with vulnerabilities.
• using not so secure Basic Authentication

However, the packages cannot easily be upgraded.
Reason: the pubsub library was never meant to be used via an API. If a listener reads a message from SB it caches it on that instance, if via an API call you try to complete a message and it is not on the same instance as the listener it fails (it's in-memory stateful).

Best to rework them.

Add correlation tracking to the dashboard and all the Invictus Components.

[Components]
This includes two parts: handling incoming correlated requests and adding the correct correlation headers to the callback. W3C correlation means that we should be able to handle the trace-parent request header either from HTTP or RabbitMQ.

[Dashboard]
W3C correlation means that we should be able to handle the trace-parent Service Bus property, and link it to the storage account we're using to store the received records.

Sequence controller Framework component seems to be missing from the navigation sidebar:
https://invictus-integration.github.io/docs-ifa/#/framework/components/sequencecontroller

[Anton for client Vervaeke]
Small improvement for visibility: Is it possible to add a keyword to all action buttons? Right now, only the ‘Resume’ and ‘Resubmit’ have a description next to it.

Is your feature related to a problem? Please describe.
Currently, after the Helm deploy, we provide a suggestion to do 'something' with Azure ACR, but the purpose or reason for this is not explained.

What solution do you propose?
We should make sure that after the Helm deploy, we provide more info on why to use Azure ACR. I'm guessing it is to more easily manage future deployments on on-premise machines? A clear explanation on this and a visual confirmation of such deployment result would be helpful.

• Freeze the scope of V2
• Release it officially
• Make a companywide communication

Is your feature related to a problem? Please describe.
Currently, we still use the insensitive master branch instead of the more universal and inclusive main branch name.

What solution do you propose?
Rename the default master branch to main.

Currently when you set the date filter on one flow in the dashboard and then switch flows, the filter gets reset to 24 hours.

Would be a useful addition to have this date range remembered when switching.

From the documentation it is not clear what the difference between the Folder Admin and Operator roles are.
See:
https://invictus-integration.github.io/docs-ifa/#/dashboard/role-management?id=folder-admin
https://invictus-integration.github.io/docs-ifa/#/dashboard/role-management?id=operator

Folder admin states: this role can perform certain administrative tasks only within the folder where he/she is assigned to, and cannot manage users on the dashboard in general.
Operator states: Users with this role can perform certain tasks within the folder and its flows.

What are these "certain tasks" per role. To me it is not immediately clear when to choose which role.

On the page https://invictus-integration.github.io/docs-ifa/#/dashboard/installation/dashboard-buildpipeline?id=yaml-pipeline we have a link to https://invictus-integration.github.io/docs-ifa/#/dashboard/installation/pipelines/dashboard.build.yaml, this however results in a 404.

If I check the docs in the GitHub repo this does work: https://github.com/invictus-integration/docs-ifa/blob/master/dashboard/installation/dashboard-buildpipeline.md#yaml-pipeline points to https://github.com/invictus-integration/docs-ifa/blob/master/dashboard/installation/pipelines/dashboard.build.yaml which works.

I don't really understand why this works in the GitHub repo but not on the website...

Here are all the URL's that are used that do not work from the https://invictus-integration.github.io/docs-ifa page:
https://invictus-integration.github.io/docs-ifa/#/dashboard/installation/pipelines/dashboard.build.yaml
https://invictus-integration.github.io/docs-ifa/#/dashboard/installation/pipelines/dashboard.release.yaml
https://invictus-integration.github.io/docs-ifa/#/framework/installation/pipelines/framework.build.yaml
https://invictus-integration.github.io/docs-ifa/#/framework/installation/pipelines/framework.release.yaml

[Anton Everaert]
The transco config used a SQL stored procedure that was managed by the customer and returned all fields from a certain table, of which I only needed a particular one (not the first one). I experimented and investigated for a bit, but it turns out that it's not possible with the current transco. Although it is possible with the Biztalk Transco Component.

Eventually I was able to convince the customer to create a separate stored proc that only returned that particular field.

So, it could be useful if it was possible to select a particular field from the SQL stored procedure response, especially for SQL databases that we're not managing.

https://teams.microsoft.com/l/message/19:[email protected]/1686296028441?tenantId=7517bc42-bcf8-4916-a677-b5753051f846&groupId=f2ebbeb0-4e8b-4764-9835-98011ae154e9&parentMessageId=1686292940787