microsoft / innereye-inference Goto Github PK

After deployment to dev, trigger an update in InnerEye-Gateway using repository_dispatch

The build-and-deploy job is currently failing due to invalid Azure credentials. It's like that these have expired and need to be updated.

A recent package update for one of the dependent packages has left the app unable to authenticate with Azure.

Add a brief explanation of how to run a quick test that the server is working.

e.g.

CURL -i -H "API_AUTH_SECRET: <val of CUSTOMCONNSTR_API_AUTH_SECRET>" http://localhost:5000/v1/ping

The build-linux GitHub workflow is currently failing to be automatically triggered. The triggers may need to be reviewed.

The following improvements should be added to the inference documentation:

• Add instructions on how to run pytest tests
• Clarify where logs can be found in a deployed App Service (Monitoring -> Log Stream)
• Add explanation and instructions for the WAF setup

I updated the InnerEye-Inference project to the newest version and now it doesn't run at all - I'm using the same commands that worked before, which are mentioned in README.md

az login --use-device-code
az webapp up --sku S1 --name inferenceapp --subscription <subscription_name> -g InnerEyeInference --location <location>

Which gives me the output:
Could not auto-detect the runtime stack of your app.
HINT: Are you in the right folder?
For more information, see 'https://go.microsoft.com/fwlink/?linkid=2109470'

I created set_environment.sh with all variables set with correct data (they have worked the whole time for inference).

Do you have any idea what could be the problem here and how we could solve it? (for example, should we define any additional path in the configs?)

Make sure that the action is rerun daily to pick up any possible problems with 3rd party packages changing.

Unable to deploy code to an app, service principal pair due to an issue with conflicting module version requirements. Tail of error message is shown below:

ERROR: Cannot install -r requirements.txt (line 12), -r requirements.txt (line 9) and azure-core==1.22.1 because these package versions have conflicting dependencies.
[10:22:51+0000] INFO: pip is looking at multiple versions of absl-py to determine which version is compatible with other requirements. This could take a while.
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
[10:22:51+0000]
[10:22:51+0000] The conflict is caused by:
[10:22:51+0000] The user requested azure-core==1.22.1
[10:22:51+0000] azure-identity 1.7.0 depends on azure-core<2.0.0 and >=1.11.0
[10:22:51+0000] azure-mgmt-core 1.3.1 depends on azure-core<2.0.0 and >=1.23.0
[10:22:51+0000]
[10:22:51+0000] To fix this you could try to:
[10:22:51+0000] 1. loosen the range of package versions you've specified
[10:22:51+0000] 2. remove package versions to allow pip attempt to solve the dependency conflict
[10:22:51+0000]
WARNING: You are using pip version 22.0.4; however, version 22.3.1 is available.
You should consider upgrading via the '/tmp/8dadc2ab9883def/antenv/bin/python -m pip install --upgrade pip' command. ", 'Timestamp': '2022-12-12 10:22:52', 'ExitCode': '1', 'Recommendation': 'Please review your requirements.txt', 'UrlToGetMoreInformation': 'https://aka.ms/troubleshoot-python'}], 'warnings': []}}. Please run the command az webapp log deployment show -n inf-app-osairis-net -g rg-osairis-net

To improve reproducibility and stability, repository needs environment locking in the same way as the InnerEye-DeepLearning repo.

Documentation is a little spread out - could do with being hosted on Readthedocs to unify things a little more.

... like temporary network problems.

When running the inference service behind an Azure Application Gateway, health checks are needed to ensure that a healthy connection is being maintained. However, the health checks default to the root path, /, which on a healthy inference service returns a 404 as there is nothing configured in app.py to return anything. It would be useful to have this path instead return a 200 response to show that the inference service is alive and running.

Currently there is no system test / GitHub workflow to verify if the app can be successfully deployed to Azure via the az webapp up command. Not sure if this is possible or not but is worth investigating, as it would prevent issues such as #42.

this downloads and checks 70_driver_log.txt. This is generated by the old AML runtime. The new runtime writes a file user_logs/std_log.txt that should be checked instead.

Upgrade to support new models InnerEye-DICOM-RT 1.0.1

https://docs.microsoft.com/en-us/azure/machine-learning/how-to-setup-authentication#use-managed-identity-authentication

from azureml.core.authentication import MsiAuthentication

msi_auth = MsiAuthentication()

ws = Workspace(subscription_id="your-sub-id",
resource_group="your-resource-group-id",
workspace_name="your-workspace-name",
auth=msi_auth
)

The environment locking allows the web app to be deployed, however it does not run successfully and will return 404 responses to curl requests as specified in the README. The locking script needs to be updated in order to place the pip requirements into the requirements.txt file if the Inference service is to run successfully.