Setting up this application as a development environment is currently a fraught process. There are PRs in flight (see: #803), but this only gets you to an environment where the unit tests pass and the application starts. There remains a problem with interacting with Auth0, AWS, Slack, Kubernetes. It has been suggested (here) that a "developer" mode/configuration should be available so that the application works without needing to be plugged into these other services.

We should discuss how to do this on this ticket. I would like to know:

• Is my list of problem third party "services" complete.
• For each list item, how could they be reasonably be coded around.
• What further "institutional" information is needed for this to work in terms of settings and configuration.

Thanks in advance..!

Description

There is an error when attempting to run the "Reset home directory" helm chart via the Control Panel UI. The error may be related to a failure to update the helm repo, as the sentry log shows that the "mojanalytics/reset-user-efs-home" chart is not found.

The full stack trace of the error can be found in sentry:
https://ministryofjustice.sentry.io/issues/3941226983/?query=is%3Aunresolved&referrer=issue-stream&stream_index=17

Note: in sentry there are multiple Helm related errors, that all may be linked

To reproduce

1. Log in to control panel
2. Go to the tools page
3. Click the "reset home directory" link
4. Reset home directory
5. To a user it appears it completed successfully. This is because the task is run in the background via django-channels. So check Sentry to see the stack trace of the background task

Expected behaviour

The task to run the helm helm chart should run successfully, and the users home directory is reset. The bug is fixed when:

1. the steps to "reset home directory" can be completed via Control Panel (as steps above) and there is no error reported in sentry
2. the commands defined in the helm chart are completed. This can be checked by connecting to the rstudio tools pod and checking that the files have been moved to the backup folder (defined in the helm chart command)

The logs for cpanel are flooded with these errors:

2018-09-13 10:45:32,267 django.template DEBUG Exception while resolving variable 'code_style' in template 'rest_framework/login.html'.
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/django/template/base.py", line 835, in _resolve_lookup
    current = current[bit]
  File "/usr/lib/python3.6/site-packages/django/template/context.py", line 83, in __getitem__
    raise KeyError(key)
KeyError: 'code_style'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/django/template/base.py", line 841, in _resolve_lookup
    if isinstance(current, BaseContext) and getattr(type(current), bit):
AttributeError: type object 'RequestContext' has no attribute 'code_style'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/django/template/base.py", line 849, in _resolve_lookup
    current = current[int(bit)]
ValueError: invalid literal for int() with base 10: 'code_style'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/django/template/base.py", line 856, in _resolve_lookup
    (bit, current))  # missing attribute
django.template.base.VariableDoesNotExist: Failed lookup for key [code_style] in [{'True': True, 'False': False, 'None': None}, {'csrf_token': <SimpleLazyObject: <function csrf.<locals>._get_val at 0x7fd937388488>>, 'user': <SimpleLazyObject: <function AuthenticationMiddleware.process_request.<locals>.<lambda> at 0x7fd9373f9158>>, 'perms': <django.contrib.auth.context_processors.PermWrapper object at 0x7fd93723c668>, 'STATIC_URL': '/static/', 'TIME_ZONE': 'UTC'}, {}, {'form': <AuthenticationForm bound=False, valid=Unknown, fields=(username;password)>, 'view': <django.contrib.auth.views.LoginView object at 0x7fd93722da20>, 'next': '', 'site': <django.contrib.sites.requests.RequestSite object at 0x7fd93723cac8>, 'site_name': '100.122.14.82:8000'}, {'block': <Block Node: head. Contents: [<TextNode: '\n\n      '>, <Block Node: meta. Contents: [<TextNode: '\n        <meta http-equiv'>]>, <TextNode: '\n\n      <title>'>, <Block Node: title. Contents: [<IfNode>, <TextNode: 'Django REST framework'>]>, <TextNode: '</title>\n\n      '>, <Block Node: style. Contents: [<TextNode: '\n        '>, <Block Node: bootstrap_theme. Contents: [<TextNode: '\n          <link rel="sty'>, <django.templatetags.static.StaticNode object at 0x7fd9372bbda0>, <TextNode: '"/>\n          <link rel="'>, <django.templatetags.static.StaticNode object at 0x7fd9372bbe48>, <TextNode: '"/>\n        '>]>, <TextNode: '\n\n        <link rel="styl'>, <django.templatetags.static.StaticNode object at 0x7fd9372bbeb8>, <TextNode: '"/>\n        <link rel="st'>, <django.templatetags.static.StaticNode object at 0x7fd9372bbf60>, <TextNode: '"/>\n        '>, <IfNode>, <TextNode: '\n      '>]>, <TextNode: '\n\n    '>]>}, {'block': <Block Node: style. Contents: [<TextNode: '\n        '>, <Block Node: bootstrap_theme. Contents: [<TextNode: '\n          <link rel="sty'>, <django.templatetags.static.StaticNode object at 0x7fd9372bbda0>, <TextNode: '"/>\n          <link rel="'>, <django.templatetags.static.StaticNode object at 0x7fd9372bbe48>, <TextNode: '"/>\n        '>]>, <TextNode: '\n\n        <link rel="styl'>, <django.templatetags.static.StaticNode object at 0x7fd9372bbeb8>, <TextNode: '"/>\n        <link rel="st'>, <django.templatetags.static.StaticNode object at 0x7fd9372bbf60>, <TextNode: '"/>\n        '>, <IfNode>, <TextNode: '\n      '>]>}]

This occurs every 5 seconds, suggesting the readiness probe, which is configured:

    Readiness:  http-get http://:http/auth/login delay=5s timeout=1s period=5s #success=1 #failure=3

And sure enough, the readiness is failing:

Events:
  Type     Reason     Age                From                                                   Message
  ----     ------     ----               ----                                                   -------
  Warning  Unhealthy  58m (x20 over 1h)  kubelet, ip-192-168-14-128.eu-west-1.compute.internal  Readiness probe failed: Get http://100.101.194.119:8000/auth/login: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

When you try to revoke a user's access to a bucket when the user no longer has an IAM role, it results in a 'sorry, there is a problem with the service error'.

Also cannot delete apps when a user with no IAM policy is on the access list.

Dependabot Preview will be shut down on August 3rd, 2021.

This repository has been configured to only receive security updates from Dependabot Preview; however, the GitHub-native version of Dependabot does not provide security updates through a configuration file. If you wish to keep receiving security updates, please enable them via your repository's settings.

Once you have enabled Dependabot security updates, you can remove Dependabot Preview from your organization or personal account and your migration is complete.

The job succeeds even if an invalid image tag is provided and the helm chart install fails with ImagePullBackoff or similar.

We should add a check for the existence of the specified image tag.

When we create a user, most of the time it seems that auth0 creates a nickname on the auth0 user object using the first part of their email address. This doesn't happen in a small minority of cases (<5%). To avoid this breaking for those unfortunate users we should explicitly set nickname when creating a user.
Related to:
ministryofjustice/analytics-platform#98

Steps to reproduce

1. A user is granted access to /folder_1/* within a bucket
2. The user has their access to /folder_1/* removed but they are granted access to /folder_2/*

The user will still have access to /folder_1/*, that is, user IAM policies are not updated when path-specific access is modified.

When a user is given path-specific access to a folder, say /folder_1/* as per the user guidance, their IAM policy will provide access to arn:aws:s3:::bucket/folder_1/*/*.

The user should have access to arn:aws:s3:::bucket/folder_1/*.

When a user selects the 'Ministry of Justice' or 'Analytical Platform Control Panel' links in the control panel header, they are not taken to their home page as they should be.

Notification banners currently span the full width of the page (example below).

Their width should be constrained to the same width as the rest of the page content.

Data warehouse to do:

Data warehouse bucket details page:

• Typeahead that searches and shows the user name and the github name in branch i.e. Robin Linacre (RobinL)

• Filter out nulls and jenkins from the list

Copy

• Convert 'make user admin of this data source' to 'Grant access to this data to other users'. This only shows if you are an admin. Three options to select from:

  • Read access (default option)
  • Read write access
  • Make admin (if this is selected, a prompt shows 'Making user admin will allow them to confer access rights on additional users')

• Change 'users with admin access' to 'Data access group'

Main /data_warehouse page

• On the /warehouse_data, need a button that clicks through to AWS. "View data in AWS"

• Delete 'delete warehouse data source' button in /warehouse_data - this should be available in the data source page, but not in the main listing

Question mark popuops by the headers of the table in /warehouse_data

• Name: "The name of the bucket in AWS"
• "User has admin access and :user has read write". Convert to "Your access level". Question mark next to your access level, with a popup that explains what the access levels mean

Create data source page

Main header: "Create secure data storage folder"

Subheader text: "Create a new folder to store data, with an associated data access group"

"Folder name" rather than "warehoue data source bucket name"

Hi there

We now have a process to manage github collaborators in code:
https://github.com/ministryofjustice/github-collaborators/blob/main/README.md#github-external-collaborators

Please follow the procedure described there to grant @aidanmews access to this repository.

If you have any questions, please post in #ask-operations-engineering on Slack.

If you create an app in controlpanel and view it in controlpanel before the app has been deployed (by Concourse) then you get an exception:
https://sentry.service.dsd.io/mojds/control-panel-api/issues/36471/

According to the guidance this is the correct order: https://moj-analytical-services.github.io/platform_user_guidance/deploying-an-r-shiny-app.html#basic-deployment but whatever the order we should probably fail better.

I experienced this on https://github.com/moj-analytical-services/sdt-email
https://controlpanel.services.alpha.mojanalytics.xyz/webapps/125/ "Sorry, there is a problem with the service"
until it was deployed.

--
BTW there seems a related but separate error for the same thing in the old cpanel:
https://controlpanel.services.alpha.mojanalytics.xyz/webapps/125/ "Sorry, there is a problem with the service"
https://sentry.service.dsd.io/mojds/control-panel-frontend/issues/34769/

Viewing any app in dev cpanel gives an error: https://cpanel-master.services.dev.mojanalytics.xyz/apps/11

Found 0 ingresses for app kpi-s3-proxy
Error: Found 0 ingresses for app kpi-s3-proxy

👻 Brief Description

Current users cannot view items in S3 buckets that have versioning on.

🐕 Steps To Reproduce

Steps to reproduce the behaviour:

1. Login to the console and navigate to S3 as a alpha_user_*
2. Any versioned bucket
3. The error below appears

🚓 Expected behaviour

Users should be able to view and download files

➕ Additional context

When a user is given path-specific access they are unable to list files in the S3 console. This is because the ListBucket operation applies to a path resource not the bucket itself.

The ListBucket operation should apply to the whole bucket not just the specific path.