Follow up to #1.

We could also have a workflow that takes an arbitrary command and opens a new PR with the changes (could be under the maintainer GitHub name).

For example this would be useful to update dependencies with commands like jlpm run update:dependency --regex @lumino/ latest

We could reduce time it takes to update snapshots by only installing the browser we are actually using. By default playwrigtht install installs all default browsers (https://playwright.dev/docs/browsers); npx playwright install browser-name will only install browser-name browser.

Problem

We will want to easily keep this repo and its v1 tag up to date.

Proposed Solution

We should adopt Jupyter Releaser and copy the pattern it uses to update its v1 tag.

When asking bot to perform snapshot update or backport it would help if it reacted with thumbs up on the comment that triggered the action. This is what dependabot does when asked to rebase.

Problem

Some packages put rather important functionality behind [extras], and these are not followed up by install-minimums, which can leave a broken test state if not specified directly.

Proposed Solution

• figure out how to make the spec building discover these when solving
• add something similar to test-sdist's package_spec

Alternatives

• document options for install-minimums users
  • avoid using [extras] 😝
  • re-capture the upstream's [extra] as another extra

Additional context

• example log for a PR that exists mostly to opt in to jsonschema[format-nongpl].

Description

The runs are picking up the pytest config of the host package.

cf https://github.com/jupyter/jupyter_client/runs/5740132235?check_suite_focus=true

rootdir: /home/runner/work/jupyter_client/jupyter_client, configfile: pyproject.toml

We should make sure that the test takes place in a directory that does not share a root directory with the host package.

Problem

It is hard to standardize best practices for the large number of Jupyter repos.

Proposed Solution

Add maintainability audit checklist to maintainer-tools

• Point to example implementation PRs and current config files as appropriate
• Advise annual audit for updated list and current best practices - upgrade version tags, etc.

Ideas to include:

• Use Jupyter Releaser
• Triage Label enforcement
• Use "Base Setup" GitHub Action
• Cron job for daily builds
• Precommit setup
  • Add minimal pre-commit file without auto-formatters
  • Enable and run the auto-formatters as a separate commit
  • Merge the PR
  • Add a new PR that adds the .git-blame-ignore-revs file
  • Add a new PR for flake8 and/or eslint
  • Run autoformatters on backport branches to make backporting easier
• ReadTheDocs setup with PR builds
• Pydata theme with MyST
• Example binder and PR commenter using a workflow or jupyterlab-probot config
• 4 kinds of docs
• Coverage shown on PRs
• Coverage thresholds enforced in pytest
• Tests for downstream libraries
• CI job minimum dependency versions
• CI job against prerelease dependency versions
• Test with warnings as errors
• Test with warnings as errors in docs
• No upper version constraints on dependencies
• Include tests in sdist but not wheel - tests should be at the top level of the repo and excluded when using find_packages
• Run the test suite on the conda-forge recipe
• Run pre-commit autoupdate or use pre-commit.ci
• Add .github/dependabot.yml file with weekly updates
• Consider adding mypy - copy pip config, add py.typed file
• Use dev requirements for anything that isn't strictly required to run tests, e.g. pre-commit, and recommend using pip install -e ".[dev,test]"

Perhaps tie this to an annual update of supported Python versions.

Problem

The only way to set the versions is using a matrix. For workflows that don't otherwise need a matrix, we should be able to provide these versions as inputs.

Proposed Solution

Add inputs to the base setup, add docs, and use the input in the minimum versions example.

Problem

Right now the PR script workflows creates a new commit with "Run maintainer script" as the message.

It would be nice to be able to pass a custom commit message as part of the workflow inputs. For example "Lint" when running a lint script.

Proposed Solution

Probably we can pass the input down to the script, so it can be picked up here:

Additional context

Example commit currently generated: jupyterlab/jupyterlab@6971c03

Problem

At the moment the PR script runs the following command which is specific to Python repos:

Proposed Solution

This could be similar to the jupyter_releaser choosing which command to run based on the existence of pyproject.toml / setup.py / package.json.

Additional context

This would make the PR script more generic.

Description

If a dependency includes the > directive, we currently convert it to == here. This is technically incorrect, as we should instead technically find the next available version greater than that version and constrain to that version.

Reproduce

Depend on a >, see example failure.

Expected behavior

We get the correct min version.

Context

We can use the PyPI JSON API to get the right version.

import requests
data = requests.get('https://pypi.org/pypi/notebook/json').json()
versions = list(data['releases'])

• Operating System and version:
• Browser and version:
• JupyterLab version:

Paste the output from running `jupyter troubleshoot` from the command line here.
You may want to sanitize the paths in the output.

Paste the output from your command line running `jupyter lab` here, use `--debug` if possible.

Paste the output from your browser Javascript console here, if applicable.

Description

Using the "Run Script on PR" workflow from the GitHub Actions directly does not seem to work anymore:

Reproduce

Example run: https://github.com/jtpio/maintainer-tools/runs/5173329507?check_suite_focus=true

Expected behavior

Ideally it should still be possible to run one-off commands without having to configure the target repo.

This is useful when maintainers know they have push rights to the repo.

It looks like depend-a-bot automatically adds the dependencies label when it bumps deps.

Could that be one of the allowed enforced labels?

e.g. mamba-org/quetz#484

Problem

We would like to be able to test against the minimum versions we claim to support. There is discussion about adding this to pip, but it has stalled.

Proposed Solution

We can add an action that does the following:

• Creates a wheel using python -m build --wheel
• Uses pkginfo to parse the wheel file to get its requirements, e.g. (Wheel("./dist/jupyter_server-1.14.0.dev0-py3-none-any.whl").requires_dist)
• Parses the requirements using packaging to parse the requirements, looking for ~ or >. e.g.

from packaging.requirements import Requirement
r = Requirement('anyio (>=3.1.0)')
if not r.extras:
   for specifier in r.specifier:
       if '~' in specifer.operator or `>` in specifier.operator:
              # save as a constraint

• Generate a constraints file
• Install the current package using the constraints file, with optional extras (defaults to test).

Additional context

cf jupyter-server/jupyter_server#678 where we discussed adding this capability to jupyter_server itself.

Description

The GH Action test-sdist currently install the SDist package (line 30) before testing. This installation step however generates a BDist (from the SDist) and installs that instead (you can see it happening in this failed job).

Problem

Testing a wheel is problematic because the project may remove test files from binaries, making the test step impossible. I found this issue when doing exactly that in jupyter/nbconvert#1822.

Possible Solution

I'm not too versed in pip, but installing in --editable/-e mode does keep the source files intact, so it could be inserted directly in pip invocation (line 30). If forcing editable mode is not desired, only the default package_spec (line 6) could be modified.

Problem

By default the check-links action fails without too much information when there is a broken link.

Example run: https://github.com/jupyter/notebook/actions/runs/3570433572/jobs/6001397371

Proposed Solution

Ideally the broken link(s) should be printed in the logs so it's easier to find and fix them.

Additional context

Noticed in https://github.com/jupyter/notebook/actions/runs/3570433572/jobs/6001397371 on https://github.com/jupyter/notebook

Problem

We often have to go back and re-label PRs that are missing appropriate labels so they are appropriately labeled.

Proposed Solution

We could include an action here that wraps enforce-label-action with the labels required by github-activity and document how to use it on other repos that use Jupyter Releaser.

Problem

In the scope of a PR, seeing a long-broken URL is not helpful for getting a contributor to a ✔️ .

Proposed Solution

Add a switch, (e.g. default changed_files_only: false) which will only check files that are changed on a PR

Additional context

jupyter/accessibility#139

Description

The enforce label action is using the original github context to check the label, which means that our 60 second delay is having no effect. We need to write our own GitHub Script that gets the current set of labels.

Reproduce

Create a ChangeLog PR using Jupyter Releaser.
Releaser adds a "documentation" label after creating the PR.
The Enforce Label action fails.

Expected behavior

The enforce label workflow should pass.

Problem

We have a lot of duplication of our workflow files for thing like installing and configuring Python, caching, etc.

Proposed Solution

Provide a base action here that is similar to the one used in jupyter_releaser

Problem

We should be able to trigger a PR script using a comment. That way it is transparent when it is triggered and is easier for the maintainer to invoke.

Proposed Solution

We can make the PR script into an action, and show an example of triggering it using https://github.com/Khan/pull-request-comment-trigger. The action is run using the commenter's credentials. This can also be a general pattern of issue comments that trigger actions.

Problem

We have a pattern of testing downstream packages, but it is currently brittle. We are installing in the same base environment, and the testing dependencies may differ between the downstream package(s) and/or the package under test.

Proposed Solution

We can provide an action here that has the following inputs:

package_name: required
package_version: optional
test_command: optional
env_values: optional

It will:

• Create a virtual environment
• Install the given package in the test environment with the given version specification, with the default being pip install --pre -U --upgrade-strategy=only-if-needed <package>
• Install the package under test (not test dependencies) in the test environment using pip install . --force-reinstall
• Run the test command, by default pytest --pyargs <package_name> with the given environment values

Description

The CI fails in jupyter-collaboration. Since it's happening at the "Base Setup" step, I'm opening an issue here, but I'm not sure what is going on.

Problem

Snapshot updates take a long time and can introduce regressions in presence of flaky scenarios

Proposed Solution

Allow to specify a subset of snapshots to consider for updating (adding an argument in actions/update-snapshots/action.yml).

Additional context

I envision that JupyterLab action would parse out the search term out of a comment.

Problem

Some of our options are booleans that are presented as text fields

Proposed Solution

Use the new github.blog/changelog/2021-11-10-github-actions-input-types-for-manual-workflows

Problem

We often approve PRs and then have to monitor until CI passes before merging.

Proposed Solution

Add an action and example workflow that waits for a check suite to finish, and also check for an automerge label on the pull request. If true, it merges the PR. We would use the action on this repo as well. Note that for repos like JupyterLab, that allow anyone to apply labels by invoking meekseeksdev, branch protection rules should be in affect to ensure at least one approval, and potentially have approvals revoked when subsequent changes are pushed. In general repos should be using that 😄.

Additional context

Conda-forge has a similar action available using their bot. This could also be implemented as part of jupyterlab-probot, but then it is not as widely useful since not all orgs want to/can install that bot.

Problem

Snapshots on my PR are failing prior to taking a snapshots which I cannot repro locally. The PR job contains both assets and snapshots but the update job only assets, compare:

from https://github.com/jupyterlab/jupyterlab/actions/runs/4682777719?pr=14356

with

from https://github.com/jupyterlab/jupyterlab/actions/runs/4682788709

Proposed Solution

Upload both on failure

Additional context

In lab repo we use:

      - name: Upload Galata Test assets
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: jupyterlab-galata-test-assets
          path: |
            galata/test-jupyterlab-results
            galata/test-results

      - name: Upload Galata Test report
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: jupyterlab-galata-report
          path: |
            galata/playwright-report

Description

We are currently assuming that the tests ship with the package and can be run from site-packages.
We are also re-using the virtual environment.

Reproduce

cf https://github.com/jupyter-server/jupyter_server/runs/5527570612

Expected behavior

We should have test isolation and test the project in-place.

Context

We need to remove the virtual env directory between each run.
We need to download the sdist, install in editable mode, and
run the tests from that folder

Problem

Not every org wants to use jupyterlab-probot to add binder links.

Proposed Solution

Provide an action here that accepts an optional urlpath.

Problem

Checkouts can be slow for larger repos like JupyterLab.

Proposed Solution

We should add an option to do a shallow checkout and make it the default. You shouldn't need a full checkout unless you are manipulating tags or otherwise using git history.

Currently the PR_HEAD_REF is taken as is and passed to the Binder link:

However this can create issues when a branch has a special character, for example #, as noticed in jupyter/notebook#7178. The Binder link becomes https://mybinder.org/v2/gh/haok1402/notebook/issue#7147?urlpath=tree, but Binder fails to build it:

However Binder can build the repo fine if the special character is URL encoded: https://mybinder.org/v2/gh/haok1402/notebook/issue%237147

So this binder-link action should probably try to encode the PR_HEAD_REF before creating the link.