Code Monkey home page Code Monkey logo

maintainer-tools's Introduction

Installation | Documentation | Contributing | License | Team | Getting help |

PyPI version Downloads Build Status Build Status Documentation Status Crowdin GitHub Discourse Gitter Gitpod

Binder

An extensible environment for interactive and reproducible computing, based on the Jupyter Notebook and Architecture.

JupyterLab is the next-generation user interface for Project Jupyter offering all the familiar building blocks of the classic Jupyter Notebook (notebook, terminal, text editor, file browser, rich outputs, etc.) in a flexible and powerful user interface.

JupyterLab can be extended using npm packages that use our public APIs. The prebuilt extensions can be distributed via PyPI, conda, and other package managers. The source extensions can be installed directly from npm (search for jupyterlab-extension) but require an additional build step. You can also find JupyterLab extensions exploring GitHub topic jupyterlab-extension. To learn more about extensions, see the user documentation.

Read the current JupyterLab documentation on ReadTheDocs.

Important

JupyterLab 3 will reach its end of maintenance date on May 15, 2024, anywhere on Earth. To help us make this transition, fixes for critical issues will still be backported until December 31, 2024. If you are still running JupyterLab 3, we strongly encourage you to upgrade to JupyterLab 4 as soon as possible. For more information, see JupyterLab 3 end of maintenance on the Jupyter Blog.


Getting started

Installation

If you use conda, mamba, or pip, you can install JupyterLab with one of the following commands.

  • If you use conda:
    conda install -c conda-forge jupyterlab
  • If you use mamba:
    mamba install -c conda-forge jupyterlab
  • If you use pip:
    pip install jupyterlab
    If installing using pip install --user, you must add the user-level bin directory to your PATH environment variable in order to launch jupyter lab. If you are using a Unix derivative (e.g., FreeBSD, GNU/Linux, macOS), you can do this by running export PATH="$HOME/.local/bin:$PATH". If you are using a macOS version that comes with Python 2, run pip3 instead of pip.

For more detailed instructions, consult the installation guide. Project installation instructions from the git sources are available in the contributor documentation.

Installing with Previous Versions of Jupyter Notebook

When using a version of Jupyter Notebook earlier than 5.3, the following command must be run after installing JupyterLab to enable the JupyterLab server extension:

jupyter serverextension enable --py jupyterlab --sys-prefix

Running

Start up JupyterLab using:

jupyter lab

JupyterLab will open automatically in the browser. See the documentation for additional details.

If you encounter an error like "Command 'jupyter' not found", please make sure PATH environment variable is set correctly. Alternatively, you can start up JupyterLab using ~/.local/bin/jupyter lab without changing the PATH environment variable.

Prerequisites and Supported Browsers

The latest versions of the following browsers are currently known to work:

  • Firefox
  • Chrome
  • Safari

See our documentation for additional details.


Getting help

We encourage you to ask questions on the Discourse forum. A question answered there can become a useful resource for others.

Bug report

To report a bug please read the guidelines and then open a Github issue. To keep resolved issues self-contained, the lock bot will lock closed issues as resolved after a period of inactivity. If a related discussion is still needed after an issue is locked, please open a new issue and reference the old issue.

Feature request

We also welcome suggestions for new features as they help make the project more useful for everyone. To request a feature please use the feature request template.


Development

Extending JupyterLab

To start developing an extension for JupyterLab, see the developer documentation and the API docs.

Contributing

To contribute code or documentation to JupyterLab itself, please read the contributor documentation.

JupyterLab follows the Jupyter Community Guides.

License

JupyterLab uses a shared copyright model that enables all contributors to maintain the copyright on their contributions. All code is licensed under the terms of the revised BSD license.

Team

JupyterLab is part of Project Jupyter and is developed by an open community. The maintenance team is assisted by a much larger group of contributors to JupyterLab and Project Jupyter as a whole.

JupyterLab's current maintainers are listed in alphabetical order, with affiliation, and main areas of contribution:

  • Mehmet Bektas, Netflix (general development, extensions).
  • Alex Bozarth, IBM (general development, extensions).
  • Eric Charles, Datalayer, (general development, extensions).
  • Frédéric Collonval, WebScIT (general development, extensions).
  • Martha Cryan, Mito (general development, extensions).
  • Afshin Darian, QuantStack (co-creator, application/high-level architecture, prolific contributions throughout the code base).
  • Vidar T. Fauske, JPMorgan Chase (general development, extensions).
  • Brian Granger, AWS (co-creator, strategy, vision, management, UI/UX design, architecture).
  • Jason Grout, Databricks (co-creator, vision, general development).
  • Michał Krassowski, Quansight (general development, extensions).
  • Max Klein, JPMorgan Chase (UI Package, build system, general development, extensions).
  • Gonzalo Peña-Castellanos, QuanSight (general development, i18n, extensions).
  • Fernando Perez, UC Berkeley (co-creator, vision).
  • Isabela Presedo-Floyd, QuanSight Labs (design/UX).
  • Steven Silvester, MongoDB (co-creator, release management, packaging, prolific contributions throughout the code base).
  • Jeremy Tuloup, QuantStack (general development, extensions).

Maintainer emeritus:

  • Chris Colbert, Project Jupyter (co-creator, application/low-level architecture, technical leadership, vision, PhosphorJS)
  • Jessica Forde, Project Jupyter (demo, documentation)
  • Tim George, Cal Poly (UI/UX design, strategy, management, user needs analysis).
  • Cameron Oelsen, Cal Poly (UI/UX design).
  • Ian Rose, Quansight/City of LA (general core development, extensions).
  • Andrew Schlaepfer, Bloomberg (general development, extensions).
  • Saul Shanabrook, Quansight (general development, extensions)

This list is provided to give the reader context on who we are and how our team functions. To be listed, please submit a pull request with your information.


Weekly Dev Meeting

We have videoconference meetings every week where we discuss what we have been working on and get feedback from one another.

Anyone is welcome to attend, if they would like to discuss a topic or just listen in.

Notes are archived on GitHub Jupyter Frontends team compass.

maintainer-tools's People

Contributors

3coins avatar blink1073 avatar brichet avatar dependabot[bot] avatar fcollonval avatar gabalafou avatar github-actions[bot] avatar jasonweill avatar jtpio avatar krassowski avatar max-schaefer avatar ophie200 avatar pre-commit-ci[bot] avatar tiagodepalves avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

maintainer-tools's Issues

Add a Workflow to Create a PR and Run a Script

Follow up to #1.

We could also have a workflow that takes an arbitrary command and opens a new PR with the changes (could be under the maintainer GitHub name).

For example this would be useful to update dependencies with commands like jlpm run update:dependency --regex @lumino/ latest

Use Jupyter Releaser

Problem

We will want to easily keep this repo and its v1 tag up to date.

Proposed Solution

We should adopt Jupyter Releaser and copy the pattern it uses to update its v1 tag.

Upvote comments asking for actions

When asking bot to perform snapshot update or backport it would help if it reacted with thumbs up on the comment that triggered the action. This is what dependabot does when asked to rebase.

Follow [extras] in install-minimums

Problem

Some packages put rather important functionality behind [extras], and these are not followed up by install-minimums, which can leave a broken test state if not specified directly.

Proposed Solution

Alternatives

  • document options for install-minimums users
    • avoid using [extras] 😝
    • re-capture the upstream's [extra] as another extra

Additional context

  • example log for a PR that exists mostly to opt in to jsonschema[format-nongpl].

Add Maintainability Audit Checklist for Repos

Problem

It is hard to standardize best practices for the large number of Jupyter repos.

Proposed Solution

Add maintainability audit checklist to maintainer-tools

  • Point to example implementation PRs and current config files as appropriate
  • Advise annual audit for updated list and current best practices - upgrade version tags, etc.

Ideas to include:

  • Use Jupyter Releaser
  • Triage Label enforcement
  • Use "Base Setup" GitHub Action
  • Cron job for daily builds
  • Precommit setup
    • Add minimal pre-commit file without auto-formatters
    • Enable and run the auto-formatters as a separate commit
    • Merge the PR
    • Add a new PR that adds the .git-blame-ignore-revs file
    • Add a new PR for flake8 and/or eslint
    • Run autoformatters on backport branches to make backporting easier
  • ReadTheDocs setup with PR builds
  • Pydata theme with MyST
  • Example binder and PR commenter using a workflow or jupyterlab-probot config
  • 4 kinds of docs
  • Coverage shown on PRs
  • Coverage thresholds enforced in pytest
  • Tests for downstream libraries
  • CI job minimum dependency versions
  • CI job against prerelease dependency versions
  • Test with warnings as errors
  • Test with warnings as errors in docs
  • No upper version constraints on dependencies
  • Include tests in sdist but not wheel - tests should be at the top level of the repo and excluded when using find_packages
  • Run the test suite on the conda-forge recipe
  • Run pre-commit autoupdate or use pre-commit.ci
  • Add .github/dependabot.yml file with weekly updates
  • Consider adding mypy - copy pip config, add py.typed file
  • Use dev requirements for anything that isn't strictly required to run tests, e.g. pre-commit, and recommend using pip install -e ".[dev,test]"

Perhaps tie this to an annual update of supported Python versions.

Base Setup Should Accept Python and Node Versions

Problem

The only way to set the versions is using a matrix. For workflows that don't otherwise need a matrix, we should be able to provide these versions as inputs.

Proposed Solution

Add inputs to the base setup, add docs, and use the input in the minimum versions example.

Custom commit message when running a PR script

Problem

Right now the PR script workflows creates a new commit with "Run maintainer script" as the message.

It would be nice to be able to pass a custom commit message as part of the workflow inputs. For example "Lint" when running a lint script.

Proposed Solution

Probably we can pass the input down to the script, so it can be picked up here:

run(f"git commit -a -m 'Run maintainer script' -m 'by {maintainer}' -m '{json.dumps(script)}'")

Additional context

Example commit currently generated: jupyterlab/jupyterlab@6971c03

Support for other types of repos for the PR script

Problem

At the moment the PR script runs the following command which is specific to Python repos:

run("pip install -e '.[test]'")

Proposed Solution

This could be similar to the jupyter_releaser choosing which command to run based on the existence of pyproject.toml / setup.py / package.json.

Additional context

This would make the PR script more generic.

Min Version Check Should Handle Greater Than

Description

If a dependency includes the > directive, we currently convert it to == here. This is technically incorrect, as we should instead technically find the next available version greater than that version and constrain to that version.

Reproduce

Depend on a >, see example failure.

Expected behavior

We get the correct min version.

Context

We can use the PyPI JSON API to get the right version.

import requests
data = requests.get('https://pypi.org/pypi/notebook/json').json()
versions = list(data['releases'])
  • Operating System and version:
  • Browser and version:
  • JupyterLab version:
Troubleshoot Output
Paste the output from running `jupyter troubleshoot` from the command line here.
You may want to sanitize the paths in the output.
Command Line Output
Paste the output from your command line running `jupyter lab` here, use `--debug` if possible.
Browser Output
Paste the output from your browser Javascript console here, if applicable.

Add a minimum version test

Problem

We would like to be able to test against the minimum versions we claim to support. There is discussion about adding this to pip, but it has stalled.

Proposed Solution

We can add an action that does the following:

  • Creates a wheel using python -m build --wheel
  • Uses pkginfo to parse the wheel file to get its requirements, e.g. (Wheel("./dist/jupyter_server-1.14.0.dev0-py3-none-any.whl").requires_dist)
  • Parses the requirements using packaging to parse the requirements, looking for ~ or >. e.g.
from packaging.requirements import Requirement
r = Requirement('anyio (>=3.1.0)')
if not r.extras:
   for specifier in r.specifier:
       if '~' in specifer.operator or `>` in specifier.operator:
              # save as a constraint
  • Generate a constraints file
  • Install the current package using the constraints file, with optional extras (defaults to test).

Additional context

cf jupyter-server/jupyter_server#678 where we discussed adding this capability to jupyter_server itself.

test-sdist is run with BDist files

Description

The GH Action test-sdist currently install the SDist package (line 30) before testing. This installation step however generates a BDist (from the SDist) and installs that instead (you can see it happening in this failed job).

Problem

Testing a wheel is problematic because the project may remove test files from binaries, making the test step impossible. I found this issue when doing exactly that in jupyter/nbconvert#1822.

Possible Solution

I'm not too versed in pip, but installing in --editable/-e mode does keep the source files intact, so it could be inserted directly in pip invocation (line 30). If forcing editable mode is not desired, only the default package_spec (line 6) could be modified.

More verbose `check-links` to find broken links more easily

Problem

By default the check-links action fails without too much information when there is a broken link.

Example run: https://github.com/jupyter/notebook/actions/runs/3570433572/jobs/6001397371

image

Proposed Solution

Ideally the broken link(s) should be printed in the logs so it's easier to find and fix them.

Additional context

Noticed in https://github.com/jupyter/notebook/actions/runs/3570433572/jobs/6001397371 on https://github.com/jupyter/notebook

Add Action to Enforce Labels for PRs

Problem

We often have to go back and re-label PRs that are missing appropriate labels so they are appropriately labeled.

Proposed Solution

We could include an action here that wraps enforce-label-action with the labels required by github-activity and document how to use it on other repos that use Jupyter Releaser.

Enforce Label Action Does Not Properly Delay

Description

The enforce label action is using the original github context to check the label, which means that our 60 second delay is having no effect. We need to write our own GitHub Script that gets the current set of labels.

Reproduce

Create a ChangeLog PR using Jupyter Releaser.
Releaser adds a "documentation" label after creating the PR.
The Enforce Label action fails.

Expected behavior

The enforce label workflow should pass.

Add an Action for Workflow Prep

Problem

We have a lot of duplication of our workflow files for thing like installing and configuring Python, caching, etc.

Proposed Solution

Provide a base action here that is similar to the one used in jupyter_releaser

Make PR Script an Action and Show Its Usage

Problem

We should be able to trigger a PR script using a comment. That way it is transparent when it is triggered and is easier for the maintainer to invoke.

Proposed Solution

We can make the PR script into an action, and show an example of triggering it using https://github.com/Khan/pull-request-comment-trigger. The action is run using the commenter's credentials. This can also be a general pattern of issue comments that trigger actions.

Add Action to Test Downstream Package

Problem

We have a pattern of testing downstream packages, but it is currently brittle. We are installing in the same base environment, and the testing dependencies may differ between the downstream package(s) and/or the package under test.

Proposed Solution

We can provide an action here that has the following inputs:

package_name: required
package_version: optional
test_command: optional
env_values: optional

It will:

  • Create a virtual environment
  • Install the given package in the test environment with the given version specification, with the default being pip install --pre -U --upgrade-strategy=only-if-needed <package>
  • Install the package under test (not test dependencies) in the test environment using pip install . --force-reinstall
  • Run the test command, by default pytest --pyargs <package_name> with the given environment values

Add ability to auto-merge PRs

Problem

We often approve PRs and then have to monitor until CI passes before merging.

Proposed Solution

Add an action and example workflow that waits for a check suite to finish, and also check for an automerge label on the pull request. If true, it merges the PR. We would use the action on this repo as well. Note that for repos like JupyterLab, that allow anyone to apply labels by invoking meekseeksdev, branch protection rules should be in affect to ensure at least one approval, and potentially have approvals revoked when subsequent changes are pushed. In general repos should be using that 😄.

Additional context

Conda-forge has a similar action available using their bot. This could also be implemented as part of jupyterlab-probot, but then it is not as widely useful since not all orgs want to/can install that bot.

When snapshot update fails, not only assets but also report should be uploaded

Problem

Snapshots on my PR are failing prior to taking a snapshots which I cannot repro locally. The PR job contains both assets and snapshots but the update job only assets, compare:

Screenshot from 2023-04-14 18-22-29
from https://github.com/jupyterlab/jupyterlab/actions/runs/4682777719?pr=14356

with

Screenshot from 2023-04-14 18-22-36
from https://github.com/jupyterlab/jupyterlab/actions/runs/4682788709

Proposed Solution

Upload both on failure

Additional context

In lab repo we use:

      - name: Upload Galata Test assets
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: jupyterlab-galata-test-assets
          path: |
            galata/test-jupyterlab-results
            galata/test-results

      - name: Upload Galata Test report
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: jupyterlab-galata-report
          path: |
            galata/playwright-report

Fix the way downstream tests work

Description

We are currently assuming that the tests ship with the package and can be run from site-packages.
We are also re-using the virtual environment.

Reproduce

cf https://github.com/jupyter-server/jupyter_server/runs/5527570612

Expected behavior

We should have test isolation and test the project in-place.

Context

We need to remove the virtual env directory between each run.
We need to download the sdist, install in editable mode, and
run the tests from that folder

Add Binder Badge Action

Problem

Not every org wants to use jupyterlab-probot to add binder links.

Proposed Solution

Provide an action here that accepts an optional urlpath.

Add the ability to do a shallow checkout

Problem

Checkouts can be slow for larger repos like JupyterLab.

Proposed Solution

We should add an option to do a shallow checkout and make it the default. You shouldn't need a full checkout unless you are manipulating tags or otherwise using git history.

Encode branch name when creating a Binder link

Currently the PR_HEAD_REF is taken as is and passed to the Binder link:

var PR_HEAD_REF = process.env.PR_HEAD_REF;
var URL_PATH = process.env.URL_PATH;
var BODY = `[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/${PR_HEAD_USERREPO}/${PR_HEAD_REF}?urlpath=${URL_PATH}) :point_left: Launch a Binder on branch _${PR_HEAD_USERREPO}/${PR_HEAD_REF}_`;

However this can create issues when a branch has a special character, for example #, as noticed in jupyter/notebook#7178. The Binder link becomes https://mybinder.org/v2/gh/haok1402/notebook/issue#7147?urlpath=tree, but Binder fails to build it:

image

However Binder can build the repo fine if the special character is URL encoded: https://mybinder.org/v2/gh/haok1402/notebook/issue%237147

image

So this binder-link action should probably try to encode the PR_HEAD_REF before creating the link.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.