This repository contains a Python package providing development tools that span the OpenXLA suite of projects under the openxla org.

It is primarily intended to be installed by developers. In everyday use, it should not be necessary to actual modify the tools themselves.

pip install git+https://github.com/openxla/openxla-devtools.git

The openxla-workspace command is used to:

• Setup and maintain an "OpenXLA workspace" (collection of individual project repositories).
• Checkout individual project repositories.
• Sync repositories to pinned versions.
• Update pinned version.

Quick start to create an openxla repository and start developing:

mkdir ~/openxla
cd ~/openxla
openxla-workspace init
openxla-workspace checkout --sync openxla-pjrt-plugin

The above will initialize a new workspace and checkout the openxla-pjrt-plugin plus its dependencies, syncing to the dependency version pins that the project specifies. There is nothing special about this project except that it is a leaf project and illustrates dependency resolution.

The --sync option tells checkout to behave as if the sync command was run on the checked out repository. This has the effect of checking out the pinned revisions of all deps recursively.

NOTE: Some projects manage their dependencies via submodules. Such cases are not covered here (just use normal git tools).

If a project depends on other OpenXLA projects, it expresses these dependencies via sync_deps.py script in the root of its repository. This script is both importable (in order to get programmatic access to dependencies) and runnable (so that end-developers and CI systems have the simplest possible mechanism to check everything out needed to build the project).

When it is time to update the pins in this file, bring the dependencies of interest to the desired commit (and land it in the corresponding dependency repo). Then, from the project to be updated, run openxla-workspace pin.

Example:

cd ~/openxla/openxla-pjrt-plugin
openxla-workspace pin --require-upstream

Note that the --require-upstream flag enables an (expensive) safety check to ensure that the chosen commits are on the appropriate upstream tracking branch.

This will overwrite the sync_deps.py script with a new version. Creating a PR in the repo with changes to this file will trigger the CI to validate the chosen commits.

This same basic procedure will be applied by automation which periodically moves all dependencies forward when there are no build/test issues.

A repository may be configured to have dependency rolling schedules. These schedules can be invoked by both humans and automation to bump dependencies according to different policies.

Typically, a repository will define two schedules (if it has any):

• continuous: Makes "inter-day" updates to core dependencies. Used to integrate rapidly changing deps automatically.
• nightly: Makes "once per day" updates to all dependencies. These can be done multiple times per day if desired but typically this involves "big jumps". Usually this schedule will be responsible for updating pinned binaries (i.e. pip packages) and such as well (which may be derived from nightly releases).

Once dependencies are rolled, sync must be performed and any project specific steps for upgrading packages.

Invoke a dependency roll with a command like this:

cd ~/openxla/openxla-pjrt-plugin
openxla-workspace roll nightly