Code Monkey home page Code Monkey logo

monodist's Introduction

monodist

This is a demonstration repo that proves out:

  1. a monorepo using pnpm
  2. a private NPM registry hosted on GitHub
  3. changesets using @atlassian/changesets
  4. CI through GitHub actions to tie this all together

Usage

There are scripts in the root package.json as well as the scripts that a package defines in its package.json. Many of the scripts in the root package.json can be applied to only packages which have changed since the most common ancestor with origin/master (or a git ref supplied by the user).

As a utility, you can retrieve the common ancestor of HEAD with origin/master by running:

pnpm --silent echo:ancestor

Or, to find a common ancestor against any other commit, you can supply a git ref or commit id as the environment variable ANCESTOR_REF:

ANCESTOR_REF=$(git rev-parse HEAD~20) pnpm echo:ancestor

hint: use --silent to supress additional NPM logging

ANCESTOR_REF is also supported by other scripts. For example, the following script will test only the packages which have changed since the common ancestor of HEAD and twenty commits before HEAD (that is, HEAD~20).

ANCESTOR_REF=$(git rev-parse HEAD~20) pnpm test:ancestor

For example:

  • pnpm build:since will build all packages that have changed since the divergence with origin/master
  • SINCE=$(git rev-parse HEAD~2) pnpm build:since will build all packages that have changed in the last two commits.

Changes

For a new package

Create a new package, and version it to "0.0.0" in its package.json, and apply a change (pnpm change in the repo root).

For an existing package

Make a change, do not manually update the version, and apply a change (pnpm change in the repo root).

Then open a PR and merge to master.

GitHub Actions Workflows

There are two workflows:

This runs when a PR to master is created. It runs the following steps (grouped here for clarity):

  1. Setup
    1. Checkout the repo
    2. Setup node
    3. Cache the pnpm-store
    4. Downloads the pnpm self-installer
    5. Installs the workspace dependencies
  2. Fetches the master branch (so pnpm can build-test-lint only the changed packages)
  3. Runs build-test-lint
    1. Check formatting
    2. Builds all packages
    3. Tests all packages
    4. Lints all packages
    5. Ensures dependencies are correctly

This runs when a push to master happens (e.g. a PR is merged into master). It runs the following steps (grouped here for clarity):

  1. Setup
    1. Checkout the repo
    2. Setup node
    3. Cache the pnpm-store
    4. Downloads the pnpm self-installer
    5. Installs the workspace dependencies
  2. Check formatting (before any mutations in the repo)
  3. Version
    1. Updates the git config (adding committer details)
    2. Get the changeset status (determines if there was a change), and if there are changes:
      1. Run changeset version (updating changelogs, package.jsons and auto-committing)
      2. Reflect changeset version (update the pnpm-lock.yml, runs prettier against changes, and ammends the former commit)
  4. Runs build-test-lint
    1. Builds all packages
    2. Tests all packages
    3. Lints all packages
    4. Ensures dependencies are correctly
  5. Push version commit (if a changeset was found)
  6. Publishes (if a changeset was found)
    1. Checkout with monodist-worker's personal access token
    2. Sets the .npmrc authToken to publish to the GitHub NPM Registry
    3. Run changeset publish (publishes to the registry, creates git tags)
    4. Push release git changes (the "version" commit from above) and git tags from the previous step

Issues / Considerations

Changeset race condition

Because a push to master kicks off:

  • integration workflow (e.g. build / test / lint)
  • pnpx changeset version
  • pnpx changeset publish
  • git push origin master
  • git push --follow-tags

And there is currently no support for preventing parallel workflows with GitHub Actions, there is a race condition when merging multiple branches into master in rapid succession will cause the slower merges to conflict when running git push origin master and push the wrong tags when running git push --follow-tags.

This is because pnpx changeset version and pnpx changeset publish each produce a commit on top of master, and won't reflect any intermediate commits. For example:

Consider the situation where 97dd2ae is the master before a commit.

  • merge of feature/a produces a new HEAD for master (formerly 97dd2ae) as 0657719
  • the GitHub Action workflow for push:master begins on 0657719
  • merge of feature/b produces a new HEAD for master (formerly 97dd2ae) as 08ee9ae
  • the GitHub Action workflow for push:master begins on 0657719
  • the commits from version and publish for 0657719 (the resultant HEAD of merging feature/a) are pushed back to master as 0fa08cc (correctly).
  • the commits from version and publish for 08ee9ae (the resultant HEAD of merging feature/b) are attempted to be pushed back to master as 0fa08cc (incorrectly, causing a conflict and perhaps overwriting tags from 0fa08cc - the versioned and published HEAD from feature/a).

The solutions to this problem are:

  1. don't use GitHub Actions
  2. optimistically run pnpx changeset version and pnpx changeset publish first, before any other integration workflow runs
  3. acknowledge the problem, and don't merge to master if we're currently running a workflow on it.
  4. use a solution like @softprops/turnstyle as a mutex for actions

The second approach simply narrows the span of time the race condition could present itself, it does not solve the race condition.

References:

TODOs

monodist's People

Contributors

dfee avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.