Code Monkey home page Code Monkey logo

monorepo-split-github-action's Introduction

GitHub Action for Monorepo Split

Version 2.0 now supports split to GitHub and Gitlab private repositories!


Do you have a monorepo project on GitHub and need split packages to many repositories? Add this GitHub Action to your workflow and let it split your packages on every commit and tag.

How does the Split Result Look Like?

This repository splits tests into symplify/monorepo-split-github-action-test repository.

Not on every commit, but only if contents of /tests/packages/some-package directory changes. Try it yourself - send PR with change in that directory.


Docs

Config

Split is basically git push or local directory to remote git repository. This remote repository can be located on GitHub or Gitlab. To be able to do that, it needs GITHUB_TOKEN or GITLAB_TOKEN with write repository access:

env:
    GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN }}
    # or
    GITLAB_TOKEN: ${{ secrets.GITLAB_TOKEN }}

Make sure to add this access token in "Secrets" of package settings at https://github.com///settings/environments


Define your GitHub Workflow

name: 'Packages Split'

on:
    push:
        branches:
            - main
        tags:
            - '*'

env:
    # 1. for Github split
    GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN }}

    # 2. for Gitlab split
    GITLAB_TOKEN: ${{ secrets.GITLAB_TOKEN }}

jobs:
    packages_split:
        runs-on: ubuntu-latest

        strategy:
            fail-fast: false
            matrix:
                # define package to repository map
                package:
                    -
                        local_path: 'easy-coding-standard'
                        split_repository: 'easy-coding-standard'

        steps:
            -   uses: actions/checkout@v2

            # no tag
            -
                if: "!startsWith(github.ref, 'refs/tags/')"
                uses: "symplify/[email protected]"
                with:
                    # ↓ split "packages/easy-coding-standard" directory
                    package_directory: 'packages/${{ matrix.package.local_path }}'

                    # ↓ into https://github.com/symplify/easy-coding-standard repository
                    repository_organization: 'symplify'
                    repository_name: '${{ matrix.package.split_repository }}'

                    # [optional, with "github.com" as default]
                    repository_host: git.private.com:1234

                    # ↓ the user signed under the split commit
                    user_name: "kaizen-ci"
                    user_email: "[email protected]"

            # with tag
            -
                if: "startsWith(github.ref, 'refs/tags/')"
                uses: "symplify/[email protected]"
                with:
                    tag: ${GITHUB_REF#refs/tags/}

                    # ↓ split "packages/easy-coding-standard" directory
                    package_directory: 'packages/${{ matrix.package.local_path }}'

                    # ↓ into https://github.com/symplify/easy-coding-standard repository
                    repository_organization: 'symplify'
                    repository_name: '${{ matrix.package.split_repository }}'

                    # [optional, with "github.com" as default]
                    repository_host: git.private.com:1234

                    # ↓ the user signed under the split commit
                    user_name: "kaizen-ci"
                    user_email: "[email protected]"

monorepo-split-github-action's People

Contributors

ajardin avatar ferror avatar jakubtobiasz avatar janmikes avatar tacman avatar tomasvotruba avatar torounit avatar zepfietje avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

monorepo-split-github-action's Issues

Commit messages with single quote throw a syntax error.

Received the following message from the Github Action -

[NOTE] Pushing git commit with "Add a github action to split the plugin into it's own repository in order to release composer packages." message to "main"

sh: 1: Syntax error: Unterminated quoted string

I believe the commit message needs to be escaped when used in the git commit.

https://github.com/symplify/monorepo-split-github-action/blob/e5a3284a0bf07e0df6c564399ff0a31bf105bd4e/entrypoint.php#L117

GitLab: Support CI_JOB_TOKEN

For GitLab it would be better to support the CI_JOB_TOKEN (https://docs.gitlab.com/ee/ci/jobs/ci_job_token.html) instead of a personal access token as this is generated by GitLab for the current user with the user's permissions and removes the need of the project owner/maintainer to create a PAT for the project which other users might be able to see.

I tried it but couldn't get it set up correctly to have permissions for pushing to the split repository.

Split command fails to detect new files (maybe like #13)

Hi, I think I'm facing the same issue as #13, but even with the fix applied.

Here is the output log from my split action: https://github.com/mep-agency/web-toolkit/runs/3760142721?check_suite_focus=true#step:4:63 (it's a new opensource project so everything is public)
The output shows an untracked file, but then nothing is added to the commit.

I'm splitting my monorepo to 3 repos. One is synced correctly (the skeleton) the other 2 are missing the latest commit where I just added the LICENCE/README.md files.
Unfortunately I don't know neither how to test this locally, nor how to test a custom version of this action, with changed source code, in order to give you more insights.

Any suggestion is welcome.

Thank you for your time.

Cannot get the Artifact files built on the before step

this is content of my packages-release.yml:

but when the Actions done, the split repo https://github.com/example/administration.git does not have the src/Administration/Resources/public

so how can i get the new generated directory or files be splitted to the new repo ?

best regards.

image
name: 'Packages Release'
on:
  push:
    branches:
      - 'trunk'
    paths:
      - src/Administration/**/*
env:
  GITHUB_TOKEN: ${{ secrets.TOKEN }}
jobs:
  jobs-split: 
    runs-on: ubuntu-latest
    strategy:
      matrix:
        package:
          -
            local_path: 'Administration'
            split_repository: 'administration'
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v4
      - name: Build Administration 
        run: |
          cd src/Administration/Resources
          mkdir -p public/static
          echo ${{ github.sha }} > public/static/Release.txt
      - name: Test Administration
        run: |
          ls -lht src/Administration/Resources/public 
      - name: Split Monorepo Without Tag 
        if: "!startsWith(github.ref, 'refs/tags/')"
        uses: "danharrin/[email protected]"
        with:
          package_directory: 'src/Administration'
          repository_organization: 'example'
          repository_name: '${{ matrix.package.split_repository }}'
          branch: 'trunk'
          user_name: "example"
          user_email: "[email protected]"

After switching to PHP, the CI run is always successful, even when it should fail

In this code in entrypoint.php:

exec($copyGitDirectoryCommandLine, $outputLines, $exitCode);

if ($exitCode === 1) {
    die('Command failed');
}

...the php die is only printing the message on screen, but it does not produce an exit 1, so the script still does not fail.

Demonstration:

I also tried executing this (https://github.com/leoloso/monorepo-split-github-action/blob/8795fc8/entrypoint.php#L98):

exec('exit ' . $exitCode);

But it also doesn't work.

Cannot login to github.com: could not read Password for 'https://***@github.com': No such device or address

fatal: could not read Password for 'https://***@github.com': No such device or address

https://github.com/roadiz/core-bundle-dev-app/actions/runs/4343986118/jobs/7586720997

I've tryied to add a personal access-token as ACCESS_TOKEN via my monorepo settings/environment section or by using automatic GITHUB_TOKEN. But action fails both ways when it try to push code to package repository.

Everything is fine before this step, new files changes have been found and a new commit is created into package repository.

[NOTE] Changing directory from "/github/workspace" to "/tmp/monorepo_split/build_directory"

On branch develop
Your branch is up to date with 'origin/develop'.

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   src/Node/NodeDuplicator.php
	modified:   src/Node/NodeNameChecker.php
	modified:   src/Node/NodeNamePolicyInterface.php
	modified:   src/Node/UniqueNodeGenerator.php
	deleted:    var/files/fonts/.gitkeep
	deleted:    var/files/private/.gitkeep
	deleted:    var/secret/.gitkeep

no changes added to commit (use "git add" and/or "git commit -a")

[NOTE] Adding git commit



[NOTE] Pushing git commit with "ci: Force GITHUB_TOKEN (https://docs.github.com/en/actions/security-guides/automatic-token-authentication)" message to "develop"

fatal: could not read Password for 'https://***@github.com': No such device or address

Can't get it running

I've created a new repository with an readme file and try to run the action, but'll get following error

[NOTE] Resolving configuration...



[NOTE] Cloning "https://github.com/MY_ORG_NAME/Base.AssetList.git" repository to "/tmp/monorepo_split/clone_directory" directory



[NOTE] Running: git clone -- https://***@github.com/MY_ORG_NAME/Base.AssetList.git /tmp/monorepo_split/clone_directory

Cloning into '/tmp/monorepo_split/clone_directory'...


[NOTE] Cleaning destination repository of old files



[NOTE] Copying contents to git repo of "49d3bc83602b61bd989e49efa87fd24b132b8f98" branch



[NOTE] Files that will be pushed

total 28
drwxr-xr-x 6 1001  123 4096 Jul  7 20:32 .
drwxr-xr-x 3 root root 4096 Jul  7 20:32 ..
drwxr-xr-x 8 root root 4096 Jul  7 20:32 .git
drwxr-xr-x 2 1001  123 4096 Jul  7 20:32 Configuration
drwxr-xr-x 2 1001  123 4096 Jul  7 20:32 NodeTypes
drwxr-xr-x 3 1001  123 4096 Jul  7 20:32 Resources
fatal: detected dubious ownership in repository at '/github/workspace'
To add an exception for this directory, call:

	git config --global --add safe.directory /github/workspace
-rw-r--r-- 1 1001  123  328 Jul  7 20:32 composer.json

[NOTE] Changing directory from "/github/workspace" to "/tmp/monorepo_split/build_directory"

fatal: detected dubious ownership in repository at '/tmp/monorepo_split/build_directory'
To add an exception for this directory, call:

	git config --global --add safe.directory /tmp/monorepo_split/build_directory
fatal: detected dubious ownership in repository at '/tmp/monorepo_split/build_directory'
To add an exception for this directory, call:

	git config --global --add safe.directory /tmp/monorepo_split/build_directory


[NOTE] No files to change



[NOTE] Changing directory from "/tmp/monorepo_split/build_directory" to "/github/workspace"

What I am doing wrong? I've set up everything just like in the readme

Since using PHP, a `git push` error does not produce the run to fail

I had set-up the wrong credentials in ACCESS_TOKEN. With version 1.1 when doing git push, it printed the Git error, and the process failed.

With this new version in PHP, doing exec('git push --quiet origin ' . $config->getBranch()); also produces a Git error, but it is just printed on screen; the GitHub Action does not fail, so I get the ✔️ at the end of the run.

To fix it, something like this could work:

$output = [];
$result_code = 0;
$result = $exec('git push --quiet origin ' . $config->getBranch(), $output, $result_code);
if ($result === false) {
    die('Command failed: ' . implode(PHP_EOL, $output));
}

Or return $result_code?

Split with new files

Hello and thanks for your amazing work (@rectorphp user here)! 😉

From what I understand, the split only works with existing files on both sides. If I add a new file, the split does nothing ([NOTE] No files to change instead of [NOTE] Adding git commit). Is it possible to force the indexation of new changes?

I assume it's related to this line, but I wanted your point of view before submitting any changes.

Question/Readme: Split without GitHub Actions

First of all, thanks to everyone involved working on those tools, really helpful! I hope it is okay to ask a question here since I could not get a clear answer from the readme. The whole monorepo topic is new for me as I used multiple repositores until now, which is problematic in a current project.

As far as I could find out, there was a feature in symplify v8 that allowed to split up the monorepo into multiple repositories. It was removed in v9 (I've read the blog post) and replaced by a github workflow action. Now, I want to set up a mono-repository on a private git server (not on GitHub), which means I can't use the new workflow. What would be the way to go to split the mono-repo into multiple repos on a private git server (e. g. Bitbucket or GitLab)? For starters without a CI that does the whole release, a local command to run as it was possible in v8 would be sufficient. Would you advise to make a custom script for it or is there still something in symplify that would help to achieve this?

Looking for a maintainer to take over in 2023

Hi everyone,

to be honest, I don't have time to maintain this package and keep it high quality.
If you find it useful and want to improve it to the bright future, let me know.

Tomas

GitHub: How can it be used with privates repos?

I've configured the workflow in this way

name: 'Packages Split'

on:
  push:
    branches:
      - production

env:
  GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN }}

jobs:
  packages_split:
    runs-on: ubuntu-latest

    strategy:
      fail-fast: false
      matrix:
        # define package to repository map
        packages:
          - local_path: 'Base'
            split_repository: 'oms-base'


    steps:
      -   uses: actions/checkout@master

      # no tag
      -
        if: "!startsWith(github.ref, 'refs/tags/')"
        uses: "symplify/monorepo-split-github-action@main"
        with:
          # ↓ split "packages/easy-coding-standard" directory
          package_directory: 'app/code/Oms/${{ matrix.packages.local_path }}'

          # ↓ into https://github.com/symplify/easy-coding-standard repository
          repository_organization: 'my-organization'
          repository_name: '${{ matrix.packages.split_repository }}'

          branch: 'production'

          # ↓ the user signed under the split commit
          user_name: "my-username"
          user_email: "[email protected]"

log when action runs

[NOTE] Resolving configuration...



[NOTE] Cloning "https://github.com/my-organization/oms-base.git" repository to "/tmp/monorepo_split/clone_directory" directory



[NOTE] Running: git clone -- https://***@github.com/my-organization/oms-base.git /tmp/monorepo_split/clone_directory

Cloning into '/tmp/monorepo_split/clone_directory'...
remote: Repository not found.
fatal: repository 'https://github.com/my-organization/oms-base.git/' not found

Warning: chdir(): No such file or directory (errno 2) in /splitter/entrypoint.php on line 39
fatal: detected dubious ownership in repository at '/github/workspace'
To add an exception for this directory, call:


	git config --global --add safe.directory /github/workspace

[NOTE] Trying to checkout production branch

fatal: detected dubious ownership in repository at '/github/workspace'
To add an exception for this directory, call:

	git config --global --add safe.directory /github/workspace


[NOTE] Creating branch "production" as it doesn't exist

fatal: detected dubious ownership in repository at '/github/workspace'
To add an exception for this directory, call:

	git config --global --add safe.directory /github/workspace
fatal: detected dubious ownership in repository at '/github/workspace'
To add an exception for this directory, call:

	git config --global --add safe.directory /github/workspace


[NOTE] Cleaning destination repository of old files

cp: can't stat '/tmp/monorepo_split/clone_directory/.git': No such file or directory
Command failed

The indicated repository is private within the organization, the token is configured within the main repository as follows

image

is there anything that needs to be changed in this case?

Job failed: exit code 127

It seems after the recent update this error has started to show... code has never been touched in over a year, i have noticed there has been an update on the repo so maybe it has something to do with it!

Error message:

$ apt-get update -y > /dev/null # collapsed multi-line command
/bin/sh: eval: line 125: apt-get: not found
Cleaning up file based variables
00:00
ERROR: Job failed: exit code 127

I am using Ubuntu with gitlab-ci. Any thoughts?

Screenshot 2022-10-13 at 15 21 43

Creating branches if branch is missing?

Hi I'm using this github action on my monorepo. Since it's still a WIP all I had is a main branch and I was pushing to other repositories to the main branch. Now I have a 1.0 branch so I can work on making a stable version and I run into issues with the monorepo split.

In the workflow I added:

-
        name: Monorepo Split of ${{ matrix.package }} (no tag)
        if: "!startsWith(github.ref, 'refs/tags/')"
        uses: "symplify/[email protected]"
        with:
          package_directory: 'packages/${{ matrix.package }}'
          repository_organization: '<organization name>'
          repository_name: '${{ matrix.package }}'
          user_name: "<username>"
          user_email: "<email>"
          branch: '${{ github.ref_name }}'

This fails however as the repository I'm pushing to has no 1.0 branch (yet). Which results in this error:

[NOTE] Adding git commit



[NOTE] Pushing git commit with "<commit message>" message to "1.0"

error: src refspec 1.0 does not match any
error: failed to push some refs to 'https://github.com/<repository-link>.git'


[NOTE] Changing directory from "/tmp/monorepo_split/build_directory" to "/github/workspace"

Do I need to manually add these branches to all the child repositories? Or can this be done automatically?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.