svanboxel / gitlab-mirror-and-ci-action Goto Github PK

License: Apache License 2.0

Dockerfile 14.38% Shell 85.62%

gitlab-mirror-and-ci-action's Introduction

Mirror to GitLab and trigger GitLab CI

A GitHub Action that mirrors all commits to GitLab, triggers GitLab CI, and returns the results back to GitHub.

This action uses active polling to determine whether the GitLab pipeline is finished. This means our GitHub Action will run for the same amount of time as it takes for GitLab CI to finish the pipeline.

Example workflow

This is an example of a pipeline that uses this action:

name: Mirror and run GitLab CI

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v1
    - name: Mirror + trigger CI
      uses: SvanBoxel/gitlab-mirror-and-ci-action@master
      with:
        args: "https://gitlab.com/<namespace>/<repository>"
      env:
        FOLLOW_TAGS: "false"
        FORCE_PUSH: "false"
        GITLAB_HOSTNAME: "gitlab.com"
        GITLAB_USERNAME: "svboxel"
        GITLAB_PASSWORD: ${{ secrets.GITLAB_PASSWORD }} // Generate here: https://gitlab.com/profile/personal_access_tokens
        GITLAB_PROJECT_ID: "<GitLab project ID>" // https://gitlab.com/<namespace>/<repository>/edit
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} // https://docs.github.com/en/actions/reference/authentication-in-a-workflow#about-the-github_token-secret

Be sure to define the GITLAB_PASSWORD secret in https://github.com/<namespace>/<repository>/settings/secrets
Before setup a token to use as GITLAB_PASSWORD here https://gitlab.com/profile/personal_access_tokens
The token must have read_api, read_repository & write_repository permissions in GitLab.
For granular permissions create seperate users and tokens in GitLab with restricted access.

If you're rewriting history in the primary repo (e.g by using git rebase), you'll need to force push. Set the FORCE_PUSH environment variable to true to enable this. This will overwrite history in the mirror as well, so be careful with this (just like any time you're using git push --force).

If you want to mirror repository tags too, you can define FOLLOW_TAGS environment variable to true.

gitlab-mirror-and-ci-action's People

Contributors

Stargazers

Watchers

Forkers

ailisp involic aperloff stenongithub repat ekee-io masterleinad inkitt vvolkl bsedin arashfarahani-com wjossey ozon2 krauthosting cochiseruhulessin auernige vivekschauhan sandykadam trolologuy ve-meet aehart ontra-ai yds05238 crewshin melkio pratikbin karnadixyz aarriagadac cms-l1tk kaydoh asac gallium70 sbalci pathologyatlas socialgouv karlinator pancetta grogou someblackmagic lutengda hackershohag amirali108 numericalalgorithmsgroup acsimon33 simeonehrig timothystewart6 klinkeklinke tiggi austinwilcox rosiel guillaume iamjohnnym kourb

gitlab-mirror-and-ci-action's Issues

Broken due to Detected Dubious Ownership

The action seems to have started to fail sometime in the last 3 days with the following error message:

fatal: detected dubious ownership in repository at '/github/workspace'
To add an exception for this directory, call:

	git config --global --add safe.directory /github/workspace

Unhelpfully, it doesn't show up as a failure in GitHub Actions (likely due to #16 and #10), so the only way I found out it had stopped working was by not seeing any recent commits on my GitLab mirrors.

Action fails when triggered by a tag.

We are using the FOLLOW_TAGS option and have set up our Github workflow to trigger on tag creation.

The result is, after creating tag 2.9.0,

error: pathspec '.9.0' did not match any file(s) known to git
fatal: ref HEAD is not a symbolic ref

I suspect that this is caused by line 25, git checkout "${GITHUB_REF:11}". When the GITHUB_REF is refs/heads/<branch_name> this works, but when it's refs/tags/<tag_name> it cuts the first letter off the tag name.

I think this could be solved by using $GITHUB_REF_NAME which is "The short ref name of the branch or tag that triggered the workflow run. This value matches the branch or tag name shown on GitHub. For example, feature-branch-1" Source: https://docs.github.com/en/actions/learn-github-actions/variables

Include / Exclude specific files or dirs

Is there a way to exclude any specific directory that won't appear in gitlab but requires in github like the github workflow files which doesn't requires in gitlab? If there's any custom way to add/remove any specific files / dirs, it would be great.

possible bug when namespace between github and gitlab differ

Firstly, thank you for sharing your github action workflow script in the market place.

There may be a minor bug in this GitHub action when you have the same git repo name but different namespaces (usernames). For example, in my particular cases I have:
https://github.com/naromero77/rte-rrtmgp (source)
https://software.nersc.gov/naromero/rte-rrtmgp (mirror)

/usr/bin/docker run --name bb10364e20a41b3e45908f1d05c1dcc59fab_afdf04 --label 54bb10 --workdir /github/workspace --rm -e GITLAB_HOSTNAME -e GITLAB_USERNAME -e GITLAB_PASSWORD -e GITLAB_PROJECT_ID -e GITHUB_TOKEN -e INPUT_ARGS -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/rte-rrtmgp/rte-rrtmgp":"/github/workspace" 54bb10:364e20a41b3e45908f1d05c1dcc59fab https://software.nersc.gov/naromero/rte-rrtmgp
Branch 'omp-45-cray' set up to track remote branch 'omp-45-cray' from 'origin'.
Switched to a new branch 'omp-45-cray'
pushing to naromero77/rte-rrtmgp/omp-45-cray branch at https://software.nersc.gov/naromero/rte-rrtmgp
error: src refspec naromero77/rte-rrtmgp/omp-45-cray does not match any.
error: failed to push some refs to 'https://software.nersc.gov/naromero/rte-rrtmgp'
Triggered CI for branch naromero77%2Frte-rrtmgp%2Fomp-45-cray
Working with pipeline id #null
Poll timeout set to 10
Current pipeline status: null
Pipeline finished with status null
Fetching all GitLab pipeline jobs involved
jq: error (at <stdin>:0): Cannot index string with string "id"
Posting output from all GitLab pipeline jobs
Debug problems by unfolding stages/jobs above

Is there a simple fix for this github action? Here is the workflow yaml file which I based on your template.
https://github.com/naromero77/rte-rrtmgp/tree/omp-45-cray/.github/workflows

SSL certificate problem due to not updated trust pool

Got lately those errors which we assumed might have to due with non updated trust pools.
The new signing CA of Let's Encrypt is not in old CA trust pools like in EOLed Ubuntu 16.04.
Saw similar problems outside our fleet when folks are running still on unpatched Linux distros.

Run krauthosting/gitlab-mirror-and-ci-action@master
/usr/bin/docker run --name XXX --label YYY --workdir /github/workspace --rm -e GITLAB_HOSTNAME -e GITLAB_USERNAME -e GITLAB_PASSWORD -e GITLAB_PROJECT_ID -e GITHUB_TOKEN -e INPUT_ARGS -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_RUN_ATTEMPT -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_NAME -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/atsw.de/atsw.de":"/github/workspace" YYY:XXX https://gitlab.domain.tld/path/repo
Switched to a new branch 'main'
Branch 'main' set up to track remote branch 'main' from 'origin'.
pushing to main branch at https://gitlab.domain.tld/path/repo
fatal: unable to access 'https://gitlab.domain.tld/path/repo/': SSL certificate problem: certificate has expired
Triggered CI for branch main
Working with pipeline id #
Poll timeout set to 10
Current pipeline status: null
Pipeline finished with status null
Fetching all GitLab pipeline jobs involved
Posting output from all GitLab pipeline jobs
Debug problems by unfolding stages/jobs above

@SvanBoxel Wonder what we can do for this project or if this is a known issue with GitHub actions and ubuntu-latest?

Fork repositories

Does it mirror branches/PRs from forked repos to GitLab and return statuses for them?

Pass custom env. variables specified from the github action to the gitlab pipeline

Hi!

Very nice github action, we use it quite a bit, but noticed that we would like to split the behavior of the gitlab pipelines based on the trigger source of the github action. We would like to do this via a custom provided env variable, which could be specified during this job and sent over to gitlab, which is supported by the API: https://docs.gitlab.com/ee/ci/triggers/#pass-cicd-variables-in-the-api-call.

Thank you!

No action executed

When i call this action my job don´t excute nothing
I get this message: "Waiting for a runner to pick up this job..."

Action doesn't fail in case of errors

Hi @SvanBoxel,

Thanks for this great action. The only problem is that the action doesn't fail in case if anything doesn't work in the process of interacting with Gitlab. I'm running the action with the standard parameters written in Readme, including using the master version of the code:

uses: SvanBoxel/gitlab-mirror-and-ci-action@master

When everything is configured correctly on Gitlab the action works great. But when something expires the action still pretends everything works with a success status. Here are logs from 2 of such cases:

A Gitlab access token expires:

Switched to a new branch 'my-branch'
Branch 'my-branch' set up to track remote branch 'my-branch' from 'origin'.
pushing to my-branch branch at <my gitlab repo>
remote: HTTP Basic: Access denied
fatal: Authentication failed for '<my gitlab repo>'
Triggered CI for branch my-branch
Working with pipeline id #null
Poll timeout set to 10
Current pipeline status: null
Pipeline finished with status null
Fetching all GitLab pipeline jobs involved
jq: error (at <stdin>:0): Cannot index string with string "id"
Posting output from all GitLab pipeline jobs
Debug problems by unfolding stages/jobs above

Push to Gitlab fails because the given access token doesn't have a permission to push to a protected branch:

Switched to a new branch 'master'
Branch 'master' set up to track remote branch 'master' from 'origin'.
pushing to master branch at <my gitlab repo>
warning: redirecting to <my gitlab repo>
remote: GitLab: You are not allowed to push code to protected branches on this project.
To <my gitlab repo>
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to '<my gitlab repo>'
Triggered CI for branch master
Working with pipeline id #167976
Poll timeout set to 10
Current pipeline status: success
Pipeline finished with status success
Fetching all GitLab pipeline jobs involved
Posting output from all GitLab pipeline jobs
Stage test / Job my_integration_job
Debug problems by unfolding stages/jobs above
{
  "message": "Resource not accessible by integration",
  "documentation_url": "https://docs.github.com/rest/reference/repos#create-a-commit-status"
}

(Note: in the above logs I just masked the actual gitlab repo link as it is not important for debugging.)

This is probably very related to #10

Run with `actions/checkout@v3`

Hi! Very nice action, took me a while to find something like that, thanks a lot.

Just a note: if you want to/need to run this with actions/checkout@v3 instead of actions/checkout@v1, you need to specify fetch-depth: 0 in the checkout action. Otherwise this will result in the error

! [remote rejected] my_branch -> my_branch (shallow update not allowed)

when pushing to the mirror.

Action doesn't return "fail" when GitLab has changes GitHub doesn't have

Action-logs:

Switched to a new branch 'master'
Branch 'master' set up to track remote branch 'master' from 'origin'.
pushing to master branch at https://gitlab.com/zuedev/*********
warning: redirecting to https://gitlab.com/zuedev/*********.git/
To https://gitlab.com/zuedev/*********
 ! [rejected]            master -> master (fetch first)
error: failed to push some refs to 'https://gitlab.com/zuedev/*********'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
Triggered CI for branch master
Working with pipeline id #null
Poll timeout set to 10
Current pipeline status: null
Pipeline finished with status null
Fetching all GitLab pipeline jobs involved
jq: error (at <stdin>:0): Cannot index string with string "id"
Posting output from all GitLab pipeline jobs
Debug problems by unfolding stages/jobs above