Code Monkey home page Code Monkey logo

remote-file-plugin's Introduction

Remote Jenkinsfile Provider Plugin


Important: Plugin name had changed to "Remote Jenkinsfile Provider Plugin". The previous name was "Remote File Plugin".

This plugin enables definition of external Jenkinsfile from another repository for MultiBranch Pipelines.

Description

Jenkins MultiBranch Pipeline feature is great to manage multi branched repositories which need to have dynamically created Pipeline Jobs on-demand. To use this feature you need to define Jenkins Pipeline Script in the repository alongside with the source code.

However, when it comes to adding new definitions/stages/steps or maintaining the Jenkins Pipeline Script for other reasons, you need to make all the changes within all branches or make changes in master branch and wait for other branches getting the update. For eliminating this you can use Jenkins Shared Library. However, when you are in a large-scaled/enterprise environments with hundreds/thousands of developers with lots of repositories, you need to somehow protect the content/stability of the Jenkins Pipeline Scripts to avoid failures.

In this point Remote Jenkinsfile Provider Plugin comes in. With this plugin you can define/set Jenkins files from another repository while still able to use MultiBranch Pipeline Project features. This way you will be able to centralize all Jenkins files in another repository where you can review or restrict changes and use MultiBranch Pipeline for multi branched repositories.

In summary, Using MultiBranch Pipeline Jobs with Remote Jenkinsfile Provider Plugin you can;

  • Centralize Jenkins Files in another repository
  • Easily maintain
  • Review or restrict changes
  • Apply changes to all Pipelines in seconds

Environment Variables

Remote Jenkinsfile Provider plugin injects two environment variables into every build automatically. These variables are described below,

  • RJPP_SCM_URL: URl of the repository as String which is defined in Remote Jenkinsfile Provider. Comma separated if there are more than one URL definition.
  • RJPP_JENKINSFILE: Script Path of the Jenkinsfile as defined in Remote Jenkinsfile Provider.
  • RJPP_BRANCH: Branch name of the Jenkinsfile where it is cloned.
  • RJPP_LOCAL_MARKER: Path to the local marker file as defined in Remote Jenkinsfile Provider.

These variables can be retrieved same as other environment variables in Jenkins.

echo "${env.RJPP_SCM_URL}"
echo "${env.RJPP_JENKINSFILE}"
echo "${env.RJPP_BRANCH}"
echo "${env.RJPP_LOCAL_MARKER}"

This feature only works for Git SCM definitions

Configuration

To use this Plugin first you need to install it from Jenkins Plugin Site, for details see Managing Plugins

After installing plugin for using the plugin you can follow the steps with examples below,

Assume that you have a project to build, named "MyCrazyProject", which is stored in a repository https://github.com/aytuncbeken/MyCrazyProject.git

  • Create a new repository for storing Jenkins files for your projects. For example, it is named "MyJenkinsLibrary" and stored in https://github.com/aytuncbeken/MyJenkinsLibrary.git .
  • Create a directory in this repository for your project. For example, create a directory called "MyCrazyProject".
  • Commit/Push your Jenkins File, named "Jenkinsfile" under this directory
  • Open you Jenkins and create a new MultiBranch Project. For example, named "MyCrazyProject Job".
  • Add Source for your project to build. For example, fill information for "MyCrazyProject".
  • In the "Build Configuration", change "Mode" to "by Remote Jenkins File Plugin". You will see new SCM definition is available.
  • Define your script path. For example, it will be "MyCrazyProject/Jenkinsfile".
  • Define your SCM for Jenkins file. For example, it will be the repository https://github.com/aytuncbeken/MyJenkinsLibrary.git .
  • Click Save.
  • In the "Scan Multi Branch Pipeline Log", you will see logs "Ignoring Jenkins file checking in Source Code SCM" which is normal as this plugin ignores Jenkins files in your repository and inject Jenkins File from another repository which you stored your Jenkins files.

Match Branches

If you select option Remote Jenkinsfile Provider Plugin tries to find matching branch in a repository which defined in Plugin SCM definition with in the Branch Sources SCM definition.

By this, Jenkins Files will be used in the branch of Jenkins File repository which have the same branch name in code source repository.

Branch Specifier will be ignored if this options is checked.

If plugin cannot find same branch name in the Jenkins File repository, it will automatically fall back to master branch and search for Jenkins file in it.

This feature only works for Git SCM definitions

You can see screenshot of example definition.

Local Marker Project Recogniser

When using Remote Jenkinsfile Provider Plugin, you might want to build only repositories and branches which contain a certain file or directory. In this case, you may specify a Local Marker in the Plugin SCM Definition. This way, Jenkins will create projects only for repositories and branches containing the file or directory you specified.

In the example below, only repositories and branches containing a file named pom.xml would be recognised:

Exclude From Change Set

In default, SCM changes which are coming from Jenkins File repository are included in Job Change Set. For excluding these changes from change set, add behaviour to this plugin's SCM definition as shown below.

Important Note: For applying behaviour changes, generated branch jobs must be run once

Important Note: This feature is supported only for Git

Exclude From SCM Poll

In default, SCM poll will trigger jobs for building if new commits are found in Jenkins File repository. For excluding these commits from poll, add behaviour to this plugin's SCM definition as shown below.

Important Note: For applying behaviour changes, generated branch jobs must be run once

Important Note: This feature is supported only for Git

Note

For filtering branches in the project, you can use "Filter by name" feature.

Configuration with Job DSL

Remote Jenkinsfile Plugin Provider can be configured with Job DSL.

For configuring, please use the Job DSL script below.

multibranchPipelineJob('example') {
  // Below definiton comes from Job DSL. This is a basic example. 
  // For details please visit https://jenkinsci.github.io/job-dsl-plugin/#method/javaposse.jobdsl.dsl.jobs.MultibranchWorkflowJob.branchSources
  branchSources {
    git {
      id('1')
      remote('https://github.com/jenkinsci/remote-file-plugin.git')
      includes('master')
    }
  }

  // Below section for configuring Remote Jenkinsfile Provider Plugin with defaults
  factory{
    remoteJenkinsFileWorkflowBranchProjectFactory{
      localMarker("")
      matchBranches(true)
      remoteJenkinsFile("Jenkinsfile")
      remoteJenkinsFileSCM{
        gitSCM{
          userRemoteConfigs{
            userRemoteConfig{
              name("MyRepo") //Custom Repository Name or ID
              url("https://github.com/aytuncbeken/multibranch-action-triggers-test.git") //URL for the repository
              refspec("master") // Branch spec
              credentialsId("") // Credential ID. Leave blank if not required
            }
            browser{} // Leave blank for default Git Browser
            gitTool("") //Leave blank for default git executable
          }
        }
      }
    }
  }
}

For viewing other definition alternatives with different SCMs, please follow the steps below.

  1. Install Job DSL plugin on your Jenkins
  2. Navigate to address http://YOUR_JENKINS_INSTANCE/plugin/job-dsl/api-viewer/index.html#method/javaposse.jobdsl.dsl.helpers.workflow.BranchProjectFactoryContext.remoteJenkinsFileWorkflowBranchProjectFactory*
  3. Expand remoteJenkinsFileSCM section. This will show available SCM definition options.

Reporting Issues

Please create issue in this repository.

Create Issue

Thank You!

If you feel your self generous today, you can buy me a coffee : )
, Or you can star the project. Thanks.

remote-file-plugin's People

Contributors

aytuncbeken avatar basil avatar dependabot[bot] avatar leonardo-filipe avatar nop00 avatar offa avatar roesh avatar sepulrator avatar sghill-rewrite avatar vanroy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

remote-file-plugin's Issues

Checkout fail with subversion

Describe the bug
11:19:10 ERROR: Checkout failed
11:19:10 java.lang.ClassCastException: class hudson.scm.SubversionSCM cannot be cast to class hudson.plugins.git.GitSCM (hudson.scm.SubversionSCM is in unnamed module of loader hudson.ClassicPluginStrategy$AntClassLoader2 @630bb5c4; hudson.plugins.git.GitSCM is in unnamed module of loader hudson.ClassicPluginStrategy$AntClassLoader2 @664f6f36)
11:19:10 at org.jenkinsci.plugins.workflow.multibranch.extended.scm.ExcludeFromChangeSet$ScmListener.onCheckout(ExcludeFromChangeSet.java:59)
11:19:10 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:150)
11:19:10 at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:154)
11:19:10 at org.jenkinsci.plugins.workflow.multibranch.extended.scm.ExtendedSCMBinder.create(ExtendedSCMBinder.java:83)
11:19:10 at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:309)
11:19:10 at hudson.model.ResourceController.execute(ResourceController.java:97)
11:19:10 at hudson.model.Executor.run(Executor.java:429)

To Reproduce
Configure multibranch
mode -> by Remote Jenkinsfile Provider Plugin
Jenkinsfile SCM -> Subversion

Screenshots
grafik

dynamic script path

is there a way to have a dynamic script path in this plugin?
for example, if I commit on the test-project repo its going on my Jenkinsfile repo and find a folder that name is test-project and inside that folder look for Jenkinsfile.

i using GitLab Branch Source plugin.

Environment variable RJPP_BRANCH is initialized with wrong value

Describe the bug
Environment variable RJPP_BRANCH is initialized with wrong value when setting multiple values in 'Branches to build' section (image attached)

To Reproduce
Steps to reproduce the behavior:

Consider the following git branch tree:

*-----*-----* (master)
 \
  *-----*-----*-----* (feature1)
  1. Create new Multibranch Pipeline
  2. Define the 'Mode' in 'Build Configuration' to by Remote Jenkinsfile Provider Plugin
  3. Configure script path - Pipeline should include a sh step to print env vars as follows
    sh "printenv RJPP_BRANCH"
  4. Set 'Jenkinsfile SCM settings' to Git
  5. Add multiple existing branches to 'Branches to build', master being the first
    5.a. Note the order of configured branches matters - make sure the first configured branch's HEAD isn't committed last
  6. Finish configuration according to your settings
  7. Submit & Scan Job
  8. Run the pipeline in any of the scanned branches
  9. The pipeline runs, reading (remotely) Jenkinsfile from the most-recently-committed branch HEAD
  10. View Console Output, see 1st occurrence of 'Checking out Revision': (related to JRPP)
    10.a. Commit ID selected is the most recent commit, as expected
  11. See printenv step output:
    11.a. Printed RJPP_BRANCH value matches the 1st element configured in 'Branches to build' (master according to the git tree provided above), regardless if it was checked-out or not.

Expected behavior
RJPP_BRANCH value should match the actually checked-out Jenkinsfile branch.

Screenshots
Screenshot 2022-11-20 at 15 06 17

Desktop (please complete the following information):
IMO irrelevant, running on cloud VM as Docker container, image pulled from DockerHub

Extra Info:
In the attached image, until I moved the REDACTED branch to the top of the list, '*/master' was printed as JRPP_BRANCH's value.
I'm trying to define some logic based on the branch used for reading the Jenkinsfile, so the current statue forces me to reorder the list for every development I need to make on the given pipeline, rather just adding a development branch to the list 'knowing' it'll be used and initialize JRPP_BRANCH, and remove it when done.

Blue Ocean compatibility

Any way / thoughts to make this more compatible with Blue Ocean plugin; specifically the editor? The pipelines run fine but if you edit the pipeline using the editor it edits the Jenkinsfile in the working repo and not the repo that is set up to be the Jenkensfile host.
Screen Shot 2021-10-21 at 12 26 42 AM

Expose Jenkinsfile repository checkout path

Is your feature request related to a problem? Please describe.
I'd like to package build support files, along with the Jenkinsfile, in the external repository. I'd like to refer to those files from within the Jenkinsfile. There are two places where access to these files is needed: during a "bootstrapping" stage running on the Jenkins server, and from a Jenkins agent.

Describe the solution you'd like
A possible solution would be if this plugin provided environment variables related to the remote-file-plugin.

  • One variable might contain the path to the location of the Jenkinsfile or checkout of the Jenkinsfile repository. That variable could then be used to construct paths pointing to other files in that repository for reference by the build. This could be used from the Jenkins service for things like dynamic provision of worker agents on a cloud platform.
  • Another variable might contain the SCM URL configured in the remote-file-plugin. This could be used from the Jenkins agent to checkout a copy of the Jenkinsfile repository into the workspace.

Build status is reported via Bitbucket Branch Source plugin

I have a common use case:

  • Code repository hosted on Bitbucket
  • Jenkinsfile repository hosted on Bitbucket
  • Jenkins uses Bitbucket Branch Source plugin which (1) providers a webhook that is called by Bitbucket and (2) allows cloning from Bitbucket
  • Jenkins has a multibranch pipeline project to discover branches of the code repository and uses the remote jenkinsfile plugin to fetch the Jenkinsfile from the Jenkinsfile repository

The Remote Jenkinsfile Plugin can use any SCM source plugin to actually retrieve the Jenkinsfile. It can use the "normal" git plugin as well as the Bitbucket Branch Source.

Now, when the Bitbucket Branch Source is used, an interesting behaviour occurs:

  • Jenkins checks out the Jenkinsfile from the Jenkinsfile repository
  • Jenkins/the Bitbucket Branch Source plugin posts the build status ("in progress") for the Jenkinsfile repository to Bitbucket
  • Jenkins checks out the code from the code repository
  • Jenkins/the Bitbucket Branch Source plugin posts the build status ("in progress") for the code repository to Bitbucket
  • Jenkins runs the pipeline
  • Jenkins/the Bitbucket Branch Source plugin posts the build status to Bitbucket... from what I can see it occurs in the Jenkinsfile repository but not in the code repository

When changing the SCM source for the Remote Jenkinsfile Plugin to git, every runs as expected, meaning that the build status is only reported for the code repository.

I'm not sure if this is something which can be solved in this plugin. But maybe there is a parameter that can be passed to SCM plugin that prevents the use of webhooks?

use cache for Jenkins remote files

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

we have a heavy Jenkins with more than 10,000 runs a day. it would be nice to try and reduce some loads from it :

currently it checkout every branch build

Checking out git https://github.com/XXXX/jenkins-files.git into /var/jenkins_home/workspace/aaa_main@script/23af8263fcd014953cdff9bf6785fd52eb5b74f67310a854c0195f1b4d89dd8a to read Jenkinsfiles/aaa/Jenkinsfile

instead of doing the checkout over and over maybe we can use some cache , like in the jenkins-lib mechanism

Loading library jenkins@main
Library jenkins@main is cached. Copying from home.

another issue , is missing of cleanup , those remote files stay forever , need to add a weekly / daily cleanup

How to work with Organization folder

Hi,
I'm using Jenkins with an Organization folder linked to my Gitea installation.
This is awesome because Jenkins scan the organization and create a multibranch pipeline for each repository that I have.

Now I would to use your plugin but I don't know how to do.

I created a repository with a branch for every repositories in organization. In each branch I have a folder called same the branch of the repository.

For example, this is the struct of the organization folder:

  • Organization "Org1"
    • Repository "Repo1"
      • Branch "master"
      • Branch "Branch1"
    • Repository "Repo2"
      • Branch "master"
    • Repository ....

The repository of jenkins files is:

  • Repository "RemoteJenkinsFile"
    • Branch "Repo1"
      • folder "master"
      • folder "branch1"
    • Branch "Repo2"
      • folder "master"

I am able to specify the branch name in Script Path field with the env variable BRANCH_NAME.
I throught to use the Branch Specifier to identiy the correct branch but I cannot have the repository name. There isn't in Jenkins a env variable for this value. I have think to use the env JOBS_NAME but is complete and the regex in Branch Specifier don't work.

How I can do this?

Tests rely on 'master' default branch

Describe the bug
Executing tests on systems, where master isn't set es default git branch fails. The tests are going to use the configured branch name instead, thus failing on everything based on master assumptions.

Since the setup is done by calling init() of GitSampleRepoRule (git plugin), it's better fixed there (--> I'm going to submit a PR).

To Reproduce
Steps to reproduce the behavior:

  1. Set another branch, eg. main, as default branch globally
  2. Run tests
  3. Some tests fail due to missing master

Expected behavior
Tests should pass and not depend on machine settings.

Screenshots
N/A

Desktop (please complete the following information):
N/A

Smartphone (please complete the following information):
N/A

Additional context
This issue is used for documentation, as others may run into the problem too.

The plugin changes the variables that are required for the Bitbucket server integration plugin

Describe the bug
We have a multibranch pipeline configured to watch bitbucket repo A. Also have a bitbucket repo B with the jenkinsfile mapped in this plugin (on branch master).

When jenkins triggers a job for a change on a branch in repo A (example branch "develop") the job first checkout repo B to get Jenkinsfile and "Bitbucket server integration plugin" notifies bitbubket about de build job (This should not be so but at this point it does not matter) to IN PROGRESS:
"Posting build status of INPROGRESS to Bitbucket LAB for commit id [755bc3ee2c84a8a5baf8e80c9d111f8cd6a3c3aa] and ref 'refs/heads/master'"

after checkouts repo A to get the real source code and ocurrs the first problem... the bitbucket plugin notifies the job but for the branch "master" (from the B repo):
Posting build status of INPROGRESS to Bitbucket LAB for commit id [240c87ff09e5c93caaede92d13a265f64e39b278] and ref 'refs/heads/master'
(this should be branch "develop")

and after the execution of all the steps in pipeline only notifies to repo B commit in bitbucket:

Posting build status of SUCCESSFUL to Bitbucket LAB for commit id [755bc3ee2c84a8a5baf8e80c9d111f8cd6a3c3aa] and ref 'refs/heads/master'

If we add the jenkinsfile to repo A and dont use "remote-file-plugin" the "Bitbucket server integration plugin" notifies correctly the build status to bitbucket but something changes when adding "remote-file-plugin" that change the branch and commit data to the pipeline.

To Reproduce
Steps to reproduce the behavior:
Multibranch config for repo A
image

Remote file plugin config (repo B):
image

Expected behavior
job notifies to bitbucket about the correct repo, branch and commits

Desktop (please complete the following information):

  • OS: Windows server

Multibrach with scan with remote Jenkinsfile failed at java.lang.NullPointerException: Cannot invoke "java.lang.Boolean.booleanValue()" because "this.matchBranches" is null

Describe the bug
From time to time I get the following error:

Connecting to https://bitbucket.org/ using bitbucket
Repository type: Git
Looking up xxx/module-xxx-yyy for branches
Checking branch master from xxx/module-xxx-yyy
No local file defined. Skipping Source Code SCM probe, since Jenkinsfile will be provided by Remote Jenkins File Plugin
    Met criteria
ERROR: [Fri Oct 27 12:09:19 GMT 2023] Could not fetch branches from source c6c3dd8409142dcf4990b018e60cab8298ebad74b5b103e13b018271a50e1fae2
[Fri Oct 27 12:09:19 GMT 2023] Finished branch indexing. Indexing took 0.96 sec
FATAL: Failed to recompute children of YYY Jobs » YYY: module-xxx-yyy (module-xxx-yyy)
java.lang.NullPointerException: Cannot invoke "java.lang.Boolean.booleanValue()" because "this.matchBranches" is null

Different repos. But the next run can be fine without issues.

Additional context

Jenkins 2.414.3
All plugins up&date

Null value not allowed as an environment variable: RJPP_LOCAL_MARKER

Recently rebuilt our Jenkins instance and updating plugins. Upon updating to the latest version 1.22 of this plugin we started seeing the error for all builds using this remote Jenkinsfile.

Null value not allowed as an environment variable: RJPP_LOCAL_MARKER

Our current config has never used a local marker in the config options. It's always been null.

image

image

Jenkins 2.319.2

It seems like there was a recent PR #92 merged with this new env var. Rolling back to version 1.21 seems to be working fine.

"Script Path" configuration element doesn't handle well undefined variables

Describe the bug
When running a job for a new branch (in multi branch pipeline) that has a parameter (defined in the declerative pipeline we are running in the job) as the value of the "Script Path" plugin configuration, the build fails since the plugin treats this parameter (name) as a literal, instead of an empty value. Only after we put a constant string "e.g. Jenkinsfile" in the "Script Path" field, and the build complete successfully, from that point on we can replace the constant with the parameter (that is defined in the job's pipeline)

To Reproduce
Steps to reproduce the behavior:

  1. Define a multibranch job, with a parameter to hold the name of the pipeline to run (e.g. a list with "Jenkinsfile" and "Jenkinsfile2", and assuming you have Jenkinsfile and Jenkinsfile2 in the repo you are working on)
  2. Write the the parameter as the value of the "Script Path" plugin configuration (e.g. Script Path -> $TARGET_JENKINSFILE_FILE_NAME)
  3. Run the build
  4. The build will fail saying it can't find a pipeline named $TARGET_JENKINSFILE_FILE_NAME

Expected behavior
As long as the $TARGET_JENKINSFILE_FILE_NAME isn't defined (the first run), treat it as empty and go with the default Jenkinsfile literal
Note: Maybe the plugin should inspect this parameter and in case it starts with "$" - try to resolve it (as environment or parameter variable) and if it's undefined, use the "Jenkinsfile" default

Screenshots
image
image

Desktop (please complete the following information):

  • OS: Windows 10
  • Browser: Microsoft Edge
  • Version: Version 89.0.774.54 (Official build) (64-bit)

Additional context
Jenkins version: 2.263.3
The intended pipeline has the above mentioned parameter defined as:

	parameters {
		choice (
			name: 'TARGET_JENKINSFILE_FILE_NAME',
			choices: infraJenkinsfileFilesNamesListWithNewLineDelimiter,
			description: 'The desired Jenkinsfile to run'
		)

We like the plugin very much and find it very powerful, but this behaviour is very troubling and any assistance would be very much appreciated.

How to use the plugin via declarative jenkinsfile?

We are keen on using this plugin to manage our jenkinsfiles in a separate repo and so far, have found the plugin to work very well. Only problem is that currently we are setting it up via the jenkins UI, which may be a limitation for us to allow other users to do. Is there a sample available on how we do the setup in a declarative pipeline/jenkinsfile?

Build runs on the remote repo with the Jenkinsfile, not on the main repository (BitBucket)

We have a large git repository stored in BitBucket, let's call it main_repo. I created a new repository for a Jenkinsfile, named jenkinsfile_repo. In the jenkinsfile_repo I created a directory named main_repo and put our Jenkinsfile in it. I created a Multibranch Pipline project in Jenkins, according to the documentation here:

https://plugins.jenkins.io/remote-file/

When I save it, then the scan process starts, collects our branches from the main_repo correctly. But if a change triggers a build on one of those branches, the build process looks like this:
-branch change triggers a build
-it checks out the jenkinsfile_repo
-then tries to run the Jenkinsfile on the jenkinsfile_repo/ not on the main_repo/.
Here is the console log:

Branch indexing
Checking out com.atlassian.bitbucket.jenkins.internal.scm.BitbucketSCM into C:\jenkins<main_repo branch name>@script to read main_repo/Jenkinsfile
The recommended git tool is: NONE
using credential 27cc7c94-f898-4cf8-ad4a-51e5cdb95c0f
Cloning the remote Git repository
Cloning repository /scm/sw/jenkinsfile_repo.git

git init C:\jenkins<main_repo branch name>@script # timeout=10
Fetching upstream changes from /scm/sw/jenkinsfile_repo.git
git --version # timeout=10
git --version # 'git version 2.27.0.windows.1'
using GIT_ASKPASS to set credentials
git fetch --tags --force --progress -- /scm/sw/jenkinsfile_repo.git +refs/heads/:refs/remotes/jenkinsfile_repo/ # timeout=10
git config remote.jenkinsfile_repo.url /scm/sw/jenkinsfile_repo.git # timeout=10
git config --add remote.jenkinsfile_repo.fetch +refs/heads/:refs/remotes/jenkinsfile_repo/ # timeout=10
Avoid second fetch
Seen branch in repository jenkinsfile_repo/<main_repo branch name>
Seen branch in repository jenkinsfile_repo/master
Seen 2 remote branches
git show-ref --tags -d # timeout=10
Multiple candidate revisions
Checking out Revision 33938ad54089375aa59ea6c9434f4048a3276707 (jenkinsfile_repo/<main_repo branch name>)
git config core.sparsecheckout # timeout=10
git checkout -f 33938ad54089375aa59ea6c9434f4048a3276707 # timeout=10
Commit message: ""
First time build. Skipping changelog.
Posting build status of INPROGRESS to Bitbucket for commit id [33938ad54089375aa59ea6c9434f4048a3276707] and ref 'null'
[Checks API] No suitable checks publisher found.
Posting build status of FAILED to Bitbucket for commit id [33938ad54089375aa59ea6c9434f4048a3276707] and ref 'null'
[Checks API] No suitable checks publisher found.
ERROR: C:\jenkins<main_repo branch name>@script\main_repo\Jenkinsfile not found

To Reproduce
Steps to reproduce the behavior:
Create a new repository named "jenkinsfile_repo" for storing Jenkins files. It is stored in "/jenkinsfile_repo.git"
Create a directory in the "jenkinsfile_repo" for your project. Name it "main_repo"
Create your Jenkins File, named "Jenkinsfile" under "main_repo" directory, and commit/push it to the "jenkinsfile_repo" repository
Open you Jenkins and create a new MultiBranch Pipeline Project, named "main_repo_job"
Add Branch Sources for your project to build. Fill information for "main_repo".
In the "Build Configuration", change "Mode" to "by Remote Jenkins File Plugin".
Define your script path: "main_repo/Jenkinsfile".
Define your SCM for Jenkins file. For example, it will be the repository /jenkinsfile_repo.git .
Click Save.
Now when a branch changes in the main_repo repository, a build will start. After the Jenkinsfile checkout, the build tries to run Jenkinsfile on their own repository (jenkinsfile_repo), not on the main_repo repository. It will fail, of course, because there isn't any source code in the jenkinsfile_repository.

Expected behavior
If a branch changes in the main_repo repository, then check out that branch, then checkout the Jenkinsfile from jenkinsfile_repo repository and start the compile steps for main_repo described in the Jenkinsfile came from jenkinsfile_repo.

Desktop (please complete the following information):

  • Windows 10, Fedora Linux
  • BitBucket server as git source
  • Jenkins version: 2.249.1, Remote Jenkinsfile Provider plugin version: 1.11

Don't work with Gitlab Branch Source in multibranch pipeline

Describe the bug
Pipeline don't work with Gitlab Branch Source in multibranch pipeline. Don't work code (in Remote Jenkins file):

stage('Checkout SCM') {
    steps {
        script {
            scmInfo = checkout scm
        }
    }
}

Jenkins 2.361.4
Gitlab Branch Source Plugin version 660.vd45c0f4c0042
Remote Jenkinsfile Provider Plugin version 1.23

Build runs on the remote repo with the Jenkinsfile, not on the main repository (GitHub)

Describe the bug
Given a multibranch pipeline with a configuration similar to:

When a build is triggered by a GitHub webhook from the mainRepo
Then only the code from the remoteRepo is checked out, while I would have expected a checkout (and build) of the mainRepo as well?

This issue seems similar to #17

To Reproduce
With the configuration described above, push a change to the main repo to trigger a multibranch build.

Expected behavior
A checkout and build of the mainRepo.

Additional context
Versions used:

  • Jenkins 2.303.1
  • Pipeline: Multibranch: 2.26
  • Remote Jenkinsfile Provider: 1.17
  • GitHub Branch Source 2.11.1

Expose the git branch of the Jenkinsfile repository that was checked out

Per #21 the the environment variables RJPP_SCM_URL and RJPP_JENKINSFILE describe the checkout of the Jenkinsfile used for the current build. The plugin will made a decision as to what branch/git ref to check out, but this information is not available. Please include this detail as an additional environment variable. Thanks!

Tier 2 upgrade

I like your plugin as it helps us centralise management of our pipelines, but as per the maintenance policies - plugins below Tier 2 from Cloudbees are not supported to be installed at our organisation.

Are there any plans to move the plugin to Tier 2 within Cloudbees?

Plugin uses now-deprecated dependent plugin (causes yellow bar in Jenkins)

Describe the bug
The https://plugins.jenkins.io/workflow-cps-global-lib has been deprecated.
Remote Jenkinsfile plugin uses the now-obsolete plugin. Please migrate to the
new plugin which is https://plugins.jenkins.io/pipeline-github-lib

Thank you!


Pipeline: Deprecated Groovy Libraries
Version 588.v576c103a_ff86
Hosting of Pipeline Groovy libraries inside a Jenkins Git server. Deprecated. Use Pipeline: Groovy Libraries instead. If you see this plugin installed just because you upgraded, you can probably uninstall it now. This plugin should only be used if you have historically pushed libraries to a Git server inside Jenkins.
Report an issue with this plugin
This plugin is deprecated. In general, this means that it is either obsolete, no longer being developed, or may no longer work. Learn more.

Use Remote Jenkins Provider in multi-node pipelines

Note: I asked this question in stackoverflow, but I got no answers yet. I'm sharing here because even if this is not really something related to the remote provider's behavior, I think it is common use-case and not been able to support it makes the remote provider less usable.


I'm using the very handy Remote Jenkins Provider plugin for retrieving the Jenkinsfile from a different repository than the source code. The Jenkinsfile script and some helper files are downloaded in the @scripts directory in the job workspace. This works as expected.

The problem I'm facing is that the pipeline runs in multiple nodes and the @script directory is only available in the master node (the one that retrieved it from the repository). I'm trying to use stash/unstash for sharing this directory across nodes:

node('master'){
    stage('save script'){
        sh "ls -l ${env.WORKSPACE}@script/"
        stash name: 'scripts', includes: "${env.WORKSPACE}@script/**"
    }
}

pipeline {
    agent { label 'worker' }
    stages {
        stage('build') {
            steps {
                unstash name: 'scripts'
                sh "${env.WORKSPACE}@script/helper.sh"
            }
        }
    }
}

The output of the ls step shows that both the Jenkinsfile and the helper script are there:

-rw-r--r-- 1 jenkins jenkins 166 Jul 10 16:53 Jenkinsfile
-rwxr-xr-x 1 jenkins jenkins  29 Jul 10 15:18 helper.sh

However, the build fails in the stash step with this message: ERROR: No files included in stash ‘scripts’

Notice that if I run the whole pipeline in the same master node, without stash/unstash, it works.

How can I share the @script directory across nodes?

Null value not allowed as an environment variable: RJPP_BRANCH

Describe the bug
After upgrading from version 1.13 to 1.14, job fails with following error message:
java.lang.IllegalArgumentException: Null value not allowed as an environment variable: RJPP_BRANCH at hudson.EnvVars.put(EnvVars.java:379) at org.jenkinsci.plugins.workflow.multibranch.extended.RemoteJenkinsFileItemListener.buildEnvironmentFor(RemoteJenkinsFileItemListener.java:43) at hudson.model.Run.getEnvironment(Run.java:2442) at org.jenkinsci.plugins.workflow.job.WorkflowRun.getEnvironment(WorkflowRun.java:492) at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:113) at org.jenkinsci.plugins.workflow.multibranch.extended.scm.ExtendedSCMBinder.create(ExtendedSCMBinder.java:83) at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:309) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429)

Screenshots
image

Additional context
Configured job is a multibranch pipeline.
Jenkins version: 2.279

Cron-configuration in Jenkinsfile ignored when obtained via Remote Jenkinsfile Provider plugin?

Describe the bug
Build is not triggered at expected time, even though Jenkins shows the time-based trigger in the GUI correctly: H H(16-18) * * 0-5
We suspect that this is ignored since the Jenkinsfile needs to be obtained from Git by the plugin first before it can be interpreted.
Push-based triggering works fine though.

To Reproduce
Steps to reproduce the behavior:

  1. Create a Jenkinsfile with a 'triggers'-section containing a 'cron'-statement
  2. Upload Jenkinsfile to a Git-repo
  3. Configure a build to use the Remote Jenkinsfile Provider plugin to obtain the Jenkinsfile from the Git-repo
  4. Wait until build is triggered at expected time - which won't happen

Expected behavior
Build triggers at times specified in Jenkinsfile.

Versions:

  • Jenkins version 2.246.1
  • Plugin version: 1.24

Support Pull-Requests in MatchBranches feature

Hi,
From what I see in the documentation/code, MatchBranches feature will fallback on master branch when called with a Pull-Request branch, unless some kind of hook is implemented to make sure Pull-Request branches are equally created in the remote Jenkinsfile repository...
Would be nice if the MatchBranches feature could handle Pull-Request branches by using the branch they are based on (env.CHANGE_BRANCH) when trying to match with the remote Jenkinsfile repo.
Thanks!

Build runs on Jenkinsfile repo

Describe the bug

Here is a configuration:

Multibranch pipeline:

Branch sources (Bitbucket Cloud): bitbucket.org:xxx/servicecore.git
Global shared library repo (Bitbucket Cloud) : bitbucket.org:xxx/xxx-infra-cicd.git
Remote Jenkinsfile Provider Plugin (Bitbucket Cloud) : bitbucket.org:xxx/xxx-infra-cicd.git

Build is triggered by Jenkins UI. The code from bitbucket.org:xxx/xxx-infra-cicd.git for Jenkinsfile_common is checked out, then the same repo is checked out for global shared library, but the main repo bitbucket.org:xxx/servicecore.git is never checked out. So the build fails.

To Reproduce

Just start build in the main repo bitbucket.org:xxx/servicecore.git via Jenkins UI.

Expected behavior

A checkout and build of the bitbucket.org:xxx/servicecore.git as a step after any other checkouts.

Additional context

Here is a log file:

`
Checking out git [email protected]:xxx/xxx-infra-cicd.git into /var/lib/jenkins/workspace/servicecore_master@script to read Jenkinsfile_common
The recommended git tool is: NONE
using credential xx-ssh

git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/servicecore_master@script/.git # timeout=10
Fetching changes from the remote Git repository
git config remote.origin.url [email protected]:xxx/xxx-infra-cicd.git # timeout=10
Fetching upstream changes from [email protected]:xxx/xxx-infra-cicd.git
git --version # timeout=10
git --version # 'git version 2.25.1'
using GIT_SSH to set credentials
git fetch --tags --force --progress -- [email protected]:xxx/xxx-infra-cicd.git +refs/heads/:refs/remotes/origin/ # timeout=10
git rev-parse refs/remotes/origin/master^{commit} # timeout=10
Checking out Revision 2245883c9e05a219fb48c39e3dca1f79e68e8afa (refs/remotes/origin/master)
git config core.sparsecheckout # timeout=10
git checkout -f 2245883c9e05a219fb48c39e3dca1f79e68e8afa # timeout=10
Commit message: "Fix typo"
git rev-list --no-walk 2245883c9e05a219fb48c39e3dca1f79e68e8afa # timeout=10
Loading library xxx-cicd@master
Attempting to resolve master from remote references...
git --version # timeout=10
git --version # 'git version 2.25.1'
using GIT_SSH to set credentials
git ls-remote -h -- [email protected]:xxx/xxx-infra-cicd.git # timeout=10
Found match: refs/heads/master revision 2245883c9e05a219fb48c39e3dca1f79e68e8afa
The recommended git tool is: NONE
using credential xx-ssh
git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/servicecore_master@libs/xxx-cicd/.git # timeout=10
Fetching changes from the remote Git repository
git config remote.origin.url [email protected]:xxx/xxx-infra-cicd.git # timeout=10
Fetching without tags
Fetching upstream changes from [email protected]:xxx/xxx-infra-cicd.git
git --version # timeout=10
git --version # 'git version 2.25.1'
using GIT_SSH to set credentials
git fetch --no-tags --force --progress -- [email protected]:xxx/xxx-infra-cicd.git +refs/heads/:refs/remotes/origin/ # timeout=10
Checking out Revision 2245883c9e05a219fb48c39e3dca1f79e68e8afa (master)
git config core.sparsecheckout # timeout=10
git checkout -f 2245883c9e05a219fb48c39e3dca1f79e68e8afa # timeout=10
Commit message: "Fix typo"
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/lib/jenkins/workspace/servicecore_master
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Build image)
[Pipeline] sh

  • git log -1 --pretty=%h
    fatal: not a git repository (or any of the parent directories): .git
    [Pipeline] }
    [Pipeline] // stage
    [Pipeline] sh
  • docker images --filter label=build_tag=jenkins-servicecore-master-34 -q
  • uniq
  • xargs -r docker rmi -f
    [Pipeline] }
    [Pipeline] // node
    [Pipeline] End of Pipeline
    ERROR: script returned exit code 128
    Finished: FAILURE
    `

Remote Jenkinsfile repo fails: ERROR: Error cloning remote repo 'origin' status code 128

Describe the bug
In a Jenkins pipeline job fails to clone the repo for the remote Jenkinsfile
The fix for the error "stderr: fatal: detected dubious ownership in repository at '/var/jenkins_home/workspace/" says to set the directory as 'safe' in git. The directory is set to safe in the controller's Dockerfile. Controller runs in AWS ECS container, agent nodes are AWS EC2 instances.

To Reproduce
Steps to reproduce the behavior:

  1. Create a remote Jenkinsfile that starts like this:
  • note I have at other times configured the agent to run on specific nodes with a label, the error persists
import org.jenkinsci.plugins.pipeline.modeldefinition.Utils

pipeline {
    agent any
    environment {
        GIT_CREDENTIALS = credentials('auth-jenkins')
    }
    parameters {
        text(name: "CONFIG_JSON", defaultValue: "{}", description: "JSON String for Pipeline Configuration")
        string(name: 'root_message', defaultValue: '')
    }
    options {
        timeout(time: 30, unit: 'MINUTES')
        skipDefaultCheckout(true)
    }
  1. In a Jenkins Pipeline job configure Remote Jenkinsfile Plugin as below
    image

  2. Run Jenkins Pipeline build

  3. See error in Jenkins console log:
    image

NOTE: Dockerfile for Jenkins controller sets /var/jenkins_home/workspace as 'safe':
image

Jenkins Job Config XML

<org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject plugin="[email protected]">
	<actions />
	<description />
	<displayName>
		Pluto2
	</displayName>
	<properties>
		<org.jenkinsci.plugins.docker.workflow.declarative.FolderConfig plugin="[email protected]">
			<dockerLabel />
			<registry plugin="[email protected]" />
		</org.jenkinsci.plugins.docker.workflow.declarative.FolderConfig>
	</properties>
	<folderViews class="jenkins.branch.MultiBranchProjectViewHolder" plugin="[email protected]">
		<owner class="org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject" reference="../.." />
	</folderViews>
	<healthMetrics />
	<icon class="jenkins.branch.MetadataActionFolderIcon" plugin="[email protected]">
		<owner class="org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject" reference="../.." />
	</icon>
	<orphanedItemStrategy class="com.cloudbees.hudson.plugins.folder.computed.DefaultOrphanedItemStrategy" plugin="[email protected]">
		<pruneDeadBranches>
			true
		</pruneDeadBranches>
		<daysToKeep>
			-1
		</daysToKeep>
		<numToKeep>
			-1
		</numToKeep>
		<abortBuilds>
			false
		</abortBuilds>
	</orphanedItemStrategy>
	<triggers />
	<disabled>
		false
	</disabled>
	<sources class="jenkins.branch.MultiBranchProject$BranchSourceList" plugin="[email protected]">
		<data>
			<jenkins.branch.BranchSource>
				<source class="jenkins.scm.impl.subversion.SubversionSCMSource" plugin="[email protected]">
					<id>
						fd603df5-2c48-4926-8509-d9a61610cad6
					</id>
					<remoteBase>
						https://server.corp.com/a427f7f3-c37f-485a-8e5f-ef2f18c14abf/
					</remoteBase>
					<credentialsId>
						255680d7-f207-4f34-b68e-bbc9d9f0bbef
					</credentialsId>
					<includes>
						trunk
					</includes>
					<excludes />
					<workspaceUpdater class="hudson.scm.subversion.CheckoutUpdater" />
				</source>
				<strategy class="jenkins.branch.DefaultBranchPropertyStrategy">
					<properties class="empty-list" />
				</strategy>
			</jenkins.branch.BranchSource>
		</data>
		<owner class="org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject" reference="../.." />
	</sources>
	<factory class="org.jenkinsci.plugins.workflow.multibranch.extended.RemoteJenkinsFileWorkflowBranchProjectFactory" plugin="[email protected]">
		<owner class="org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject" reference="../.." />
		<scriptPath>
			Jenkinsfile
		</scriptPath>
		<localMarker />
		<remoteJenkinsFile>
			./cicd/jenkins_pipeline/Jenkinsfile
		</remoteJenkinsFile>
		<remoteJenkinsFileSCM class="hudson.plugins.git.GitSCM" plugin="[email protected]">
			<configVersion>
				2
			</configVersion>
			<userRemoteConfigs>
				<hudson.plugins.git.UserRemoteConfig>
					<url>
						https://[email protected]/corp-realty/corp-cicd.git
					</url>
					<credentialsId>
						corp-jenkins
					</credentialsId>
				</hudson.plugins.git.UserRemoteConfig>
			</userRemoteConfigs>
			<branches>
				<hudson.plugins.git.BranchSpec>
					<name>
						fix-build-failures
					</name>
				</hudson.plugins.git.BranchSpec>
			</branches>
			<doGenerateSubmoduleConfigurations>
				false
			</doGenerateSubmoduleConfigurations>
			<submoduleCfg class="empty-list" />
			<extensions>
				<hudson.plugins.git.extensions.impl.WipeWorkspace />
				<hudson.plugins.git.extensions.impl.CloneOption>
					<shallow>
						true
					</shallow>
					<noTags>
						true
					</noTags>
					<reference />
					<depth>
						1
					</depth>
					<honorRefspec>
						true
					</honorRefspec>
				</hudson.plugins.git.extensions.impl.CloneOption>
				<hudson.plugins.git.extensions.impl.CloneOption>
					<shallow>
						true
					</shallow>
					<noTags>
						false
					</noTags>
					<reference />
					<depth>
						1
					</depth>
					<honorRefspec>
						true
					</honorRefspec>
				</hudson.plugins.git.extensions.impl.CloneOption>
				<hudson.plugins.git.extensions.impl.WipeWorkspace />
			</extensions>
		</remoteJenkinsFileSCM>
		<matchBranches>
			false
		</matchBranches>
		<scmSourceBranchName>
			master
		</scmSourceBranchName>
		<fallbackBranch />
		<lookupInParameters>
			false
		</lookupInParameters>
	</factory>
</org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject>

Expected behavior
Expected remote Jenkinsfile repository to clone and run the configured job

Screenshots
See above

Jenkins Version
2.426.2

Remote Jenkinsfile Provider
Version 1.24

Additional context
For other Jenkins Pipeline jobs with identical configuration the build succeeds
image

Bitbucket Notifications not working with Remote Jenkinsfile

Describe the bug
Hi. I've detected that when using the Remote Jenkinsfile plugin with a bitbucket repository, the default bitbucket notification that comes with multibranch pipelines stops working. Even using the Bitbucket Build Status Notifier to force the notification doesn't work either.

To Reproduce
Steps to reproduce the behavior:

  1. Having a vanilla multibranch pipeline with a bitbucket repo
  2. A bitbucket notification system working on this multibranch pipeline
  3. Getting the Remote Jenkinsfile Plugin and changing the Jenkins file system to the remote Jenkinsfile.
  4. Bitbucket notications stops working.

Expected behavior
The bitbucket notifications should work with the Remote Jenkinsfile Plugin

Jenkins Version

  • [Jenkins 2.346.3]

Remote Jenkinsfile Plugin Version

  • 1.23

The RJPP_BRANCH env variable is set to the source's branch, not the remote file's branch

It appears the variable that was supposed to indicate the remote file branch (or, hopefully, also tags, which is our use case), actually points to the source branch.

This line in particular is suspicious

Steps to reproduce the behaviour:

  1. Create a job using the Remote Jenkinsfile build configuration
  2. Change the default branch to any other branch
  3. Use the RJPP_BRANCH variable in the pipeline
  4. Notice that the value of $RJPP_BRANCH is always the source branch (the same as BRANCH_NAME).

This is the pipeline on my master branch:

pipeline {
    agent any

    stages {
        stage('Test') {
            steps {
                echo 'Hello!'
            }
        }
        stage('Branch') {
            steps {
                echo "Branch is: ${env.RJPP_BRANCH}"
            }
        }
    }
}

This is the pipeline on my "not-master" branch:

pipeline {
    agent any

    stages {
        stage('Test') {
            steps {
                echo 'Hello! This is the not-master pipeline!'
                echo "Current source branch is ${env.BRANCH_NAME}"
            }
        }
        stage('Branch') {
            steps {
                echo "Current RJPP_BRANCH branch is: ${env.RJPP_BRANCH}"
            }
        }
    }
}

Screenshots:
image

image

image

Remote Jenkinsfile Provider takes exception with Subversion SCM

When a generated Multibranch Pipeline job is executed that uses a jenkinsfile obtained using Remote Jenkinsfile Provider plug-in, the following occurs early in the run of the generated project:

ERROR: Checkout failed
java.lang.ClassCastException: hudson.scm.SubversionSCM cannot be cast to hudson.plugins.git.GitSCM
at org.jenkinsci.plugins.workflow.multibranch.extended.scm.ExcludeFromChangeSet$ScmListener.onCheckout(ExcludeFromChangeSet.java:59)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:150)
at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:155)
at org.jenkinsci.plugins.workflow.multibranch.extended.scm.ExtendedSCMBinder.create(ExtendedSCMBinder.java:83)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:309)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:427)
ERROR: Maximum checkout retry attempts reached, aborting
Finished: FAILURE

Errors on plugin

Describe the bug
Don't know if this is related but the error is triggered by this plugin

To Reproduce
Steps to reproduce the behavior:
Configure Jenkins LTS multbranch pipeline and scan gitlab

Expected behavior
Should scan gitlab projects and branches without errors

Screenshots
Error:

1 branches were processed (query completed)
ERROR: Failed to create or update a subproject commons / wtr-ecomm-spring-sleuth-baggage
java.lang.NullPointerException
	at org.jenkinsci.plugins.workflow.multibranch.extended.RemoteJenkinsFileWorkflowMultiBranchProjectFactory.getMatchBranches(RemoteJenkinsFileWorkflowMultiBranchProjectFactory.java:178)
	at org.jenkinsci.plugins.workflow.multibranch.extended.RemoteJenkinsFileWorkflowMultiBranchProjectFactory.customize(RemoteJenkinsFileWorkflowMultiBranchProjectFactory.java:160)
	at org.jenkinsci.plugins.workflow.multibranch.AbstractWorkflowMultiBranchProjectFactory.doCreateProject(AbstractWorkflowMultiBranchProjectFactory.java:51)
	at org.jenkinsci.plugins.workflow.multibranch.AbstractWorkflowMultiBranchProjectFactory.doCreateProject(AbstractWorkflowMultiBranchProjectFactory.java:46)
	at jenkins.branch.MultiBranchProjectFactory$BySCMSourceCriteria.createNewProject(MultiBranchProjectFactory.java:235)
	at jenkins.branch.OrganizationFolder$SCMSourceObserverImpl$1.completeNew(OrganizationFolder.java:1451)
	at jenkins.branch.OrganizationFolder$SCMSourceObserverImpl$1.complete(OrganizationFolder.java:1395)
	at jenkins.scm.api.trait.SCMNavigatorRequest.process(SCMNavigatorRequest.java:254)
	at jenkins.scm.api.trait.SCMNavigatorRequest.process(SCMNavigatorRequest.java:204)
	at io.jenkins.plugins.gitlabbranchsource.GitLabSCMNavigator.visitSources(GitLabSCMNavigator.java:290)
	at jenkins.branch.OrganizationFolder.computeChildren(OrganizationFolder.java:536)
	at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:278)
	at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:166)
	at jenkins.branch.OrganizationFolder$OrganizationScan.run(OrganizationFolder.java:918)
	at hudson.model.ResourceController.execute(ResourceController.java:101)
	at hudson.model.Executor.run(Executor.java:442)

We are not sure if this is caused by this plugin, any help is greatly appreciated . Thanks

CASC configuration

Hello. How can i write casc configuration for your plugin. Thank you.

Upon changing agents, plugin tries do checkout the scanned repository

Describe the bug
Plugin going into default behavior when changing agents. It begins without issue, checks out the Remote Pipeline and executes, but when a new agents is called in, before any pipeline action is taken, the plugin attempts to checkout the entire repo that is set in the Multibranch Pipeline, instead of the Remote one.

To Reproduce
Steps to reproduce the behavior:
Make a multi agent pipeline

Expected behavior
The defined remote pipeline to be downloaded and used on all agents.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: Linux and Windows
  • Browser Chrome
  • Version Latest at time of writting
13:56:56  Started by user Paulo Godinho
13:56:56  Checking out git [email protected]:company/consoles-engine-client-dll-builder.git into /var/jenkins_home/workspace/l-builder_feature_dll-auto-build@script/0c715dbd54feefe571a700899ea82cf5b6aa729750a2fa95ff7d94927bbadab9 to read test.jenkinsfile
13:56:56  The recommended git tool is: NONE
13:56:56  using credential tooling/jenkins/companybot_github_ssh/privatekey
13:56:56   > git rev-parse --resolve-git-dir /var/jenkins_home/workspace/l-builder_feature_dll-auto-build@script/0c715dbd54feefe571a700899ea82cf5b6aa729750a2fa95ff7d94927bbadab9/.git # timeout=10
13:56:56  Fetching changes from the remote Git repository
13:56:56   > git config remote.origin.url [email protected]:company/consoles-engine-client-dll-builder.git # timeout=10
13:56:56  Fetching upstream changes from [email protected]:company/consoles-engine-client-dll-builder.git
13:56:56   > git --version # timeout=10
13:56:56   > git --version # 'git version 2.30.2'
13:56:56  using GIT_SSH to set credentials SSH Private Key to access Github as company Bot
13:56:56   > git fetch --tags --force --progress -- [email protected]:company/consoles-engine-client-dll-builder.git +refs/heads/*:refs/remotes/origin/* # timeout=10
13:56:57   > git rev-parse refs/remotes/origin/main^{commit} # timeout=10
13:56:57  Checking out Revision 7339ad9459ee27371c00d4576f777c01fae2937e (refs/remotes/origin/main)
13:56:57   > git config core.sparsecheckout # timeout=10
13:56:57   > git checkout -f 7339ad9459ee27371c00d4576f777c01fae2937e # timeout=10
13:56:57  Commit message: "Fixing cleanup step"
13:56:57   > git rev-list --no-walk d075d6413d5521a0cc6bff7452a8d249676b6d3d # timeout=10
13:56:58  [Pipeline] Start of Pipeline
13:56:58  [Pipeline] withEnv
13:56:58  [Pipeline] {
13:56:59  [Pipeline] stage
13:56:59  [Pipeline] { (Verify if Build DLL is Required)
13:56:59  [Pipeline] node
13:56:59  Running on EC2 (PortUnit-EC2) - (Ubuntu) General purpose Ubuntu machine (i-0a154708cfa8e92da) in /home/ubuntu/Jenkins/workspace/l-builder_feature_dll-auto-build

[...] Running a bunch of stages and steps
[...] Time to do something in another machine

13:57:08  [Pipeline] { (Build DLLs)
13:57:09  [Pipeline] node
13:57:09  Running on EC2 (PortUnit-EC2) - (Windows) Persistent - 4.27 company-develop-plus-consoles (i-05ef20281730f8501) in C:\Jenkins\workspace\l-builder_feature_dll-auto-build
13:57:09  [Pipeline] {
13:57:09  [Pipeline] checkout
13:57:09  The recommended git tool is: git
13:57:11  using credential tooling/jenkins/companybot_github_ssh/privatekey
13:57:11   > git rev-parse --resolve-git-dir C:\Jenkins\workspace\l-builder_feature_dll-auto-build\.git # timeout=10
13:57:12  Fetching changes from the remote Git repository
13:57:12  Fetching without tags
13:57:12   > git config remote.origin.url [email protected]:company/my-project.git # timeout=10
13:57:12  Fetching upstream changes from [email protected]:company/my-project.git
13:57:12   > git --version # timeout=10
13:57:14  Checking out Revision cfa90907af9e63736a616eaeb976f80bf413d773 (feature/dll-auto-build)
13:57:13   > git --version # 'git version 2.36.0.windows.1'
13:57:13  using GIT_SSH to set credentials SSH Private Key to access Github as company Bot
13:57:13   > git fetch --no-tags --force --progress -- [email protected]:company/my-project.git +refs/heads/*:refs/remotes/origin/* # timeout=10
13:57:14   > git config core.sparsecheckout # timeout=10
13:57:14   > git checkout -f cfa90907af9e63736a616eaeb976f80bf413d773 # timeout=10

Is there something I may be missing? I will try to solve this by forwarding the actions to a secondary job.

Scan for branches fails after pipeline creation by JobDSL

Describe the bug
Scan for branches fails after pipeline creation by JobDSL job.

To Reproduce
Steps to reproduce the behavior:

  1. Create a pipeline by JobDSL job
  2. Go to the pipeline
  3. Push the button ('Scan Multibranch Pipeline Now')
  4. Take a look into the log of the scan and see java.lang.NullPointerException

Workaround
Go to Configure and push Save button without any changes.

Log

Fri Dec 17 11:08:25 UTC 2021] Starting branch indexing...
 > git --version # timeout=10
 > git --version # 'git version 2.25.1'
using GIT_ASKPASS to set credentials service-user
 > git ls-remote --symref -- <REPOSITORY_URL> # timeout=10
 > git rev-parse --resolve-git-dir /var/lib/jenkins/caches/git-fe9dc345026856a1354c034869a81bf1/.git # timeout=10
Setting origin to <REPOSITORY_URL>
 > git config remote.origin.url <REPOSITORY_URL> # timeout=10
Fetching & pruning origin...
Listing remote references...
 > git config --get remote.origin.url # timeout=10
 > git --version # timeout=10
 > git --version # 'git version 2.25.1'
using GIT_ASKPASS to set credentials service-user
 > git ls-remote -h -- <REPOSITORY_URL> # timeout=10
Fetching upstream changes from origin
 > git config --get remote.origin.url # timeout=10
using GIT_ASKPASS to set credentials service-user
 > git fetch --tags --force --progress --prune -- origin +refs/heads/*:refs/remotes/origin/* # timeout=10
Checking branches...
  Checking branch <BRANCH_1>
No local file defined. Skipping Source Code SCM probe, since Jenkinsfile will be provided by Remote Jenkins File Plugin
    Met criteria
ERROR: [Fri Dec 17 11:08:26 UTC 2021] Could not fetch branches from source 1
[Fri Dec 17 11:08:26 UTC 2021] Finished branch indexing. Indexing took 0.49 sec
FATAL: Failed to recompute children of Folder » Pipeline
java.lang.NullPointerException
	at org.jenkinsci.plugins.workflow.multibranch.extended.RemoteJenkinsFileWorkflowBranchProjectFactory.createDefinition(RemoteJenkinsFileWorkflowBranchProjectFactory.java:97)
	at org.jenkinsci.plugins.workflow.multibranch.AbstractWorkflowBranchProjectFactory.setBranch(AbstractWorkflowBranchProjectFactory.java:63)
	at org.jenkinsci.plugins.workflow.multibranch.AbstractWorkflowBranchProjectFactory.newInstance(AbstractWorkflowBranchProjectFactory.java:54)
	at org.jenkinsci.plugins.workflow.multibranch.AbstractWorkflowBranchProjectFactory.newInstance(AbstractWorkflowBranchProjectFactory.java:44)
	at jenkins.branch.MultiBranchProject$SCMHeadObserverImpl.observeNew(MultiBranchProject.java:2055)
	at jenkins.branch.MultiBranchProject$SCMHeadObserverImpl.observe(MultiBranchProject.java:1986)
	at jenkins.scm.api.trait.SCMSourceRequest.process(SCMSourceRequest.java:357)
	at jenkins.plugins.git.AbstractGitSCMSource$8.discoverBranches(AbstractGitSCMSource.java:705)
	at jenkins.plugins.git.AbstractGitSCMSource$8.run(AbstractGitSCMSource.java:609)
	at jenkins.plugins.git.AbstractGitSCMSource$8.run(AbstractGitSCMSource.java:588)
	at jenkins.plugins.git.AbstractGitSCMSource.doRetrieve(AbstractGitSCMSource.java:394)
	at jenkins.plugins.git.AbstractGitSCMSource.doRetrieve(AbstractGitSCMSource.java:350)
	at jenkins.plugins.git.AbstractGitSCMSource.retrieve(AbstractGitSCMSource.java:588)
	at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373)
	at jenkins.scm.api.SCMSource.fetch(SCMSource.java:283)
	at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:641)
	at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:278)
	at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:166)
	at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:1032)
	at hudson.model.ResourceController.execute(ResourceController.java:99)
	at hudson.model.Executor.run(Executor.java:432)
Finished: FAILURE

JobDSL code

branchSources {
  git {
    id('1')
    remote('<CODE_REPOSITORY_URL>')
    credentialsId('service-user')
  }
}
factory {
  remoteJenkinsFileWorkflowBranchProjectFactory{
    localMarker('')
    remoteJenkinsFile('folder/subfolder/Jenkinsfile.groovy')
    remoteJenkinsFileSCM{
      gitSCM{
        userRemoteConfigs{
          userRemoteConfig{
            name('origin')
            url('<JENKINSFILES_REPOSITORY_URL>')
            refspec('')
            credentialsId('service-user')
          }
          browser{
            gitLab {
              repoUrl('<REPOSITORY_URL>')
            }
          }
          gitTool('')
        }
      }
    }
  }
}
orphanedItemStrategy {
  discardOldItems {
    daysToKeep(90)
  }
}

Expected behavior
A repository can be scanned for branches right after creation by JobDSL.

Versions
Jenkins 2.319.1
Job DSL 1.78.1
Remote Jenkinsfile Provider 1.21

Make bitbucket branch source dependency optional

Is your feature request related to a problem? Please describe.

The plugin requires the bitbucket branch source as a dependency. I do not use bitbucket. Even worse, the bitbucket branch source plugin has a series of dependencies that are not relevant to me (e.g. Mercurial, which requires an update of the Credential plugin)

Describe the solution you'd like

This plugin should be installed if at least one branch source plugin (e.g. githb) is installed instead of requiring all of them to be installed.

Describe alternatives you've considered
The alternative is to bloat my Jenkins installation with unused plugins.

Download remote Jenkinsfile at specific git tag

How to download a specific tag from the JenkinsFile repo?

I want to create a structure with multiple Jenkinsfile in the same centralized repo for different platforms:

Flutter/Jenkinsfile
Android/Jenkinsfile
iOS/Jenkinsfile
Web/Jenkinsfile

I have to tag a specific version of the Jenkinsfiles to prevent breaking all the projects which does not handle every version of the same pipeline.

I've tried setting refs/tags/0.0.1 in Fallback branch property but is not working.
What is the property I should set? Currently it always get a Jenkinsfile from latest commit of master branch

Thank you,
Mattia

Support for groovy `load`

I would like to be able to load another groovy file relative to the remote Jenkinsfile, not the locally cloned repo.

For example, suppose the repo containing the Jenkinsfile also contains file_to_be_included.groovy.
When I attempt to load "file_to_be_included.groovy", I get a java.nio.file.NoSuchFileException as jenkins is looking in the source code repo, not the jenkinsfile repo.

As a workaround, I can use the RJPP_BRANCH, RJPP_JENKINSFILE, and RJPP_SCM_URL to re-clone the remote repo and then attempt the load (relative to where I re-cloned the repo).

New commits to remote jenkinsfile repository triggers builds

Describe the bug
Jenkins polls remote jenkinsfile repository even though it should not. I have followed the documentation and added Additional behaviors "Exclude from Poll" but commits pushed to the remote jenkinsfile repository triggers build anyway. I can see in the Scan Multibranch Pipeline Log that Jenkins polls remote jenkinsfile repository.

Expected behavior
The remote jenkinsfile repository should be ignored and not trigger builds on new commits.

Screenshots
bild

Additional context
Using Remote Jenkinsfile Provider version 1.21

recreate branch on in multibranch will use the old jenkinsfile

In a multibranch pipeline, we encountered a situation where there was a branch named "aaa" with a corresponding remote Jenkinsfile repository having the same branch name.

After merging the "aaa" branch into the multibranch pipeline, deleting the remote Jenkinsfile repository for the "aaa" branch, and subsequently recreating the "aaa" branch, the pipeline started using an outdated Jenkinsfile.

couldn't find any cleanup mechanism in place for managing Jenkinsfiles in such scenarios.

To Reproduce

  1. create branch aaa in multibranch + the same aaa branch in remote repo
  2. merge aaa / or delete aaa in multi branch project
  3. delete aaa branch from remote jenkinsfiles repo
    4 recreate aaa branch in multibranch - it use the very old Jenkinsfile

Expected behavior
use the latest jenkinsfile

Screenshots
If applicable, add screenshots to help explain your problem.

Jenkins LTS - 2.414.3
plugin version : 1.24

Plugin is just cloning the JenkinsFile remote repository

In a multi branch pipeline job. When the pipeline has been executed this is just cloning the remote repository where JenkinsFile is hosted, ignoring the configured branch source (with GitLab)

To Reproduce

  1. Create a multi branch pipeline
  2. Add any "CrazyProject" url as branch source. (with default params selected)
  3. In build configuration section
  4. Select By RemoteJenkinsFile Provider Plugin:
    a. Fallback = main
    b. JenkinsFile SCM = Git
    c. Repository URL = MyJenkinsLibrary url.
    c. Provide valid credentials.
    d. Branch specifier = */main
  5. Use below code as reference for JenkinsFile
    #!/usr/bin/env groovy
pipeline {
    agent any
    options {
        buildDiscarder(
            logRotator(artifactDaysToKeepStr:'50', artifactNumToKeepStr: '10', daysToKeepStr: '100', numToKeepStr: '10')
        )
    }
    tools {
        // Install the Maven version configured as "M3" and add it to the path.
        maven "local_maven"
    }
    stages {
      stage('Build'){
        steps{
            echo "${env.RJPP_SCM_URL}"
            echo "${env.RJPP_JENKINSFILE}"
            echo "${env.RJPP_BRANCH}"
            echo "${env.RJPP_LOCAL_MARKER}"
            echo "${env.BRANCH_NAME}"
            sh 'printenv'

            sh "mvn -Dmaven.test.failure.ignore=true clean compile"
        }
      }

    }
}

Expected behavior
When printing the RJPP_SCM_URL variable, it should contain two urls. (According to source code) and maven goal should succeed. Instead, RJPP_SCM_URL just prints one remote repo, and maven goal fails.

PrinEnv extract:
image

Maven build status:
image

Source code extracted from remote provider plugin:
image

Desktop (please complete the following information):

  • OS: Mac OS Ventura 13.4.1

  • Browser Safari 16.5.1

  • Jenkins 2.361.2

  • Plugins used:

    • GitLab Branch Source Version 650.va_d1ce6d01959
    • Remote Jenkinsfile Provider Version 1.23

Github Branch Notifications not working with Jenkins Remote

Describe the bug
A clear and concise description of what the bug is.
When using Jenkins remote, Github Branch Source notifications are stuck pending.
To Reproduce
Steps to reproduce the behavior:

  1. Create standard multibranch pipeline with Github repo
  2. Have Github Branch Source installed
  3. Set remote jenkinsfile
  4. continuous-inegration/jenkins-pr/pr-merge status stuck on Pending - Commit is scheduled to be built regardless of build status.

Expected behavior
Status on Github updates according to the status of the current build.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):
[Jenkins 2.361.4]
Remote Jenkinsfile Provider 1.23

Additional context
Add any other context about the problem here.

Expose branch where Jenkinsfile was found in environment variable

Is your feature request related to a problem? Please describe.
In my pipelines, I almost always need to check out the repository where the Jenkinsfile resides because I need some accompanying file.
Since I don't know on which branch the plugin found the Jenkinsfile, I'm always checking out the fallback branch first (because I know it exists), and then I run some git commands to know if the branch I'm building also exists on the Jenkinsfile repository and check it out if it does.

I feel like my pipelines are doing work that was already done by the plugin.

Describe the solution you'd like
If the plugin exposed the name of the branch it got the Jenkinsfile from (in something like env.RJPP_BRANCH maybe?), I would simply have to check out this branch and I would be done.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.