jenkinsci / acceptance-test-harness Goto Github PK

Acceptance tests cases for Jenkins and its plugins based on selenium and docker.

Shell 0.50% Groovy 0.46% Java 97.51% JavaScript 0.08% HTML 0.01% CSS 0.01% C# 0.02% Dockerfile 1.30% Batchfile 0.08% Io 0.02% Ruby 0.02%

hacktoberfest

acceptance-test-harness's Introduction

Acceptance tests for Jenkins

End to end test suite for Jenkins automation server and its plugins.

The scenarios are described in the form of tests controlling Jenkins under test (JUT) through UI / REST APIs. Clean instances are started for individual tests to isolate the tests. The harness provides convenient docker support so integration tests can be written easily.

Contributing to acceptance tests

Follow the contributing guidelines if you want to propose new tests for this project.

Getting Started

The simplest way to start the harness is calling BROWSER=firefox JENKINS_VERSION=2.73 mvn test. The complete test suite takes hours to run due to the number of covered components/use-cases, the cost of Jenkins setup, and selenium interactions. That can be avoided by selecting a subset of tests to be run - smoke tests for instance.

Here is a walkthrough for running ATH tests on changes made to a local version of Jenkins.

Running tests

The harness provides a variety of ways to configure the execution including:

Selecting web browser
Specifying test(s) to run
Managing the versions of Jenkins and plugins
Using a http proxy
Prelaunching Jenkins
Selecting how to launch Jenkins
Obtaining a report of plugins that were exercised
Running tests in container
Running tests on Java 11 (WIP)
Debugging tests in container
Capture a support bundle
Selecting tests based on plugins they cover (TODO)

Creating tests

Given how long it takes for the suite to run, test authors are advised to focus on the most popular plugins and use-cases to maximize the value of the test suite. Tests that can or already are written as a part of core/plugin tests should be avoided here as well as tests unlikely to catch future regressions (reproducers for individual bugs, boundary condition testing, etc.). Individual maintainers are expected to update their tests reflecting core/plugin changes as well as ensuring the tests does not produce false positives. Tests identified to violate this guideline might be removed without author's notice for the sake of suite reliability.

Selenium test in plugin repository
Docker fixtures
Page objects
- Mix-ins
Guice is our glue
Writing tests
- Video tutorial by Kohsuke on how to write tests
- Writing JUnit test
Testing agents
Hamcrest matchers
EC2 provider configuration
Investigation

Areas where acceptance-tests-harness is more suitable than jenkins-test-harness are:

Installing plugins for cross-plugin integration
Running tests in a realistic classloader environment
Verifying UI behaviour in an actual web browser

acceptance-test-harness's People

Contributors

Stargazers

Watchers

Forkers

kleopatra999 ovanes appls tobiwan88 hm-ende jenkins-acceptance-test-group itgkarl eidottermihi jlidder sschulzehm mubbashir rsandell ibratanov thors manishya christ66 olivergondza phagunbaya elig twestling trinidad8516 orjanpercy houchker msrb jglick thomaswdonnelly srve4 oleg-nenashev recena praqma tfennelly amuniz kostyasha scoheb kevinckw lordofthejars andresrc adini121 gitter-badger manishgoyaler geethsrk armfergom clara-kang jtnord recampbell projectarchives abayer vlatombe carlossg mafraba paula810 raul-arabaolaza alexsomai jcechace varmenise alvarolobato sumitnagal alexandarchan varyvol nagyistge myusufcse pjanouse vilacides batmat franckg dondyabla mypravite-project clguiman uhafner darenegade sikelerd restlessreality engelmi maxock pmwar maximilian-zollbrecht peter-mueller mreinbol wlash wlash-kmfas stefanschuhbaeck spathare21 koceluch stephenc daniel-beck kshultzcb witokondoria kuisathaverat josiahhaswell lvotypko fcojfernandez seandst v1v reeznik rtyler westerjn martinda dhivyaa21 dwnusbaum ksenia-nenasheva

acceptance-test-harness's Issues

`plugins.DashboardViewPluginTest#unstableJobsPortlet_showOnlyFailedJobs` fails on 2.415

https://github.com/jenkinsci/acceptance-test-harness/pull/1292/checks?check_run_id=15378238749

and

https://github.com/jenkinsci/acceptance-test-harness/runs/15378336134

Error
Expected: Text matching \QProject varied_insurance\E
     but: was "http://127.0.0.1:40465/view/good_battlefield/job/varied_insurance/"
"1\n2\nDashboard\ngood_battlefield\nvaried_insurance\nStatus\nChanges\nWorkspace\nBuild Now\nConfigure\nDelete Project\nRename\nBuild History\ntrend\n#1\nJul 26, 2023, 10:55 PM\nAtom feed for all\nAtom feed for failures\nvaried_insurance\nAdd description\nDisable Project\nPermalinks\nLast build (#1), 0.69 sec ago\nLast failed build (#1), 0.69 sec ago\nLast unsuccessful build (#1), 0.69 sec ago\nLast completed build (#1), 0.69 sec ago\nREST API\nJenkins 2.416"
Stacktrace
java.lang.AssertionError: 
Expected: Text matching \QProject varied_insurance\E
     but: was "http://127.0.0.1:40465/view/good_battlefield/job/varied_insurance/"
"1\n2\nDashboard\ngood_battlefield\nvaried_insurance\nStatus\nChanges\nWorkspace\nBuild Now\nConfigure\nDelete Project\nRename\nBuild History\ntrend\n#1\nJul 26, 2023, 10:55 PM\nAtom feed for all\nAtom feed for failures\nvaried_insurance\nAdd description\nDisable Project\nPermalinks\nLast build (#1), 0.69 sec ago\nLast failed build (#1), 0.69 sec ago\nLast unsuccessful build (#1), 0.69 sec ago\nLast completed build (#1), 0.69 sec ago\nREST API\nJenkins 2.416"

(fyi @yaroslavafenkin just in case related to security fix)

Version 3.0 of Matrix Authorization Strategy breaks acceptance-test-harness

Latest version (3.0) of Matrix Authorization Strategy splits from one button "Add user or group" to two buttons "Add user" and "Add group" respectively. This breaks the selector in org.jenkinsci.test.acceptance.plugins.matrix_auth.MatrixAuthorizationStrategy

Version report

Jenkins and plugins versions report: Matrix Authorization Strategy version 3, Jenkins 2.289.1 and later

Because the selector is broken tests that rely on it are now failing with a stack trace like: (snipped as it is quite a long stack trace)

*** Element info: {Using=xpath, value=.//input[./@type = 'submit' or ./@type = 'reset' or ./@type = 'image' or ./@type = 'button'][((./@id = 'Add user or group…' or ./@name = 'Add user or group…' or contains(./@value, 'Add user or group…')) or contains(./@title, 'Add user or group…'))] | .//input[./@type = 'image'][contains(./@alt, 'Add user or group…')] | .//button[(((./@id = 'Add user or group…' or contains(./@value, 'Add user or group…')) or contains(normalize-space(.), 'Add user or group…')) or contains(./@title, 'Add user or group…'))] | .//input[./@type = 'image'][contains(./@alt, 'Add user or group…')]}
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.openqa.selenium.remote.http.W3CHttpResponseCodec.createException(W3CHttpResponseCodec.java:187)
at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:122)
at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:49)
at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:158)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:552)
at org.openqa.selenium.remote.RemoteWebElement.execute(RemoteWebElement.java:285)
at org.openqa.selenium.remote.RemoteWebElement.findElement(RemoteWebElement.java:185)
at org.openqa.selenium.remote.RemoteWebElement.findElementByXPath(RemoteWebElement.java:261)
at org.openqa.selenium.By$ByXPath.findElement(By.java:353)
at org.openqa.selenium.remote.RemoteWebElement.findElement(RemoteWebElement.java:181)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.openqa.selenium.support.events.EventFiringWebDriver$EventFiringWebElement.lambda$new$0(EventFiringWebDriver.java:404)
at com.sun.proxy.$Proxy41.findElement(Unknown Source)
at org.openqa.selenium.support.events.EventFiringWebDriver$EventFiringWebElement.findElement(EventFiringWebDriver.java:496)
at org.jenkinsci.test.acceptance.plugins.matrix_auth.MatrixAuthorizationStrategy.lambda$addUser$0(MatrixAuthorizationStrategy.java:23)
at org.jenkinsci.test.acceptance.po.CapybaraPortingLayerImpl.runThenHandleAlert(CapybaraPortingLayerImpl.java:385)
at org.jenkinsci.test.acceptance.po.CapybaraPortingLayerImpl.runThenHandleAlert(CapybaraPortingLayerImpl.java:379)
at org.jenkinsci.test.acceptance.plugins.matrix_auth.MatrixAuthorizationStrategy.addUser(MatrixAuthorizationStrategy.java:23)

GitRepo does not work on windows

Jenkins and plugins versions report

GitRepo fails on windows because

posixPermissions are not supported and it throws a java.lang.UnsupportedOperationException
even if that silently worked or was swallowed, the SSH executable is using /bin/sh

What Operating System are you using (both controller, and any agents involved in the problem)?

Windows

Reproduction steps

the github template is not suitable for this project

Expected Results

the github template is not suitable for this project

Actual Results

the github template is not suitable for this project

Anything else?

No response

Plugin installing is flakey

          > Closing, the DeployPluginTest issues are preexisting.

This new functionality is flakey unfortunately, e.g. lots failed here:
https://ci.jenkins.io/blue/organizations/jenkins/Core%2Facceptance-test-harness/detail/PR-1261/6/tests

cc @janfaracik / @NotMyFault

https://ci.jenkins.io/blue/organizations/jenkins/Core%2Facceptance-test-harness/detail/PR-1261/11/tests/

Originally posted by @timja in #1274 (comment)

ATH always installs form-element-path plugin which causes failures due to detached plugins

Jenkins and plugins versions report

the ATH tests start by installing form-element-path.jpi

as this is not built with the weekly (or last LTS) this often causes detached plugins to be installed at their bundled versions.

Then when any test requires a plugin that depends on a newer version of these detached plugins the test blows up.

e.g. https://ci.jenkins.io/job/Core/job/acceptance-test-harness/job/PR-772/2/testReport/junit/plugins/OwnershipPluginTest/java_11_jenkins_latest_split7___explicitly_set_ownership/

Installing form-element-path.jpi
master00000|Running from: /home/jenkins/workspace/e_acceptance-test-harness_PR-772/jenkins-latest.war
...snip...
master00000|2022-05-18 13:56:59.254+0000 [id=48]	INFO	hudson.PluginManager#considerDetachedPlugin: Loading a detached plugin as a dependency: /home/jenkins/workspace/e_acceptance-test-harness_PR-772/target/jenkins7533342259552178954home/plugins/javax-activation-api.jpi
master00000|2022-05-18 13:56:59.284+0000 [id=48]	INFO	hudson.PluginManager#considerDetachedPlugin: Loading a detached plugin as a dependency: /home/jenkins/workspace/e_acceptance-test-harness_PR-772/target/jenkins7533342259552178954home/plugins/javax-mail-api.jpi
...snip...
master00000|2022-05-18 13:57:15.876+0000 [id=22]	INFO	hudson.PluginManager#install: Starting installation of a batch of 2 plugins plus their dependencies
master00000|2022-05-18 13:57:15.877+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of apache-httpcomponents-client-4-api for plugin ownership
master00000|2022-05-18 13:57:15.878+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of jaxb for plugin apache-httpcomponents-client-4-api
master00000|2022-05-18 13:57:15.878+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of javax-activation-api for plugin jaxb
master00000|2022-05-18 13:57:15.880+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of sshd for plugin jaxb
master00000|2022-05-18 13:57:15.880+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of jdk-tool for plugin apache-httpcomponents-client-4-api
master00000|2022-05-18 13:57:15.881+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of trilead-api for plugin jdk-tool
master00000|2022-05-18 13:57:15.881+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of command-launcher for plugin apache-httpcomponents-client-4-api
master00000|2022-05-18 13:57:15.881+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of script-security for plugin command-launcher
master00000|2022-05-18 13:57:15.881+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of caffeine-api for plugin script-security
master00000|2022-05-18 13:57:15.882+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of workflow-cps for plugin ownership
master00000|2022-05-18 13:57:15.882+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of ace-editor for plugin workflow-cps
master00000|2022-05-18 13:57:15.882+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of bouncycastle-api for plugin ace-editor
master00000|2022-05-18 13:57:15.882+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of workflow-scm-step for plugin workflow-cps
master00000|2022-05-18 13:57:15.883+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of workflow-step-api for plugin workflow-scm-step
master00000|2022-05-18 13:57:15.883+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of structs for plugin workflow-step-api
master00000|2022-05-18 13:57:15.883+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of scm-api for plugin workflow-cps
master00000|2022-05-18 13:57:15.883+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of workflow-support for plugin workflow-cps
master00000|2022-05-18 13:57:15.884+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of workflow-api for plugin workflow-support
master00000|2022-05-18 13:57:15.884+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of mailer for plugin ownership
master00000|2022-05-18 13:57:15.884+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of display-url-api for plugin mailer
master00000|2022-05-18 13:57:15.884+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of matrix-auth for plugin ownership
master00000|2022-05-18 13:57:15.885+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of matrix-project for plugin ownership
master00000|2022-05-18 13:57:15.885+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of junit for plugin matrix-project
master00000|2022-05-18 13:57:15.885+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of jackson2-api for plugin junit
master00000|2022-05-18 13:57:15.885+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of snakeyaml-api for plugin jackson2-api
master00000|2022-05-18 13:57:15.886+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of bootstrap5-api for plugin junit
master00000|2022-05-18 13:57:15.886+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of font-awesome-api for plugin bootstrap5-api
master00000|2022-05-18 13:57:15.886+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of plugin-util-api for plugin font-awesome-api
master00000|2022-05-18 13:57:15.886+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of popper2-api for plugin bootstrap5-api
master00000|2022-05-18 13:57:15.887+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of echarts-api for plugin junit
master00000|2022-05-18 13:57:15.887+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of jquery3-api for plugin echarts-api
master00000|2022-05-18 13:57:15.887+0000 [id=22]	INFO	hudson.model.UpdateSite$Plugin#deploy: Adding dependent install of checks-api for plugin junit
master00000|2022-05-18 13:57:15.931+0000 [id=114]	INFO	h.model.UpdateCenter$DownloadJob#run: Starting the installation of javax-activation-api on behalf of anonymous
master00000|2022-05-18 13:57:15.932+0000 [id=114]	INFO	h.m.UpdateCenter$UpdateCenterConfiguration#download: Downloading javax-activation-api
master00000|2022-05-18 13:57:15.940+0000 [id=114]	INFO	h.model.UpdateCenter$DownloadJob#run: Starting the installation of sshd on behalf of anonymous
master00000|2022-05-18 13:57:15.940+0000 [id=114]	INFO	h.m.UpdateCenter$UpdateCenterConfiguration#download: Downloading sshd
master00000|2022-05-18 13:57:16.037+0000 [id=114]	INFO	h.model.UpdateCenter$DownloadJob#run: Installation successful: sshd
master00000|2022-05-18 13:57:16.037+0000 [id=114]	INFO	h.model.UpdateCenter$DownloadJob#run: Starting the installation of jaxb on behalf of anonymous
master00000|2022-05-18 13:57:16.037+0000 [id=114]	INFO	h.m.UpdateCenter$UpdateCenterConfiguration#download: Downloading jaxb
master00000|2022-05-18 13:57:16.079+0000 [id=114]	SEVERE	h.model.UpdateCenter$DownloadJob#run: Failed to install jaxb
master00000|java.io.IOException: Failed to load: JAXB plugin (jaxb 2.3.6-1)
master00000| - Update required: JavaBeans Activation Framework (JAF) API (javax-activation-api 1.2.0-2) to be updated to 1.2.0-3 or higher
master00000|	at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:994)
master00000|	at hudson.PluginManager.dynamicLoad(PluginManager.java:926)
master00000|Caused: java.io.IOException: Failed to install jaxb plugin
master00000|	at hudson.PluginManager.dynamicLoad(PluginManager.java:940)
master00000|	at hudson.model.UpdateCenter$InstallationJob._run(UpdateCenter.java:2217)
master00000|Caused: java.io.IOException: Failed to dynamically deploy this plugin
master00000|	at hudson.model.UpdateCenter$InstallationJob._run(UpdateCenter.java:2221)
master00000|	at hudson.model.UpdateCenter$DownloadJob.run(UpdateCenter.java:1867)
master00000|	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
master00000|	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
master00000|	at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:121)
master00000|	at java.base/java.lang.Thread.run(Thread.java:829)

When using a LocalController we are already setting the flag to enable the form-path so we no longer need this plugin, so it can / should be skipped.

not sure what can / should happen for other Controllers

What Operating System are you using (both controller, and any agents involved in the problem)?

ci.jenkins.io - no idea if it is CentOs (stream) / Debian / Ubuntu / Amazon linux or whatever

Reproduction steps

mvn test -Dtest=OwnershipPluginTest

Expected Results

test passes

Actual Results

test barfs

Anything else?

No response

MatrixAuth plugin PO no longer works with recent matrix-auth plugin

Jenkins and plugins versions report

Environment

Paste the output here

What Operating System are you using (both controller, and any agents involved in the problem)?

N/A

Reproduction steps

run MatrixAuthPluginTest

Expected Results

tests pass

Actual Results

tests fail to add users

Anything else?

No response

Cannot make changes to the ATH image and its consumers in the same code change

As of #1168, the ATH image used for a test run is not the same as the code under test (CUT), thereby breaking the ability to make changes to the ATH image and its consumers in the same code change. This effectively nullifies testing of changes to the ATH image. Rather the ATH image should be built once from the code under test (CUT) and then consumed from each of the parallel test branches.

ATH does not startup with BROWSER=chrome-container (or firefox-container) anymore

In https://github.com/jenkinsci/acceptance-test-harness/releases/tag/5527.vf5d62a_388c39 the containers have been adapted to match the Selenium 4 upgrade (#1077).

This update does not work for my plugins, it breaks all of my tests:

plugins.OwnershipPluginTest#correct_redirect_after_save failing

Reproduction steps

Run plugins.OwnershipPluginTest#correct_redirect_after_save failing

Expected Results

Passes

Actual Results

Failing

Anything else?

Caused by jenkinsci/jenkins#7608

cc @Vlatombe

Screen recording can't be opened natively on Mac

Jenkins and plugins versions report

What Operating System are you using (both controller, and any agents involved in the problem)?

Reproduction steps

Open a ui-recording.mov from a test result
If you're on mac it will default to quicktime player, it fails to open

Expected Results

It opens

Actual Results

It doesn't

Anything else?

I'm using VLC to open them currently which works but it's the only thing I need VLC for these days...

Code is around here:

acceptance-test-harness/src/main/java/org/jenkinsci/test/acceptance/recorder/TestRecorderRule.java

Lines 72 to 87 in 200aafd

    
           String mimeType = FormatKeys.MIME_QUICKTIME; 
        
           String videoFormatName = VideoFormatKeys.ENCODING_QUICKTIME_ANIMATION; 
        
           String compressorName = VideoFormatKeys.COMPRESSOR_NAME_QUICKTIME_ANIMATION; 
        
           Dimension outputDimension = gc.getBounds().getSize(); 
        
           int bitDepth = BIT_DEPTH; 
        
           float quality = QUALITY_RATIO; 
        
           int screenRate = FRAME_RATE_PER_SEC; 
        
           Format outputFormatForScreenCapture = getOutputFormatForScreenCapture(videoFormatName, 
        
                   compressorName, outputDimension, 
        
                   bitDepth, quality, screenRate); 
        
           this.screenRecorder = new JUnitScreenRecorder 
        
                   (gc, gc.getBounds(), getFileFormat(mimeType), 
        
                           outputFormatForScreenCapture, null, null, diagnostics); 
        
           this.screenRecorder.start();

Found a blog post that's newer than our code here which may have some updated format:
https://www.ontestautomation.com/creating-a-video-capture-of-your-selenium-tests-using-monte-screen-recorder/

plugins.DiskUsagePluginTest#update_disk_usage is failing

https://ci.jenkins.io/blue/organizations/jenkins/Core%2Facceptance-test-harness/detail/PR-759/46/tests/

java.lang.AssertionError: 
Expected: disk usage reloaded
     but: Expected condition failed: Finished Calculation of builds disk usage.* to be logged (tried for 0 second(s) with 500 milliseconds interval)
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03'
System info: host: '72512dfe3a53', ip: '172.17.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1022-azure', java.version: '11.0.17'
Driver info: driver.version: unknown

Likely caused by jenkinsci/jenkins#7281

Failure in ownership tests

https://ci.jenkins.io/job/Core/job/acceptance-test-harness/job/master/570/testReport/junit/plugins/OwnershipPluginTest/java_11_jenkins_latest_split7___explicitly_set_ownership/

https://ci.jenkins.io/job/Core/job/acceptance-test-harness/job/master/570/testReport/junit/plugins/OwnershipPluginTest/java_11_jenkins_latest_split7___implicitly_set_job_ownership/

Caused by a recent merge :/

Majority of ATH tests are failing on CI

Reproduction steps

see https://ci.jenkins.io/blue/organizations/jenkins/Core%2Facceptance-test-harness/detail/master/685/tests

Expected Results

0 failing

Actual Results

368 failing

Anything else?

I ran core.FreestyleJobTest#buildParametrized locally against 2.392 in Firefox and it passed fine

plugins.SshSlavesPluginTest.newAgent fails

Named test fails with https://ci.jenkins.io/job/Core/job/acceptance-test-harness/job/master/524/testReport/junit/plugins/SshSlavesPluginTest/java_11_jenkins_latest_split4___newAgent/ on the default branch.

Additional test coverage

What feature do you want to see added?

CloudBees has maintained it's own extended set of tests for some open source plugins.

These internal tests don't benefit the community as much as they could as there is no visibility of them outside of CloudBees.

They are run on every build of our product (many times per day) without any surefire re-runs enabled and so are considered as stable as the ATHs in this repository.

We intend to open source these tests by migrating them to this repository where the tests do not significantly overlap.

This ticket is so you are aware of this upcoming work @timja @NotMyFault

Upstream changes

No response

`core.PluginManagerTest#uninstall_plugin` is flakey

I've run this ~5 times locally and haven't managed to reproduce it but it's failing quite a bit on CI atm:

https://github.com/jenkinsci/acceptance-test-harness/runs/4958889552
https://github.com/jenkinsci/acceptance-test-harness/pull/730/checks?check_run_id=4965433217

cc @jtnord if you have any ideas

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Location: .github/renovate.json
Error type: Invalid JSON (parsing failed)
Message: Syntax error: expecting end of expression or separator near {

Automate updates of LTS version

What feature do you want to see added?

#1097 introduces managing of the LTS version as configuration rather than automatically figuring it out. This allows us to get more deterministic results and review when breakages occur.

#1097 (comment) shows where needs to be automatically updated

The easiest way is likely with updatecli, see:
https://github.com/jenkins-infra/jenkins-infra/blob/production/updatecli/weekly.d/jenkins-lts.yaml

and could be run with a GitHub action, see https://github.com/jenkinsci/bom/blob/master/.github/workflows/action-updatecli.yaml

Upstream changes

No response

`FormValidation` API should allow asserting for text in expected location rather than attempting to retrieve the validation area

What feature do you want to see added?

The FormValidation API attempts to select the text in the validation area.

acceptance-test-harness/src/main/java/org/jenkinsci/test/acceptance/po/FormValidation.java

Line 67 in 2524148

public static FormValidation await(Control control, boolean silent) {

But nothing in the API says whether it expects content to be there.

If the JavaScript hasn't fired by the time the selenium code executes then it can sometimes fail to find the content causing flakey tests, one such example was worked around in 650531a

My suggested fix would be either:
a) Matchers query element directly for expected text, that way they can retry if text is not what is expected
b) minor change to API to say whether validation content is expected or not, that way it can retry if getText() doesn't return what it should

Upstream changes

No response

Dependabot uses wrong release notes

The release notes are somehow broken. When Dependabot updates ATH in my project, then it shows the release notes of something unrelated. Is this a bug in dependabot or a bug here?

Example: jenkinsci/warnings-ng-plugin#1470

Adapt scriptler changes

The scriptler tests need to be updated to adapt changes made in jenkinsci/scriptler-plugin#59

One change needed has already been pushed in #1237, but I lack the time to update all test UI changes.

`plugins.JobDslPluginTest#should_create_job` failed on 2.415

org.openqa.selenium.NoSuchElementException: Unable to locate By.xpath: //*[@id="main-panel"]/h1 in http://127.0.0.1:44119/job/MyJob/
	at org.jenkinsci.test.acceptance.po.CapybaraPortingLayerImpl.find(CapybaraPortingLayerImpl.java:197)
	at org.jenkinsci.test.acceptance.po.TopLevelItem.getDisplayName(TopLevelItem.java:102)
	at plugins.JobDslPluginTest.should_create_job(JobDslPluginTest.java:800)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
	at org.jenkinsci.test.acceptance.junit.WithPlugins$RuleImpl$1.evaluate(WithPlugins.java:160)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
	at org.jenkinsci.test.acceptance.junit.JenkinsAcceptanceTestRule$1$1.evaluate(JenkinsAcceptanceTestRule.java:153)
	at org.jenkinsci.test.acceptance.junit.FilterRule$1.evaluate(FilterRule.java:61)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
	at org.jenkinsci.test.acceptance.junit.JenkinsAcceptanceTestRule$1.evaluate(JenkinsAcceptanceTestRule.java:53)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:316)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:240)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:214)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:155)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495)
Caused by: org.openqa.selenium.TimeoutException: Expected condition failed: waiting for Wait for the element (By.xpath: //*[@id="main-panel"]/h1) to become visible (tried for 1 second(s) with 500 milliseconds interval)
Build info: version: '4.10.0', revision: 'c14d967899'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1042-azure', java.version: '17.0.7'
Driver info: driver.version: unknown
	at org.openqa.selenium.support.ui.FluentWait.timeoutException(FluentWait.java:262)
	at org.jenkinsci.test.acceptance.junit.Wait.timeoutException(Wait.java:188)
	at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:230)
	at org.jenkinsci.test.acceptance.junit.Wait.until(Wait.java:119)
	at org.jenkinsci.test.acceptance.po.CapybaraPortingLayerImpl.find(CapybaraPortingLayerImpl.java:181)
	... 40 more

SECURITY-1468's solution impacts ATH builds

After investigating into the failure of #876, ATH is impacted by the security fix in the git-client-plugin too, jenkinsci/git-client-plugin#875, because ATH runs on CentOS 7 and doesn't know how to deal with "accept new" as strategy.

Later reading over JENKINS-69149, am I correct assuming that jenkinsci/git-client-plugin#882 restores the functionality of ATH, once merged and released?

Don't build ATH image on every split

It takes ~3.5 minutes to build and has flaked at least once.

It would be good to build it once and run a smoke test on it and then use a published image for the rest, or something like that.
Or even build and publish and then use the published version.

`plugins.WorkflowPluginTest#parallelTests` fails on Java 17

When running this test after calling set-java.sh 17:

[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] Source option 6 is no longer supported. Use 7 or later.
[ERROR] Target option 6 is no longer supported. Use 7 or later.
[INFO] 2 errors 
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.017 s
[INFO] Finished at: 2023-04-25T20:51:10Z
[INFO] ------------------------------------------------------------------------

WorkflowPluginTest#linearFlow is flakey

In:

https://ci.jenkins.io/blue/organizations/jenkins/Core%2Facceptance-test-harness/detail/PR-726/1/pipeline

master60361|2021-12-24 09:59:11.449+0000 [id=82]	INFO	hudson.slaves.CommandLauncher#launch: agent launched for organic_salad
master60361|2021-12-24 09:59:12.386+0000 [id=115]	WARNING	o.e.j.s.h.ContextHandler$Context#log: Error while serving http://127.0.0.1:60361/job/digital_comfort/1/input/30df4afa64a9373296bec35feb317cee/proceedEmpty
master60361|java.lang.NullPointerException
master60361|	at org.jenkinsci.plugins.workflow.support.steps.input.InputStepExecution.canSettle(InputStepExecution.java:305)
master60361|	at org.jenkinsci.plugins.workflow.support.steps.input.InputStepExecution.canSubmit(InputStepExecution.java:296)
master60361|	at org.jenkinsci.plugins.workflow.support.steps.input.InputStepExecution.preSubmissionCheck(InputStepExecution.java:262)
master60361|	at org.jenkinsci.plugins.workflow.support.steps.input.InputStepExecution.doProceedEmpty(InputStepExecution.java:219)
master60361|	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
master60361|	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:398)
master60361|Caused: java.lang.reflect.InvocationTargetException
master60361|	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:402)
master60361|	at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:410)
master60361|	at org.kohsuke.stapler.interceptor.RequirePOST$Processor.invoke(RequirePOST.java:78)
master60361|	at org.kohsuke.stapler.PreInvokeInterceptedFunction.invoke(PreInvokeInterceptedFunction.java:26)
master60361|	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:208)
master60361|	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:141)
master60361|	at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:558)
master60361|	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:59)
master60361|	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766)
master60361|	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898)
master60361|	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:475)
master60361|	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766)
master60361|	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898)
master60361|	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:475)
master60361|	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766)
master60361|	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898)
master60361|	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:475)
master60361|	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766)

cc @car-roll @jglick

JobConfigHistoryPluginTest is flaky

Jenkins and plugins versions report

Latest version of jenkins weekly as used by the pr builder

What Operating System are you using (both controller, and any agents involved in the problem)?

Whatever is used on ci.jenkins.io

Reproduction steps

JobConfigHistoryPluginTest is flaky, on several runs of #1037 it has been randomly failing or working without clear errors due to changes absolutely unrelated to the test itself. Like fixing the SVN docker container to work with ubuntu instead of centos.

Flaky tests are a pain for two reasons:

It decreases the confidence in PR builders failing by default
It reduces the possibility of a proper pr be automerged

Expected Results

The test works consistently

Actual Results

It fails randomly

Anything else?

No response

MockUpdateCenter regularly has corrupted plugin downloads

Jenkins and plugins versions report

Environment

Paste the output here

What Operating System are you using (both controller, and any agents involved in the problem)?

Windows 11

Reproduction steps

with a m2 repository with several required plugins already available (from regular maven builds pulling dependencies) installed.

mvn -P lts test

Expected Results

tests install the required plugins for the test as determined by @WithPlugins or other code

Actual Results

tests fails, resolved HPIs are corrupted.

master1740766363|2023-07-26 19:51:58.975+0000 [id=136]  INFO    h.m.UpdateCenter$UpdateCenterConfiguration#download: Downloading jackson2-api
master1740766363|2023-07-26 19:51:59.067+0000 [id=136]  SEVERE  h.model.UpdateCenter$DownloadJob#run: Failed to install jackson2-api
master1740766363|java.io.IOException: Downloaded file C:\workarea\source\github\jenkinsci\acceptance-test-harness\target\jenkinshome8417413392245346575\plugins\jackson2-api.jpi.tmp does not match expected SHA-256, expected 'QceK1GpM4/xAs1O/TNQlqhjSpNv+gmYkxig05LGf2q0=', actual 'yMow6XWcRzp7+NIfKfPykb4AMPGDv9FzrbBrjFVbbYI='
master1740766363|       at hudson.model.UpdateCenter.throwVerificationFailure(UpdateCenter.java:2042)
master1740766363|       at hudson.model.UpdateCenter.verifyChecksums(UpdateCenter.java:2077)
master1740766363|       at hudson.model.UpdateCenter$InstallationJob.replace(UpdateCenter.java:2275)
master1740766363|       at hudson.model.UpdateCenter$UpdateCenterConfiguration.install(UpdateCenter.java:1341)
master1740766363|       at hudson.model.UpdateCenter$DownloadJob._run(UpdateCenter.java:1879)
master1740766363|       at hudson.model.UpdateCenter$InstallationJob._run(UpdateCenter.java:2188)
master1740766363|       at hudson.model.UpdateCenter$DownloadJob.run(UpdateCenter.java:1850)
master1740766363|       at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
master1740766363|       at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
master1740766363|       at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:121)
master1740766363|       at java.base/java.lang.Thread.run(Thread.java:829)
master1740766363|2023-07-26 19:51:59.067+0000 [id=136]  INFO    h.model.UpdateCenter$DownloadJob#run: Starting the installation of commons-lang3-api on behalf of anonymous

Anything else?

I verified that the plugin in c/Users/jnord/.m2/repository/org/jenkins-ci/plugins/jackson2-api/2.15.2-350.v0c2f3f8fc595 has an incorrect hash according to https://updates.jenkins.io/download/plugins/jackson2-api/#2.15.2-350.v0c2f3f8fc595

removing plugins and reruning the tests move the tests forward (there are generally several plugins that become corrupted).

Given that the tests pass in CI and on many other users machines I am guessing that there is some buggy file handling code that is triggered on windows.

replace `dumpling`

What feature do you want to see added?

replace dumpling with a trivial call to jstack -l

dumpling is causing errors when trying to obtain the stack trace of a Jenkins under test.
We do not use any feature of dumpling other than obtaining a stack trace so its parsing of the stack is immaterial to us - which we can trivially replace with a invocation of jstack -l <thePid>

org.opentest4j.MultipleFailuresError: 
Multiple Failures (2 failures)
	org.openqa.selenium.TimeoutException: Expected condition failed: waiting for org.jenkinsci.test.acceptance.po.PluginManager$$Lambda$835/0x00000008405b3840@20b54cfe (tried for 10 second(s) with 500 milliseconds interval)
Build info: version: '4.9.1', revision: 'eb2032df7f'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.13.0-1021-aws', java.version: '11.0.18'
Driver info: driver.version: unknown
	com.github.olivergondza.dumpling.factory.IllegalRuntimeStateException: IN_OBJECT_WAIT thread declares they are waiting to acquire same lock:
"Finalizer" #3 daemon prio=8 os_prio=0 cpu=2.04ms elapsed=44.48s tid=0x00007f47f02c1000 nid=0x23f in Object.wait()  [0x00007f47c43e0000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait([email protected]/Native Method)
	- waiting on <no object reference available>
	at java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:155)
	- waiting to re-lock in wait() <0x000000008aa01758> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:176)
	at java.lang.ref.Finalizer$FinalizerThread.run([email protected]/Finalizer.java:170)

   Locked ownable synchronizers:
	- None

	at org.junit.vintage.engine.execution.TestRun.getStoredResultOrSuccessful(TestRun.java:200)
	at org.junit.vintage.engine.execution.RunListenerAdapter.fireExecutionFinished(RunListenerAdapter.java:248)
	at org.junit.vintage.engine.execution.RunListenerAdapter.testFinished(RunListenerAdapter.java:214)
	at org.junit.vintage.engine.execution.RunListenerAdapter.testFinished(RunListenerAdapter.java:88)
	at org.junit.runner.notification.SynchronizedRunListener.testFinished(SynchronizedRunListener.java:87)
	at org.junit.runner.notification.RunNotifier$9.notifyListener(RunNotifier.java:225)
	at org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72)
	at org.junit.runner.notification.RunNotifier.fireTestFinished(RunNotifier.java:222)
	at org.junit.internal.runners.model.EachTestNotifier.fireTestFinished(EachTestNotifier.java:38)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:372)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42)
	at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80)
	at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:147)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:127)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:90)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:55)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:102)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:54)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
	at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
	at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:50)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495)
	Suppressed: org.openqa.selenium.TimeoutException: Expected condition failed: waiting for org.jenkinsci.test.acceptance.po.PluginManager$$Lambda$835/0x00000008405b3840@20b54cfe (tried for 10 second(s) with 500 milliseconds interval)
Build info: version: '4.9.1', revision: 'eb2032df7f'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.13.0-1021-aws', java.version: '11.0.18'
Driver info: driver.version: unknown
		at org.openqa.selenium.support.ui.FluentWait.timeoutException(FluentWait.java:263)
		at org.jenkinsci.test.acceptance.junit.Wait.timeoutException(Wait.java:190)
		at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:231)
		at org.jenkinsci.test.acceptance.junit.Wait.until(Wait.java:121)
		at org.jenkinsci.test.acceptance.po.PluginManager.tickPluginToInstall(PluginManager.java:273)
		at org.jenkinsci.test.acceptance.po.PluginManager.installPlugins(PluginManager.java:230)
		at org.jenkinsci.test.acceptance.junit.WithPlugins$RuleImpl$1.installPlugins(WithPlugins.java:195)
		at org.jenkinsci.test.acceptance.junit.WithPlugins$RuleImpl$1.evaluate(WithPlugins.java:142)
		at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
		at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
		at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
		at org.jenkinsci.test.acceptance.junit.JenkinsAcceptanceTestRule$1$1.evaluate(JenkinsAcceptanceTestRule.java:155)
		at org.jenkinsci.test.acceptance.junit.FilterRule$1.evaluate(FilterRule.java:63)
		at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
		at org.jenkinsci.test.acceptance.junit.JenkinsAcceptanceTestRule$1.evaluate(JenkinsAcceptanceTestRule.java:55)
		at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
		at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
		at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
		... 31 more
	Suppressed: com.github.olivergondza.dumpling.factory.IllegalRuntimeStateException: IN_OBJECT_WAIT thread declares they are waiting to acquire same lock:
"Finalizer" #3 daemon prio=8 os_prio=0 cpu=2.04ms elapsed=44.48s tid=0x00007f47f02c1000 nid=0x23f in Object.wait()  [0x00007f47c43e0000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait([email protected]/Native Method)
	- waiting on <no object reference available>
	at java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:155)
	- waiting to re-lock in wait() <0x000000008aa01758> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:176)
	at java.lang.ref.Finalizer$FinalizerThread.run([email protected]/Finalizer.java:170)

   Locked ownable synchronizers:
	- None

		at com.github.olivergondza.dumpling.factory.ThreadDumpFactory.initStacktrace(ThreadDumpFactory.java:341)
		at com.github.olivergondza.dumpling.factory.ThreadDumpFactory.thread(ThreadDumpFactory.java:184)
		at com.github.olivergondza.dumpling.factory.ThreadDumpFactory.fromStream(ThreadDumpFactory.java:119)
		at com.github.olivergondza.dumpling.factory.PidRuntimeFactory.createRuntime(PidRuntimeFactory.java:188)
		at com.github.olivergondza.dumpling.factory.PidRuntimeFactory.fromProcess(PidRuntimeFactory.java:85)
		at org.jenkinsci.test.acceptance.controller.LocalController.getThreaddump(LocalController.java:334)
		at org.jenkinsci.test.acceptance.controller.LocalController.diagnose(LocalController.java:228)
		at org.jenkinsci.test.acceptance.junit.DiagnosticRule.failed(DiagnosticRule.java:37)
		at org.junit.rules.TestWatcher.failedQuietly(TestWatcher.java:90)
		at org.junit.rules.TestWatcher.access$300(TestWatcher.java:52)
		at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:68)
		at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
		at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
		at org.jenkinsci.test.acceptance.junit.JenkinsAcceptanceTestRule$1$1.evaluate(JenkinsAcceptanceTestRule.java:155)
		at org.jenkinsci.test.acceptance.junit.FilterRule$1.evaluate(FilterRule.java:63)
		at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
		at org.jenkinsci.test.acceptance.junit.JenkinsAcceptanceTestRule$1.evaluate(JenkinsAcceptanceTestRule.java:55)
		at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
		at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
		at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
		... 31 more

Upstream changes

olivergondza/dumpling#93

wizardInstallSuggestedTest fails if ATH runs several times

Originally observed in #794, core.InstallWizardTest.wizardInstallSuggestedTest fails if a job runs several times:
References: https://ci.jenkins.io/job/Core/job/acceptance-test-harness/job/PR-794/3/testReport/junit/core/InstallWizardTest/java_11_jenkins_latest_split4___wizardInstallSuggestedTest/

Conditional build: do we need to rebuild the full ATH for all Renovabot / Dependabot PRs?

As per jenkins-infra/helpdesk#3551 (comment), the Infra team would like to control the costs on ci.jenkins.io.

We saw a lot of builds (using a LOT of highmem VM agents) for PR changing Dockerfile dependencies such as #1164 or #1149.

At first sight, it seems a waste to execute the whole ATH for such small changes. Is there any means today to conditionnaly build?

Also, it does not look a good idea to run the ATH on ci.jenkins.io if a PR changes the file src/main/resources/ath-container/Dockerfile: it's built by infra.ci and pushed to the DockerHub.

HAR recording on ci.jenkins.io does not record interaction with Jenkins

Jenkins and plugins versions report

Environment

Paste the output here

What Operating System are you using (both controller, and any agents involved in the problem)?

https://ci.jenkins.io/job/Core/job/acceptance-test-harness/job/PR-1159/3/testReport/junit/plugins/WorkflowPluginTest/lts_linux_jdk11_firefox_split2___parallelTests/

Reproduction steps

have a flaky / failing test
on the build in ci.jenkins.io open the test report and download the HAR for the test
open the HAR

Expected Results

the HAR contains all interactions with the system under test

Actual Results

there are no interactions recorded with Jenkins

Anything else?

Jenkins is running on 127.0.0.1 and as such by default firefox will not use the proxy - so the HAR is useless

WorkflowPluginTest#testSharedLibraryFromGithub() is flaky

Jenkins and plugins versions report

WorkflowPluginTest#testSharedLibraryFromGithub() is flaky.

This is due to the library being downloaded using unaithenticated APIs which have a rate limit imposed, (per IP address) and this can cause the retreival of the libary to stall.

e.g.

Expected condition failed: Build unexpected_roof #1 is finished (tried for 120 second(s) with 500 milliseconds interval). Console output:
Started by user unknown or anonymous
Loading library Greeting@master
Jenkins-Imposed API Limiter: Current quota for Github API usage has 48 remaining (1 over budget). Next quota of 60 in 53 min. Sleeping for 4 min 53 sec.
Jenkins is attempting to evenly distribute GitHub API requests. To configure a different rate limiting strategy, such as having Jenkins restrict GitHub API requests only when near or above the GitHub rate limit, go to "GitHub API usage" under "Configure System" in the Jenkins settings.
Jenkins-Imposed API Limiter: Still sleeping, now only 1 min 51 sec remaining.
Jenkins-Imposed API Limiter: Current quota for Github API usage has 42 remaining (4 over budget). Next quota of 60 in 48 min. Sleeping for 9 min 14 sec.
Jenkins is attempting to evenly distribute GitHub API requests. To configure a different rate limiting strategy, such as having Jenkins restrict GitHub API requests only when near or above the GitHub rate limit, go to "GitHub API usage" under "Configure System" in the Jenkins settings.

The build then times out due to the sleeping...

org.openqa.selenium.TimeoutException: 
Expected condition failed: Build unexpected_roof #1 is finished (tried for 120 second(s) with 500 milliseconds interval). Console output:

What Operating System are you using (both controller, and any agents involved in the problem)?

N/A

Reproduction steps

while mvn test WorkflowPluginTest#testSharedLibraryFromGithub; do :; done

Expected Results

it reliably passes

Actual Results

it will fail after a while

Anything else?

No response

PluginManager.install(File) returns before the plugin has installed

Jenkins and plugins versions report

Environment

Jenkins 2.346.1

What Operating System are you using (both controller, and any agents involved in the problem)?

N/A but windows

Reproduction steps

if a test calls PluginManager.installPlugin(File) the method returns before the plugin has been installed.

in a test have the following (not using the mock update center)

final File plugin = new File(getClass().getResource("foobar-1.0.hpi").toURI());
jenkins.getPluginManager().installPlugin(plugin);
jenkins.restart()
assertThat(/* the plugin is installed */)

Expected Results

the assertion will always pass

Actual Results

on my laptop it reliably fails. the plugin has been uploaded but some of the background tasks to install it have not occured - which causes it to be "lost" after the restart.

Anything else?

The method does not use selenium either so actually diagnosing what was going on was not so easy.
Given #780 we should use selenium and wait for the install to complete (regardless of if that completion requires a restart)

`plugins.ExternalWorkspaceManagerPluginTest` failing

Legitimate issue https://issues.jenkins-ci.org/browse/JENKINS-70693

Fixed in jenkinsci/external-workspace-manager-plugin#77

Needs review and release.

This is a tracking issue for ATH to remove any ignored tests if #1043 is merged before the plugin is fixed

`plugins.GroovyPluginTest#run_groovy` and `plugins.GroovyPluginTest#use_custom_groovy_version` fail on Java 17

When running these tests after calling set-java.sh 17:

java.lang.reflect.InaccessibleObjectException: Unable to make protected void java.lang.Object.finalize() throws java.lang.Throwable accessible: module java.base does not "opens java.lang" to unnamed module @763d9750
 at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
 at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
 at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:199)
 at java.base/java.lang.reflect.AccessibleObject.setAccessible(AccessibleObject.java:130)
 at org.codehaus.groovy.reflection.CachedClass$3$1.run(CachedClass.java:86)
 at java.base/java.security.AccessController.doPrivileged(AccessController.java:318)
 at org.codehaus.groovy.reflection.CachedClass$3.initValue(CachedClass.java:81)
 at org.codehaus.groovy.reflection.CachedClass$3.initValue(CachedClass.java:79)
 at org.codehaus.groovy.util.LazyReference.getLocked(LazyReference.java:46)
 at org.codehaus.groovy.util.LazyReference.get(LazyReference.java:33)
 at org.codehaus.groovy.reflection.CachedClass.getMethods(CachedClass.java:250)
 at org.codehaus.groovy.reflection.CachedClass$3.initValue(CachedClass.java:116)
 at org.codehaus.groovy.reflection.CachedClass$3.initValue(CachedClass.java:79)
 at org.codehaus.groovy.util.LazyReference.getLocked(LazyReference.java:46)
 at org.codehaus.groovy.util.LazyReference.get(LazyReference.java:33)
 at org.codehaus.groovy.reflection.CachedClass.getMethods(CachedClass.java:250)
 at org.codehaus.groovy.reflection.CachedClass$3.initValue(CachedClass.java:116)
 at org.codehaus.groovy.reflection.CachedClass$3.initValue(CachedClass.java:79)
 at org.codehaus.groovy.util.LazyReference.getLocked(LazyReference.java:46)
 at org.codehaus.groovy.util.LazyReference.get(LazyReference.java:33)
 at org.codehaus.groovy.reflection.CachedClass.getMethods(CachedClass.java:250)
 at org.codehaus.groovy.reflection.stdclasses.CachedClosureClass.<init>(CachedClosureClass.java:29)
 at org.codehaus.groovy.reflection.ClassInfo.createCachedClass(ClassInfo.java:269)
 at org.codehaus.groovy.reflection.ClassInfo.access$400(ClassInfo.java:36)
 at org.codehaus.groovy.reflection.ClassInfo$LazyCachedClassRef.initValue(ClassInfo.java:441)
 at org.codehaus.groovy.reflection.ClassInfo$LazyCachedClassRef.initValue(ClassInfo.java:432)
 at org.codehaus.groovy.util.LazyReference.getLocked(LazyReference.java:46)
 at org.codehaus.groovy.util.LazyReference.get(LazyReference.java:33)
 at org.codehaus.groovy.reflection.ClassInfo.getCachedClass(ClassInfo.java:89)
 at org.codehaus.groovy.reflection.ReflectionCache.getCachedClass(ReflectionCache.java:107)
 at org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.registerMethods(MetaClassRegistryImpl.java:189)
 at org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.<init>(MetaClassRegistryImpl.java:93)
 at org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.<init>(MetaClassRegistryImpl.java:71)
 at groovy.lang.GroovySystem.<clinit>(GroovySystem.java:33)
 at org.codehaus.groovy.runtime.InvokerHelper.<clinit>(InvokerHelper.java:61)
 at groovy.lang.GroovyObjectSupport.<init>(GroovyObjectSupport.java:32)
 at groovy.lang.Binding.<init>(Binding.java:33)
 at groovy.lang.GroovyShell.<init>(GroovyShell.java:67)
 at groovy.ui.GroovyMain.processOnce(GroovyMain.java:582)
 at groovy.ui.GroovyMain.run(GroovyMain.java:375)
 at groovy.ui.GroovyMain.process(GroovyMain.java:361)
 at groovy.ui.GroovyMain.processArgs(GroovyMain.java:120)
 at groovy.ui.GroovyMain.main(GroovyMain.java:100)
 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.base/java.lang.reflect.Method.invoke(Method.java:568)
 at org.codehaus.groovy.tools.GroovyStarter.rootLoader(GroovyStarter.java:106)
 at org.codehaus.groovy.tools.GroovyStarter.main(GroovyStarter.java:128)
Caught: java.lang.ExceptionInInitializerError
java.lang.ExceptionInInitializerError
 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make protected void java.lang.Object.finalize() throws java.lang.Throwable accessible: module java.base does not "opens java.lang" to unnamed module @763d9750
 ... 3 more

I suggest updating src/main/tool_installers/updates/hudson.plugins.groovy.GroovyInstaller with the latest crawler metadata and then applying the following patch:

diff --git a/src/main/java/org/jenkinsci/test/acceptance/plugins/groovy/GroovyInstallation.java b/src/main/java/org/jenkinsci/test/acceptance/plugins/groovy/GroovyInstallation.java
index 6b4893ff..ae17f6d0 100644
--- a/src/main/java/org/jenkinsci/test/acceptance/plugins/groovy/GroovyInstallation.java
+++ b/src/main/java/org/jenkinsci/test/acceptance/plugins/groovy/GroovyInstallation.java
@@ -33,7 +33,7 @@ public class GroovyInstallation extends ToolInstallation {
     public static final String DEFAULT_GROOVY_ID = "default_Groovy";
 
     public static void installSomeGroovy(Jenkins jenkins) {
-        installGroovy(jenkins, DEFAULT_GROOVY_ID, "2.2.1");
+        installGroovy(jenkins, DEFAULT_GROOVY_ID, "4.0.11");
     }
 
     public static void installGroovy(Jenkins jenkins, String name, String version) {
diff --git a/src/test/java/plugins/GroovyPluginTest.java b/src/test/java/plugins/GroovyPluginTest.java
index f7d5aac9..62bb3406 100644
--- a/src/test/java/plugins/GroovyPluginTest.java
+++ b/src/test/java/plugins/GroovyPluginTest.java
@@ -99,17 +99,17 @@ public class GroovyPluginTest extends AbstractJUnitTest {
 
     @Test
     public void use_custom_groovy_version() {
-        GroovyInstallation.installGroovy(jenkins, "groovy-2.2.1", "Groovy 2.2.1");
+        GroovyInstallation.installGroovy(jenkins, "groovy-4.0.11", "Groovy 4.0.11");
 
         configureJob();
 
         final GroovyStep step = job.addBuildStep(GroovyStep.class);
-        step.version.select("groovy-2.2.1");
+        step.version.select("groovy-4.0.11");
         step.script(
                 "println 'version: ' + groovy.lang.GroovySystem.getVersion()"
         );
         job.save();
-        shouldReport("version: 2.2.1");
+        shouldReport("version: 4.0.11");
     }
 
     @Test @Native("groovy")

`plugins.AntPluginTest#testAdvancedConfiguration` fails on Java 17

When running this test after calling set-java.sh 17:

java.lang.AssertionError: 

Expected: a string containing "17.0.6"
     but: was "[…]11.0.18[…]"
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:6)
	at plugins.AntPluginTest.testAdvancedConfiguration(AntPluginTest.java:155)

Use Jenkins project mirrors / caches

What feature do you want to see added?

Ideally we should utilise maven caches if available (without forcing people to use them who use the framework)

This build failed:
https://ci.jenkins.io/blue/organizations/jenkins/Core%2Facceptance-test-harness/detail/master/601/pipeline/

Because of:

[2022-10-02T22:45:56.251Z] [ERROR] Error executing Maven.
[2022-10-02T22:45:56.251Z] [ERROR] Extension io.jenkins.tools.incrementals:git-changelist-maven-extension:1.4 or one of its dependencies could not be resolved: Could not transfer artifact org.apache.maven.resolver:maven-resolver-util:jar:1.0.3 from/to central (https://repo.maven.apache.org/maven2): transfer failed for https://repo.maven.apache.org/maven2/org/apache/maven/resolver/maven-resolver-util/1.0.3/maven-resolver-util-1.0.3.jar
[2022-10-02T22:45:56.251Z] [ERROR] Caused by: Could not transfer artifact org.apache.maven.resolver:maven-resolver-util:jar:1.0.3 from/to central (https://repo.maven.apache.org/maven2): transfer failed for https://repo.maven.apache.org/maven2/org/apache/maven/resolver/maven-resolver-util/1.0.3/maven-resolver-util-1.0.3.jar
[2022-10-02T22:45:56.251Z] [ERROR] Caused by: Could not transfer artifact org.apache.maven.resolver:maven-resolver-util:jar:1.0.3 from/to central (https://repo.maven.apache.org/maven2): transfer failed for https://repo.maven.apache.org/maven2/org/apache/maven/resolver/maven-resolver-util/1.0.3/maven-resolver-util-1.0.3.jar
[2022-10-02T22:45:56.251Z] [ERROR] Caused by: transfer failed for https://repo.maven.apache.org/maven2/org/apache/maven/resolver/maven-resolver-util/1.0.3/maven-resolver-util-1.0.3.jar
[2022-10-02T22:45:56.251Z] [ERROR] Caused by: Connect to repo.maven.apache.org:443 [repo.maven.apache.org/146.75.32.215] failed: Connection timed out (Connection timed out)
[2022-10-02T22:45:56.251Z] [ERROR] Caused by: Connection timed out (Connection timed out)

Upstream changes

No response

replace dependabot with renovate

What feature do you want to see added?

Currently dependencies are managed partially by dependabot and partially by renovate with some overlap.

This is sub optimal as we can end up with duplicate PRs, or missed dependencies, and keeping config in sync/ optimal is cumbersome.

As renovate is more powerfull than dependabot we should switch to renovate.

#1195 (review)

Upstream changes

No response

Support running on Java 17

What feature do you want to see added?

Java 17 support

Upstream changes

I was trying to upgrade the media library to possibly fix #1041
But the new version on central only works with Java 17.

Trying to upgrade to Java 17 (just to see if it works, not to require 17 in this project yet) hits:

java.lang.IllegalAccessError: class com.cloudbees.sdk.extensibility.AnnotationLiteral (in unnamed module @0x33833882) cannot access class sun.reflect.annotation.AnnotationParser (in module java.base) because module java.base does not export sun.reflect.annotation to unnamed module @0x33833882

	at com.cloudbees.sdk.extensibility.AnnotationLiteral.of(AnnotationLiteral.java:49)
	at com.cloudbees.sdk.extensibility.AnnotationLiteral.of(AnnotationLiteral.java:45)
	at com.cloudbees.sdk.extensibility.AnnotationLiteral.of(AnnotationLiteral.java:41)
	at com.cloudbees.sdk.extensibility.ExtensionLoaderModule$Default.configure(ExtensionLoaderModule.java:86)
	at com.google.inject.AbstractModule.configure(AbstractModule.java:66)
	at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:409)
	at com.google.inject.AbstractModule.install(AbstractModule.java:108)
	at com.cloudbees.sdk.extensibility.ExtensionFinder.bind(ExtensionFinder.java:76)
	at com.cloudbees.sdk.extensibility.ExtensionFinder.configure(ExtensionFinder.java:60)
	at com.google.inject.AbstractModule.configure(AbstractModule.java:66)
	at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:409)
	at com.google.inject.spi.Elements.getElements(Elements.java:108)
	at com.google.inject.util.Modules$OverrideModule.configure(Modules.java:236)
	at com.google.inject.AbstractModule.configure(AbstractModule.java:66)
	at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:409)
	at com.google.inject.spi.Elements.getElements(Elements.java:108)
	at com.google.inject.util.Modules$OverrideModule.configure(Modules.java:213)
	at com.google.inject.AbstractModule.configure(AbstractModule.java:66)
	at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:409)
	at com.google.inject.AbstractModule.install(AbstractModule.java:108)
	at org.jenkinsci.test.acceptance.guice.World.configure(World.java:86)
	at com.google.inject.AbstractModule.configure(AbstractModule.java:66)
	at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:409)
	at com.google.inject.spi.Elements.getElements(Elements.java:108)
	at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:160)
	at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:107)
	at com.google.inject.Guice.createInjector(Guice.java:87)
	at com.google.inject.Guice.createInjector(Guice.java:69)
	at com.google.inject.Guice.createInjector(Guice.java:59)
	at org.jenkinsci.test.acceptance.guice.World.getInjector(World.java:45)
	at org.jenkinsci.test.acceptance.junit.JenkinsAcceptanceTestRule$1.evaluate(JenkinsAcceptanceTestRule.java:49)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)

Which appears to be an archived CloudBees project:
https://github.com/cloudbees/extensibility-api

`plugins.JacocoPluginTest#checkSuccessfulExecutionAndsummary` fails on Java 17

When running this test after calling set-java.sh 17:

Exception in thread "main" java.lang.reflect.InvocationTargetException
 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.base/java.lang.reflect.Method.invoke(Method.java:568)
 at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:491)
 at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:503)
Caused by: java.lang.RuntimeException: Class java/lang/UnknownError could not be instrumented.
 at org.jacoco.agent.rt.internal_28bab1d.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:140)
 at org.jacoco.agent.rt.internal_28bab1d.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:101)
 at org.jacoco.agent.rt.internal_28bab1d.PreMain.createRuntime(PreMain.java:55)
 at org.jacoco.agent.rt.internal_28bab1d.PreMain.premain(PreMain.java:47)
 ... 6 more
Caused by: java.lang.NoSuchFieldException: $jacocoAccess
 at java.base/java.lang.Class.getField(Class.java:2117)
 at org.jacoco.agent.rt.internal_28bab1d.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:138)
 ... 9 more
*** java.lang.instrument ASSERTION FAILED ***: "result" with message agent load/premain call failed at ./src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 422
FATAL ERROR in native method: processing of -javaagent failed, processJavaStart failed
Aborted (core dumped)

I suggest the following patch:

diff --git a/src/test/java/plugins/JacocoPluginTest.java b/src/test/java/plugins/JacocoPluginTest.java
index fe2a69de..5666dd71 100644
--- a/src/test/java/plugins/JacocoPluginTest.java
+++ b/src/test/java/plugins/JacocoPluginTest.java
@@ -31,7 +31,6 @@ public class JacocoPluginTest extends AbstractJUnitTest {
 
             MavenBuildStep maven = job.addBuildStep(MavenBuildStep.class);
             maven.targets.set("clean package -B");
-            maven.properties("jacoco.version=" + getJacocoLibraryVersion(), true);
 
             JacocoPublisher publisher = job.addPublisher(JacocoPublisher.class);
             publisher.changeBuildStatus.check();
@@ -57,17 +56,4 @@ public class JacocoPluginTest extends AbstractJUnitTest {
         assertEquals(50.0, pkg.methodCoverage(), 0.01);
         assertEquals(100.0, pkg.classCoverage(), 0.01);
     }
-
-    // https://wiki.jenkins-ci.org/display/JENKINS/JaCoCo+Plugin
-    // Unfortunately JaCoCo 0.7.5 breaks compatibility to previous binary formats of the jacoco.exec files. The JaCoCo plugin up to version
-    // 1.0.19 is based on JaCoCo 0.7.4, thus you cannot use this version with projects which already use JaCoCo 0.7.5 or newer. JaCoCo plugin
-    // starting with version 2.0.0 uses JaCoCo 0.7.5 and thus requires also this version to be used in your projects. Please stick to JaCoCo plugin
-    // 1.0.19 or lower if you still use JaCoCo 0.7.4 or lower
-    private String getJacocoLibraryVersion() {
-        boolean old = jenkins.getPlugin("jacoco").getVersion().isOlderThan(new VersionNumber("2.0.0"));
-        return old ? OLD_VERSION: NEW_VERSION;
-    }
-
-    private static final String OLD_VERSION = "0.7.4.201502262128";
-    private static final String NEW_VERSION = "0.8.2";
 }
diff --git a/src/test/resources/jacoco/test/pom.xml b/src/test/resources/jacoco/test/pom.xml
index 1828b075..10544948 100644
--- a/src/test/resources/jacoco/test/pom.xml
+++ b/src/test/resources/jacoco/test/pom.xml
@@ -29,7 +29,7 @@
       <plugin>
         <groupId>org.jacoco</groupId>
         <artifactId>jacoco-maven-plugin</artifactId>
-        <version>${jacoco.version}</version>
+        <version>0.8.8</version>
         <executions>
           <execution>
             <id>default-prepare-agent</id>

Use a fixed version of core in tests and automate updates to it

What feature do you want to see added?

I would like to get a PR created by updatecli / whatever that bumps the version of core that ATH tests against, currently it just always runs against the last weekly but that means we don't recognise regressions when they happen and there's no notification to check the test suite.

In bom we do:

https://github.com/jenkinsci/bom/blob/master/.github/workflows/updatecli.yaml
https://github.com/jenkinsci/bom/blob/master/updatecli/updatecli.d/weekly-apply.sh

Currently the version is set in https://github.com/jenkinsci/acceptance-test-harness/blob/master/src/main/resources/ath-container/run.sh#L48 where it just uses latest and

I think what we want to do is change the Jenkinsfile to read a file with the version number we're testing against.

The reason to use a file is to allow 'untrusted' contributors to test against different versions without needing write access

Upstream changes

No response

Replace `docker-fixtures` with `testcontainers`

docker-fixtures has been deprecated for a long time, https://www.testcontainers.org is the recommended replacement.

We should switch to it.

1.116 and later unusable with `runATH`

Jenkins and plugins versions report

Environment

Paste the output here

What Operating System are you using (both controller, and any agents involved in the problem)?

Ubuntu 22.04.1 LTS x86_64

Reproduction steps

Create an essentials.yml file with …

---
ath:
  useLocalSnapshots: false
  athRevision: "acceptance-test-harness-1.117"
  athImage: "jenkins/ath:1.97-pre"
  categories:
    - org.jenkinsci.test.acceptance.junit.SmokeTest
  jdks:
    - 11

… noting that these are the very latest versions of the official Acceptance Test Harness (1.117) and its official Docker image (1.97-pre) at the time of this writing. Then in your Jenkinsfile add runATH metadataFile: metadataPath and run the build.

Expected Results

The build passes.

Actual Results

The build fails with:

[2022-08-01T16:46:14.524Z] [INFO] --- maven-enforcer-plugin:3.1.0:enforce (display-info) @ acceptance-test-harness ---
[2022-08-01T16:46:14.904Z] [WARNING] The artifact org.slf4j:slf4j-log4j12:jar:1.7.36 has been relocated to org.slf4j:slf4j-reload4j:jar:1.7.36
[2022-08-01T16:46:14.904Z] [INFO] Adding ignore: module-info
[2022-08-01T16:46:15.296Z] [WARNING] The artifact org.slf4j:slf4j-log4j12:jar:1.7.36 has been relocated to org.slf4j:slf4j-reload4j:jar:1.7.36
[2022-08-01T16:46:23.044Z] [ERROR] Rule 1: org.apache.maven.plugins.enforcer.RequireMavenVersion failed with message:
[2022-08-01T16:46:23.045Z] 3.8.1 required to no longer download dependencies via HTTP (use HTTPS instead).
[2022-08-01T16:46:23.045Z] [INFO] ------------------------------------------------------------------------
[2022-08-01T16:46:23.045Z] [INFO] BUILD FAILURE
[2022-08-01T16:46:23.045Z] [INFO] ------------------------------------------------------------------------
[2022-08-01T16:46:23.045Z] [INFO] Total time:  51.341 s
[2022-08-01T16:46:23.045Z] [INFO] Finished at: 2022-08-01T16:46:22Z
[2022-08-01T16:46:23.045Z] [INFO] ------------------------------------------------------------------------
[2022-08-01T16:46:23.045Z] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.1.0:enforce (display-info) on project acceptance-test-harness: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1]
[2022-08-01T16:46:23.045Z] [ERROR] 
[2022-08-01T16:46:23.045Z] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[2022-08-01T16:46:23.045Z] [ERROR] Re-run Maven using the -X switch to enable full debug logging.
[2022-08-01T16:46:23.045Z] [ERROR] 
[2022-08-01T16:46:23.045Z] [ERROR] For more information about the errors and possible solutions, please read the following articles:
[2022-08-01T16:46:23.045Z] [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
script returned exit code 1

Anything else?

Evaluation

The version of Maven in the latest official Docker image is 3.6.3, but as of #851 (and thefefore jenkinsci/pom#198) Maven 3.8.1 or later is required. As a result, Acceptance Test Harness 1.116 or later is unusable with https://github.com/jenkins-infra/pipeline-library/blob/5d1f824b90a231752fa543c69b5615b1a1a5884f/vars/runATH.groovy.

ATH doesn't work on recent Firefox

What feature do you want to see added?

ATH to either have Firefox unpinned or automatically updated.
ATH using the latest version of Firefox

Upstream changes

see #1043
and
#1039

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

Update docker Docker tag to v26

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Update ubuntu Docker tag to v24

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Detected dependencies

dockerfile

src/main/resources/ath-container/Dockerfile

ubuntu 22.04

src/test/resources/docker_build_step/context.dir/Dockerfile

github-actions

.github/workflows/cd.yml

jenkins-infra/github-reusable-workflows v1

.github/workflows/verify-gradle-wrapper.yaml

actions/checkout v4

gradle/wrapper-validation-action v3

gradle

src/test/resources/artifactory_plugin/quickstart/build.gradle

commons-collections:commons-collections 3.2.2

junit:junit 4.+

src/test/resources/gradle_plugin/script.gradle

com.gradle.build-scan 1.16

src/test/resources/gradle_plugin/scriptNoPlugins.gradle

gradle-wrapper

src/test/resources/artifactory_plugin/quickstart/gradle/wrapper/gradle-wrapper.properties

gradle 8.7

maven

.mvn/extensions.xml

io.jenkins.tools.incrementals:git-changelist-maven-extension 1.8

pom.xml

org.jenkins-ci:jenkins 1.114

org.gitlab4j:gitlab4j-api 5.5.0

args4j:args4j 2.37

com.browserup:browserup-proxy-core 2.1.2

com.cloudbees:extensibility 1.7

com.github.jnr:jffi 1.3.13

com.github.jnr:jffi 1.3.13

com.github.jnr:jnr-unixsocket 0.38.22

com.github.stephenc.monte:monte-screen-recorder 0.7.7.0

com.google.guava:guava 33.2.0-jre

com.google.inject:guice 6.0.0

com.jcraft:jsch 0.1.55

commons-logging:commons-logging 1.3.2

commons-net:commons-net 3.10.0

org.apache.commons:commons-configuration2 2.10.1

org.apache.commons:commons-lang3 3.14.0

org.bouncycastle:bcpkix-jdk18on 1.78.1

org.codehaus.groovy:groovy 3.0.21

org.codehaus.groovy:groovy-console 3.0.21

org.jenkins-ci:crypto-util 1.9

org.jenkins-ci:groovy-guice-binder 1.2

org.jenkins-ci:jira-api 1.3

org.jenkins-ci:version-number 1.11

org.jenkins-ci.main:remoting 3248.v65ecb_254c298

org.jenkins-ci.test:docker-fixtures 190.vd6a_e600cb_775

org.json:json 20240303

org.kohsuke:wordnet-random-name 1.5

org.seleniumhq.selenium:selenium-api 4.21.0

org.seleniumhq.selenium:selenium-chrome-driver 4.21.0

org.seleniumhq.selenium:selenium-firefox-driver 4.21.0

org.seleniumhq.selenium:selenium-java 4.21.0

org.seleniumhq.selenium:selenium-safari-driver 4.21.0

org.seleniumhq.selenium:selenium-support 4.21.0

org.slf4j:slf4j-jdk14 2.0.13

org.zeroturnaround:zt-zip 1.17

org.jenkins-ci.main:jenkins-core 2.458

jakarta.mail:jakarta.mail-api 2.1.3

com.fasterxml.jackson:jackson-bom 2.17.1

io.netty:netty-bom 4.1.109.Final

commons-io:commons-io 2.16.1

net.bytebuddy:byte-buddy 1.14.15

org.javassist:javassist 3.30.2-GA

org.apache.commons:commons-text 1.12.0

org.apache.httpcomponents:httpclient 4.5.14

org.apache.httpcomponents:httpcore 4.4.16

org.slf4j:slf4j-api 2.0.13

org.slf4j:jcl-over-slf4j 2.0.13

org.codehaus.mojo:appassembler-maven-plugin 2.1.0

org.jenkins-ci.main:jenkins-war 2.458

src/test/resources/artifactory_plugin/multimodule/module_a/pom.xml

src/test/resources/artifactory_plugin/multimodule/module_b/pom.xml

src/test/resources/artifactory_plugin/multimodule/pom.xml

src/test/resources/jacoco/test/pom.xml

junit:junit 4.13.2

org.jacoco:jacoco-maven-plugin 0.8.12

org.apache.maven.plugins:maven-surefire-plugin 3.2.5

src/test/resources/maven_plugin/multimodule/module_a/pom.xml

src/test/resources/maven_plugin/multimodule/module_b/pom.xml

src/test/resources/maven_plugin/multimodule/pom.xml

regex

src/main/java/org/jenkinsci/test/acceptance/FallbackConfig.java

org.seleniumhq.selenium:selenium-java 4.21.0

org.seleniumhq.selenium:selenium-java 4.21.0

src/main/resources/ath-container/Dockerfile

mozilla/geckodriver 0.34.0

src/main/resources/ath-container/Dockerfile

org.apache.maven:maven-core 3.9.6

src/main/resources/ath-container/Dockerfile

docker 25.0.5

Check this box to trigger a request for Renovate to run again on this repository

`plugins.PipelineTest#testCompletePipeline` fails on Java 17

When running this test after calling set-java.sh 17:

[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] Source option 6 is no longer supported. Use 7 or later.
[ERROR] Target option 6 is no longer supported. Use 7 or later.
[INFO] 2 errors 
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.251 s
[INFO] Finished at: 2023-04-25T20:42:34+00:00
[INFO] Final Memory: 17M/70M
[INFO] ------------------------------------------------------------------------

Gradle plugin is failing to install in ATH

java.net.SocketException: Connection reset
	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
	at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
	at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:292)
	at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:351)
	at java.base/sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:788)
	at java.base/sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:723)
	at java.base/sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:747)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1620)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1525)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:3129)
	at java.base/java.net.URLConnection.getHeaderFieldLong(URLConnection.java:636)
	at java.base/java.net.URLConnection.getContentLengthLong(URLConnection.java:508)
	at java.base/java.net.URLConnection.getContentLength(URLConnection.java:492)
	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1277)
Caused: java.net.SocketException: Connection reset
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1989)
	at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1984)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1983)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1541)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1525)
	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1296)
Caused: java.io.IOException: Failed to load http://0.0.0.0:56148/gradle.hpi to /Users/timja/code/jenkins/acceptance-test-harness/target/jenkinshome15427209656703364098/plugins/gradle.jpi.tmp
	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1307)
Caused: java.io.IOException: Failed to download from http://0.0.0.0:56148/gradle.hpi
	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1341)
	at hudson.model.UpdateCenter$DownloadJob._run(UpdateCenter.java:1898)
	at hudson.model.UpdateCenter$InstallationJob._run(UpdateCenter.java:2210)
	at hudson.model.UpdateCenter$DownloadJob.run(UpdateCenter.java:1872)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:121)
	at java.base/java.lang.Thread.run(Thread.java:829)

Causes all Gradle plugin tests to fail and also any that depend on the plugin e.g. artifactory plugin

Master branch build broken down to UnsupportedClassVersionError expectations of dependencies

While observing the stalled builds of the master branch, it appears the following test case fails, because core now requires Java 11 but something picks up Java 8 still:
https://ci.jenkins.io/job/Core/job/acceptance-test-harness/job/master/476/consoleFull

10:45:24  <===[JENKINS REMOTING CAPACITY]===>channel started
10:45:24  Jul 01, 2022 7:53:53 AM hudson.remoting.UserRequest perform
10:45:24  WARNING: LinkageError while performing UserRequest:hudson.slaves.SlaveComputer$SlaveVersion@730ead12
10:45:24  java.lang.UnsupportedClassVersionError: Failed to load hudson.slaves.SlaveComputer$SlaveVersion
10:45:24   at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:460)
10:45:24   at hudson.remoting.RemoteClassLoader.loadRemoteClass(RemoteClassLoader.java:292)
10:45:24   at hudson.remoting.RemoteClassLoader.loadWithMultiClassLoader(RemoteClassLoader.java:269)
10:45:24   at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:228)
10:45:24   at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
10:45:24   at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
10:45:24   at java.lang.Class.forName0(Native Method)
10:45:24   at java.lang.Class.forName(Class.java:348)
10:45:24   at hudson.remoting.MultiClassLoaderSerializer$Input.resolveClass(MultiClassLoaderSerializer.java:132)
10:45:24   at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1986)
10:45:24   at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1850)
10:45:24   at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2160)
10:45:24   at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1667)
10:45:24   at java.io.ObjectInputStream.readObject(ObjectInputStream.java:503)
10:45:24   at java.io.ObjectInputStream.readObject(ObjectInputStream.java:461)
10:45:24   at hudson.remoting.UserRequest.deserialize(UserRequest.java:289)
10:45:24   at hudson.remoting.UserRequest.perform(UserRequest.java:189)
10:45:24   at hudson.remoting.UserRequest.perform(UserRequest.java:54)
10:45:24   at hudson.remoting.Request$2.run(Request.java:376)
10:45:24   at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
10:45:24   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
10:45:24   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
10:45:24   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
10:45:24   at java.lang.Thread.run(Thread.java:748)
10:45:24  Caused by: java.lang.UnsupportedClassVersionError: hudson/slaves/SlaveComputer$SlaveVersion has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
10:45:24   at java.lang.ClassLoader.defineClass1(Native Method)
10:45:24   at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
10:45:24   at java.lang.ClassLoader.defineClass(ClassLoader.java:635)
10:45:24   at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:458)
10:45:24   ... 23 more

docker based tests no longer run on CI builds

Jenkins and plugins versions report

Environment

ci.jenkins.io

What Operating System are you using (both controller, and any agents involved in the problem)?

ci.jenkins.io

Reproduction steps

run a build of the ATH on ci.jenkins.io

Expected Results

tests that require docker are run

Actual Results

tests that require docker are skipped

Anything else?

#878 (comment)

	String mimeType = FormatKeys.MIME_QUICKTIME;
	String videoFormatName = VideoFormatKeys.ENCODING_QUICKTIME_ANIMATION;
	String compressorName = VideoFormatKeys.COMPRESSOR_NAME_QUICKTIME_ANIMATION;
	Dimension outputDimension = gc.getBounds().getSize();
	int bitDepth = BIT_DEPTH;
	float quality = QUALITY_RATIO;
	int screenRate = FRAME_RATE_PER_SEC;

	Format outputFormatForScreenCapture = getOutputFormatForScreenCapture(videoFormatName,
	compressorName, outputDimension,
	bitDepth, quality, screenRate);

	this.screenRecorder = new JUnitScreenRecorder
	(gc, gc.getBounds(), getFileFormat(mimeType),
	outputFormatForScreenCapture, null, null, diagnostics);
	this.screenRecorder.start();