Comments (10)
targos
commented on September 17, 2024
9
I think the cause is nodejs/node#53627, but this issue is not about the regression itself. It's about the fact that CITGM somehow reported successful runs instead of failures.
from citgm.
targos
commented on September 17, 2024
1
That's better, thanks ! Now do we need to investigate why it "thinks" there are no results ?
from citgm.
ljharb
commented on September 17, 2024
This is also filed as npm/cli#7657, and nodejs/node#53462 was pointed to me as a possible cause.
from citgm.
targos
commented on September 17, 2024
@nodejs/build-infra I believe this is an issue with the Jenkins job. If I run node bin/citgm-all.js --includeTags binary-split, I can reproduce the regression and it correctly exits with code 1.
from citgm.
richardlau
commented on September 17, 2024
Possibilities:
- The job runs citgm via
eval:
# Using eval so we can use a complicated CITGM_COMMAND such as 'NODE_OPTIONS=--xzy citgm-all'
eval $CITGM_COMMAND "--nodedir=$npm_config_nodedir -v $CITGM_LOGLEVEL -x $PWD/report.xml -q $NPM_LOGLEVEL --tmpDir $temp"
- We have
Do not fail the build on empty test resultsturned on
I'll turn off Do not fail the build on empty test results.
from citgm.
richardlau
commented on September 17, 2024
Rerunning https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/3457/nodes=debian12-x64/ with Do not fail the build on empty test results disabled:
https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/3459/nodes=debian12-x64/console
from citgm.
richardlau
commented on September 17, 2024
Rerunning https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/3457/nodes=debian12-x64/ with
Do not fail the build on empty test resultsdisabled: https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/3459/nodes=debian12-x64/console
With Do not fail the build on empty test results disabled, the CITGM run for 22.5.0 now fails due to having no test results.
from citgm.
richardlau
commented on September 17, 2024
https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/3459/nodes=debian12-x64/console
13:55:12 Recording test results
13:55:12 ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
which likely means smoker/report.xml wasn't generated:
we use -x to specify outputting to report.xml.
# Using eval so we can use a complicated CITGM_COMMAND such as 'NODE_OPTIONS=--xzy citgm-all'
eval $CITGM_COMMAND "--nodedir=$npm_config_nodedir -v $CITGM_LOGLEVEL -x $PWD/report.xml -q $NPM_LOGLEVEL --tmpDir $temp"
from citgm.
targos
commented on September 17, 2024
Here's the report.xml I get with Node.js 22.5.0 using the following command: node bin/citgm-all.js --includeTags binary-split -x $PWD/report.xml
<testsuite name="citgm">
<testcase name="binary-split-v1.0.5" time="12.196">
<system-out><![CDATA[ > [email protected] test
> standard --verbose && node test.js
npm error Exit handler never called!
npm error This is an error with npm itself. Please report this error at:
npm error <https://github.com/npm/cli/issues>
npm error A complete log of this run can be found in: /var/folders/9n/05y6s4ls78515vmnbd86hktm0000gn/T/9987debd-ab8e-402c-a690-08daa9c9d7de/home/.npm/_logs/2024-07-20T09_32_11_815Z-debug-0.log
sh: standard: command not found]]></system-out>
<failure message="module test suite failed">Error: The canary is dead: </failure>
</testcase>
<testcase name="clinic" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="debug" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="ember-cli" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="eslint" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="glob" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="got" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="gulp" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="jest" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="q" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="underscore" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="watchify" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="yeoman-generator" time="NaN">
<system-out/>
<skipped/>
</testcase>
</testsuite>
And with Node.js 22.5.1:
<testsuite name="citgm">
<testcase name="binary-split-v1.0.5" time="9.769">
<system-out><![CDATA[ added 267 packages in 7s
> [email protected] test
> standard --verbose && node test.js
TAP version 13
# ldjson file
ok 1 should be equal
# custom matcher
ok 2 should be equal
ok 3 should be equal
# long matcher
ok 4 should be equal
ok 5 should be equal
ok 6 should be equal
# matcher at index 0 check
ok 7 should be equal
ok 8 should be equal
ok 9 should be equal
# chunked input
ok 10 should be equal
# chunked input with long matcher
ok 11 should be equal
ok 12 should be equal
# lookbehind in multi character matcher
ok 13 should be equal
ok 14 should be equal
ok 15 should be equal
1..15
# tests 15
# pass 15
# ok]]></system-out>
</testcase>
<testcase name="clinic" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="debug" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="ember-cli" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="eslint" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="glob" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="got" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="gulp" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="jest" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="q" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="underscore" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="watchify" time="NaN">
<system-out/>
<skipped/>
</testcase>
<testcase name="yeoman-generator" time="NaN">
<system-out/>
<skipped/>
</testcase>
</testsuite>
from citgm.
Related Issues (20)
- Suggestion: `citgm-smoker-lite` job for PRs HOT 6
- citgm-smoker-nobuild seems broken HOT 23
- `esmocha` broken by ESM hook changes? HOT 3
- import-in-the-middle failing HOT 3
- `jest` flaky tests HOT 2
- v21.x failing modules HOT 11
- Add `readable-stream` to the CITGM
- Add `react` to the CITGM HOT 2
- `[email protected]` chromium binary is not available for arm64 HOT 1
- citgm-smoke-nobuild osx11 fails to run x64 binary on arm HOT 3
- CI failures on Windows
- ppc64le incompatibility HOT 3
- Undici failures HOT 21
- Modules failing on Node.js 20.x HOT 1
- Modules failing on Node.js 18.x HOT 1
- Add to citgm: astro HOT 1
- CITGM failures in node 22 HOT 1
- Node.js latest CITGM results HOT 3
- remove alpine from CITGM runs HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from citgm.