Comments (6)
https://crater-reports.s3.amazonaws.com/pr-116284/index.html also has quite a few of these.
from crater.
Also happened here: https://crater.rust-lang.org/ex/pr-124636.
from crater.
Also happened here: https://crater.rust-lang.org/ex/pr-124636.
I see 213 entries correctly sorted under spurious-regressed with build no space left on device
.
And 9 or 10 under regressed with build failed (unknown)
For the 10 the errors are
[INFO] [stderr] Error response from daemon: mkdir /var/lib/docker/overlay2/d0392f4632b22de6528ebe4efcc10f002656b8723c4a1b710ed4458e6b02ea3e-init: no space left on device
[INFO] [stderr] Error response from daemon: mkdir /var/lib/docker/overlay2/bd2885f2ca702c73f2089fd560ca5812020f20a703dd920f909fcf68b043099e/diff: no space left on device
[INFO] [stderr] Error response from daemon: symlink ../1744f2c6d6c1b7751a30adc89afb973fac6960b6b240c59be797348b8d713855/diff /var/lib/docker/overlay2/l/YQMKEEA6H4GSSXCHI2Y5LO3XBH: no space left on device
[INFO] [stderr] Error response from daemon: mkdir /var/lib/docker/overlay2/0fa118698fba3bdf766053a429fda884e20458aa114597c09bd675a92e77adcb-init: no space left on device
[INFO] [stderr] Error response from daemon: mkdir /var/lib/docker/overlay2/35445b29951a97ad96f344bd33abb3c7976dcd0a7db9620eb7ebfaed05284b94-init: no space left on device
[INFO] [stderr] Error response from daemon: mkdir /var/lib/docker/overlay2/5cd8254e4074e5a5dc108cf93c21d54d17667b05d9d70dd70a7404d4a2600f55-init: no space left on device
[INFO] [stderr] Error response from daemon: mkdir /var/lib/docker/overlay2/d1f525505d6a64c02a2fc42e1e7e0fa163877b4776a8c7ca3c94d69358ca848e-init: no space left on device
[INFO] [stderr] Error response from daemon: write /var/lib/docker/containers/08cdd65fbdc00b34e781e9ffe505de039f45012ef9e480a41d13c7b4780739d5/.tmp-config.v2.json2346282875: no space left on device
- log truncated, no error show, so uncertain if this is even a
no space left on device
error [INFO] [stderr] Error response from daemon: mkdir /var/lib/docker/overlay2/9404567f07eaeadab3dc07a00c6d4ea16806738d6eaa4b83b6ed78c401868307-init: no space left on device
So it looks like for all these the docker daemon failed to perform some action due to the full device.
Another source not seen here is described in #715. i.e. when a dependency fails to compile it is always a DependsOn failure even if a no space left on device
error occurred.
This can be seen in crater result posted above by RalphJung under regressed: dependencies
https://crater-reports.s3.amazonaws.com/pr-116284/index.html also has quite a few of these.
from crater.
What leads to the disk being full for crater?
from crater.
To reduce computation time crater does not reset to a pristine environment between crates so that the builds of dependencies can be reused. So build artifacts accumulate and can fill up the disk.
There is supposed to be a disk space every now and then see this comment https://github.com/rust-lang/crater/blob/master/src/runner/tasks.rs#L172-L185 that is supposed to cleanup when that happens.
The disc-space-watcher is started over here https://github.com/rust-lang/crater/blob/master/src/runner/mod.rs#L98 and appear to be currently configured to run every 30 seconds with a threshold of 80%.
from crater.
I see, thank you.
So in a sense, disk-full errors are spurious, but shouldn't be ignored, but and the crates maybe retried after cleaning up some disk space.
from crater.
Related Issues (20)
- Have a separate category in the report for crates using nightly features
- Report generation is slow and prone to failure HOT 4
- The SQLite database is hard to manage in production HOT 1
- Proactively delete old Docker images and dangling containers
- runner going down with scheduled work stalls experiment HOT 1
- "Temporary failure in name resolution" is a popular source of spurious failures
- Latest commit (9503083) compiles forever, eventually OOMs HOT 2
- Triaging/improving the number of crates classified as build-fail HOT 1
- Completion percentage can be over 100%
- Tons of spurious failures due to build scripts failing to run rustc HOT 11
- Automatic verification of potential regressions
- Should `tokio` really be marked as broken?
- Automatic detection of spurious regressions
- Numbers in final reports are confusing
- test timout to compiler error marked as spurious
- "root results" does not list the biggest root regression HOT 2
- Limit available parallelism to 1 per 2GB of RAM HOT 1
- Crater seems to sometimes test outdated crates HOT 3
- Crater log of "current run" always begins with "previous run"
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from crater.