Feature request: retry policy about download-artifact HOT 8 CLOSED

Retries have been added to all HTTP calls during artifact download with the latest v2 release. ECONNRESETs, 503 and a host of other errors should now be gracefully retried that previously were not. Each call will be retried up to 5 times with exponential back-off. If this issue comes up again please open up a new issue.

@wsnyder Backoff due to too many request (HTTP 429) can happen if a user is uploading or downloading a large amounts of artifacts or logs (both are effectively powered by the same backend infrastructure). It's a throttling mechanism so that a single user/org doesn't over overload our service or try to cause some sort of DDOS. If a 429 is received, it's treated as a normal retry retry and we just back-off and wait before trying again in a bit. It's very rare that a user upload/downloads so many artifacts that they see that error.

from download-artifact.

I see one more instantiation of network problem, that retry should fix (log are here):

Starting download for ispc_llvm11_linux
Error: read ECONNRESET

from download-artifact.

So most http calls during artifact download are retried. What you linked to is the webpack version of what is in this repo, the source code is here: https://github.com/actions/toolkit/blob/73d5917a6b5ea646ac3173cfceb727ee914ff6ed/packages/artifact/src/internal/download-http-client.ts#L173-L177

Upload-artifact faces a similar problem where a small percentage of calls are not retried. Those that do get retried have exponential back-off plus up to 5 retries. Will try to fix it this week 🤞

from download-artifact.

Looking at index.js I see retry logic, but it seems to be not configurable externally.

@konradpabjan, you have committed the code, which does retry under the hood. Is it possible to make the retry parameters externally configurable?

This becomes really painful to see fails related to network issues daily. In the workflow I have 21 jobs running in parallel, which consume the same artifact. One of them is failing with good chances.

from download-artifact.

If you could make it externally configurable, that would be even better! This might be especially important for self-hosted runners (we are going to start using them soon).

from download-artifact.

Thanks for work towards this, but our project just turned on additional actions and we are seeing about 1 in 50 times where download-artifact give what appears the same "Error: read ECONNRESET". Any hope to get this retry policy knob to turn up the backoff?

BTW I don't see the "info Backoff due to too many request" messages described actions/toolkit#675, I presume some verbosity flag is needed to see them. how do I get those?

from download-artifact.

Same issue here.

Run actions/download-artifact@v2
No artifact name specified, downloading all artifacts
Creating an extra directory for each artifact that is being downloaded
Total number of files that will be downloaded: 8
Total number of files that will be downloaded: 2
Error: read ECONNRESET

This is especially a trouble for the lengthy tests.

from download-artifact.

Just got the connection reset issue. I investigated after a job was taking a while to run. Glad to have found this issue and the knowledge that it was a considered scenario. 👍

from download-artifact.