Code Monkey home page Code Monkey logo

Comments (97)

chrispat avatar chrispat commented on June 3, 2024 268

Caches will all expire eventually so I don’t think being able to clear specific ones is going to be very useful.

from cache.

sjackman avatar sjackman commented on June 3, 2024 79

I would scream with joy for

  • a button to clear the cache
  • a button to Re-run jobs without cache
  • stretch goal: a UI to inspect the number and size of cache tarballs, and which branches/PRs/commits they're associated with, and delete them selectively from the web interface

from cache.

chrispat avatar chrispat commented on June 3, 2024 73

We will evict the least recently accessed cache automatically. Eventually we will add a way to clear out the cache from the UI.

from cache.

ahdbilal avatar ahdbilal commented on June 3, 2024 69

Hello folks! We finally have started looking into it and will soon begin engineering on it. We’d greatly appreciate it if you can complete a 2-min survey to provide us your feedback on a few of the decisions variables! Thank you

from cache.

vsvipul avatar vsvipul commented on June 3, 2024 51

Good News folks, we have shipped the delete caches by key and delete cache by id APIs which will help you to delete caches for a repository. This is in addition to the API which lets you list caches for a repository. Go ahead and try it out and let us know feedback if any in this issue itself.
Here are the official docs-
List - https://docs.github.com/en/rest/actions/cache#list-github-actions-caches-for-a-repository
Delete by ID - https://docs.github.com/en/rest/actions/cache#delete-a-github-actions-cache-for-a-repository-using-a-cache-id
Delete by Key - https://docs.github.com/en/rest/actions/cache#delete-github-actions-caches-for-a-repository-using-a-cache-key

P.S. This is currently not shipped to GHES and is expected to ship in GHES 3.7.

from cache.

eregon avatar eregon commented on June 3, 2024 36

In ruby/setup-ruby#7 we noticed an issue because using subtly different prebuilt Ruby binaries corrupts the cache.
Being able to clear the cache when it's corrupted or contains errors would be very useful as a debugging mechanism.

Right now the only workaround seems to use another key for the cache action.

from cache.

vsvipul avatar vsvipul commented on June 3, 2024 34

@adamant-pwn This is being targeted to be done by end of this quarter. You can track the progress at the public roadmap at github/roadmap#502 .

from cache.

firefart avatar firefart commented on June 3, 2024 30

Same here. Dealing with a corrupted binary in a gem cache and the only way to get rid of it is to use another key. A method to clear the cache would be nice

from cache.

beatngu13 avatar beatngu13 commented on June 3, 2024 30

One can abuse secrets to clear the cache via the UI, i.e. without editing the workflow file. For instance, create a secret using the key CACHE_VERSION and as value a Unix timestamp, a counter, a version string or something else. Then use it as follows:

key: ${{ runner.os }}-mycache-${{ secrets.CACHE_VERSION }}-${{ hashFiles(...) }}

Whenever the secret is updated via the UI, the workflow will use a new cache.

from cache.

vsvipul avatar vsvipul commented on June 3, 2024 20

@Be-ing Eventually yes, this will be brought to UI as well, but we don't have an expected ship date for that. Keeping this issue open for that particular reason.

from cache.

vsvipul avatar vsvipul commented on June 3, 2024 19

@JavierSegoviaCordoba Cache management experience is currently targeted for end of Q3 2022, and ability to view cache usage is targeted for end of Q1 2022. Details are updated at public roadmap for github here - https://github.com/orgs/github/projects/4247/views/1?filterQuery=cache

from cache.

0x2b3bfa0 avatar 0x2b3bfa0 commented on June 3, 2024 18

GitHub will remove any cache entries that have not been accessed in over 7 days. There is no limit on the number of caches you can store, but the total size of all caches in a repository is limited to 10 GB. If you exceed this limit, GitHub will save your cache but will begin evicting caches until the total size is less than 10 GB.

Crystal clear, GitHub! :octocat: Inspired by #2 (comment)

on: workflow_dispatch
jobs:
  flush:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/cache@v2
        with:
          path: /tmp/flush
          key: ${{ github.run_id }}-${{ github.run_attempt }}
      - run: dd if=/dev/random of=/tmp/flush bs=1M count=10000

Data is compressed with zstd so fallocate(1) won't do the trick.

from cache.

kcgen avatar kcgen commented on June 3, 2024 16

@chrispat said: "Eventually we will add a way to clear out the cache from the UI."

Coming up on a year; @chrispat, can you please give us an update on this?

I'm not sure why something so trivial yet sought-after hasn't been implemented.

Forget the GUI for now (I don't have more years to wait for it) The MVP for this is a single API call that can be added to ones YAML and rm -rf the project's cache. Done.

from cache.

Jared-Dev avatar Jared-Dev commented on June 3, 2024 16

As others have mentioned. We have a corrupt package that is failing the build. I appreciate the "hack" listed above, but this should really be added into the UI.

Could the "Re-run jobs" dropdown not have an option to "Re-run all jobs & clear cache" added?!

image

from cache.

matks avatar matks commented on June 3, 2024 15

Would be awesome to have a way to clear cache from the UI 👍

I am setting up a GA using Composer dependencies (using dev-master) so I need Composer to grab the very latest versions from my github repo instead of the one he has in cache

from cache.

kubukoz avatar kubukoz commented on June 3, 2024 13

This seems like a really obvious feature, I really wish it was available :)

any way to clear the cache without editing the workflow files would be awesome!

from cache.

ThiefMaster avatar ThiefMaster commented on June 3, 2024 13

How is this still open over a year later?! The workarounds - even with the variable from secrets - are super ugly... I'm tempted to just run a dummy action that dumps 5GB of garbage in the cache to evict old data but that's ugly as well (and a waste of space).

from cache.

ellneal avatar ellneal commented on June 3, 2024 11

+1

Just encountered a situation where I need to clear the cache to solve a build failure, and the lack of support for it is frustrating (especially given how long this issue has been open).

from cache.

dhadka avatar dhadka commented on June 3, 2024 9

@knubie Change the key (and restore keys). For example, you could add -v2- to the key.

from cache.

ahmadnassri avatar ahmadnassri commented on June 3, 2024 9

2 years in, still no manual cache control... because the team who built this clearly never uses it in their own work.

allow user to clear cache (like every other system in the planet) is bad ... yet charging every penny possible for storage of said cache is somehow okay?

from cache.

ocelotl avatar ocelotl commented on June 3, 2024 9

As others have mentioned. We have a corrupt package that is failing the build. I appreciate the "hack" listed above, but this should really be added into the UI.

Could the "Re-run jobs" dropdown not have an option to "Re-run all jobs & clear cache" added?!

image

Yes, please. This seems intuitive and practical, users will greatly appreciate this feature 👍

from cache.

WolfgangFahl avatar WolfgangFahl commented on June 3, 2024 9

Ubuntu had bug #1. github actions has bug # 2

from cache.

edaemon avatar edaemon commented on June 3, 2024 9

While searching for a solution or workaround to this I came across this Stack Overflow answer which suggested including a secret in the cache key, e.g.:

key: ${{ runner.os }}-example-${{ secrets.CACHE_VERSION }}

This allows you to change the cache key without changing the workflow file. It isn't quite as nice as a native feature but it gets the job done; if you need to re-run a job with a fresh cache just update the CACHE_VERSION secret and re-run the job.

from cache.

stevencpp avatar stevencpp commented on June 3, 2024 8

My use case for clearing only specific caches is that I have a huge dependency that takes two hours to build, and while it may eventually expire, I really don't want to have to rebuild that unless absolutely necessary. I also have a bunch of other smaller caches, and in some cases getting the right hash at the end of the key to make it rebuild exactly when it needs to can be difficult to implement, so I'd like to just use a key either without a hash at the end, or with an imperfect hash, and manually clear only that specific cache when I know it needs to be rebuilt. My current workaround is to rename the key when I want to rebuild it, but it would be better to avoid those dummy commits and just do it from the UI.

from cache.

ahmadnassri avatar ahmadnassri commented on June 3, 2024 8

funny how everything I posted is getting a reply except for the original topic: allowing clearing cache

from cache.

lavfilip avatar lavfilip commented on June 3, 2024 8

How is this still not done

from cache.

jvacek avatar jvacek commented on June 3, 2024 8

Would be nice to have an option to auto-clear caches in case an action fails, so that the next run can be "from scratch"

from cache.

knubie avatar knubie commented on June 3, 2024 6

I need to clear the cache to get my yarn installation working again. An interface would be nice, but in lieu of that, how can i clear it manually?

from cache.

JavierSegoviaCordoba avatar JavierSegoviaCordoba commented on June 3, 2024 5

Is there an ETA about this? The problem is not only with this action itself, other actions can be using this action to manage cache with internal keys, so can be hard to trigger cleaning up the cache without knowing the keys and so on...

from cache.

vsvipul avatar vsvipul commented on June 3, 2024 5

@alexislefebvre @mkurz Thanks for adding the announcement here.
https://github.blog/changelog/2022-10-20-manage-caches-in-your-actions-workflows-from-web-interface/
We have released the UI to manage your caches from web interface. Do check it out. I'm going to go ahead and close this issue now.

For bulk delete, we might support it in the future, and I would suggest you to open a separate issue as well for that for more discussion. Thank you.

from cache.

bbimber avatar bbimber commented on June 3, 2024 4

Having the ability to clear cache has been an open issue forever with no change. What about allowing some kind of flag that simply tell the cache plugin "dont query the cache on this job". I know that's not the same thing as clearing a bad cache, but if a given job was prevented from querying the existing cache, completed, and then re-populated the cache on success, this might be a pragmatic solution to clear that bad cache. Presumably implementing this kind of thing is a different level of complexity than true cache clearing. It could probably be implemented on this plugin.

I dont know if there are more elegant ways to pass per-job environment or config, but the same concept as the 'hack' for ${{SECRETS.CACHE_VERSION}} could be applied. This plugin could support:

key:
restore-keys: $<RESTORE_KEYS>
skipCacheLoad: <SOME CONDITION or ${{ secrets.SKIP_QUERY_CACHE }}>

from cache.

Rashmi-278 avatar Rashmi-278 commented on June 3, 2024 4

Would love to have this feature

from cache.

beatngu13 avatar beatngu13 commented on June 3, 2024 3

2 years in, still no manual cache control... because the team who built this clearly never uses it in their own work.

allow user to clear cache (like every other system in the planet) is bad ... yet charging every penny possible for storage of said cache is somehow okay?

Yes, the feature request should get more attention, but that comment is a bit harsh and unobjective IMO. Also, there are various practical workarounds, so it's not really a blocker.

from cache.

samwightt avatar samwightt commented on June 3, 2024 3

Is this being actively considered / worked on? It's the second issue on the repo and one of the most requested features.

from cache.

OmgImAlexis avatar OmgImAlexis commented on June 3, 2024 3

@chrispat can we get an update on if this is even being looked into? Your first comment suggested that you’d be adding it at some point. Well we’re over a year later and still nothing.

Edit: even if you don’t add something in the UI please atleast get it added to your official cli.

from cache.

vsvipul avatar vsvipul commented on June 3, 2024 3

@jacksongoode Try this -

gh api \
  --method DELETE \
  -H "Accept: application/vnd.github+json" \
  /repos/OWNER/REPO/actions/caches\?key\=YOUR_KEY_HERE

from cache.

wpbonelli avatar wpbonelli commented on June 3, 2024 3

Thanks for this feature. Would it be possible to add the ability to match partial keys, like the cache action's restore-keys attribute does? It would be nice to be able to invalidate any cache entries matching a prefix. Naively, this seems compatible with the API's current implementation, since it already returns a list actions_caches of deleted entries.

from cache.

cdce8p avatar cdce8p commented on June 3, 2024 3

The current workaround I've found is to key on the commit hash:

- uses: actions/cache@v3
   with:
     key: ${{ github.workflow }}.${{ github.job }}.${{ github.event.after }}
     restore-keys: |
       ${{ github.workflow }}.${{ github.job }}.${{ github.event.before }}
     path: ...

The workflow tries to pull "CI.tests.", doesn't find it, instead pulls "CI.tests.", and then creates an entry for "CI.tests.". The next time it runs, the new before is the old after, so it repeats the same process.

I've implemented something similar to cache downloaded dependencies. However instead github.event.after, I use a timestamp. The result is the same though. There is never an actual cache hit, the action will just restore the last saved cached.

      - name: Generate partial restore key
        id: generate-key
        run: >-
          echo "::set-output name=key::$(date -u '+%Y-%m-%dT%H:%M:%s')"
      - users: actions/cache@v3
        with:
          path: ${{ env.CACHE }}
          key: >-
            ${{ runner.os }}-${{ env.CACHE_VERSION }}-${{ steps.generate-key.outputs.key }}
          restore-keys: |
            ${{ runner.os }}-${{ env.CACHE_VERSION }}-

Since I only need the latest cache entry, it would save a lot of storage if all old cache entries (which didn't match) could be deleted, with a config option.

from cache.

jcornaz avatar jcornaz commented on June 3, 2024 2

Caches will all expire eventually so I don’t think being able to clear specific ones is going to be very useful.

Please consider the following use case:

A gradle project caching the gradle caches like this:

    - name: Gradle caches
      uses: actions/cache@v1
      with:
        path: ~/.gradle/caches
        key: gradle-cache-${{ hashFiles('**/*.kt*') }}
        restore-keys: |
          gradle-cache-

Each build will:

  • download a recent gradle build cache
  • perform the build (using and updating the build cache)
  • upload the updated cache

Which means the build cache will grow over time and will eventually reach the size limit.

But expiring the old caches doesn't help, because the build always re-upload it.

Being able to manually clear the cache from time to time, would definitely be nice and useful.

I would personally do when:

  • I just upgraded gradle to a new major version
  • I just released a new major version of my software

from cache.

dhadka avatar dhadka commented on June 3, 2024 2

@ahmadnassri Cache storage is free and is not included in the "shared storage for Actions & Packages" shown on the billing page.

also, somewhat related: have some ghost workflows that were deleted months ago, yet still show up in the UI... seemingly cached forever!

Can you still view the logs and any artifacts created by that workflow? I believe the record of the workflow will still exist (so links to old workflows still work), but old logs and artifacts will be deleted after the workflow expires.

If you don't need to keep artifacts around for the full 90 days, you can also consider using an action like gha-remove-artifacts to clean up artifacts sooner as a way to reduce storage costs.

I wish the billing page showed more breakdown, but currently it's a total cost of "shared storage between both Actions & Packages" then I'd be able to share exactly how much storage is being used for caching, cuz it clearly not all packages

I'll track down the team that works on billing and open an issue for this. Please also consider sending this request via contact us.

from cache.

beatngu13 avatar beatngu13 commented on June 3, 2024 2

There's a big caveat to this vs actually clearing the cache and that's if you keep doing this you'll likely hit the cache size limit.

That's true and worth noting. Is there much downside to hitting the limit, though? As far as I understand it, once you hit the limit your least-recently-used cache items are evicted, so I don't think workflows would be held up or impaired by reaching the cache size limit.

If, for instance, you maintain different caches for multiple scopes such as branches, you might evict valid caches due to the overall size limit of 5 GB.

It depends on your project/workflow if that is actually a "big caveat", but currently I'm not aware of any other workaround other than enforcing a cache miss by changing the key(s).

from cache.

ahdbilal avatar ahdbilal commented on June 3, 2024 2

@sideeffffect the format is definitely not ideal. You just have to pick one response option per row and column.

Thanks to everyone that have completed the survey. Really appreciate it!

from cache.

vsvipul avatar vsvipul commented on June 3, 2024 2

I'll look into getting this changed in the docs.

from cache.

alexklibisz avatar alexklibisz commented on June 3, 2024 2

I would find it very useful to specify that the cache key can be overwritten from the post action.

Something like:

with:
  overwrite-key: true

Example: I have a cache keyed on "${{ github.workflow }}.${{ github.job }}.${{ github.ref_name }}". Call it "CI.tests.main" for the CI workflow, running the tests job, on the main branch. My job pulls the cache and runs the tests, which results in some modifications to the cache (maybe from compiling new files). In the post-action, the current behavior results in a message like:

Cache hit occurred on the primary key CI.tests.main, not saving cache.

If my job made significant changes to the cache (e.g., from a big refactor), then I'd like to update the cache. Otherwise, the next time it runs, it's going to pull the 7-day-old copy, and it's going to re-do a lot of work.

The current workaround I've found is to key on the commit hash:

- uses: actions/cache@v3
   with:
     key: ${{ github.workflow }}.${{ github.job }}.${{ github.event.after }}
     restore-keys: |
       ${{ github.workflow }}.${{ github.job }}.${{ github.event.before }}
     path: ...

The workflow tries to pull "CI.tests.", doesn't find it, instead pulls "CI.tests.", and then creates an entry for "CI.tests.". The next time it runs, the new before is the old after, so it repeats the same process.

This works, but it's confusing and is going to create a lot of churn in the cache. There will basically be zero cache hits on the key.

from cache.

mkurz avatar mkurz commented on June 3, 2024 2

Check this announcement: Manage caches in your Actions workflows from Web Interface.

Thanks, however it is not possible to clean the whole cache of a repo right now, only single entries.
That's why I created

Please vote for it, thanks!

from cache.

jmgilman avatar jmgilman commented on June 3, 2024 2

Adding cache management to the UI doesn't address the use case where we need to overwrite the contents of a specific cache programmatically. We have a bit of code that does an extended evaluation, and we would like to cache the results across runs; however, trying to get the cache key unique enough to do the right thing has been non-trivial. It'd be preferable to maintain a single cache across runs and overwrite it as required, but the current implementation of the action does not support this.

For now, it seems the best solution is calling the API to delete the cache from within the action when we need to overwrite its contents.

from cache.

marco-schmidt avatar marco-schmidt commented on June 3, 2024 1

After a version upgrade a plugin did somehow not properly update its internal database kept in the cache (NVD, rather large): dependency-check/dependency-check-gradle#196 On my machine, rm -rf ~/.gradle/dependency-check-data/ was enough to solve this, forcing the plugin to recreate that database. My attempt to add that call to rm to my GitHub action failed for some reason after less than a minute, no idea what caused a cancellation:

##[error]The operation was canceled.

https://github.com/marco-schmidt/am/runs/1087579674?check_suite_focus=true

With Travis CI I could simply delete the cache, a new build then leads to the database being recreated and that solves the issue. It would be very helpful to be able to do this in GitHub actions as well.

from cache.

dhadka avatar dhadka commented on June 3, 2024 1

@marco-schmidt While it's not quite the same as deleting a cache, you can change the key to effectively start with a new cache. For example, it's common to add a version number, such as ${{ runner.os }}-gradle-v2-${{ hashFiles('**/*.gradle') }}. Please make sure to update the key as well as any restore-keys with the change.

As for the operation was canceled, that's caused by the fail-fast behavior. Since the windows-latest job failed, it canceled the ubuntu-latest job. You can change this behavior by setting fail-fast to false. Here's an example: https://github.com/actions/cache/blob/main/.github/workflows/workflow.yml#L23

from cache.

OmgImAlexis avatar OmgImAlexis commented on June 3, 2024 1

While searching for a solution or workaround to this I came across this Stack Overflow answer which suggested including a secret in the cache key, e.g.:

key: ${{ runner.os }}-example-${{ secrets.CACHE_VERSION }}

This allows you to change the cache key without changing the workflow file. It isn't quite as nice as a native feature but it gets the job done; if you need to re-run a job with a fresh cache just update the CACHE_VERSION secret and re-run the job.

There's a big caveat to this vs actually clearing the cache and that's if you keep doing this you'll likely hit the cache size limit.

from cache.

sideeffffect avatar sideeffffect commented on June 3, 2024 1

Are the radio buttons in the forms for the questions

  • How would you rate the following cache management scenarios?
  • How would you prefer to interact with the cache management service?

working as intended?

from cache.

t2y avatar t2y commented on June 3, 2024 1

@vsvipul I can access. Thank you. I'm looking forward to using that feature. 😃

from cache.

jacksongoode avatar jacksongoode commented on June 3, 2024 1

@vsvipul Thanks! Would be nice to have that mentioned in the docs as well :)

from cache.

ihostage avatar ihostage commented on June 3, 2024 1

@AntoinePrv You can see my example that I linked above. And two things in this example are related with your question.

  1. You need to add permission
permissions:
  actions: write # this permission is needed to delete cache
  1. GITHUB_TOKEN works only in protected branches. If you want to delete cache entry in the job for PR for example, you need to create a personal token with right permissions.

from cache.

stevencpp avatar stevencpp commented on June 3, 2024

Maybe it could be a part of something to list the individual caches and their sizes in the UI and allow clearing only specific caches ?

from cache.

adithyabsk avatar adithyabsk commented on June 3, 2024

Thought I'd ping this as well. Running into cache issues after the repo's name was changed.

from cache.

ioquatix avatar ioquatix commented on June 3, 2024

Running into the same issue. Cached the wrong directory, the cache archive is empty and cannot regenerate it.

from cache.

ahmadnassri avatar ahmadnassri commented on June 3, 2024

yes, that was harsh, however no less harsh and unobjective than the Staff Engineer for Actions stating:

Caches will all expire eventually so I don’t think being able to clear specific ones is going to be very useful.

all while, GH continues to charge for storage without any indication on cache expiry rules (or control over it)

I've been steadily updating the $ limits in one account month over month as storage usage keeps going up, and had to migrate away from caching big items (like docker image builds)

also, somewhat related: have some ghost workflows that were deleted months ago, yet still show up in the UI... seemingly cached forever!

the only workaround is essentially changing the cache key, but that does not give you control over cache.

  • can't delete old cached items, thus they continue to count toward your monthly cost, until the mysterious and undocumented automated cache clearing occurs.

  • can't re-run workflows which point to older cache key

from cache.

filips123 avatar filips123 commented on June 3, 2024

all while, GH continues to charge for storage without any indication on cache expiry rules (or control over it)

What about this:

GitHub will remove any cache entries that have not been accessed in over 7 days. There is no limit on the number of caches you can store, but the total size of all caches in a repository is limited to 5 GB. If you exceed this limit, GitHub will save your cache but will begin evicting caches until the total size is less than 5 GB.

from cache.

ahmadnassri avatar ahmadnassri commented on June 3, 2024

the 5 GB number doesn't square with what I'm seeing, I wish the billing page showed more breakdown, but currently it's a total cost of "shared storage between both Actions & Packages" then I'd be able to share exactly how much storage is being used for caching, cuz it clearly not all packages ... 🤷‍♂️

thanks for sharing that, I clearly missed it

from cache.

ahmadnassri avatar ahmadnassri commented on June 3, 2024

Can you still view the logs and any artifacts created by that workflow?

no logs & artifacts, but the workflow entry and the execution history still show, as well as a cached version of the workflow file (4+ months after they've been deleted)

I realize this particular issue is not related to this particular action though, so can post more details elsewhere..

from cache.

Borda avatar Borda commented on June 3, 2024

I guess that the problem is that cache is handled by another/independent action/bot and as a user you can set it anywhere, so there is probably not simple way how to tell which cacher you want to drop... well maybe drop all? :]

from cache.

 avatar commented on June 3, 2024

Can confirm: this needs to be implemented at some point. :/

from cache.

gustavopch avatar gustavopch commented on June 3, 2024

@ellneal You can use this workaround for now: #2 (comment)

from cache.

Milo123459 avatar Milo123459 commented on June 3, 2024

Is there a way to just clear it via the action? For example use actions/cache@v2
with: clear or smth.

from cache.

WolfgangFahl avatar WolfgangFahl commented on June 3, 2024

My need derives from this. The original tests had an issue leading to a 3 GByte download. After a fix it's only 2.5 MByte The download results seem to be in the cache - the machine fails before even trying the new download.

from cache.

jpvajda avatar jpvajda commented on June 3, 2024

Adding to the pile of requests, this would be a very useful feature. 🙏

from cache.

Milo123459 avatar Milo123459 commented on June 3, 2024

Not sure maybe the maintainers have lives?

from cache.

kubukoz avatar kubukoz commented on June 3, 2024

How is this still not done

@lavfilip pull requests are open.

from cache.

NyCodeGHG avatar NyCodeGHG commented on June 3, 2024

Would be awesome with an easy way to clear the cache. I know there are some workarounds, but that would be easier with something in the ui.

from cache.

edaemon avatar edaemon commented on June 3, 2024

There's a big caveat to this vs actually clearing the cache and that's if you keep doing this you'll likely hit the cache size limit.

That's true and worth noting. Is there much downside to hitting the limit, though? As far as I understand it, once you hit the limit your least-recently-used cache items are evicted, so I don't think workflows would be held up or impaired by reaching the cache size limit.

from cache.

nulano avatar nulano commented on June 3, 2024

There's a big caveat to this vs actually clearing the cache and that's if you keep doing this you'll likely hit the cache size limit.

Isn't that a good thing? IIUC when you hit the cache limit, the least recently used cache entry is deleted, so this should just clear the cache entry you want to get rid of.


key: ${{ runner.os }}-example-${{ secrets.CACHE_VERSION }}

It's probably better to put the secret at the front so that you can use something like

key: ${{ secrets.CACHE_VERSION }}-example-${{ runner.os }}-${{ hashFiles("example") }}
restore-keys:
  ${{ secrets.CACHE_VERSION }}-example-${{ runner.os }}-
  ${{ secrets.CACHE_VERSION }}-example-

from cache.

gugaiz avatar gugaiz commented on June 3, 2024

What if you do something like this?

name: manual deploy with invalidate cache

on: 
  workflow_dispatch:     
     inputs:      
        invalidate_cache:        
           description: Invalidate GitHub Action cache        
           required: true        
           default: 'false'      

jobs:
 tests:
    steps:
      - uses: actions/checkout@v2
      - name: Cache Libs
        id: cache-libs
        uses: actions/cache@v2
        with:
          path: |
            node_modules
            public
          key: cache-node-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            cache-node-

      - name: Invalidate cache
        if: ${{ github.event.inputs.invalidate_cache != 'false' && steps.cache-libs.outputs.cache-hit == 'true' }}
        run: |
          rm -rf node_modules
          rm -rf public

from cache.

Be-ing avatar Be-ing commented on June 3, 2024

Edit: even if you don’t add something in the UI please atleast get it added to your official cli.

Yeah something like adding an invalidateCache input to the action might work.

from cache.

burn2delete avatar burn2delete commented on June 3, 2024

We ran into an issue when using Github Packages, then deleted the package and re-deployed new packages (with the same versions, but different hash) - now our cache is corrupted with no "official" way to clear it.

Since we are using the node-setup action, I don't believe the options above will work for us, since we don't have direct access to the cache action keys or restore-keys. Our only option is to disable caching in node-setup until we pass the package versions that are corrupted in cache.

from cache.

Be-ing avatar Be-ing commented on June 3, 2024

That is also why I stopped using the run-vcpkg action.

from cache.

t2y avatar t2y commented on June 3, 2024

@vsvipul I tried to access your link, but I couldn't. Is that project private?

from cache.

vsvipul avatar vsvipul commented on June 3, 2024

@t2y Yes thanks for pointing that out. Updated to the public one now - but that doesn't have management experience added yet. might be added later.

from cache.

adamant-pwn avatar adamant-pwn commented on June 3, 2024

Still nothing? :(

from cache.

hovancik avatar hovancik commented on June 3, 2024

If anyone creates action with this, please, share :)

from cache.

Be-ing avatar Be-ing commented on June 3, 2024

Good news this was finally added to the GitHub API! Will it be added to the web UI too?

from cache.

chenrui333 avatar chenrui333 commented on June 3, 2024

Good News folks, we have shipped the delete caches by key and delete cache by id APIs which will help you to delete caches for a repository. This is in addition to the API which lets you list caches for a repository. Go ahead and try it out and let us know feedback if any in this issue itself. Here are the official docs- List - https://docs.github.com/en/rest/actions/cache#list-github-actions-caches-for-a-repository Delete by ID - https://docs.github.com/en/rest/actions/cache#delete-a-github-actions-cache-for-a-repository-using-a-cache-id Delete by Key - https://docs.github.com/en/rest/actions/cache#delete-github-actions-caches-for-a-repository-using-a-cache-key

P.S. This is currently not shipped to GHES and is expected to ship in GHES 3.7.

@vsvipul thanks for adding this endpoint, but however I am not able to use CACHE_ID to do the cache pruning.

$ curl -X DELETE -H "Accept: application/vnd.github+json" -H "Authorization: token $GITHUB_CACHE_TOKEN" https://api.github.com/repos/xxx/xxxx/actions/caches/67233
{"message":"Not Found","documentation_url":"https://docs.github.com/rest/actions/cache#delete-a-github-actions-cache-for-a-repository-using-a-cache-id"}

Also the purging cache by using CACHE KEY seems a bit wrong.

from cache.

chenrui333 avatar chenrui333 commented on June 3, 2024

Actually the cache purging works as expected, (I used gh cli earlier in the same session, which confused myself), sorry about the noise.

However, the documentation for cache purge using CACHE KEY could still use some help.

from cache.

jacksongoode avatar jacksongoode commented on June 3, 2024

However, the documentation for cache purge using CACHE KEY could still use some help.

Has anyone figured out how to use this endpoint? I keep getting Missing required query parameter key (HTTP 422).

from cache.

vsvipul avatar vsvipul commented on June 3, 2024

@jacksongoode key is an essential query parameter. Can you please tell how are you passing it and I might be able to help more.

from cache.

jacksongoode avatar jacksongoode commented on June 3, 2024

Sure, I'm just not even sure where to put it in regards to the docs.

gh api \
  --method DELETE \
  -H "Accept: application/vnd.github+json" \
  /repos/OWNER/REPO/actions/caches

I'm not sure where the key ought to go in this query?

from cache.

oddhack avatar oddhack commented on June 3, 2024

Clarification request: can I use this API to force pulling a new image from Dockerhub used in the 'container:' field of a job? Currently having problems with a out-of-date cached image vs. a PR that requires the latest Dockerhub version.

from cache.

alexislefebvre avatar alexislefebvre commented on June 3, 2024

Check this announcement: Manage caches in your Actions workflows from Web Interface.

from cache.

mkurz avatar mkurz commented on June 3, 2024

For bulk delete, we might support it in the future, and I would suggest you to open a separate issue as well for that for more discussion. Thank you.

Is https://github.com/orgs/community/discussions/36878 enough? Or do you mean in this repo here?

from cache.

vsvipul avatar vsvipul commented on June 3, 2024

@mkurz That should be enough but if you want you can open an issue in this repo as well.

from cache.

seh avatar seh commented on June 3, 2024

It's worth noting that if you've been interpolating Actions secret values into your cache names in order to be able invalidate them forcibly by changing that secret value, the Web interface now reveals your secret values, embedded within the displayed cache names.

Now, that was an abuse of the secret system, and the values were immaterial—such as "v2"—but still, it's worth a warning that GitHub isn't protecting against revealing these secret values here.

from cache.

nulano avatar nulano commented on June 3, 2024

I believe the cache names were already visible in the log of the cache step, e.g.:

Run actions/cache@v3
Received 7637907 of 7637907 (100.0%), 16.7 MBs/sec
Cache Size: ~7 MB (7637907 B)
C:\Windows\System32\tar.exe -z -xf D:/a/_temp/06f09ba7-bec2-461a-a241-d11afa9480f0/cache.tgz -P -C D:/a/Pillow/Pillow
Cache restored successfully
Cache restored from key: fbc4ecc14e7c86fb662b1be66ad2567466c767e957ec2f89ba03ac810e92d716-b1599657002fbcb4f133085efc810231ed266be17f248422ad42405170e10ac3-C:\hostedtoolcache\windows\Python\3.10.8\x86-17.3.32929.385

from cache.

seh avatar seh commented on June 3, 2024

I believe the cache names were already visible in the log of the cache step

But weren’t the secret values masked, as they are in other logs from job steps using secret values?

from cache.

nulano avatar nulano commented on June 3, 2024

Oh right, they would be masked in the log. I always forget about that detail as I find it very unintuitive. But yes, I would not expect a secret value to be used as part of the cache key.

from cache.

Betristor avatar Betristor commented on June 3, 2024

I tried to manage the caches using timestamps and seek to find a way which could erase those outdated caches within a workflow.
Motivation: I'm developing some optimization models. And I cached some base results for my code to compare every time I made a pull request so I could compare the difference between previous version and current version.
Problem: The cached base results would not update as some changes are introduced and approved into the base results.
Solution: Cache the results with timestamps and use the latest version of results. New cache will be triggered when a pull request is merged which means everything is approved including changes to the code and influence onto optimization model results.
Only one thing left: The cache size will grow larger and larger. Some outdated caches will take the github quota.
Fix: According to github official website, "GitHub will remove any cache entries that have not been accessed in over 7 days."

Everything done.

from cache.

ihostage avatar ihostage commented on June 3, 2024

Only one thing left: The cache size will grow larger and larger. Some outdated caches will take the github quota.

@Betristor If you are sure, that cache entry is outdated, you can remove it itself with help GitHub CLI.

For example in Playframework we remove outdated cache entries by simple script in schedule workflow
https://github.com/playframework/playframework/blob/3a78ae68acb10e614e8b1f1411ff4f7b93cd909e/.github/workflows/delete-caches.yml

from cache.

Betristor avatar Betristor commented on June 3, 2024

For example in Playframework we remove outdated cache entries by simple script in schedule workflow https://github.com/playframework/playframework/blob/3a78ae68acb10e614e8b1f1411ff4f7b93cd909e/.github/workflows/delete-caches.yml

@ihostage That's quite an elegant way for my situation. Much thanks. Maybe temporarily github could do the thing for me and I will introduce this feature in later development. And each time when a new cache is generated, whether its contents are updated or not, it's outdated for me. That's why I used time stamp to tag them.

from cache.

AntoinePrv avatar AntoinePrv commented on June 3, 2024

Hi there, anyone managed to call this in GHA?
It works without any issue locally, but in the CI I keep getting Resource not accessible by integration (HTTP 403).
I feel like I tried all possible combination of GITHUB_TOKEN GH_TOKEN, secrets.GITHUB_TOKEN, github.token, permissions and so on...

from cache.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.