Comments (31)
Certainly not all ecosystems, but for some major ones.
Compare the config for Python's pip with Travis CI:
cache: pip
With GitHub Actions:
- name: Ubuntu cache
uses: actions/cache@v1
if: startsWith(matrix.os, 'ubuntu')
with:
path: ~/.cache/pip
key:
${{ matrix.os }}-${{ matrix.python-version }}-${{ hashFiles('**/setup.py')
}}
restore-keys: |
${{ matrix.os }}-${{ matrix.python-version }}-
- name: macOS cache
uses: actions/cache@v1
if: startsWith(matrix.os, 'macOS')
with:
path: ~/Library/Caches/pip
key:
${{ matrix.os }}-${{ matrix.python-version }}-${{ hashFiles('**/setup.py')
}}
restore-keys: |
${{ matrix.os }}-${{ matrix.python-version }}-
- name: Windows cache
uses: actions/cache@v1
if: startsWith(matrix.os, 'windows')
with:
path: ~\AppData\Local\pip\Cache
key:
${{ matrix.os }}-${{ matrix.python-version }}-${{ hashFiles('**/setup.py')
}}
restore-keys: |
${{ matrix.os }}-${{ matrix.python-version }}-
10 characters vs. 1,000 characters!
I know which I'm be more confident using, and prefer suggesting for projects to add :)
Travis also has shortcuts like:
cache: bundler # Ruby
cache: cocoapods # Objective-C
cache: npm # npm
cache: yarn # yarn
cache: ccache # C or other C/C++ variants
cache: packages # R
cache: cargo # Rust
The flexibility of GHA's caching is good, but it would be even better to have some user-friendly shortcuts. Thanks!
from cache.
We made a scoping call for the v1 to not support multiple paths. This is something we do plan to look at for the future.
from cache.
👋 With #212, multiple paths are now supported.
This is currently available in the default branch (-uses: actions/cache@master
) and we'll be pushing a v2
tag next week, see #323 for details.
from cache.
Would it be possible to use a glob pattern for path?
The use case that I have in mind is caching the typescript build output in a monorepo.
For example:
path: packages/*/{build,tsconfig.tsbuildinfo}
from cache.
It might also be good if this didn't require config for something like Yarn
I don't think it is good for this project to have tool-specific behavior. I think it should be agnostic of whatever tools we use.
from cache.
Given that a lot of high-profile open source projects are monorepos, and many use Yarn, I still think it would be great to get in.
Examples include:
- Babel
- Jest
- Storybook
- React and Create React App
from cache.
paths
instead of path
would be a really nice addition.
I think anything beyond that should be a separate issue around simplification.
from cache.
For those who in need of configuring multiple cache targets for multiple languages or simply want more flexibility, I created an action called Cached Dependencies that allows you to manage all dependencies and caches in one build step.
This action adds two additional config files for you to define reusable commands and avoid copypasta across workflows:
.github/workflows/caches.js
: for cache configs.github/workflows/bashlib.sh
: for bash command shortcuts
Then you assemble the commands in one simple build step like this:
steps:
- uses: actions/checkout@v2
- uses: ktmud/cached-dependencies@v1
with:
parallel: true
run: |
cache-restore npm
npm install
cache-save npm
cache-restore pip
pip install -r requirements.txt
cache-save pip
cache-restore cypress
cd cypress/
npm install
cache-save cypress
Obviously, bashlib is not needed if you only have one workflow. You can read more documentation here or see it in action here.
from cache.
It might also be good if this didn't require config for something like Yarn, where you could just say workspaces: true
.
from cache.
Yes, I understand that, but do we create options for all ecosystems? That's not realistic.
I would go farther than saying "it is not realistic". I would say "it is not desirable".
I think, this this project should strive to be flexible and useful for as many different ecosystems as possible. Introducing yarn-specific behavior, would add maintenance overhead that do not bring benefits to the other ecosystems.
from cache.
10 characters vs. 1,000 characters!
@hugovk, your example is not using the multiple path feature that we are discussing in this issue!
The whole point of this issue, is to simplify such configuration, by letting the user to specify multiple paths at once. Of course, we all want to have a simpler and more concise way to handle multiple paths.
That does not necessarily imply adding tool specific behavior.
And I understand that you are not happy that pip
uses different cache folders depending on the OS. But that's a problem you should report to pip
, not here. pip
could use a path that remains consistent across operating systems (~/.cache/pip
for instance). I don't think it is this responsibility of this project to pay for the design issues of pip
.
from cache.
Sorry, you're right, that was somewhat off-topic, but it is related: the point is that shortcuts can be provided for common setups (multiple paths for some, different OS paths for others). I do know the point of this issue, I opened it :)
Some tool shortcut would include multiple paths; some would include different paths; whatever it needs.
I'm fine with pip using different paths depending on OS (and I have reported an issue to pip to have a way to find out the path on the current OS, and am working on a PR now :)
However, I'd like the GHA interface to be as simple as possible for the users of GHA. I know some maintainers will be put off by having to include and maintain great big chunks of YAML for things which will be common for many projects.
Like my first post in this issue shows 74 vs. 332 characters.
For example, we could close this issue and let the user write their own code to copy all their cache files and directories into a single directory just before the cache upload, and similarly re-instate them right after the cache download. But shortcuts would be helpful.
from cache.
I agree with @hugovk, supporting the biggest tools out of the box is ideal.
@jcornaz and @imbsky, that being said, if the end result was like GitLab's, that would be OK:
cache:
paths:
- node_modules/
- packages/*/node_modules/
- .yarn
But Travis has a simpler:
cache:
yarn: true
from cache.
Ok @mrmckeb and @imbsky I think that the first step is include something like this:
cache:
paths:
- node_modules/
- packages/*/node_modules/
- .yarn
And in a future, make something simpler like you said:
cache:
yarn: true
We are moving from CircleCI to Github Actions, and it's very important for us that some points are ready in the earlier release.
Features like this, or for example, don't have a scope in events like "deployments" make to us consider Github Actions a real Continuous Integration flow, or just something "funny" to auto add Labels and other stuff to our continuous development process.
I don't want to be rude, and sorry if I do, but it's very important for a new product see how it's growing and implementing minimal requirements. Remember that for a business make the change to Github Actions is always dangerous for us.
Have a nice day!!
from cache.
Same requirement for Elixir where we commonly cache multiple paths:
paths:
- _build
- deps
For now something like this is required:
- uses: actions/cache@preview
with:
path: deps
key: ${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }}
restore-keys: |
${{ runner.os }}-mix-
- uses: actions/cache@preview
with:
path: _build
key: ${{ runner.os }}-build-${{ hashFiles('**/mix.lock') }}
restore-keys: |
${{ runner.os }}-build-
from cache.
I'm not sure I can agree. That would mean considerable overhead for projects - the bigger they are, the more work someone is going to need to put in to configure their CI.
It would make more sense for the action to do that heavy lifting, as that's exactly what actions exist for.
from cache.
@mrmckeb Yes, I understand that, but do we create options for all ecosystems? That's not realistic.
from cache.
I don't disagree @chemitaxis, the product is new and a lot will need to change. I'm just pointing out that that the goal should be simplicity, and I'd be happy to contribute that code in to help.
from cache.
cache: paths: - node_modules/ - packages/*/node_modules/ - .yarn
Would a pr in that direction be considered? Working with node, you usually have a lot of caching directories (besides the quoted ones we might have some build cache, test cache(jest/cypress/whatever), lint cache, global package cache, ....
While caching them one by one is certainly possible, it's also very much bloated & unreadable. I'd appreciate solution(like the suggested one) where I would only have to type one line per cache folder, not 5-7 😅
I guess one could live without the glob matching, although it would be nice for workspaces & lerna.
from cache.
Rather than directly support different languages in the cache action, would it make more sense if the cache action could be used by other actions (#55)?
E.g. actions/[email protected]
could cache the deps and _build folders.
Then cache action only needs to be specified directly if caching folders outside of those that the language actions support.
from cache.
I've created a PR for this - see #180
from cache.
This will definitely be required for many open source projects. We want to trial this on some Storybook projects, but caching for a monorepo (Yarn) would be needed for that - along with #6.
But that's not to discourage this great first-step.
from cache.
This would be very important, for example if you build a web app for android in node, you will add node_modules and also gradle to the cache.
from cache.
Is anyone already working on this?
from cache.
I agree with @jcornaz.
from cache.
Is this proposal so far from your goal? #16 (comment)
from cache.
Well, I already say that in #70...
from cache.
@imbsky, that's a great start - but something like Yarn workspaces has more complex caching requirements.
from cache.
Is that something serious? What if the cache directory changes from a specific version in each ecosystem? I don't think it's necessary...
from cache.
Honestly, do you change it so often? I don't think the benefit equals the cost of maintaining it forever. I'm not saying that Node.js projects don't need caching (I also use.), but projects that are really large to need caching are usually not simple to specify in templates like that.
from cache.
The Actions team likes to keep things simple. and I completely agree with that. and if you need a Cache action that supports Monorepo, you can fork this repository or create it from scratch. but, it will take a lot of effort. I guess.
from cache.
Related Issues (20)
- cache restore on versions 3 and 4 not working HOT 6
- engineering finace tshingombe cash flow HOT 2
- Restore action doesn't actually fail workflow even when fail-on-cache-miss is true HOT 2
- N claims? failed Accusations and it is funny
- Using cache with git tags HOT 1
- 🎁 #SubscribersGift HOT 2
- Repo looks empty
- workarounds for cache size quotas/limits?
- zstdmt does not exist HOT 1
- Net efx ptions HOT 1
- Home | Task
- Failed to restore: getCacheEntry failed: Cache service responded with 500 HOT 5
- Signing certificate with thumbprint *** not found in store HOT 7
- Incomplete key restores arbitrary cache object
- > ## Description HOT 3
- Sinlee94 HOT 1
- [ HOT 4
- > > * > [2024-03-01T17:38:57.4236128Z ##[debug]Starting: Set up job
- > ## Description
- Duplicated cache due to concurrency HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cache.