Comments (2)
Hello, thanks I was pretty pleased with it too :)
There's some relevant discussion in google/jax#17107 (comment). Essentially we're not officially supporting it outside the GraphCast project for now, as it does have some rough edges (see the thread for some examples) and it is in part a bit of a stop-gap measure until JAX supports the new python array API standard which will allow it to integrate better with xarray.
If you wanted to speed things along here, one really valuable thing could be working with the JAX folks to push along support for the array API in JAX. From what I gather one of the remaining obstacles is figuring out how to handle the mutable parts of the array API, since JAX doesn't support mutation. (One thing I've done in JaxArrayWrapper here is just to drop / ignore the out=
kwargs which show up when ufuncs are run in-place, meaning it returns a fresh array and the mutation doesn't happen in-place. However this isn't perfect and it might be better to throw an error; I'm also not sure if the same approach would work for the new array API). IMO it would be OK just to omit support for in-place mutation.
@shoyer may be a good person to talk to about this (or might at least know who to ask) as he has a good visibility of both xarray and jax projects.
In the long run I'd definitely like to improve this and make a standalone release, although I was waiting for the array API implementation in JAX before investing a lot more in it as a stand-alone project, as it could change the approach required quite a bit, simplify it a bit and remove some (not all) of the rough edges.
If you wanted to maintain your own fork in the meantime I won't object and others may find it useful, although I can't promise we'd switch to or co-maintain it ourselves if that proved disruptive to our current research. By the nature of these things with slightly rough edges, if you make changes to the rough edges it can end up breaking things downstream in slightly subtle and annoying ways. (We could admittedly also use more test coverage for some of these things). But would be interested to follow what you do anyway and could maybe join up with it at some later date.
Hope that makes sense anyway and thanks for volunteering!
from graphcast.
Just coming back to this now -- thanks so much @mjwilson for your response!
Conforming to the Array API standard definitely makes sense before investing too much effort, so totally with you on that (and on not supporting mutation: seems like this isn't part of the array API standard anyway!). Had a shop round the existing issues, and it seems like there's a couple roadblocks here and there, but they looked a little stale. Will look at finding an entrypoint there, I think it would be great for this to go through (even without the xarray support!).
RE: maintaining a fork -- totally makes sense. We're not 100% concrete on our use case or if we'll have the engineering effort to maintain something, but will definitely be trialling the great work you've done here. If, however, you start to push on making this more mature at any point, I'd definitely be interested in lending a hand :)
Thanks again for your thorough answer, and hope to talk again on this! I'll leave the issue open for the time being since there's potential directions here, but feel free to close if you'd rather keep it clean until other things pop up.
from graphcast.
Related Issues (20)
- Predicting Forecast for 10 Days , 5 Days HOT 2
- Obtaining successive forecasts based on previous predictions HOT 2
- Are forcing variables repeated? HOT 2
- Haiku needs all `hk.Module` must be initialized inside an `hk.transform` HOT 1
- About loss weights HOT 3
- GPU / TPU memory requirements for training HOT 3
- [GraphCast Operational Model] Issue with Negative Precipitation Data in GraphCast Operational Model Output HOT 2
- How to get the data in the paper? HOT 1
- weights license - use of graphcast HOT 5
- Graphcast error on Mac os HOT 1
- Problems feeding data to operational model: Target variable geopotential_at_surface must be time-dependent HOT 1
- when is the prediction result of this demo? HOT 2
- Forecasting beyond 10 days HOT 8
- Cyclone tracking
- There are some questions about forecasting. HOT 1
- Fine-Tuning Strategy for the GraphCast Operational Model HOT 2
- About the atmospheric variable “Vertical velocity”
- about the autoregressive finetuning HOT 1
- How to train a model by myself HOT 6
- How get the value from model dataset, such as norm_prediction?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from graphcast.