Comments (2)
I took one of the targets
examples and transferred it one-to-one to hamilton
to see how the concepts compare. Both workflows are implemented in modules and make use of helper functions from a separate module. Then both are started interactively from quarto documents and their results and graphs visualized in the rendered output of said notebooks: http://jmbuhr.de/targets-hamilton-comparison/ (source code: https://github.com/jmbuhr/targets-hamilton-comparison)
Again, this is for exploration of possibilities, not to impose paradigms on you :)
In this first pass I noticed two things I was missing in hamilton compared to targets when it comes to caching:
- changing a function that is used by a node, but is not itself a node, should also invalidated the cache of the node
- loading the cached result from any node independently of the
dr.execute
run as withtar_load(<name of node>)
(https://docs.ropensci.org/targets/reference/tar_load.html) is super helpful for interactively picking up where you left of with a workflow and working on different parts of it.
from hamilton.
For inspiration, the developer documentation of how targets
does caching might come in handy: https://books.ropensci.org/targets-design/data.html#skipping-up-to-date-targets
from hamilton.
Related Issues (20)
- save_to.parquet strips URI prefix when decorated from a resolve_from_config wrapper HOT 2
- Add helm chart for running Hamilton UI
- Add podman compatible instructions for running Hamilton UI
- Programmatic project creation for Hamilton UI
- Add google collab links for all notebooks HOT 2
- Enable outputs to access all inputs HOT 2
- `PandasExcelWriter` overwrites file
- Workflow stuck with parallelizable node HOT 2
- `Value Error` When Applying Multiple Data Validation Decorators to the Same Function HOT 7
- chore: Support notebooks in docs
- display `DataLoader` as materializer in viz HOT 3
- Enable Hamilton UI to run in google colab
- Misleading error reporting for `parameterize.validate` on `group` dependency with empty list
- GracefulErrorAdapter fails with Parallelizable HOT 6
- Schema tracker does not track column position
- Detect cycles at graph build time HOT 1
- Multiple result builders in async adapter
- AsyncDriver doesn't prune graph based on overrides HOT 1
- Include `injectable` in library comparison
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hamilton.