Comments (18)
from tfx.
from tfx.
Any update on this? I'm building a similar image right now and could use this.
I believe that this idea has become obsolete as reported in that pr. If that is the situation maybe we could consider closing this issue.
from tfx.
While I liked this idea and have mentioned it from times to times, we need to determine how much we will keep a docker based tutorial similar to the step by step workshop . Specifically, if we want to preserve the step oriented progression, how will the workflow looks like for the user?
from tfx.
Also to think about how the development flow will look like if user started the tutorial w/ a simplified docker based tutorial, but decided to proceed to use TFX to solve their own problem (on their own data and code). We want to make sure that the tutorial serves as a step stone towards that.
from tfx.
I'm interested in working on this tutorial. Is anybody already working on it?
I believe that to solve the issues mentioned by the @zhitaoli would need to separate the installation section from the usage example. Thinking as a developer start using the application he could look for installation guide that would suit him best and then move on to the get started page. That way we could maintain the installation and get started documentation separately.
from tfx.
Searching in dockerhub I find that image (that I believe it's official). The image is prepared to run the run executor script. Since that the image is not referenced by the official documentation should I use it as a base ? I mean, that is a official image ?
If that is not recognized as a official, it's a goal of the project run the system in a docker image ?
from tfx.
Hi @Bumbleblo,
Yes this image is the official TFX image. It currently runs the run_executor script and will later be converted to a (WIP) run_component.py which runs driver/executor/
I think it's the right thing to extend this image (through "FROM tensorflow/tfx" docker file clause), install apache-airflow/mysql/etc necessary to run the demo, and create another image (i.e, "tensorflow/tfx:latest-airflow-demo"). I can help once we have this image built.
from tfx.
I already create a initial version for that image (that version is using sqlite). Now I'm trying to understand what is important in default setup script of the workshop.
Is there any other step that was not in that script?
from tfx.
from tfx.
Great! Can you post the Dockerfile for the initial version? IMHI, high level things we need from the default setup script should be: - install jupyter notebook; - install notebook related dependencies; - start notebook on a different port and expose that port to browser; - make sure pipeline data and sqlite DB file is on a volume (which can be mounted to the container); The part of copying python files around should be more or less optional as long as you can get Airflow running.
…
On Fri, Jun 28, 2019 at 7:32 AM Felipe Borges @.***> wrote: I already create a initial version for that image https://hub.docker.com/r/bumbleblo/tfx (that version is using sqlite). Now I'm trying to understand what is important in default setup script of the workshop https://github.com/tensorflow/tfx/blob/master/tfx/examples/workshop/setup/setup_demo.sh . With that complete I believe I can start writing the tutorial. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#51?email_source=notifications&email_token=AAY6AZVJWEBVH3G4WGFVPCDP4YOG7A5CNFSM4HH3TOU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY2HU6Q#issuecomment-506755706>, or mute the thread https://github.com/notifications/unsubscribe-auth/AAY6AZUR54DRGM7HH3RQGDLP4YOG7ANCNFSM4HH3TOUQ .
-- Cheers, Zhitao Li
Yes, that is the Initial Dockerfile
FROM tensorflow/tfx
# write that in a file later
RUN pip install --no-cache \
tensorflow==1.13.1 \
tfx==0.13.0 \
flask==1.0.3 \
apache-airflow
# removing base script
RUN rm -rf *
WORKDIR /home
RUN airflow initdb
COPY setup.sh .
ENTRYPOINT bash -c "./setup.sh"
setup.sh script:
#!/bin/bash
airflow webserver -p 8080 &
airflow scheduler
from tfx.
Recently a new release of Flask broke the Dockerfile above. I already did an update on the post with the repair
from tfx.
Great! Can you post the Dockerfile for the initial version? IMHI, high level things we need from the default setup script should be: - install jupyter notebook; - install notebook related dependencies; - start notebook on a different port and expose that port to browser; - make sure pipeline data and sqlite DB file is on a volume (which can be mounted to the container); The part of copying python files around should be more or less optional as long as you can get Airflow running.
…
On Fri, Jun 28, 2019 at 7:32 AM Felipe Borges @.***> wrote: I already create a initial version for that image https://hub.docker.com/r/bumbleblo/tfx (that version is using sqlite). Now I'm trying to understand what is important in default setup script of the workshop https://github.com/tensorflow/tfx/blob/master/tfx/examples/workshop/setup/setup_demo.sh . With that complete I believe I can start writing the tutorial. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#51?email_source=notifications&email_token=AAY6AZVJWEBVH3G4WGFVPCDP4YOG7A5CNFSM4HH3TOU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY2HU6Q#issuecomment-506755706>, or mute the thread https://github.com/notifications/unsubscribe-auth/AAY6AZUR54DRGM7HH3RQGDLP4YOG7ANCNFSM4HH3TOUQ .
-- Cheers, Zhitao Li
I pushed a new version of the image with that volumes (one for user data and other with airflow data). Also I already update the dockerhub page description with a new usage command. All files used for build that image it's here.
from tfx.
@Bumbleblo Thanks. I'll try the image out later today and provide feedback. Since I cannot comment on your branch, consider send a PR so I can comment?
from tfx.
For sure, I'll open a pull request for that docker files and link here.
from tfx.
Hey @zhitaoli, can you send me your feedback about the latest version of tutorial image ? I made some changes to python 3 and that new version is based in the Dockerfile on master branch.
from tfx.
Any update on this? I'm building a similar image right now and could use this.
from tfx.
Closing this as per the comment trace above.Please feel free to reopen if required.Thanks.
from tfx.
Related Issues (20)
- [Request] Update to Apache Beam 2.52.0, enable Beam 2.46.0 compatibility HOT 5
- How to pass airflow task configuration to one custom component? HOT 3
- Error executing pip install tfx in new conda environment with python 3.10 HOT 6
- installing tfx 1.13.0 by pip takes so much time HOT 5
- TFX trainer component running in Kubeflow fails although it was successful in the Interactive Context HOT 8
- TFX components in GCP does not display component logs in GCP Vertex AI HOT 17
- DataFlow Job in TFX pipeline fails after running for an hour HOT 6
- TFX component never completes even though Vertex AI custom job succeeds / fails HOT 8
- Upgrade Tensorflow version HOT 3
- documentations for driver class HOT 2
- Custom driver support for KubeflowV2DagRunner HOT 3
- Error when starting Evaluator component HOT 6
- TFX 1.15.0 Issues HOT 1
- R2Score Metric is incompatible with Evaluator Component HOT 2
- Version issues with Savemodel
- Version Issues with Estimator SaveModel
- Python-snappy not found during execution of CSVExampleGen HOT 1
- Dependency Version Constraints error in release notes for version 1.15.0 HOT 1
- TFX for small, single-laptop workflows HOT 4
- TFX 1.15 docker image contains conflicting dependencies HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tfx.