Comments (14)
@zhitaoli do you have any ideas as to why this happened?
from tfx.
from tfx.
I got the same error using the setup.py libraries:
import setuptools
# LINT.IfChange
TF_VERSION = '1.12.0'
# LINT.ThenChange(train_mlengine.sh, start_model_server_mlengine.sh)
# LINT.IfChange
BEAM_VERSION = '2.11.0'
# LINT.ThenChange(setup_beam_on_flink.sh)
if __name__ == '__main__':
setuptools.setup(
name='tfx_chicago_taxi',
version='0.12.0',
packages=setuptools.find_packages(),
install_requires=[
'apache-beam[gcp]==' + BEAM_VERSION,
'jupyter==1.0',
'numpy==1.14.5',
'protobuf==3.6.1',
'tensorflow==' + TF_VERSION,
'tensorflow-data-validation==0.12.0',
'tensorflow-metadata==0.12.1',
'tensorflow-model-analysis==0.12.1',
'tensorflow-serving-api==1.12.0',
'tensorflow-transform==0.12.0',
],
python_requires='>=2.7,<3')
Traceback (most recent call last):
File "tfdv_analyze_and_validate.py", line 191, in <module>
main()
File "tfdv_analyze_and_validate.py", line 176, in main
pipeline_args=pipeline_args)
File "tfdv_analyze_and_validate.py", line 121, in compute_stats
statistics_pb2.DatasetFeatureStatisticsList)))
File "/Users/gogasca/Documents/Development/dpe/tfx_taxi_cloud/taxi/lib/python2.7/site-packages/apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "/Users/gogasca/Documents/Development/dpe/tfx_taxi_cloud/taxi/lib/python2.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", line 1238, in wait_until_finish
(self.state, getattr(self._runner, 'last_error_msg', None)), self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 773, in run
self._load_main_session(self.local_staging_directory)
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 489, in _load_main_session
pickler.load_session(session_file)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/internal/pickler.py", line 269, in load_session
return dill.load_session(file_path)
File "/usr/local/lib/python2.7/dist-packages/dill/_dill.py", line 410, in load_session
module = unpickler.load()
File "/usr/lib/python2.7/pickle.py", line 864, in load
dispatch[key](self)
File "/usr/lib/python2.7/pickle.py", line 1139, in load_reduce
value = func(*args)
File "/usr/local/lib/python2.7/dist-packages/dill/_dill.py", line 828, in _import_module
return getattr(__import__(module, None, None, [obj]), obj)
File "/usr/local/lib/python2.7/dist-packages/trainer/taxi.py", line 19, in <module>
from tensorflow_transform import coders as tft_coders
File "/usr/local/lib/python2.7/dist-packages/tensorflow_transform/__init__.py", line 19, in <module>
from tensorflow_transform.analyzers import *
File "/usr/local/lib/python2.7/dist-packages/tensorflow_transform/analyzers.py", line 39, in <module>
from tensorflow_transform import tf_utils
File "/usr/local/lib/python2.7/dist-packages/tensorflow_transform/tf_utils.py", line 24, in <module>
from tensorflow.contrib.proto.python.ops import encode_proto_op
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/__init__.py", line 48, in <module>
from tensorflow.contrib import distribute
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/distribute/__init__.py", line 34, in <module>
from tensorflow.contrib.distribute.python.tpu_strategy import TPUStrategy
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/distribute/python/tpu_strategy.py", line 27, in <module>
from tensorflow.contrib.tpu.python.ops import tpu_ops
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/tpu/__init__.py", line 73, in <module>
from tensorflow.contrib.tpu.python.tpu.keras_support import tpu_model as keras_to_tpu_model
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/tpu/python/tpu/keras_support.py", line 71, in <module>
from tensorflow.python.estimator import model_fn as model_fn_lib
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/estimator/__init__.py", line 25, in <module>
import tensorflow.python.estimator.estimator_lib
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/estimator/estimator_lib.py", line 22, in <module>
from tensorflow.python.estimator.canned.baseline import BaselineClassifier
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/estimator/canned/baseline.py", line 50, in <module>
from tensorflow.python.estimator import estimator
ImportError: cannot import name estimator
This is when running latest tfdv_analyze_and_validate_dataflow.sh
from tfx.
@zhitaoli I'm using the github master branch version. As for the pipeline, I'm using the one constructed in tfdv_analyze_and_validate_dataflow.sh and running the file with the command:
python tfdv_analyze_and_validate.py \
--input bigquery-public-data.chicago_taxi_trips.taxi_trips \
--infer_schema \
--stats_path ${TFDV_OUTPUT_PATH}/train_stats.tfrecord \
--schema_path ${SCHEMA_PATH} \
--project ${MYPROJECT} \
--region us-central1 \
--temp_location ${TEMP_PATH} \
--experiments shuffle_mode=auto \
--job_name ${JOB_ID} \
--setup_file ./setup.py \
--save_main_session \
--runner DataflowRunner \
--max_rows=${MAX_ROWS}
from tfx.
Using:
import setuptools
# LINT.IfChange
TF_VERSION = '1.13.1'
# LINT.ThenChange(train_mlengine.sh, start_model_server_mlengine.sh)
# LINT.IfChange
BEAM_VERSION = '2.12.0'
# LINT.ThenChange(setup_beam_on_flink.sh)
if __name__ == '__main__':
setuptools.setup(
name='tfx_chicago_taxi',
version='0.13.0',
packages=setuptools.find_packages(),
install_requires=[
'apache-beam[gcp]==' + BEAM_VERSION,
'jupyter==1.0',
'protobuf==3.7.1',
'tensorflow==' + TF_VERSION,
'tensorflow-data-validation==0.13.1',
'tensorflow-metadata==0.13.0',
'tensorflow-model-analysis==0.13.2',
'tensorflow-serving-api==1.13.0',
'tensorflow-transform==0.13.0',
],
python_requires='>=2.7,<3')
In a Python 2.7 virtualenv seems to be working now
from tfx.
@gogasca It worked for me too, thanks! What do we do now? Do you want to submit a PR updating the dependencies for chicago_taxi, or should I do it?
Once again, great find
from tfx.
There should be a submit from @ruoyu90 soon which will fix this. Thanks for the help.
from tfx.
This PR seems to include changes as well #91
from tfx.
This PR seems to include changes as well #91
Yes this is the PR I mentioned. Waiting for @ruoyu90 to submit.
from tfx.
Thanks, just started training and looks like just some minor updates in chicago_taxi_client.py, task.py and model.py needed in order to avoid import errors.
from tfx.
@ruoyu90 It is necessary to limit the version of scikit-learn to 0.20 in setup.py of the chicago_taxi (and probably in more places), as version 0.21.0 requires python 3.5 since today
source
from tfx.
@zhitaoli I also seem to have a problem with the try-except
construct on Dataflow - even if it's included, my Dataflow runner fails with the same ImportError. Do you have any idea why can this happen?
from tfx.
@mwalenia thanks for the notice! We are close to release 0.13 which will have Python 3.5 support.
from tfx.
@ruoyu90 Hi, when can I expect a fix for this issue? #91 seems to be stagnant, should I submit my own PR?
from tfx.
Related Issues (20)
- [Request] Update to Apache Beam 2.52.0, enable Beam 2.46.0 compatibility HOT 5
- How to pass airflow task configuration to one custom component? HOT 3
- Error executing pip install tfx in new conda environment with python 3.10 HOT 6
- installing tfx 1.13.0 by pip takes so much time HOT 5
- TFX trainer component running in Kubeflow fails although it was successful in the Interactive Context HOT 8
- TFX components in GCP does not display component logs in GCP Vertex AI HOT 17
- DataFlow Job in TFX pipeline fails after running for an hour HOT 6
- TFX component never completes even though Vertex AI custom job succeeds / fails HOT 8
- Upgrade Tensorflow version HOT 3
- documentations for driver class HOT 2
- Custom driver support for KubeflowV2DagRunner HOT 3
- Error when starting Evaluator component HOT 6
- TFX 1.15.0 Issues HOT 1
- R2Score Metric is incompatible with Evaluator Component HOT 2
- Version issues with Savemodel
- Version Issues with Estimator SaveModel
- Python-snappy not found during execution of CSVExampleGen HOT 1
- Dependency Version Constraints error in release notes for version 1.15.0 HOT 1
- TFX for small, single-laptop workflows HOT 4
- TFX 1.15 docker image contains conflicting dependencies HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tfx.