$ python _scikeras_bug.py
...
/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/sklearn/utils/validation.py:73: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
return f(**kwargs)
/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/sklearn/utils/validation.py:73: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
return f(**kwargs)
/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/sklearn/utils/validation.py:73: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
return f(**kwargs)
/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/sklearn/utils/validation.py:73: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
return f(**kwargs)
2020-07-15 15:07:29.807015: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-07-15 15:07:29.813243: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-07-15 15:07:29.818382: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-07-15 15:07:29.819346: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f8a7c1f1fe0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-07-15 15:07:29.819367: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2020-07-15 15:07:29.826941: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f8b0b201b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-07-15 15:07:29.826962: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2020-07-15 15:07:29.837069: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fdc87b75290 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-07-15 15:07:29.837096: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/sklearn/utils/validation.py:73: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
return f(**kwargs)
2020-07-15 15:07:29.916084: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
distributed.utils - ERROR - 'NoneType' object is not iterable
Traceback (most recent call last):
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/distributed/utils.py", line 656, in log_errors
yield
File "/Users/scott/Developer/stsievert/dask-ml/dask_ml/model_selection/_incremental.py", line 103, in _partial_fit
model.partial_fit(X, y, **(fit_params or {}))
File "/Users/scott/Developer/stsievert/scikeras/scikeras/wrappers.py", line 547, in partial_fit
return self.fit(
File "/Users/scott/Developer/stsievert/scikeras/scikeras/wrappers.py", line 511, in fit
self.model_ = self._build_keras_model(
File "/Users/scott/Developer/stsievert/scikeras/scikeras/wrappers.py", line 304, in _build_keras_model
model = final_build_fn(**build_args)
File "_scikeras_bug.py", line 36, in _keras_build_fn
model = Sequential(layers)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/training/tracking/base.py", line 456, in _method_wrapper
result = method(self, *args, **kwargs)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/keras/engine/sequential.py", line 129, in __init__
self.add(layer)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/training/tracking/base.py", line 456, in _method_wrapper
result = method(self, *args, **kwargs)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/keras/engine/sequential.py", line 213, in add
output_tensor = layer(self.outputs[0])
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 960, in __call__
self._set_inputs(cast_inputs, outputs)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/contextlib.py", line 120, in __exit__
next(self.gen)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 418, in inner_cm
for fn in self._scope_exit_callbacks:
TypeError: 'NoneType' object is not iterable
distributed.utils - ERROR - 'NoneType' object is not iterable
Traceback (most recent call last):
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/distributed/utils.py", line 656, in log_errors
yield
File "/Users/scott/Developer/stsievert/dask-ml/dask_ml/model_selection/_incremental.py", line 103, in _partial_fit
model.partial_fit(X, y, **(fit_params or {}))
File "/Users/scott/Developer/stsievert/scikeras/scikeras/wrappers.py", line 547, in partial_fit
return self.fit(
File "/Users/scott/Developer/stsievert/scikeras/scikeras/wrappers.py", line 511, in fit
self.model_ = self._build_keras_model(
File "/Users/scott/Developer/stsievert/scikeras/scikeras/wrappers.py", line 304, in _build_keras_model
model = final_build_fn(**build_args)
File "_scikeras_bug.py", line 36, in _keras_build_fn
model = Sequential(layers)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/training/tracking/base.py", line 456, in _method_wrapper
result = method(self, *args, **kwargs)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/keras/engine/sequential.py", line 129, in __init__
self.add(layer)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/training/tracking/base.py", line 456, in _method_wrapper
result = method(self, *args, **kwargs)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/keras/engine/sequential.py", line 213, in add
output_tensor = layer(self.outputs[0])
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 960, in __call__
self._set_inputs(cast_inputs, outputs)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/contextlib.py", line 120, in __exit__
next(self.gen)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 418, in inner_cm
for fn in self._scope_exit_callbacks:
TypeError: 'NoneType' object is not iterable
2020-07-15 15:07:29.932341: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fb0b125cc00 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-07-15 15:07:29.932392: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
distributed.worker - WARNING - Compute Failed
Function: execute_task
args: ((<function _partial_fit at 0x14b23d550>, (KerasClassifier(
build_fn=<function _keras_build_fn at 0x14b465160>
lr=0.0018805049416974935
), {'model_id': 2, 'params': {'lr': 0.0018805049416974935}, 'partial_fit_calls': 0}), array([[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]], dtype=float32), array([4, 0, 2, 1, 3, 8, 5, 0, 3, 1, 2, 7, 9, 1, 3, 4, 7, 7, 1, 8, 6, 3,
0, 9, 9, 4, 4, 5, 9, 0, 7, 6, 7, 6, 0, 0, 6, 8, 6, 3, 1, 9, 8, 1,
7, 5, 3, 8, 6, 4, 4, 9, 4, 2, 9, 9, 2, 0, 4, 0, 7, 1, 3, 7, 1, 1,
6, 3, 1, 2, 8, 6, 5, 2, 4, 7, 9, 9, 0, 8], dtype=uint8), (<class 'dict'>, [])))
kwargs: {}
Exception: TypeError("'NoneType' object is not iterable")
distributed.utils - ERROR - Expected float32 passed to parameter 'b' of op 'MatMul', got <tf.Variable 'dense_1_2/kernel:0' shape=(512, 10) dtype=float32> of type 'ResourceVariable' instead. Error: 'NoneType' object is not iterable
Traceback (most recent call last):
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/framework/op_def_library.py", line 465, in _apply_op_helper
values = ops.convert_to_tensor(
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 1341, in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 1825, in _dense_var_to_tensor
return var._dense_var_to_tensor(dtype=dtype, name=name, as_ref=as_ref) # pylint: disable=protected-access
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 1242, in _dense_var_to_tensor
return self.value()
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 550, in value
return self._read_variable_op()
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 645, in _read_variable_op
result = read_and_set_handle()
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 635, in read_and_set_handle
result = gen_resource_variable_ops.read_variable_op(self._handle,
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/ops/gen_resource_variable_ops.py", line 482, in read_variable_op
_, _, _op, _outputs = _op_def_library._apply_op_helper(
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/framework/op_def_library.py", line 759, in _apply_op_helper
return output_structure, op_def.is_stateful, op, outputs
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/contextlib.py", line 120, in __exit__
next(self.gen)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 418, in inner_cm
for fn in self._scope_exit_callbacks:
TypeError: 'NoneType' object is not iterable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/distributed/utils.py", line 656, in log_errors
yield
File "/Users/scott/Developer/stsievert/dask-ml/dask_ml/model_selection/_incremental.py", line 103, in _partial_fit
model.partial_fit(X, y, **(fit_params or {}))
File "/Users/scott/Developer/stsievert/scikeras/scikeras/wrappers.py", line 547, in partial_fit
return selTraceback (most recent call last):
File "_scikeras_bug.py", line 65, in <module>
search.fit(X, y) # fails on this line
File "/Users/scott/Developer/stsievert/dask-ml/dask_ml/model_selection/_incremental.py", line 1006, in fit
f.fit(
File "/Users/scott/Developer/stsievert/scikeras/scikeras/wrappers.py", line 511, in fit
self.model_ = self._build_keras_model(
File "/Users/scott/Developer/stsievert/scikeras/scikeras/wrappers.py", line 304, in _build_keras_model
model = final_build_fn(**build_args)
File "_scikeras_bug.py", line 36, in _keras_build_fn
model = Sequential(layers)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/training/tracking/base.py", line 456, in _method_wrapper
result = method(self, *args, **kwargs)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/keras/engine/sequential.py", line 129, in __init__
self.add(layer)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/training/tracking/base.py", line 456, in _method_wrapper
result = method(self, *args, **kwargs)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensor return super(IncrementalSearchCV, self).fit(X, y=y, **fit_params)
File "/Users/scott/Developer/stsievert/dask-ml/dask_ml/model_selection/_incremental.py", line 695, in fit
return client.sync(self._fit, X, y, **fit_params)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/distributed/client.py", line 831, in sync
flow/python/keras/engine/sequential.py", line 213, in add
output_tensor = layer(self.outputs[0])
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 922, in __call__
outputs = call_fn(cast_inputs, *args, **kwargs)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/keras/layers/core.py", line 1194, in call
outputs = gen_math_ops.mat_mul(inputs, self.kernel)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/ops/gen_math_ops.py", line 5585, in mat_mul
_, _, _op, _outputs = _op_def_library._apply_op_helper(
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/framework/op_def_library.py", line 475, in _apply_op_helper
raise TypeError(
TypeError: Expected float32 passed to parameter 'b' of op 'MatMul', got <tf.Variable 'dense_1_2/kernel:0' shape=(512, 10) dtype=float32> of type 'ResourceVariable' instead. Error: 'NoneType' object is not iterable
return sync(
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/distributed/utils.py", line 339, in sync
raise exc.with_traceback(tb)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/distributed/utils.py", line 323, in f
result[0] = yield future
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tornado/gen.py", line 735, in run
value = future.result()
File "/Users/scott/Developer/stsievert/dask-ml/dask_ml/model_selection/_incremental.py", line 641, in _fit
results = await fit(
File "/Users/scott/Developer/stsievert/dask-ml/dask_ml/model_selection/_incremental.py", line 457, in fit
return await _fit(
File "/Users/scott/Developer/stsievert/dask-ml/dask_ml/model_selection/_incremental.py", line 253, in _fit
metas = await client.gather(new_scores)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/distributed/client.py", line 1841, in _gather
raise exception.with_traceback(traceback)
File "/Users/scott/Developer/stsievert/dask-ml/dask_ml/model_selection/_incremental.py", line 103, in _partial_fit
model.partial_fit(X, y, **(fit_params or {}))
File "/Users/scott/Developer/stsievert/scikeras/scikeras/wrappers.py", line 547, in partial_fit
return self.fit(
File "/Users/scott/Developer/stsievert/scikeras/scikeras/wrappers.py", line 511, in fit
self.model_ = self._build_keras_model(
File "/Users/scott/Developer/stsievert/scikeras/scikeras/wrappers.py", line 304, in _build_keras_model
model = final_build_fn(**build_args)
File "_scikeras_bug.py", line 36, in _keras_build_fn
model = Sequential(layers)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/training/tracking/base.py", line 456, in _method_wrapper
result = method(self, *args, **kwargs)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/keras/engine/sequential.py", line 129, in __init__
self.add(layer)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/training/tracking/base.py", line 456, in _method_wrapper
result = method(self, *args, **kwargs)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/keras/engine/sequential.py", line 213, in add
output_tensor = layer(self.outputs[0])
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 960, in __call__
self._set_inputs(cast_inputs, outputs)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/contextlib.py", line 120, in __exit__
next(self.gen)
File "/Users/scott/anaconda3/envs/dask-ml-test/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 418, in inner_cm
for fn in self._scope_exit_callbacks:
TypeError: 'NoneType' object is not iterable