Support for regression models? about model-analysis HOT 9 CLOSED

You can add metrics you want to the canned Estimators using tf.estimator.add_metrics

from model-analysis.

Sorry, GitHub doesn't send notifications for edits, so I didn't realise you had made an update.

The predictions argument is the same as the predictions dictionary in the EstimatorSpec returned by your model_fn. It looks like it's a dictionary containing a single key named predictions, so you probably want to do mean_absolute_error(labels, predictions['predictions']).

from model-analysis.

Do you have any info on this? Are we missing something or is it not supported yet?

from model-analysis.

Yes, all canned Estimators are supported. Did you run into problems trying to get a regression model to work?

from model-analysis.

We have access only to average_loss, label/mean, post_export_metrics/example_count, prediction/mean. Can we have access to regression tf.metrics like mae, mse, rmse, mcd ?

EDIT :
I found this in the DNNRegressor.evaluate() doc :

For canned estimators, the dict contains the loss (mean loss per mini-batch) and the average_loss (mean loss per sample). Canned classifiers also return the accuracy. Canned regressors also return the label/mean and the prediction/mean

I also learned that DNNRegressor use L2 loss function, so the average_loss on 1 example is a mse. Can you (or is it the tf.estimator team?) provide off-the-shelf mae, rmse? Thanks

from model-analysis.

It's exactly what I want, thanks! Sorry for the inconvenience.

EDIT : Well I can't seem to get it work.
I do :
def my_mae(labels, predictions): return {'mean_absolute_error': tf.metrics.mean_absolute_error(labels, predictions)}
estimator = tf.estimator.add_metrics(estimator, my_mae)

And I get the following error:

ERROR - Failed to convert object of type <type 'dict'> to Tensor. Contents: {'predictions': <tf.Tensor 'dnn/logits/BiasAdd:0' shape=(?, 1) dtype=float32>}. Consider casting elements to a supported type.

I tried predictions['logits'], predictions['activation'] (similarly to the predictions['logistics'] for a classifier in the doc) without success. Can you help me? Thanks!

from model-analysis.

Hello, any updates on my edit? I can't find any info on the internet. Thanks!

from model-analysis.

It works, thanks!

from model-analysis.

Closing the issue due to the lack of recent activity. Please feel free to add additional information and open the issue again. Thanks!

from model-analysis.