Transforming date values about tfx HOT 10 CLOSED

Everything @Efaq Says above is correct: this should be done with Transform and must be done with TensorFlow ops (not py_func) so that it can be done in serving.

Please file an issue in the tensorflow/transform repo, and we can further discuss the exact functionality required.

from tfx.

Same issue here!

from tfx.

IIUC this is about converting 1 column into several features. You can use the transform component to do this by putting your logic into preprocessing_fn like our chicago taxi example.

from tfx.

For more information on using preprocessing_fn within Tensorflow Transform, please check Preprocessing function example here.

from tfx.

@ruoyu90 I think the issue goes through graph operations with dates, more than just converting one column into three. So:

• task: given two date columns, generate a new column with the difference between them in days.
• if we could use some python library (datetime for example), it would be trivial. Without the library, we would need to implement the knowledge about the calendar (number of days in each month etc)
• I believe we cannot use a conventional python library because if we use it, the transformation would not be written to the graph, and thus we would not be able to have it at serving time.

If there is no way such operations to the graph, then as mentioned above we would need to implement a piece of pipeline transforming the data both before training and before serving, outside of the graph.

Does it make sense, or am I missing something here?

from tfx.

@tensorflow/transform-team please take a look.

from tfx.

Thanks for all of your replies. I will post an issue in the transform repo.

from tfx.

Any update on this?

from tfx.

Any update on these features? They really would be great to have!

from tfx.

Since tfx isn't supporting this, here is a workaround. You can use https://www.tensorflow.org/addons/api_docs/python/tfa/text/parse_time to parse the string to unix date time, and then derive temporal features from the number of seconds since 1970. I would almost be tempted to have a utility function that takes as input a list of columns that are date time and transforms those features to be unix time. I'll likely need to write this, and when I do, I'll post it here.

from tfx.