Is support for k-fold cross-validation on the roadmap? about tfx HOT 6 CLOSED

Currently for ExampleGen, you are able to customize the input and output split doc, but downstream components only support train and eval split for now

We don't have a offical support yet, but you can customize the Trainer component for the k-fold and train/evaluation/validation splits, you can think tfx eval split is the validation split you mentioned, and tfx train split is the train/evaluation you mentioned, in custom Trainer component, convert the train split as train/evaluation and do k-fold for tuning the model

I will keep this issue as a feature request

from tfx.

I am gonna close this issue as it has been resolved. Please add additional comments and we can open this issue again. Thanks!

from tfx.

It is not clear to me how this issue has been resolved. Can someone advise?

from tfx.

Hi, Currently custom split is fully supported for examplegen and its downstream, you should be able to utilize the splits in GenericTrainer's run_fn based on the needs

from tfx.

I consume a balanced dataset (50% positive/50% negative labels for binary classification), what I need is the equivalent of scikit-learn's StratifiedKFold, which ensures every split remains balanced. At the moment this does not seem possible and my splits are heavily imbalanced after splitting. Also, I don't think the partition_feature_name would help this use case.

from tfx.

I see, our split is just random split, so it might because the dataset is relatively small thus the results are not perfectly balanced as input.

Tensorflow dataset has an api to split data, but I'm not sure if that will guarantee the balance. If that works, you can just pass a single train split to GenericTrainer, and in the run_fn, split the dataset and do the cross validation.

from tfx.