About comparison fairness and dataset splitting about awesome_imputation HOT 2 OPEN

Hi there 👋,

Thank you so much for your attention to PyPOTS! You can follow me on GitHub to receive the latest news of PyPOTS.
If you find our research is helpful to your work, please star⭐️ this repository. Your star is your recognition,
which can help more people notice PyPOTS and grow PyPOTS community.
It matters and is definitely a kind of contribution to the community.

I have received your message and will respond ASAP. Thank you for your patience! 😃

Best,
Wenjie

from awesome_imputation.

Hi @HowardZJU, thanks for raising the discussion here.

1. Splitting datasets chronologically will be necessary for all time-series modeling tasks if you want to use the model for future data. Splitting is not only a way to avoid data leakage but also to prevent overfitting (the two may stand for the same thing in some cases). In the time-series imputation field, there are two scenarios, i.e. in-sample and out-of-sample. For the in-sample case, as you've said, there is no necessity to split data chronologically. But for out-of-sample, it is vital because we need to ensure the generalization ability of the trained models. Obviously, we take the latter one in our experiments, not only because imputation generalization is important to deep learning algorithms, but also because out-of-sample imputation is commonly used in real-world applications;
2. In our current implementations of PyPOTS naive imputation methods, e.g. the mean imputer, they don't have the training stage. Hence, if you'd like to make them calculate empirical values on both the training set and the test set, you can merge the sets together to feed into the imputers;

You ask good questions and provide helpful insights here. We appreciate that. If you also work with POTS (partially-observed time series) data, we sincerely invite you to join our community and build PyPOTS better together ;-)

from awesome_imputation.