Overfitting because of duplicate feature-target entries in the sample about ppscore HOT 3 OPEN

Sorry, but I do not understand the intention of your issue.
Can you please go into more detail and provide a full code example?

Thank you,
Florian

from ppscore.

The DataFrame here is passed to the function _calculate_model_cv_score_ which can have lots of duplicate rows [i.e., feature-target pair] and those duplicates are not being removed at any further step. It causes overfitting and thus, the evaluation metric score is incorrect.

If you want to see those duplicates, add df.to_csv(r'C:\Users\username\Desktop\temp\\' + x + '__' + y + '.csv', index=False, encoding='utf-8') to save all dataframes locally and inspect [starting from those feature and target pairs that have low count of unique values.]

from ppscore.

So, do you propose to remove duplicates?

from ppscore.