ROC AUC scores don't match to those of sklearn about hyperparameter_hunter HOT 1 OPEN

Sorry for the delay in responding to you, @abudis!
Thanks for opening this issue. Hopefully we can clear everything up!

Although you say the disparity is too large to be explained by random splits, in my experience random seeds often have a much larger impact on scores than expected. Aside from the random splits made during CV, many algorithms also accept random state parameters, which HH also assigns and records. So this may also be contributing to the difference between your scores.

This issue is rather old, but some of the sample code may help explain how HH does things behind the scenes. Also, keep in mind that average ROC-AUC scores are computed via micro-averaging the actual predictions, rather than averaging the scores themselves.

If the scores still don't seem to make sense, I'd love to see a minimal code example demonstrating the score mismatch you're seeing (preferably with a popular toy dataset for reproducibility). That way, we can determine the cause of the difference between your scores.

Thank you again for opening this issue, and once again, I apologize for taking so long to reply. Please let me know if you're having trouble coming up with some code to reproduce the problem, and we'll figure something else out.

from hyperparameter_hunter.