skaftenicki / pl_crossvalidate Goto Github PK

Hey, I was just thinking about implementing k-fold support and saw this awesome library! Great work! Just wanted to drop you a note that I noticed that it currently doesn't work with PyTorch Lightning >= 1.6.0. It works fine with e.g., 1.5.10 but in 1.6.0 it throws an error when calling trainer.cross_validate():

It happens in both cases whether you use a datamodule or train_dataloader.

I'm having trouble installing pl_cross with poetry, I've open a question in StackOverflow describing the problem.

I noticed that there are also issues with the create_ensemble method complaining that the model argument is missing. I tried it with all possible settings providing a model only, providing checkpoints only, and providing a model and checkpoints. This issue occurs with PyTorch Lightning 1.5.10. I put together a self-contained example notebook here (https://github.com/rasbt/deeplearning-models/blob/master/pytorch-lightning_ipynb/kfold/kfold-light-cnn-mnist.ipynb) that reproduces the error if you uncomment the create_model code at the bottom.

Hi, thanks a lot for the pl_crossvalidate! I am trying to use but I keep running into some errors. The error I get is TypeError: An invalid dataloader was returned from KFoldDataModule.val_dataloader(). Found None. with trainer.cross_validate(model, train_dataloader=ds_loader).
The reason I go with train_dataloader is because I have a very small dataset and I cannot afford to split into train and val and and also num_folds. Do I need to have a val_dataloader in this case?