Include AdamW about lumin HOT 5 CLOSED

As AdamW is available in torch.optim (as mentioned in issue description), does the modification expect something more than adding 'adamW' after this as another optimizer choice?

from lumin.

Yes, adding 'adamw' to that part of the code would be enough to get it used by passing a string argument in the opt_args, however it might cause an error if a user tries to use it with a lower version of PyTorch (current requirement for lumin is torch > 1.0.0).

Either we could increase the minimum version to 1.2.0 (I'm currently running 1.5.1 and Lumin works fine), or there could be a check on the torch version to make sure that AdamW is present.

There should also be the recommendation to the user to use AdamW if they request Adam and L2 regularisation. This should probably go prior to this line
Additionally, these lines set the default optimiser to Adam, if no optimiser was specified. This could be improved to change the default optimiser to AdamW if no optimisre was specified but L2 regularisation was specified, and the torch version is high enough.

from lumin.

Alright, thanks for the pointers! I can add the changes along with the assertions/ warnings for the user. This could be a starting step, later we can just modify the requirements for PyTorch version >= 1.2.0 and mention in the readme as well.

from lumin.

That would be great, thank you!

from lumin.

@GilesStrong The updated model_builder module has AdamW included but other checks as discussed above are currently commented out. In case we use 'regularizer' as an argument, this will have to be changed accordingly.

from lumin.