[QUESTION] Ridge regression cost function about handson-ml3 HOT 4 CLOSED

Hi @UPstartDeveloper, thank you for your comment. It really solves my original question about adding 1/m factor to the Ridge case.

On the other hand, why the difference between 2*alpha and alpha in case of Elastic net is necessary is still not quite clear. Probably it just works better that way. That is, more weight on the Lasso part makes models work better.

from handson-ml3.

For me it's always been a mysterious whether it would help calculate faster/better to use any kind of constant factor such as 2, 1/m or 1/2 because, I think, theta_i would adapt to it anyway.

from handson-ml3.

The confusing points makes suddenly sense when the Elastic Net is considered. The weight of alpha is different in each penalty.

Althought now the confusion is gone, it is still not so obvious why it works better that way.

from handson-ml3.

Hi @liganega, I think part of your answer was covered in this issue in the previous Hands-On ML repo.

from handson-ml3.