Support for Sparse Group Lasso? about lightning HOT 13 OPEN

Hey, I don't have plans of implementing it in the short future but patches are welcomed :-)

from lightning.

from lightning.

Could you point me to some documentation or algorithm to follow the code? I've been looking into it (specifically the solve_l1l2 function) and I don't really get it, i was expecting something more like https://github.com/fabianp/group_lasso/blob/master/group_lasso.py but doesn't seem so. I think it's harder to follow because it's a generic solver for any objective function, isn't it?

from lightning.

Yes, in lightning there are multiple generic solvers. For sparse group lasso, you probably want to use coordinate descent or FISTA. So implementing the sparse group lasso amount to define the appropriate prox or update rule for these methods.

from lightning.

Okay thanks. Is there any difference in terms of performance between FISTA and coordinate descent? In the implementation, I mean, I see that coordinate descent is implemented using Cython while FISTA only in Python.

I'd say that both are pretty competitive, but I'm interested in large applications, will FISTA hold against the current coordinate descent implementation?

from lightning.

Simplifying a lot I would say that they both have similar performance, but this will depend depend on things like sparsity/number of features / strong convexity

from lightning.

from lightning.

I have a generic question: how do you indicate the groups to the classifier? I've been playing with this and I've implemented the Sparse Group Lasso but now that I am running some examples I don't know how to indicate which groups there are. Any pointers to doc or anything related?

from lightning.

For now we don't as the existing group lasso implementation considers that groups are equal to the coefficient associated with the different classes (is dependent of a multiclass formulation). I would add a parameter groups=[] to the class, of size n_features, where each entry specifies to which group the coefficient belongs to.

from lightning.

from lightning.

You are right, there isn't

from lightning.

Is there a plan to include it? I am playing with it but I'm afraid I may implement it in a non-optimal way, which would hurt performance on large scale applications.

Sorry for the persistence.

from lightning.

no plan for me to work on that in the short term.

from lightning.