Comments (9)
AsafManela
commented on June 15, 2024
This sounds reasonable from the user's perspective, but I'm not sure the coordinate descent algorithm will work properly in that case. Does glmnet in R or any other package provide this option so you could test against?
from lasso.jl.
barankarakus
commented on June 15, 2024
sklearn in Python does. Why would co-ordinate descent not work properly?
Get Outlook for iOS<https://aka.ms/o0ukef>
…________________________________
From: Asaf Manela <[email protected]>
Sent: Tuesday, August 18, 2020 3:02:44 PM
To: JuliaStats/Lasso.jl <[email protected]>
Cc: barankarakus <[email protected]>; Author <[email protected]>
Subject: Re: [JuliaStats/Lasso.jl] Fitting a LassoModel with a particular choice of lambda (#49)
This sounds reasonable from the user's perspective, but I'm not sure the coordinate descent algorithm will work properly in that case. Does glmnet in R or any other package provide this option so you could test against?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<#49 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIFDPKU2F64XBMGVGAAL3WTSBKCYJANCNFSM4QCDAYGQ>.
from lasso.jl.
AsafManela
commented on June 15, 2024
It works by starting from a null model (high lambda) then gradually decreasing it with small changes to coefficients. I'm just not sure how it would work if you solved directly for a specific lambda, though the API currently seems to support it.
I suggest you first experiment a bit with providing a vector with a single lambda and see what it does and how it compares with what sklearn gives in that case. It's been a while since I looked at this code carefully, so it may just work.
from lasso.jl.
barankarakus
commented on June 15, 2024
Ah, I see. I’m guessing that, given a sequence of lambdas, the algorithm solves the problem for the largest lambda (more precisely, for lambda_max, the smallest lambda giving rise to the null model), and then solves for the next largest lambda, using the solution of the current problem as an initial iterate... until we’ve solved the problem for all the lambdas.
If this is accurate, then I’m guessing that feeding it just one lambda value would simply apply coordinate descent for that one problem, with initial iterate the zero vector. This is what sklearn does too.
Are you happy with me running some tests and, if all looks good, submit a PR? I imagine an API like this:
fit(LassoModel, X, y; lambda=1.0, alpha=1.0, <possibly some other options like randomising coefficient selection or not>)
Get Outlook for iOS<https://aka.ms/o0ukef>
…________________________________
From: Asaf Manela <[email protected]>
Sent: Tuesday, August 18, 2020 3:16:09 PM
To: JuliaStats/Lasso.jl <[email protected]>
Cc: barankarakus <[email protected]>; Author <[email protected]>
Subject: Re: [JuliaStats/Lasso.jl] Fitting a LassoModel with a particular choice of lambda (#49)
It works by starting from a null model (high lambda) then gradually decreasing it with small changes to coefficients. I'm just not sure how it would work if you solved directly for a specific lambda, though the API currently seems to support it.
I suggest you first experiment a bit with providing a vector with a single lambda and see what it does and how it compares with what sklearn gives in that case. It's been a while since I looked at this code carefully, so it may just work.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<#49 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIFDPKU4RFJPDBUYM2C46BLSBKEKTANCNFSM4QCDAYGQ>.
from lasso.jl.
AsafManela
commented on June 15, 2024
Sounds good.
There may also be a way to use the select option to get what you want too, by implementing a segment selector (SegSelect) for that single lambda. Its segselect() would just give 1 always. This may or may not be simpler.
from lasso.jl.
barankarakus
commented on June 15, 2024
Sure, but we don’t want to be fitting an entire path then selecting the particular segment when we just need the problem solved for that particular segment.
That said, even in this scenario it may be faster to fit a path: choosing a sequence of lambdas lambda_0 = lambda < lambda_1 < ... < lambda_n = lambda_max (where lambda is the value we’re interested in solving the problem for), then solving the problem for lambda_n, lambda_{n-1}, ... in sequence. Though I think I’ll just keep it simple and solve the problem for lambda only.
Get Outlook for iOS<https://aka.ms/o0ukef>
…________________________________
From: Asaf Manela <[email protected]>
Sent: Tuesday, August 18, 2020 3:40:38 PM
To: JuliaStats/Lasso.jl <[email protected]>
Cc: barankarakus <[email protected]>; Author <[email protected]>
Subject: Re: [JuliaStats/Lasso.jl] Fitting a LassoModel with a particular choice of lambda (#49)
Sounds good.
There may also be a way to use the select option to get what you want too, by implementing a segment selector (SegSelect) for that single lambda. Its segselect() would just give 1 always. This may or may not be simpler.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<#49 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIFDPKX4TWHSEURKD43CIEDSBKHGNANCNFSM4QCDAYGQ>.
from lasso.jl.
BigBen-L
commented on June 15, 2024
Hey guys, have you considered about using LARS to solve Lasso problems?
barankarakus <[email protected]> 于2020年8月18日周二 上午5:34写道：
… It seems to me that the Lasso model-fitting API is not designed for the
user to simply pick a particular choice of lambda and fit a model with that
particular choice. Having looked at the source code I can see that the way
to do this would be to fit a LassoPath and specify the chosen lambda value
as a vector containing only that value. This is a bit clunky: I only want
to fit one model yet I'm using the LassoPath API.
I suggest having a fit(LassoModel, *some stuff*; \lambda) method which
allows to do this. It should fit a LassoPath only at the chosen lambda
value, then construct the LassoModel object from this. We could also give
it a default value of 1.0.
Maintainers of this project: If you give me the green light, I can go
ahead and submit a PR.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#49>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AMRJPWYD3HTS7XANYX2TM23SBGA57ANCNFSM4QCDAYGQ>
.
from lasso.jl.
barankarakus
commented on June 15, 2024
I’ve never used it and don’t know the algorithm in detail, but to the best of my knowledge the time complexity does not scale well with the number of features and the precision required of the solutions.
Moreover, I don’t think the LARS algorithm can take advantage of so-called ‘screening rules’ which (I believe) the glmnet algorithm utilises; these can provide significant speed-ups.
Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: BigBen-L <[email protected]>
Sent: Wednesday, August 19, 2020 12:26:20 AM
To: JuliaStats/Lasso.jl <[email protected]>
Cc: barankarakus <[email protected]>; Author <[email protected]>
Subject: Re: [JuliaStats/Lasso.jl] Fitting a LassoModel with a particular choice of lambda (#49)
Hey guys, have you considered about using LARS to solve Lasso problems?
barankarakus <[email protected]> 于2020年8月18日周二 上午5:34写道：
It seems to me that the Lasso model-fitting API is not designed for the
user to simply pick a particular choice of lambda and fit a model with that
particular choice. Having looked at the source code I can see that the way
to do this would be to fit a LassoPath and specify the chosen lambda value
as a vector containing only that value. This is a bit clunky: I only want
to fit one model yet I'm using the LassoPath API.
I suggest having a fit(LassoModel, *some stuff*; \lambda) method which
allows to do this. It should fit a LassoPath only at the chosen lambda
value, then construct the LassoModel object from this. We could also give
it a default value of 1.0.
Maintainers of this project: If you give me the green light, I can go
ahead and submit a PR.
―
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#49>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AMRJPWYD3HTS7XANYX2TM23SBGA57ANCNFSM4QCDAYGQ>
.
―
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<#49 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIFDPKVOBBI7JC3VQ23ULBTSBMEZZANCNFSM4QCDAYGQ>.
from lasso.jl.
GabrielPonte
commented on June 15, 2024
do you have any updates related to this? thanks!
from lasso.jl.
Related Issues (20)
- compats are outdated
- Intermittent Bounds Error HOT 3
- Accept weights that are not `AbstractVector{<:AbstractFloat}` HOT 3
- possible test failure in upcoming Julia version 1.5
- Package fails when response is zero vector.
- Bug: BoundsError: attempt to access 200-element Array{Float64,1} at index [201] HOT 2
- TagBot trigger issue HOT 6
- Bug: Bounds error thrown for small regularization parameters
- please tag a new release for Pkg resolver HOT 5
- 0.6.3 precompilation error
- upgrades and documentation HOT 1
- preferred way of saving model HOT 1
- Document maximum iterations settings. HOT 1
- Declare error exception or throw warning instead of just using error()
- Model type LassoModel doesn't support intercept HOT 1
- StackOverflow with type Vector{Union{Float64, Missing}} even if no missing values present
- matrix is not square,how to fix it? HOT 2
- Divergence from GLMnet when using a matrix with many variables HOT 2
- Improve documentation HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lasso.jl.