Comments (17)
@sebastien-villemot, I didn't contribute to the Newton code, but this seems off. From the literature I've read, the line search step size should be reset to 1 each time.
from optim.jl.
In my problem, NLsolve
returns the error "Too many iterations in backtracking_linesearch!". So it does not seem like the best choice in all cases.
from optim.jl.
PRs accepted
from optim.jl.
Arguably we should remove backtracking_linesearch since it never works well in my experience.
from optim.jl.
Any progress on this?
from optim.jl.
Should we perhaps deprecate backtracking_linesearch?
from optim.jl.
I think we should benchmark the change before doing anything, but resetting to 1.0 seems like the right thing to do. I have other stuff to finish, and limited time, but if no one else grabs it, I'll have a look at some point.
from optim.jl.
please don't remove backtracking line search; I am just in the process of rewriting it. On my first tests, I actually beat HZ and MT line search with LBFGS.
from optim.jl.
Btw, I just notices that the same 1.0 problem is also in LBFGS.
from optim.jl.
Personally, I always use an extrapolation step, which defaults to 1.0 if it ends up near 1.0. This is my current implementation; just as a quick hack to try it out:
if iteration > 1
alphaguess = 2.0 * (f_x - f_x_previous) / dphi0
alphaguess = max(alphaguess, alpha/4.0)
alphaguess = min(alphaguess, 1.0)
if alphaguess > 0.75
alphaguess = 1.0
end
else
alphaguess = 1.0
end
On Rosenbrock example:
linesearch : F-evals / G-evals
HZ : 60 / 81
MT : 76 / 76
new BT : 45 / 72
(HZ is for ConjugateGradients, the rest for LBFGS)
from optim.jl.
some variant of this might even fix #166
from optim.jl.
@cortner where are we on this, was this fixed somewhere?
from optim.jl.
An improvement to backtracking was introduced as a non-default option here: JuliaNLSolvers/LineSearches.jl#7.
Further updates that change default behaviour of backtracking will be merged with JuliaNLSolvers/LineSearches.jl#20.
The Newton algorithm still seems to be lacking the resetting state.alpha = 1.0
step, which is the best default in the "good" regions for Newton.
from optim.jl.
But this is now a LineSearches.jl issue right? Or do you need something from optim ?
from optim.jl.
No - it is more complicated. What Linesearches.jl really needs is a refactoring to make linesearches tyoes rather than functions: this will definitely affect Optim.
from optim.jl.
See #294 and JuliaNLSolvers/LineSearches.jl#9
from optim.jl.
closed by #328
from optim.jl.
Related Issues (20)
- [Feature request] Adding Hill Climbing HOT 7
- Docs Latex expressions displays as raw HOT 4
- JuMP Interface, Incorrect Status Returned HOT 5
- Mac nightly fails for some reason
- IPNewton : start is not an interior point HOT 1
- CUDA and Adam errors HOT 3
- Incorrect referencing to ipnewton_basics.ipynb in Nonlinear constrained optimization HOT 1
- `Adam` supported by `Fminbox`? HOT 2
- Add progress meter HOT 1
- Documentation Incorrect HOT 2
- Simple optimization failing HOT 1
- Not all convergence criteria mentioned in documentation HOT 2
- `eval` in OptimMOIExt fails to precompile HOT 2
- Bounds Error in Neural Network Training Process HOT 1
- Expose more optimisation parameters to users HOT 4
- Extended trace for Adam fails
- Univariate minimization given function and derivatives HOT 1
- Problem with f_calls_limit in Fminbox
- Convergence of SAMIN is converted to non-convergence.
- Optim.OptimizationState HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from optim.jl.