kthohr / optim Goto Github PK

./configure -i "usr/local" -l eigen -p
return
Summary:

• OS: linux-gnu

• C++ compiler: g++

• Build version: release

• OPTIM_LINEAR_ALG_LIB set to: eigen

• OPTIM_MATLIB_INCLUDE_PATH set to:

• BLAS and Lapack libraries set to:
  -lblas -llapack

• OpenMP features: enabled

• optimization flags:
  -fPIC -O3 -march=native -ffp-contract=fast -flto -DNDEBUG -fopenmp

• OptimLib install path:
  usr/local

• Additional notes:

Configuration completed. Creating Makefile... done.

but when I run make, it fails, return
g++ -std=c++11 -Wall -fPIC -O3 -march=native -ffp-contract=fast -flto -DNDEBUG -fopenmp -DOPTIM_ENABLE_EIGEN_WRAPPERS -I -I./include src/line_search/more_thuente.cpp -c -o src/line_search/more_thuente.o
src/line_search/more_thuente.cpp:27:10: fatal error: optim.hpp: no such file
#include "optim.hpp"
^~~~~~~~~~~
compilation terminated.
make: *** [Makefile:54：src/line_search/more_thuente.o]

The Ackley function example in the readme is very helpful for testing a fresh installation and getting started. Especially for people who have never used Armadillo before.

It would be nice to have something for Eigen as well. A simple example, without autodiff. Maybe even an equivalent Ackley function example but with Eigen?

Hi,

I'm trying to collect info on different numerical optimization libraries in C++ and I just came across yours.
Do you have an implementation of the Levenberg Marquardt algorithm in your library?

Thank you

Ive noticed theres a (log) transformation of the bounds which causes my parameters to become negative, whereby my parameters are strictly positive and Ive implemented this transformation external to the optmizer.
In lbfgs.hpp

// Line 200
    if (vals_bound) { // should we transform the parameters?
        x = transform(x, bounds_type, lower_bounds, upper_bounds);
    }

I mean sure, I could comment it out, but is there an external way to disable this? without having to change the source code?
Thanks

Hello,

when I want multithreaded parallel function evaluations for 8 Threads i do this:
#define OPTIM_USE_OMP
omp_set_dynamic(false); // omp setting
omp_set_num_threads(8); // omp setting

I have very expensive functions and some parameters take much longer than others. The problem is that often not all threads are used. Overall, this makes the optimization much longer because the CPU is often only used to 1/8 of its capacity. Therefore the effect of parallelization is only part-time.
It gives the impression that the parallelization is interrupted as soon as a "slow function execution" occurs in the overall process. As if the execution of new threads would have to wait.

What is the reason for this and is there a possibility that all 8 threads are always used and therefore the cpu is used at full capacity? This would significantly accelerate the total optimisation process.

Greetings

Setting Optim::algo_settings_t::vals_bound = true and setting lower_bounds & upper_bounds works well with ::ne() optimizer. The values are correctly resticted within the ranges defined.

Changeing optimizer to ::de() or ::pso() leads to results which are far out of the bounds defined. For example some values are restricted to 0.0 .. 1.0 range and results are in negative part.

Should ::de() & ::pso() preserve the limits? Are there some special requirements for that?

I'm trying to use this library with real dynamics systems, right now I'm facing a problem to use the API with this type of systems, because It the "optimized function" depends of a real hardware that takes time to answer and give a feedback for the next interaction. There is any plan to accomplish such thing with this library ?

Is there a possibility to use the library on Windows ?
I have built a header only version on a Linux system with the command "./configure --header-only-version". I copied it to Windows and want to use it now. With the compiler msvc2015.
I also have Armadillo only as a header only variant. This is also in the program directory of my software.

How do I get OptimLib to use Armadillo from the same workdirectory and get it running in general ?

Since there is no DLL or LIB file available, the library should be included everywhere and compiler independent.

Hi!

Thank you for your code. I would really like to test use some of your implementations in my software. However, I am struggling to link your library. If you have time please take a look at my following CMake files:

cmake_minimum_required (VERSION 2.8.8)

# set the project name and version
project(local-order-parameter)
# set the custom modules path
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/cmake/Modules/")

#add_definitions(-DOPTIM_ENABLE_EIGEN_WRAPPERS)
# resolve dependencies
find_package(Git REQUIRED)
find_package(MPI REQUIRED)
if (MPI_FOUND)
    message("MPI found!")
else (MPI_FOUND)
    message(SEND_ERROR "This application cannot compile without MPI")
endif (MPI_FOUND)
# fetch external dependencies
include(FetchEigen)
include(FetchOptim)

include_directories(${CMAKE_SOURCE_DIR}/src)
include_directories(${CMAKE_SOURCE_DIR}/include)
include_directories(${MPI_INCLUDE_PATH})
include_directories(${MPI_CXX_INCLUDE_PATH})
add_definitions(${MPI_CXX_COMPILE_FLAGS})

set(CMAKE_CXX_STANDARD 11)
add_executable(GeM
    src/main.cpp
    src/Pattern.cpp
    )
add_dependencies(GeM eigen)
add_dependencies(GeM optim)
target_link_libraries(GeM ${MPI_CXX_LIBRARIES})

Fetching is done in the following way:

set(OPTIM_REPOSITORY "https://github.com/kthohr/optim.git")
set(OPTIM_TAG master)
include(ExternalProject)
ExternalProject_Add(optim
DOWNLOAD_COMMAND ${GIT_EXECUTABLE} clone --branch=${OPTIM_TAG} ${OPTIM_REPOSITORY}
CONFIGURE_COMMAND ""
BUILD_COMMAND ""
BUILD_COMMAND ${CMAKE_COMMAND} -E copy_directory
<SOURCE_DIR>/include ${CMAKE_CURRENT_SOURCE_DIR}/include/optim
INSTALL_COMMAND ""
LOG_DOWNLOAD ON
LOG_UPDATE ON

Unfortunately, when I run a script with

#define OPTIM_ENABLE_EIGEN_WRAPPERS
#include "optim/optim.hpp"

using any function, e.g. optim::cg(), I receive the following error:

Undefined symbols for architecture x86_64:
  "optim::cg(Eigen::Matrix<double, -1, 1, 0, -1, 1>&, std::__1::function<double (Eigen::Matrix<double, -1, 1, 0, -1, 1> const&, Eigen::Matrix<double, -1, 1, 0, -1, 1>*, void*)>, void*)", referenced from:
      _main in main.cpp.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [GeM] Error 1
make[1]: *** [CMakeFiles/GeM.dir/all] Error 2
make: *** [all] Error 2

Hi,

I am running your example of pso with bounded values. I have notices that if I set:

settings.iter_max = 1;
settings.err_tol = 1000;
settings.opt_iter = 1;

Nothing works. The algorithm still runs for iterations more than 1, and it reaches the global minimum (when correct upper and lower bounds are set). I would like to stop the optimization loop at specific number of iterations. Can you help in this please?

Hi,

I'm trying to install optim-master and I got the following error after the make command :

g++ -std=c++14 -Wall -fPIC -march=native -O3 -ffp-contract=fast -flto -DNDEBUG -fopenmp -DOPTIM_FPN_TYPE=double -DOPTIM_ENABLE_EIGEN_WRAPPERS -I/usr/include/eigen3 -I./include src/line_search/more_thuente.cpp -c -o src/line_search/more_thuente.o
In file included from src/line_search/more_thuente.cpp:27:
./include/optim.hpp:28:14: fatal error: BaseMatrixOps/include/BaseMatrixOps.hpp: Aucun fichier ou dossier de ce type
#include "BaseMatrixOps/include/BaseMatrixOps.hpp"
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:54: src/line_search/more_thuente.o] Error 1

I used the configure method and everything seems to be fine before :

~/Outils/optim-master$ ./configure -i "/home/d07876/Outils/optim-master" -l eigen -p

OptimLib Configuration

Summary:

• OS: linux-gnu

• Arch: x86_64

• C++ compiler: g++

• Build version: release

• OPTIM_LINEAR_ALG_LIB set to: eigen

• OPTIM_MATLIB_INCLUDE_PATH set to:
  /usr/include/eigen3

• BLAS and Lapack libraries set to:
  -lblas -llapack

• OpenMP features: enabled

• floating-point number type: double

• optimization flags:
  -fPIC -march=native -O3 -ffp-contract=fast -flto -DNDEBUG -fopenmp

• OptimLib install path:
  /home/d07876/Outils/optim-master

• Additional notes:

Configuration completed. Creating Makefile... done.

Could you tell me if I missed something during the installation
Thank you

Hi,

The header-only version doesn't seem to work with Eigen 3.3.9. gcc prints two errors:

/home/dburov/.local/include/optimlib/unconstrained/nm.hpp: In function ‘bool optim::internal::nm_impl(optim::Vec_t&, std::function<double(const Eigen::Matrix<double, -1, 1>&, Eigen::Matrix<double, -1, 1>*, void*)>, void*, optim::algo_settings_t*)’:
/home/dburov/.local/include/optimlib/unconstrained/nm.hpp:202:69: error: no match for call to ‘(optim::Vec_t {aka Eigen::Matrix<double, -1, 1>}) (optim::VecInt_t&)’
  202 |         simplex_fn_vals = OPTIM_MATOPS_EVAL(simplex_fn_vals(sort_vec));

and

/home/dburov/.local/include/optimlib/misc/matrix_ops/access.hpp:35:50: error: ‘all’ is not a member of ‘Eigen’
   35 |     #define OPTIM_MATOPS_ROWS(x, v) (x)(v,Eigen::all) // v is a vector

The second error is relatively well-known: all is only defined in Eigen's master branch (maybe the latest RC) but not in the latest stable version, which currently is 3.3.9.

The first one is not quite clear to me. It seems like it's trying to convert vector of ints into an int, and fails (output from one of the candidates):

include/eigen3/Eigen/src/Core/DenseCoeffsBase.h:178:22: note:   no known conversion for argument 1 from ‘optim::VecInt_t’ {aka ‘Eigen::Matrix<int, -1, 1>’} to ‘Eigen::Index’ {aka ‘long int’}
  178 |     operator()(Index index) const

Unrelated, is there any way to NOT use Eigen or Arma? Those are horrible, clumsy, bloated libraries that I try to avoid like the plague. Thanks!

Hello,

I happen to have the following issue when I try to compile the library using Eigen.
I have the same issue if I try to compile the header-only or the shared-library.

It looks like there is a mismatch with Eigen and optim.

I am using Eigen 3.3.4 version.

/home/user/project/src/../header_only_version/unconstrained/nm.hpp:202:69 error: no match for call to ‘(optim::Vec_t {aka Eigen::Matrix<double, -1, 1>}) (optim::VecInt_t&)’
         simplex_fn_vals = OPTIM_MATOPS_EVAL(simplex_fn_vals(sort_vec));

/home/user/project/src/../header_only_version/misc/matrix_ops/eval.hpp:30:35: note: in definition of macro ‘OPTIM_MATOPS_EVAL’
     #define OPTIM_MATOPS_EVAL(x) (x).eval()
                                   ^
In file included from /usr/include/eigen3/Eigen/Core:414:0,
                 from /usr/include/eigen3/Eigen/StdVector:14,
                 from /usr/include/pcl-1.8/pcl/point_cloud.h:46,
                 from /usr/include/pcl-1.8/pcl/io/pcd_io.h:43,
                 from /home/kfavre/git_repository/3d-vision/recalage/PCL/mutual_information/src/mutual_information.cpp:12:
/usr/include/eigen3/Eigen/src/Core/DenseCoeffsBase.h:115:41: note: candidate: Eigen::DenseCoeffsBase<Derived, 0>::CoeffReturnType Eigen::DenseCoeffsBase<Derived, 0>::operator()(Eigen::Index, Eigen::Index) const [with Derived = Eigen::Matrix<double, -1, 1>; Eigen::DenseCoeffsBase<Derived, 0>::CoeffReturnType = const double&; Eigen::Index = long int]
     EIGEN_STRONG_INLINE CoeffReturnType operator()(Index row, Index col) const


etc....


/home/user/project/src/../header_only_version/misc/matrix_ops/access.hpp:35:50: error: ‘all’ is not a member of ‘Eigen’
     #define OPTIM_MATOPS_ROWS(x, v) (x)(v,Eigen::all) // v is a vector
                                                  ^
/home/user/project/src/../header_only_version/misc/matrix_ops/eval.hpp:30:35: note: in definition of macro ‘OPTIM_MATOPS_EVAL’
     #define OPTIM_MATOPS_EVAL(x) (x).eval()

Here is the code that I want to execute (which is pretty simple):

#define OPTIM_ENABLE_EIGEN_WRAPPERS
#include "../header_only_version/optim.hpp"
  double ackley_fn(const Eigen::VectorXd& vals_inp, Eigen::VectorXd* grad_out, void* opt_data)
  {
      const double x = vals_inp(0);
      const double y = vals_inp(1);

      double obj_val = -20*std::exp( -0.2*std::sqrt(0.5*(x*x + y*y)) ) - std::exp( 0.5*(std::cos(2*M_PI*x) + std::cos(2*M_PI*y)) ) + 22.718282L;

      return obj_val;
  }

int main (int argc, char** argv)
{
// initial values:
		Eigen::VectorXd x = Eigen::VectorXd::Zero(2,1); // (2,2)
		x[0] = 1.3;
		x[1] = 1.4;

	    bool success = optim::de(x,ackley_fn,nullptr);
            std::cout << "\nde: solution to Ackley test:\n" << x << std::endl;
}

Do not hesitate to ask if I can give any more precisions.
Thank you in advance for your help!

I wondered if you’d have any suggestions as to how to report the best guess at each iteration of the fitting loop for your NelderMead implementation? I'm interested for the purpose of investigating how our solution space is being explored but simply reporting simplex_points.row(0) (from my understanding that is the current lowest-error guess?) does not appear to yield the same results that are returned for the final iteration at the end of the method.

How easy would a feature that enables this diagnostic ability be to implement?

The document mentions that the solver enforces the constraints, g(x) >= 0. However, looking at the function that is minimized during the optimization (and the code), the constraints it enforces are, g(x) <= 0.

I've only been using this library for a couple of days (and enjoying it so far), but I ran into an issue during integration that I didn't see in a quick scan here on github. I wanted to use a header only version of the library for quick integration into an existing codebase, but ran into a bunch of 'identifier not found' compiler errors.

After a little bit of inspection, the problem seemed to be that the configure script removes the internal:: namespace labels from everywhere in the library, which causes most of the internal functions in the script to disassociate from their forward declarations.

For example, look at line_search_mt in more_thuente.hpp. This calls mt_step in several places, with no internal namespace prefix. When line_search_mt is correctly defined as being in the internal namespace, the compiler correctly infers that mt_step is also in the internal namespace. However, by killing the internal:: from the definition, the compiler won't be looking in the right namespace for its forward declarations.

Fortunately, I found an easy fix for this: remove the replacement of internal:: from the sed in the header-only-version part of the configure script, so that it reads as follows.
sed -e '1,/\[OPTIM_BEGIN\]/ d' $WDIR/src/"$dir_"/"$file_".cpp | sed -e "s|optimlib_inline|inline|" -e "s|optim::||" >> "$file_".hpp

Hi!

I would like to be able to provide an initial simplex where I can consider some background information that I have about the optimization problem. I would also like to be able to stop the optimization, continue another optimization somewhere else and then pick up where I left, continuing from the last simplex.
Would you consider implementing this?
Otherwise I can fork the library and do it myself but it would be nicer to work with the official version :)

local@MacBook-Pro optim % ./configure -l arma -p

OptimLib Configuration

Summary:

• OS: darwin19

• C++ compiler: g++

• Build version: release

• OPTIM_LINEAR_ALG_LIB set to: arma

• OPTIM_MATLIB_INCLUDE_PATH set to:
  /Users/local/armadillo-code

• BLAS and Lapack libraries set to:
  -framework Accelerate

• OpenMP features: enabled

• optimization flags:
  -O3 -march=native -ffp-contract=fast -flto -DNDEBUG -fopenmp

• OptimLib install path:
  /Users/local/optim

• Additional notes:

Configuration completed. Creating Makefile... done.

local@MacBook-Pro optim % make
g++ -std=c++11 -Wall -O3 -march=native -ffp-contract=fast -flto -DNDEBUG -fopenmp -DOPTIM_ENABLE_ARMA_WRAPPERS -DARMA_NO_DEBUG -I/Users/local/armadillo-code -I./include src/line_search/more_thuente.cpp -c -o src/line_search/more_thuente.o`

I am currently working on some constraint quadratic optimization problem and just try to install your repo on my Mac. However, when i tried to install it, there are some error message about
"clang: error: unsupported option '-fopenmp'
make: *** [src/line_search/more_thuente.o] Error 1".

It will be really helpful if you could help me take a look or give me any guidance. Thanks so much for your time and looking forward to hearing from you.

I‘m really a newbie to c++/Rcpp. Thus, I‘m not sure what causes my problem. I'm running a simulation in Rcpp using PSO to optimize parameter of an ode system in order to match my measured data.
If I use the PSO with an unbounded parameter space everything works fine. However, if the parameter space is restricted part of the parameter passed to the objective function are “nan“. Any suggestion what the problem could be?

Many thanks in advance for your help.

Hi,

I have being installing your library with no difficulty on a Linux/Ubuntu system and manage to run both examples (Ackley's function and logistic regression) : thank you very much for your work.

Now, I would like to access optim for developing R packages with Rcpp/RcppArmadillo. I am thus trying to wrap your library to R via an R package, in an attempt that can be found there. I hope it is ok to you.

To do that, I put the headers of optim into inst/include/ and the cpp files into src/. Then I tried to link everything in the src/Makevars files, and the package installation (including compilation of optim) is fine (at least, the package can be installed). The Makevars can be found there :

https://github.com/jchiquet/optimLibR/blob/414015b08a5e316de17b1e0b5385718abd23da6a/src/Makevars#L1-L41

Then I tried to create two small Rcpp functions for both Ackley and logit examples, starting with the following lines

// [[Rcpp::depends(RcppArmadillo)]]
// [[Rcpp::plugins(cpp11)]]
#define USE_RCPP_ARMADILLO
#include "optim.hpp"

However, when I call the corresponding R functions, I get the following Armadillo errors in R which I do not have wheh running them in 'pure' C++ (I am pretty sure it is not a syntax problem, the Rcpp code and the original C++ code are hopefully identical, except for the headers).

> library(optimLibR)
> ackley_function()

error: arma::memory::acquire(): out of memory

> library(optimLibR)
> logit_optimLib()

error: Mat::init(): requested size is not compatible with column vector layout

If you could have a quick glance, at least to check that the flag and libs in the Makevars are correctly set for optimr equirements, it would help a lot and I would be happy to use your library in my future R packages.

Thanks for your time

If I pass an Arma::vector (init_vals) to an optimization function with zeros only, values unequal to zero are passed.
There are values like 0.02678 etc.

System: macOS
Compile cmd: g++ -DOPTIM_ENABLE_EIGEN_WRAPPERS --std=c++11 aa.cpp -I./header_only_version/ -I ./eigen-3.3.8/

compiler output:
In file included from aa.cpp:1:
In file included from ./header_only_version/optim.hpp:51:
./header_only_version/unconstrained/nm.hpp:201:45: error: no matching function for call to object of type 'optim::Vec_t' (aka 'Matrix<double, Dynamic, 1>')
simplex_fn_vals = OPTIM_MATOPS_EVAL(simplex_fn_vals(sort_vec));
^~~~~~~~~~~~~~~
./header_only_version/misc/matrix_ops/eval.hpp:30:35: note: expanded from macro 'OPTIM_MATOPS_EVAL'
#define OPTIM_MATOPS_EVAL(x) (x).eval()
^
./eigen-3.3.8/Eigen/src/Core/DenseCoeffsBase.h:423:5: note: candidate function not viable: no known conversion from 'optim::VecInt_t' (aka 'Matrix<int, Dynamic, 1>') to 'Eigen::Index' (aka 'long') for 1st argument
operator()(Index index)
^
./eigen-3.3.8/Eigen/src/Core/DenseCoeffsBase.h:178:5: note: candidate function not viable: no known conversion from 'optim::VecInt_t' (aka 'Matrix<int, Dynamic, 1>') to 'Eigen::Index' (aka 'long') for 1st argument
operator()(Index index) const
^
./eigen-3.3.8/Eigen/src/Core/DenseCoeffsBase.h:115:41: note: candidate function not viable: requires 2 arguments, but 1 was provided
EIGEN_STRONG_INLINE CoeffReturnType operator()(Index row, Index col) const
^
./eigen-3.3.8/Eigen/src/Core/DenseCoeffsBase.h:362:5: note: candidate function not viable: requires 2 arguments, but 1 was provided
operator()(Index row, Index col)
^
In file included from aa.cpp:1:
In file included from ./header_only_version/optim.hpp:51:
./header_only_version/unconstrained/nm.hpp:202:44: error: no member named 'all' in namespace 'Eigen'
simplex_points = OPTIM_MATOPS_EVAL(OPTIM_MATOPS_ROWS(simplex_points, sort_vec));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./header_only_version/misc/matrix_ops/access.hpp:35:50: note: expanded from macro 'OPTIM_MATOPS_ROWS'
#define OPTIM_MATOPS_ROWS(x, v) (x)(v,Eigen::all) // v is a vector
~~~~~~~^
./header_only_version/misc/matrix_ops/eval.hpp:30:35: note: expanded from macro 'OPTIM_MATOPS_EVAL'
#define OPTIM_MATOPS_EVAL(x) (x).eval()
^
aa.cpp:3:24: error: use of undeclared identifier 'arma'
double sphere_fn(const arma::vec& vals_inp, arma::vec* grad_out, void* opt_data)
^
aa.cpp:3:45: error: use of undeclared identifier 'arma'
double sphere_fn(const arma::vec& vals_inp, arma::vec* grad_out, void* opt_data)
^
aa.cpp:5:22: error: use of undeclared identifier 'arma'
double obj_val = arma::dot(vals_inp,vals_inp);
^
aa.cpp:14:23: error: use of undeclared identifier 'arma'
double booth_fn(const arma::vec& vals_inp, arma::vec* grad_out, void* opt_data)
^
aa.cpp:14:44: error: use of undeclared identifier 'arma'
double booth_fn(const arma::vec& vals_inp, arma::vec* grad_out, void* opt_data)
^
aa.cpp:36:5: error: use of undeclared identifier 'arma'
arma::vec x = arma::ones(test_dim,1); // initial values (1,1,...,1)
^
aa.cpp:36:19: error: use of undeclared identifier 'arma'
arma::vec x = arma::ones(test_dim,1); // initial values (1,1,...,1)
^
aa.cpp:46:5: error: use of undeclared identifier 'arma'
arma::cout << "lbfgs: solution to sphere test:\n" << x << arma::endl;
^
aa.cpp:46:63: error: use of undeclared identifier 'arma'
arma::cout << "lbfgs: solution to sphere test:\n" << x << arma::endl;
^
aa.cpp:51:5: error: use of undeclared identifier 'arma'
arma::vec x_2 = arma::zeros(2,1); // initial values (0,0)
^
aa.cpp:51:21: error: use of undeclared identifier 'arma'
arma::vec x_2 = arma::zeros(2,1); // initial values (0,0)
^
aa.cpp:61:5: error: use of undeclared identifier 'arma'
arma::cout << "lbfgs: solution to Booth test:\n" << x_2 << arma::endl;
^
aa.cpp:61:64: error: use of undeclared identifier 'arma'
arma::cout << "lbfgs: solution to Booth test:\n" << x_2 << arma::endl;
^
15 errors generated.

Is the following Citation correct for OptimLib?

@misc{optimlib,
author = {Keith O'Hara},
title = {{OptimLib, a lightweight C++ library of numerical optimization methods for nonlinear functions}},
url = {https://www.kthohr.com/optimlib.html},
howpublished = {\url{https://www.kthohr.com/optimlib.html}},
year = {2017}
}

After configuring to header-only-library version, there are include path errors in misc/optim_misc.hpp.

Currently, the first two includes are

// structs
#include "misc/optim_structs.hpp"

// trace
#include "misc/optim_trace.hpp"

Which gives include errors when compiling. The errors are fixed by changing these lines to:

// structs
#include "optim_structs.hpp"

// trace
#include "optim_trace.hpp"

I have a number of initial parameters. For each one I have specified an upper and lower boundary.When the algorithm is complete some results are outside the boundaries.

` optim::algo_settings_t settings;

arma::vec lb = {-1,-1, -2,-2,-2,  -2,-2,-2};
arma::vec ub = {1,1,    2,2,2,     2,2,2, };
settings.pso_initial_lb = lb;
settings.pso_initial_ub = ub;
settings.pso_n_gen = 1000;

bool success = optim::pso(x, optimF,&optim_data,settings);`

Same when i use "de"

First, I would like to thank you for making this library available.

However, I was trying to use the PSO method with upper and lower bounds.
For example, the Ackley function with

X= [0.75 , 0.75]
Lower = [0.5, 0.5]
Upper = [1, 1]

After some tests I understood that I should send the pso_settings.initial_ub and pso_settings.initial_lb, because the default broke the bound limits. So that was not a big issue. (I used 0.05 instead of 0.5)

However, I think the center_particle option is a big issue with the bounds (At least in the initialize loop).
In line 147

 P.row(i) = BMO_MATOPS_COLWISE_SUM( BMO_MATOPS_MIDDLE_ROWS(P, 0, n_pop-2) ) / static_cast<fp_t>(n_pop-1); // center vector

It gets the center position of the particles using matrix P.
However, in line 162

if (vals_bound) {
      P.row(i) = transform<RowVec_t>(P.row(i), bounds_type, lower_bounds, upper_bounds);
}

The lines of P are rewritten with some values that are not inside the bounds.
So the center vector is not inside as well.

With the center_particle activated the results are very odd. After I deactivated it, the result is always good.

May I know the possibility to extend some implemented methods with preconditioning?

Following the guide, make fails.

$ make
g++ -std=c++11 -Wall -march=native -O3 -ffp-contract=fast -flto -DNDEBUG -fopenmp -DOPTIM_ENABLE_ARMA_WRAPPERS -DARMA_NO_DEBUG -I/mingw64/include -I./include src/line_search/more_thuente.cpp -c -o src/line_search/more_thuente.o
In file included from /mingw64/include/armadillo:68,
                 from ./include/misc/optim_options.hpp:78,
                 from ./include/optim.hpp:24,
                 from src/line_search/more_thuente.cpp:27:
/mingw64/include/armadillo_bits/compiler_setup.hpp:81:30: fatal error: C:/msys64/mingw64/include/hdf5.h: No such file or directory
   81 | #define ARMA_INCFILE_WRAP(x) <x>
      |                              ^
compilation terminated.
make: *** [Makefile:54: src/line_search/more_thuente.o] Error 1

Seems to be the same as msys2/MINGW-packages#6715

All packages up to date on msys.

Hello,

It would be great if you could add a conda recipe for this library.
This would allow to install it and use it easily in a jupyter notebook with xeus-cling.

More background here, it's a similar request I made.
Dobiasd/FunctionalPlus#161

Thank you

Hello, I would like to use this library to optimize a non-linear function with the opt_data feature to be passed as arguments to the function using Nelder-Mead. I could not find any example for it here. Since I'm a newbie in c++, I wasn't able to replicate it by seeing other examples in the library. Can someone help me here?

Hello,

Trying to compile the examples in VS 2022, C++17. Geting error C1017 - Invalid integer constant expression in optim_options.hpp at line 78.
The error is somewhere in this block. Based on MS documentation about this error at C1017
it says : Constants defined using #define must have values that evaluate to an integer constant if they are used in an #if, #elif, or #else directive.
Any idea how to fix this?

Thank you.

// floating point number type

#ifndef OPTIM_FPN_TYPE
#define OPTIM_FPN_TYPE double
#endif

#if OPTIM_FPN_TYPE == float //this is line 78
#undef OPTIM_FPN_SMALL_NUMBER
#define OPTIM_FPN_SMALL_NUMBER fp_t(1e-05)
#elif OPTIM_FPN_TYPE == double
#undef OPTIM_FPN_SMALL_NUMBER
#define OPTIM_FPN_SMALL_NUMBER fp_t(1e-08)
#else
#error floating-point number type must be 'float' or 'double'
#endif

The problem is in "error_reporting.ipp", line 80

//settings_inp->opt_fn_value     = opt_objfn(x_p,nullptr,opt_data);

https://drive.google.com/file/d/1XyAOyllOcnY3q10nHETtd6gvrWqGU4ov/view?usp=sharing
https://drive.google.com/file/d/12qemu3_UEhJGcZ21mwypk5RPIFuXCXgf/view?usp=sharing

Seems something was "std::moved" out of memory. Probably this is Eigen problem but may be you can work around it.

Ubuntu 21.10.
PS: could provide the full source if needed
Btw: many thanks for the library )

I build OpenBlas 0.3.3 from sourceforge through cmake on Win7. But when building optim, There are 16 unresolved external symbols.

logit_reg.obj: error: LNK2001: unresolved external symbol dgesv
bfgs.obj: error: LNK2001: unresolved external symbol dposv_
lbfgs.obj: error: LNK2001: unresolved external symbol dgesdd_
D:\Project\Opti\Opti\optim\Bin\optim.exe: error: LNK1120: 16 unresolved external symbol_

Hi,

First I'd like to thank you for creating such a wonderful library. I have created a simple example to learn how to use your library the right way. For example, I tried to estimate the parameters of the quadratic function y = 2 * x ^ 2 - 3 * x + 7 So the weights should be estimated as [2 -3 7]

When I use the line bool success = optim::bfgs(w, quadratic, &data); everything seems to be working correctly and the weights are estimated as expected. Yet when I try to manipulate the optimization settings by simply using the block below

optim::algo_settings_t settings;
settings.iter_max = 500;
bool success = optim::bfgs(w, quadratic, &data, settings);

the program crashes without even returning a success flag. The following is the code I have written. I would very much appreciate your help.

Thanks,
Davar

#include "optim.hpp"
#include <iostream> 

using std::cout;
using std::endl;
using std::vector;

struct Data {
    arma::vec x;
    arma::vec y;
};

double quadratic(const arma::vec& w, arma::vec* grad, void* opt_data) {

    Data* data = reinterpret_cast<Data*>(opt_data);
    arma::vec x = data->x;
    arma::vec y = data->y;

    double f = 0;
    (*grad)(0) = (*grad)(1) = (*grad)(2) = 0;

    for (int i = 0; i < x.n_elem; i++) {

        double t = y(i) - (w(0) * pow(x(i), 2) + w(1) * x(i) + w(2));
        f += pow(t, 2);

        (*grad)(0) += -2 * pow(x(i), 2) * t;
        (*grad)(1) += -2 * x(i) * t;
        (*grad)(2) += -2 * t;

    }

    return f;
}

int main(){

    int numOfObservations = 100;
    arma::vec x(numOfObservations, arma::fill::randn);
    arma::vec y(numOfObservations);
    for (int i = 0; i < numOfObservations; i++) {
        y(i) = 2 * pow(x[i], 2) - 3 * x[i] + 7;
    }
    Data data;
    data.x = std::move(x);
    data.y = std::move(y);

    int numOfDims = 3;
    arma::vec w(numOfDims, arma::fill::zeros);

    bool success = optim::bfgs(w, quadratic, &data);

    //optim::algo_settings_t settings;
    //settings.iter_max = 500;
    //bool success = optim::bfgs(w, quadratic, &data, settings);

    cout << "success = " << success << " --> w = " << w.t() << endl;

    return 0;
}

Hi,

I have observed that in optim/src/unconstrained/pso.cpp, in its loop line 143, the variable "err" is not being updated. This cause err_tol value inoperative, thus the loop never stops if error becomes less than err_tol. I assume it should be updated at line 201 by the following line:

err = global_best_val;

Please check and correct me if I am wrong. I have tried this and now err_tol values have an impact on the loop. I doubt, this issue is present in other algorithms too.

When trying to install with header files only, the directory BaseMatrixOps is empty. The following shell command demonstrates the issue, since the last ls statement shows that the directory is empty.

git clone https://github.com/kthohr/optim.git && cd optim && \
./configure --header-only-version && ls header_only_version/BaseMatrixOps

This further causes the following error when trying to compile with #include "optim.hpp"

fatal error: 'BaseMatrixOps/include/BaseMatrixOps.hpp' file not found
#include "BaseMatrixOps/include/BaseMatrixOps.hpp"
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Presumably the error message occurs because of this line, i.e., optim.hpp requires the BaseMatrixOps directory to not be empty.

See line 149 of nm.cpp which currently reads as follows.
simplex_points = OPTIM_MATOPS_ROWS(simplex_points, sort_vec);

If using Eigen as a linear algebra library, this presents an aliasing problem, which makes the result of this operation dramatically incorrect. For example, try it with the Booth demo listed in the documentation. It will give something like (0,0.25), far away from the correct (1,3).

Fortunately, there's a pretty easy fix. Just add an eval to the line as follows to force Eigen to behave as expected.
simplex_points = OPTIM_MATOPS_EVAL(OPTIM_MATOPS_ROWS(simplex_points, sort_vec));

This is a fantastic C++ optimization library. I have compiled the examples with MSVC compiler and they ran well under Windows OS. I wonder if there is a guide to build OptimLib into windows dynamic link library (dll)?

Hi,

Is it possible to perform bfgs without known the explicit expression of the derivative? In other words, derivatives are approximated numerically.

Thank you,
Tao

Hello I want to implement the constrained function but there seem to be some errors in the document. In that case, I don't know how to implement the constrained problem. Could anyone help fix it?

Unless I'm misreading something, it seems like the sampling bounds for DE are currently (0, (LB + (UB-LB))) = (0, UB) (while they should be (LB, UB) obviously). I think line 99 should be

        X_next.row(i) = OPTIM_MATOPS_TRANSPOSE( par_initial_lb + OPTIM_MATOPS_HADAMARD_PROD( par_initial_ub - par_initial_lb, OPTIM_MATOPS_RANDU_VEC(n_vals) ) );

In addition, it'd be safer to enforce par_initial_ub =< upper_bound and par_initial_lb >= lower_bound if users are using upper and lower bounds. I'm getting NaN errors with non-zero upper and lower bounds after X_next is log-transformed for box constraining.

When i try to create the Header-only Library i get this error: "error: unrecognized linear algebra library". Eigen is installed.
I installed the eigen library 3.4.0 via cmake/make install on debian. Eigen exist in folder /usr/local/include/eigen3/
Any idea to solve the problem? Thanks.

Would have made a pull request but didn't want to break existing code for other people, probably needs a version check for Eigen.

Hi,

If the objective function doesn't define a hessian (neither a gradiant eventually), is there a way to get an approximate of the hessian matrix at the optimal point?

Best regards

Hi, I was just going to try this library and got this error:
$ ./configure --header-only-version
sed: can't read optim-master/src/unconstrained/optim_unconstrained.cpp: No such file or directory
sed: can't read optim-master/src/zeros/optim_zeros.cpp: No such file or directory

Hello,
Would you please consider adding a function that will draw the optimization path?
Thank you for reading.

Hello,

I've faced a problem trying to optimize my cost function with the NM optimizer. The problem is that most of the time it returns true results, but sometimes they are wrong (still small but not right). I'm sure that the input data is the same (checked with hash functions) and I've set random seeds of both std and arma to 0 just to be sure. I have a guess that it somehow related to the fact that my cost function is really rough and there may take place a rounding error. How can I overcome the problem?

Thank you in advance!