Using optimparallel.minimize_parallel() can significantly reduce the optimization time. For an objective function with an execution time of more than 0.1 seconds and p parameters the optimization speed increases by up to factor 1+p when no analytic gradient is specified and 1+p processor cores with sufficient memory are available.

A similar extension of the L-BFGS-B optimizer exists in the R package optimParallel:

• optimParallel on CRAN
• R Journal article

To install the package run:

$ pip install optimparallel

Replace scipy.optimize.minimize(method='L-BFGS-B') by optimparallel.minimize_parallel() to execute the minimization in parallel:

from optimparallel import minimize_parallel
from scipy.optimize import minimize
import numpy as np
import time

# objective function
def f(x, sleep_secs=0.5):
    print('fn')
    time.sleep(sleep_secs)
    return sum((x - 14)**2)

# start value
x0 = np.array([10, 20])

# minimize with parallel evaluation of 'fun' and
# its approximate gradient.
o1 = minimize_parallel(fun=f, x0=x0, args=0.5)
print(o1)

# test against scipy.optimize.minimize()
o2 = minimize(fun=f, x0=x0, args=0.5, method='L-BFGS-B')
print(all(np.isclose(o1.x, o2.x, atol=1e-10)),
      np.isclose(o1.fun, o2.fun, atol=1e-10),
      all(np.isclose(o1.jac, o2.jac, atol=1e-10)))

The evaluated x values, fun(x), and jac(x) can be returned:

o1 = minimize_parallel(fun=f, x0=x0, args=0.5, parallel={'loginfo': True})
print(o1.loginfo)

More examples are given in example.py and the Jupyter Notebook example_extended.ipynb.

Note for Windows users: It may be necessary to run minimize_parallel() in the main scope. See example_windows_os.py.

When using this package please cite:

• Gerber, F. optimparallel - A parallel version of `scipy.optimize.minimize(method='L-BFGS-B'), 2020, http://doi.org/10.5281/zenodo.3888570
• Gerber, F., and Furrer, R. optimParallel: An R package providing a parallel version of the L-BFGS-B optimization method. The R Journal, 11(1):352–358, 2019. http://doi.org/10.32614/RJ-2019-030

• Florian Gerber, [email protected], https://user.math.uzh.ch/gerber.

• Lewis Blake, https://github.com/lewisblake

Contributions via pull requests are welcome.

• https://github.com/florafauna/optimParallel-python
• https://pypi.org/project/optimparallel