Approximate Inference for Stochastic Optimal Control

Input Inference for Control (i2c) is an inference-based optimal control algorithm. The current implementation, Gaussian i2c, can perform trajectory optimization, model predictive control and covariance control via a Gaussian approximation of the optimal state-action distribution. This yields time-varying linear (Gaussian) controllers, and is approximately equivalent to quadratic optimal control methods like differential dynamic programming, iterative-/sequential LQR.

For more information, see the following papers:

[1] J. Watson and H. Abdulsamad and R. Findeisen and J. Peters. Stochastic Control through Approximate Bayesian Input Inference. Submitted to IEEE Transactions on Automatic Control Special Issue, Learning and Control 2021. (arXiv)

[2] J. Watson and J. Peters. Advancing Trajectory Optimization with Approximate Inference: Exploration, Covariance Control and Adaptive Risk. American Control Conference (ACC) 2021 (arXiv)

[3] J. Watson and H. Abdulsamad and J. Peters. Stochastic Optimal Control as Approximate Input Inference. Conference on Robot Learning (CoRL) 2019. (arXiv)

Create environment i2c and install

    cd input-inference-for-control && conda create -y -n i2c pip python=3.7 && conda activate i2c && pip3 install -r requirements.txt && pip install -e .

To optimize pendulum swing-up with cubature quadrature, run

    python scripts/i2c_run.py pendulum_known_quad

the output directory results should look like this

Prior experiments are preserved here. All results are stored in /_results.

Section 3.1 of [3]

    python scripts/LQR_compare.py

Section 3.2 of [3], Section IV.A of [1]

    python scripts/i2c_run.py -h

results are in _results/

Linear Gaussian covariance control. Section IV.C of [2]

    python scripts/linear_covariance_control.py

Pendulum swing-up with covaraince control. Section IV.C of [2]

    python scripts/nonlinear_covariance_control.py

Runs i2c and iLQR MPC with a cubature kalman filter for an acrobatic quadropter task. Section IV.C of [1]

    python scripts/mpc_state_est/mpc_quad.py 0 --plot

For iLQR use https://github.com/hanyas/trajopt

To cite i2c, please reference the appropriate paper

@misc{watson2021stochastic,
      title={Stochastic Control through Approximate Bayesian Input Inference}, 
      author={Watson, Joe and Abdulsamad, Hany and Findeisen, Rolf and Peters, Jan},
      year={2021},
      eprint={2105.07693},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

@inproceedings{i2cacc,
	author    = {Watson, Joe and Peters, Jan},
	title     = {Advancing Trajectory Optimization with Approximate Inference: Exploration, Covariance Control and Adaptive Risk},
	booktitle = {American Control Conference},
	year      = {2021},
}

@inproceedings{i2ccorl,
	author    = {Watson, Joe and  Abdulsamad, Hany and Peters, Jan},
	title     = {Stochastic Optimal Control as Approximate Input Inference},
	booktitle = {Conference on Robot Learning},
	year      = {2019},
}