convexbrain / studynotes Goto Github PK

Study

• Luenberger汎関数最適化本 7~10章
• 数学30講シリーズ
• 積み論文リスト #3

Totsu

• logクレート
• Convex.jl(MathOptInterface)バインディング
• GPGPU化
• 実行速度ベンチマーク
• 他の行列crate連携

錐最適化

• 設計（定式化）のまとめ
• 自明な解0に行ってしまわないこと、反復がnonexpandingか、文献調査

モデリング

• SOCPでトラスのコンプライアンス最小化
• SDPで最小固有値最大化または緩和0/1問題
• Operator直接使用で画像2次微分L1最小化
• ECPでL1正則化ロジスティック回帰
• 解説記事
• 頭部伝達関数の非線形Deconvolutionで何かできないか？
• MPCまたはLMIのいい例題はないか？
• 変わった錐とその応用はないか？
• FEMやトポロジー最適化と組合せできないか？

Minimult

• cortex-m 0.6.2, num-integer 0.1.42対応
• MTSharedChでUnsafeCellを利用する
• https://keepachangelog.com/
• BLE通信ドライバ SoftDevice <-> Rust/WinRT

凸Layer関連

• Learning Convex Optimization Models
  • https://web.stanford.edu/~boyd/papers/learning_copt_models.html
• OptNet: Differentiable Optimization as a Layer in Neural Networks
  • https://arxiv.org/abs/1703.00443
• Differentiable Convex Optimization Layers
  • https://arxiv.org/abs/1910.12430
• Differentiating through the Fréchet Mean
  • https://arxiv.org/abs/2003.00335

凸NN関連

• Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-Layer Networks
  • https://arxiv.org/abs/2002.10553
• All Local Minima are Global for Two-Layer ReLU Neural Networks: The Hidden Convex Optimization Landscape
  • https://arxiv.org/abs/2006.05900
• Exact and Relaxed Convex Formulations for Shallow Neural Autoregressive Models
  • https://web.stanford.edu/~ergen/publication/convex_autorreg/
• Training Convolutional ReLU Neural Networks in Polynomial Time: Exact Convex Optimization Formulations
  • https://arxiv.org/abs/2006.14798
• Gated Linear Networks
  • https://arxiv.org/abs/1910.01526
• Convex Regularization Behind Neural Reconstruction
  • https://arxiv.org/abs/2012.05169
• Vector-output ReLU Neural Network Problems are Copositive Programs: Convex Analysis of Two Layer Networks and Polynomial-time Algorithms
  • https://arxiv.org/abs/2012.13329
• Convex Geometry and Duality of Over-parameterized Neural Networks
  • https://arxiv.org/abs/2002.11219
• Training Quantized Neural Networks to Global Optimality via Semidefinite Programming
  • https://arxiv.org/abs/2105.01420
• Hidden Convexity of Wasserstein GANs: Interpretable Generative Models with Closed-Form Solutions
  • https://arxiv.org/abs/2107.05680
• The Hidden Convex Optimization Landscape of Deep Neural Networks
  • https://youtu.be/1DWP44wKBhg

凸Solver関連

• Fast and Accurate Least-Mean-Squares Solvers
  • https://arxiv.org/abs/1906.04705
• Adaptive Gradient Methods for Constrained Convex Optimization
  • https://arxiv.org/abs/2007.08840
• Convex optimization based on global lower second-order models
  • https://arxiv.org/abs/2006.08518
• Conic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding
  • https://web.stanford.edu/~boyd/papers/scs.html "talk slides"
• 一次法の逆襲
  • http://www.orsj.or.jp/e-library/elcorsj64.html#6406
• A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms
  • https://link.springer.com/article/10.1007/s10957-012-0245-9
• COSMO: A conic operator splitting method for convex conic problems
  • https://arxiv.org/abs/1901.10887
• Distributed Proximal Splitting Algorithms with Rates and Acceleration
  • https://arxiv.org/abs/2010.00952
• Inexact basic tensor methods for some classes of convex optimization problems
  • https://www.tandfonline.com/doi/full/10.1080/10556788.2020.1854252
• Adaptive and Oblivious Randomized Subspace Methods for High-Dimensional Optimization: Sharp Analysis and Lower Bounds
  • https://arxiv.org/abs/2012.07054
• A primal-dual interior-point algorithm for nonsymmetric exponential-cone optimization
  • https://link.springer.com/article/10.1007/s10107-021-01631-4
• Adaptive first-order methods revisited: Convex optimization without Lipschitz requirements
  • https://arxiv.org/abs/2107.08011
• Neural Fixed-Point Acceleration for Convex Optimization
• https://arxiv.org/abs/2107.10254
• Newton-LESS: Sparsification without Trade-offs for the Sketched Newton Update
• https://arxiv.org/abs/2107.07480

凸拡張・非凸

• A Second-order Equilibrium in Nonconvex-Nonconcave Min-max Optimization: Existence and Algorithm
  • https://arxiv.org/abs/2006.12363
• 新世代が切り拓く連続最適化
  • http://www.orsj.or.jp/e-library/elcorsj59.html#5903
• Mixed-Projection Conic Optimization: A New Paradigm for Modeling Rank Constraints
  • https://arxiv.org/abs/2009.10395
• 新世代が切り拓く連続最適化：パート 2
  • http://www.orsj.or.jp/e-library/elcorsj65.html#6512
• Lasry-Lions Envelopes and Nonconvex Optimization: A Homotopy Approach
  • https://arxiv.org/abs/2103.08533

凸全般・凸応用

• ECE236C
  • http://www.seas.ucla.edu/~vandenbe/ee236c.html "Conic optimization"
• Lectures on Modern Convex Optimization
  • https://www2.isye.gatech.edu/~nemirovs/ "Transparencies" 応用部分
• Embedded Convex Optimization for Control
  • https://web.stanford.edu/~boyd/papers/cdc_20.html
• Reinforcement Learning via Fenchel-Rockafellar Duality
  • https://arxiv.org/abs/2001.01866

http://stanford.edu/~boyd/cvxbook/bv_cvxslides.pdf
私的（非公開）日本語訳を作りながら勉強

• 1 イントロダクション
• 2 凸集合
• 3 凸関数
• 4 凸最適化問題
• 5 双対性
• 6 近似とフィッティング
• 7 統計的推定
• 8 幾何問題
• 9 数値線形代数の背景知識
• 10 制約なし最小化
• 11 等式制約つき最小化
• 12 内点法
• 13 まとめ