The course addresses essentials in linear and nonlinear optimization, both from the theoretical as well as the numerical points of view. First, classical pattern search, evolutionary-based and gradient methods are presented, and theoretical issues are addressed. Then, advanced algorithms in nonlinear programming and in multiobjective optimization are presented and analyzed. A particular attention is paid for the python implementation of the most representative algorithms.

• Prerequisites: Basic knowledge in Linear Algebra, Matrix computation, Hilbert spaces, and Differential Calculus

1. Introduction to major problem families and solving classes in optimization
2. Gradient free methods
   1. Introduction – Motivation
   2. Pattern search methods
   3. Evolutionary-based methods; Particle Swarms
   4. Examples and Applications
3. Continuous Optimization
   1. Convexity and Differentials
   2. Optimality Conditions – Existence problems.
   3. Gradient type methods – Analysis of their convergence properties.
4. Constrained Optimization
   1. Penalization methods
   2. Projected gradient methods
   3. Lagrangian and Duality theory.
5. Advanced Algorithms in continuous optimization
   1. Conjugate and Nonlinear Conjugate gradients methods
   2. Newton and quasi-Newton methods; Spectral gradient methods
   3. Uzawa algorithms

Course slides and reading material will be available from the instructor.

Reference Books

• Algorithms for Optimization by Mykel J. Kochenderfer and Tim A. Wheeler, MIT Press, 2019
• Numerical Optimization by Jorge Nocedal, Stephen Wright, Springer, 2006.
• Optimization: Algorithms and Consistent Approximations by Elijah Polak, Springer, 1997.

1. Midterm Exam: 25%
2. Project: 25%
3. Final Exam: 50%