KELLEY
Iterative methods for optimization This book gives an introduction to optimization methods for unconstrained and bound constrained minimization problems. The style of the book is probably best described by the following quote from the book’s preface: `dots{} we treat a small number of methods in depth, giving less detailed description of only a few [dots{}]. We aim for clarity and brevity rather than complete generality and confine our scope to algorithms that are easy to implement (by the reader!) and understand.’ par This book is partitioned into two parts. The first part, occupying approximately 100 pages, is devoted to the optimization of smooth functions. The methods studied in this first part rely on the availability and accuracy of first order, and sometimes also second order derivatives of the objective function. The first part contains five chapters. The first chapter provides basic concepts. It also introduces a parameter identification problem and a discretized optimal control problem, both of which are used to demonstrate all methods discussed in the first part. Chapter 2 studies the local convergence of Newton’s method, inexact Newton methods, and the Gauss-Newton method for the solution of nonlinear least squares problems. Both, overdetermined and underdetermined nonlinear least squares problems are considered. Chapter 3 is devoted to line-search and trust-region methods, which are used to globalize convergence, i.e., remove the restriction that the starting point of the optimization iteration is sufficiently close to a solution. par The BFGS method is studied in chapter 4. A local convergence analysis is provided and implementation details are discussed. Other quasi-Newton methods are sketched. The last chapter of the first part, chapter 5, studies projection methods for the solution of bound constrained problems. All chapters conclude with a demonstration of the methods discussed in the respective chapter using the parameter identification problem and the discretized optimal control problem introduced in chapter 1, and with a set of exercises. par The second part of the book, which is approximately 50 pages long, deals with the optimization of noisy functions. Such optimization problems arise, e.g., when the evaluation of the objective function involves computer simulations. In such cases the noise often introduces artificial minimizers. Gradient information, even if available, cannot expected to be reliable. This second part contains three chapters. The first chapter provides a discussion of noisy functions, basics concepts, and three simple examples that are later used to demonstrate the behavior of optimization algorithms. Chapter 7 introduces implicit filtering, a technique due to the author and his group. Implicit filtering methods use finite difference approximations of the gradient, which are adjusted to the noise level in the function. Direct search algorithms, including the Nelder-Mead, multidirectional search, and the Hooke-Jeves algorithms are discussed in Chapter 8. Again, the latter two chapters conclude with a numerical demonstration of the methods discussed in the respective chapter, and with a set of exercises. par The treatment of both, optimization methods for smooth and for noisy functions is a unique feature of this book. Matlab implementations of all algorithms discussed in this book are easily accessible from the author’s or the publisher’s web-page.
Keywords for this software
References in zbMATH (referenced in 610 articles )
Showing results 1 to 20 of 610.
Sorted by year (- Braun, Phillip; Hare, Warren; Jarry-Bolduc, Gabriel: Limiting behavior of derivative approximation techniques as the number of points tends to infinity on a fixed interval in (\mathbbR) (2021)
- Chen, Xiaotong; Herring, James L.; Nagy, James G.; Xi, Yuanzhe; Yu, Bo: An ADMM-LAP method for total variation myopic deconvolution of adaptive optics retinal images (2021)
- Gu, Ruixue; Han, Bo; Tong, Shanshan; Chen, Yong: An accelerated Kaczmarz type method for nonlinear inverse problems in Banach spaces with uniformly convex penalty (2021)
- Hao, Wenrui: A gradient descent method for solving a system of nonlinear equations (2021)
- Jiang, H.; Gibson, N. L.: Semismooth Newton methods with a shooting-like technique for solving a constrained free-boundary HJB equation (2021)
- An, Dong; Lin, Lin: Quantum dynamics with the parallel transport gauge (2020)
- Backholm, Rasmus; Bubba, Tatiana A.; Bélanger-Champagne, Camille; Helin, Tapio; Dendooven, Peter; Siltanen, Samuli: Simultaneous reconstruction of emission and attenuation in passive gamma emission tomography of spent nuclear fuel (2020)
- Boutet, Nicolas; Haelterman, Rob; Degroote, Joris: Secant update version of quasi-Newton PSB with weighted multisecant equations (2020)
- Chakraborty, Suvra Kanti; Panda, Geetanjali: A modified coordinate search method based on axes rotation (2020)
- Chen, Liang; Ma, Yanfang: Shamanskii-like Levenberg-Marquardt method with a new line search for systems of nonlinear equations (2020)
- Chergui, Thiziri; Tas, Saadia: Existence, multiplicity and numerical examples for Schrödinger systems with nonstandard (p(x))-growth conditions (2020)
- Christof, Constantin: Gradient-based solution algorithms for a class of bilevel optimization and optimal control problems with a nonsmooth lower level (2020)
- Diniz-Ehrhardt, M. A.; Ferreira, D. G.; Santos, S. A.: Applying the pattern search implicit filtering algorithm for solving a noisy problem of parameter identification (2020)
- Frumin, Leonid L.: Linear least squares method in nonlinear parametric inverse problems (2020)
- Hare, Warren; Jarry-Bolduc, Gabriel: Calculus identities for generalized simplex gradients: rules and applications (2020)
- Hare, Warren; Planiden, Chayne; Sagastizábal, Claudia: A derivative-free (\mathcalV\mathcalU)-algorithm for convex finite-max problems (2020)
- Liao-McPherson, Dominic; Nicotra, Marco M.; Kolmanovsky, Ilya: Time-distributed optimization for real-time model predictive control: stability, robustness, and constraint satisfaction (2020)
- Manno, Andrea; Amaldi, Edoardo; Casella, Francesco; Martelli, Emanuele: A local search method for costly black-box problems and its application to CSP plant start-up optimization refinement (2020)
- Pozharskiy, Dmitry; Wichrowski, Noah J.; Duncan, Andrew B.; Pavliotis, Grigorios A.; Kevrekidis, Ioannis G.: Manifold learning for accelerating coarse-grained optimization (2020)
- Shen, Chungen; Fan, Changxing; Wang, Yunlong; Xue, Wenjuan: Limited memory BFGS algorithm for the matrix approximation problem in Frobenius norm (2020)