• LSQR

  • Referenced in 368 articles [sw00530]
  • Algorithm 583: LSQR: Sparse Linear Equations and Least Squares Problems. An iterative method is given ... equivalent to the standard method of conjugate gradients, but possesses more favorable numerical properties. Reliable ... comparing LSQR with several other conjugate-gradient algorithms, indicating that LSQR is the most reliable...
  • SCALCG

  • Referenced in 102 articles [sw08453]
  • SCALCG – Scaled conjugate gradient algorithms for unconstrained optimization. In this work we present and analyze ... scaled conjugate gradient algorithm and its implementation, based on an interpretation of the secant equation ... search conditions. The best spectral conjugate gradient algorithm SCG by Birgin and Martínez (2001), which ... show that this new scaled conjugate gradient algorithm substantially outperforms the spectral conjugate gradient...
  • AdaGrad

  • Referenced in 120 articles [sw22202]
  • ADAGRAD: adaptive gradient algorithm; Adaptive subgradient methods for online learning and stochastic optimization. We present ... earlier iterations to perform more informative gradient-based learning. Metaphorically, the adaptation allows ... proximal functions to control the gradient steps of the algorithm. We describe and analyze...
  • KELLEY

  • Referenced in 609 articles [sw04829]
  • complete generality and confine our scope to algorithms that are easy to implement ... cases the noise often introduces artificial minimizers. Gradient information, even if available, cannot expected ... used to demonstrate the behavior of optimization algorithms. Chapter 7 introduces implicit filtering, a technique ... gradient, which are adjusted to the noise level in the function. Direct search algorithms, including...
  • Adam

  • Referenced in 399 articles [sw22205]
  • Optimization. We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective ... problems with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically ... require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed...
  • SNOPT

  • Referenced in 523 articles [sw02300]
  • SNOPT: An SQP algorithm for large-scale constrained optimization. Sequential quadratic programming (SQP) methods have ... that the constraint gradients are sparse. We discuss an SQP algorithm that uses a smooth...
  • CGS

  • Referenced in 304 articles [sw08225]
  • combination of the CGS algorithm (a “squared” conjugate gradient method) with a preconditioning called ILLU...
  • CONOPT

  • Referenced in 164 articles [sw02791]
  • CONOPT is a generalized reduced-gradient (GRG) algorithm for solving large-scale nonlinear programs involving...
  • NLopt

  • Referenced in 92 articles [sw11789]
  • also algorithms exploiting user-supplied gradients. Algorithms for unconstrained optimization, bound-constrained optimization, and general...
  • darch

  • Referenced in 272 articles [sw11086]
  • with common known training algorithms like backpropagation or conjugate gradient...
  • lobpcg.m

  • Referenced in 127 articles [sw24008]
  • optimal block preconditioned conjugate gradient method. We describe new algorithms of the locally optimal block ... preconditioned conjugate gradient (LOBPCG) method for symmetric eigenvalue problems, based on a local optimization ... ideal” control algorithm, we advocate the standard preconditioned conjugate gradient method for finding an eigenvector ... preconditioned eigensolver be compared with this “ideal” algorithm on our model test problems in terms...
  • SPG

  • Referenced in 74 articles [sw00897]
  • Algorithm 813: SPG -- software for convex-constrained optimization: Fortran 77 software implementing the SPG method ... introduced. SPG is a nonmonotone projected gradient algorithm for solving large-scale convex-constrained optimization...
  • GradSamp

  • Referenced in 109 articles [sw05270]
  • robust gradient sampling algorithm for nonsmooth, nonconvex optimization The authors describe a practical and robust ... algorithm for computing the local minima of a continuously differentiable function in n real variables ... only request formulated is that the gradient of the function is easily computed where...
  • Pegasos

  • Referenced in 98 articles [sw08752]
  • simple and effective stochastic sub-gradient descent algorithm for solving the optimization problem cast ... example. In contrast, previous analyses of stochastic gradient descent methods for SVMs require ... size of the training set, the resulting algorithm is especially suited for learning from large...
  • SHASTA

  • Referenced in 239 articles [sw18281]
  • algorithms strictly maintain the positivity of actual mass densities so steep gradients and inviscid shocks ... utilizing SHASTA, a new transport algorithm for the continuity equation, which is described in detail...
  • TAO

  • Referenced in 44 articles [sw10597]
  • analyze the performance and scalabilty of algorithms for the solution of large optimization problems ... uses the GPCG (gradient projection, conjugate gradient) algorithm for solving bound-constrained convex quadratic problems...
  • TNPACK

  • Referenced in 42 articles [sw00970]
  • constraints by a truncated Newton algorithm. The algorithm is especially suited for problems involving ... version by using the preconditioned Conjugate Gradient algorithm (PCG) to solve approximately the Newton equations...
  • mboost

  • Referenced in 61 articles [sw07331]
  • package mboost: Model-Based Boosting. Functional gradient descent algorithm (boosting) for optimizing general risk functions...
  • XGBoost

  • Referenced in 71 articles [sw21035]
  • XGBoost is an optimized distributed gradient boosting library designed to be highly efficient, flexible ... portable. It implements machine learning algorithms under the Gradient Boosting framework. XGBoost provides a parallel...
  • CG_DESCENT

  • Referenced in 122 articles [sw04813]
  • Algorithm 851: CG_DESCENT. A conjugate gradient method with guaranteed descent Recently, a new nonlinear ... conjugate gradient scheme was developed which satisfies the descent condition gTkdk ... article studies the convergence behavior of the algorithm; extensive numerical tests and comparisons with other...