• KELLEY

  • Referenced in 617 articles [sw04829]
  • cases the noise often introduces artificial minimizers. Gradient information, even if available, cannot expected ... methods use finite difference approximations of the gradient, which are adjusted to the noise level...
  • Adam

  • Referenced in 436 articles [sw22205]
  • introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based ... invariant to diagonal rescaling of the gradients, and is well suited for problems that ... problems with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically...
  • SNOPT

  • Referenced in 527 articles [sw02300]
  • derivatives are available and that the constraint gradients are sparse. We discuss an SQP algorithm...
  • LSQR

  • Referenced in 372 articles [sw00530]
  • equivalent to the standard method of conjugate gradients, but possesses more favorable numerical properties. Reliable ... described comparing LSQR with several other conjugate-gradient algorithms, indicating that LSQR is the most...
  • MINOS

  • Referenced in 443 articles [sw05095]
  • automatic estimation of some or all gradients. Upper and lower bounds on the variables...
  • INTLAB

  • Referenced in 439 articles [sw04004]
  • automatic differentiation (forward mode, vectorized computations, fast) Gradients (to solve systems of nonlinear equations) Hessians...
  • CGS

  • Referenced in 305 articles [sw08225]
  • algorithm (a “squared” conjugate gradient method) with a preconditioning called ILLU (an incomplete line...
  • darch

  • Referenced in 270 articles [sw11086]
  • known training algorithms like backpropagation or conjugate gradient...
  • SHASTA

  • Referenced in 239 articles [sw18281]
  • positivity of actual mass densities so steep gradients and inviscid shocks are handled particularly well...
  • lobpcg.m

  • Referenced in 129 articles [sw24008]
  • locally optimal block preconditioned conjugate gradient method: Toward the optimal preconditioned eigensolver: Locally optimal block ... preconditioned conjugate gradient method. We describe new algorithms of the locally optimal block preconditioned conjugate ... gradient (LOBPCG) method for symmetric eigenvalue problems, based on a local optimization of a three ... algorithm, we advocate the standard preconditioned conjugate gradient method for finding an eigenvector...
  • tn

  • Referenced in 136 articles [sw05140]
  • discusses the use of the linear conjugate-gradient method (developed via the Lanczos method ... equivalent Lanczos characterization of the linear conjugate-gradient method may be exploited to define ... direction defined by a nonlinear conjugate-gradient-type method and a modified Newton direction. Numerical...
  • SCALCG

  • Referenced in 103 articles [sw08453]
  • SCALCG – Scaled conjugate gradient algorithms for unconstrained optimization. In this work we present and analyze ... scaled conjugate gradient algorithm and its implementation, based on an interpretation of the secant equation ... line search conditions. The best spectral conjugate gradient algorithm SCG by Birgin and Martínez ... Beale–Powell. The parameter scaling the gradient is selected as spectral gradient...
  • AdaGrad

  • Referenced in 120 articles [sw22202]
  • ADAGRAD: adaptive gradient algorithm; Adaptive subgradient methods for online learning and stochastic optimization. We present ... earlier iterations to perform more informative gradient-based learning. Metaphorically, the adaptation allows ... which employ proximal functions to control the gradient steps of the algorithm. We describe...
  • SPGL1

  • Referenced in 182 articles [sw08365]
  • complex domain. At each iteration, a spectral gradient-projection method approximately minimizes a least-squares...
  • NASTRAN

  • Referenced in 177 articles [sw04245]
  • behavior, dynamic and vibration response and thermal gradients in real-world systems, MSC Nastran...
  • CG_DESCENT

  • Referenced in 124 articles [sw04813]
  • Algorithm 851: CG_DESCENT. A conjugate gradient method with guaranteed descent Recently, a new nonlinear ... conjugate gradient scheme was developed which satisfies the descent condition gTkdk...
  • CONOPT

  • Referenced in 165 articles [sw02791]
  • CONOPT is a generalized reduced-gradient (GRG) algorithm for solving large-scale nonlinear programs involving...
  • Pegasos

  • Referenced in 98 articles [sw08752]
  • Pegasos: primal estimated sub-gradient solver for SVM. We describe and analyze a simple ... effective stochastic sub-gradient descent algorithm for solving the optimization problem cast by Support Vector ... example. In contrast, previous analyses of stochastic gradient descent methods for SVMs require...
  • TRON

  • Referenced in 111 articles [sw04714]
  • bound-constrained optimization problems. TRON uses a gradient projection method to generate a Cauchy step ... preconditioned conjugate gradient method with an incomplete Cholesky factorization to generate a direction...
  • GradSamp

  • Referenced in 110 articles [sw05270]
  • robust gradient sampling algorithm for nonsmooth, nonconvex optimization The authors describe a practical and robust ... only request formulated is that the gradient of the function is easily computed where...