Levenberg-Marquardt algorithm

In mathematics and computing, the Levenberg-Marquardt algorithm (or LMA) provides a numerical solution to the problem of minimizing a function, generally nonlinear, over a space of parameters of the function. These minimization problems arise especially in least squares curve fitting and nonlinear programming.
See more at Wikipedia.org...

A non-linear optimization algorithm which uses a combined strategy of linear approximation and gradient-descent to locate a minimum, actively switching between the two according to the success or failure of the linear approximation: a so-called model-trust region approach (see Levenberg, 1944; Marquardt, 1963; Bishop, 1995; Shepherd, 1997; Press et. al., 1992).
See also, the Neural Networks chapter.