Armijo condition, Levenberg-Marquardt


(a)
(b)
(c)
Figure 1. Steepest descent line search with Armijo condition on Rosenbrock function, using (a) Kelley's choice of initial step length, (b) Analytical Step Length (ASL), and (c) same as in (b), but with the diagonal of Hessian instead of full Hessian in calculating the step length. In the right column of the panels, 'misfit' denotes the objective function normalized by its initial value.


Figure 2. Levenberg-Marquardt method on Rosenbrock function. In the right panel, 'misfit' denotes the objective function normalized by its initial value.

Objective: Get acquainted with various optimization methods.

Procedure:

  1. Download lineSearch_LevenMarq.zip, from which extract all files to a working directory.
  2. Run step_length_driver.m
  3. Examine the programs and identify the code which specifies the (initial) step lengths. Which program is responsible for step length determination using cubic or quadratic interpolation?
  4. Run the script at the end of levmar.m
  5. Examine this program and identify the code which handles the trust region control.
  6. Change the starting point x0, and repeat previous steps. Explain observed behaviors of various methods.
  7. Examine the program Rosenbrock_optim.m, and note how various output arguments are dealt with. Follow this procedure in coding your own non-linear 2D functional for optimization purpose. Repeat previous steps.