The Modified Newton Method for Systems

The idea is to reduce the number of derivative evaluations and the number of LU decompositions without hurting the convergence of the method.

The modified Newton formula is

with

The modified Newton step can be equivalently written as

This means that only one Jacobian () computation is required, and only one LU decomposition is performed, regardless of the number of iterations involved. All the linear systems () share the same ``coefficient'' matrix , and therefore we can re-use the same LU decomposition.

With modified Newton, we start with computing the matrix and its LU decomposition (with pivoting!). The algorithm does not work if is singular; if this happens we need to terminate the computations.

One step of the Modified Newton algorithm follows.

0

We have available, and want to compute the next iterate, ;

1

Evaluate the vector function (we evaluate each component function individually);

2

Solve the system (). For this, we use the available LU decomposition of . Apply the back-substitution algorithm to the right hand side , to obtain the solution ;

3

Compute the next iterate as .

If after, say, iterations the speed of convergence decreases, we need to update the derivative; set

and continue with () for .

From the third iteration on we will monitor the progress of the iterations. For a good starting point, if the iterations proceed all right, we have

Estimate

(with to guard against division by zero). If then the progress of the iterations is not satisfactory; with modified Newton we have to update the derivative,

and continue with the iterations.

If for 3 or 4 consecutive steps the iteration is likely to diverge, and a better starting point should be chosen; print the proper message and exit gracefully.