The Newton Method for Systems

The Newton method is useful for solving nonlinear systems of equations of the form

There are independent variables ; in vector notation, these variables are the entries of the vector variable

The system is defined by functions (each is a function of the variables ). In vector notation,

Therefore, the system of equations () can be written compactly as

We want to find a solution of the system, i.e. a set of values such that all the equations are simultaneously satisfied

Newton's method builds a sequence of points

which converge to the true solution of the problem

In one dimension, the sequence is built using Newton's formula

For -dimensional problems, this formula generalizes to

The place of the derivative is now taken by the Jacobian matrix defined as

Let

The Newton formula () can be written as

This is a system of linear equations; is a matrix, and and are -dimensional vectors.

One step of the Newton method (say, the step) proceeds as follows:

0

We have available, and want to compute the next iterate, ;

1

Evaluate the vector function (we evaluate each component function individually);

2

Evaluate the derivative matrix (each entry is a function of and need to be evaluated individually);

3

Solve the system (). For this, we need to

3.1

compute the LU decomposition (with pivoting) of the matrix ;

3.2

apply the back-substitution algorithm to the right hand side , to obtain the solution ;

This step cannot be performed if is singular; if this happens we need to terminate the computations.

4

Compute the next iterate as .

Note that one Newton step is very expensive. We have to evaluate functions (the entries of ) and we need to calculate the LU decomposition of each step.