The Newton Method for Nonlinear Equations

The Newton method is useful for solving nonlinear equations of the form

More exactly, given , an initial, rough guess of the solution, the method builds a sequence of points

which converge to the true solution of the problem

where

We start with an intial guess . can be expanded in Taylor series around the guessed solution

Our purpose is to derive a better approximation for . From the above formula we have

In this exact formula the higher order terms of the right hand side depend on the unknown . We simply ignore them to arrive at the approximate relation

The obtained is (hopefully) a better approximation to than the initial guess was. To obtain an even better approximation we repeat the procedure with the ``guess'' and arrive at

It is clear now that we can repeat the steps as many times as we need, untill is sufficiently close to .

The sequence of succesive approximations is built recursively using Newton's formula

From the formula we infer that

• each iteration requires one evaluation of the function and one evaluation of its derivative ;
• the iteration step cannot be performed if .

Usually, if and sufficiently close to then Newton iterations converge very fast.

The order of convergence of an iterative procedure is the largest number for which

(here is some constant which does not depend on ). The larger is the faster the method converges.

For Newton's method ; we say that Newton's method converges quadratically. To see this we develop in Taylor series around :