NationMaster - Encyclopedia: Rate of convergence

In numerical analysis (a branch of mathematics), the speed at which a convergent sequence approaches its limit is called the rate of convergence. Although strictly speaking, a limit does not give information about any finite first part of the sequence, this concept is of practical importance if we deal with a sequence of successive approximations for an iterative method, as then typically fewer iterations are needed to yield a useful approximation if the rate of convergence is higher. This may e.g. even make the difference between needing ten or a million iterations. Numerical analysis is the study of algorithms for the problems of continuous mathematics (as distinguished from discrete mathematics) using basic arithmetical operations like addition. ... Mathematics is often defined as the study of topics such as quantity, structure, space, and change. ... In the absence of a more specific context, convergence denotes the approach toward a definite value, as time goes on; or to a definite point, a common view or opinion, or toward a fixed or equilibrium state. ... This is a page about mathematics. ... An iterative method attempts to solve a problem (for example an equation or system of equations) by finding successive approximations to the solution starting from an initial guess. ...

1 The definition of rate of convergence
2 The extended definition of rate of convergence
3 Examples
4 References

The definition of rate of convergence

Suppose that the sequence {x_k} converges to the number ξ.

We say that this sequence converges linearly to ξ, if

$lim_k frac{|x_{k+1}-&# 0;}{|x_k-&# 0;} = mu mbox{ with } 0 < mu < 1. quadquad (1)$

The number μ is called the rate of convergence.

If (1) holds with μ = 0, then the sequence is said to converge superlinearly. One says that the sequence converges sublinearly if it converges, but (1) does not hold for any μ < 1.

The next definition is used to distinguish superlinear rates of convergence. We say that the sequence converges with order q for q > 1 to ξ if

$lim_k frac{|x_{k+1}-&# 0;}{|x_k-&# 0;^q} = mu mbox{ with } mu > 0. quadquad (2)$

In particular, convergence with order 2 is called quadratic convergence, and convergence with order 3 is called cubic convergence.

The extended definition of rate of convergence

The drawback of the above definitions (1) and (2) is that these do not catch some sequences which still converge reasonably fast, but whose "speed" is variable, such as the sequence {b_k} below. Therefore, the definition of rate of convergence is sometimes extended as follows.

Under the new definition, the sequence {x_k} converges with at least order q if there exists a sequence {ε_k} such that

$|x_k - &# 0; le varepsilon_k quadmbox{for all } k,$

and the sequence {ε_k} converges to zero with order q according to the above "simple" definition.

Examples

Consider the following sequences:

$a_0 = 1 ,, a_1 = frac12 ,, a_2 = frac14 ,, a_3 = frac18 ,, a_4 = frac1{16} ,, a_5 = frac1{32} ,, ldots ,, a_k = frac1{2^k} ,, ldots$

$b_0 = 1 ,, b_1 = 1 ,, b_2 = frac14 ,, b_3 = frac14 ,, b_4 = frac1{16} ,, b_5 = frac1{16} ,, ldots ,, b_k = frac1{4^{operatorname{floor}(k/2)}} ,, ldots$

$c_0 = 1 ,, c_1 = frac12 ,, c_2 = frac14 ,, c_3 = frac1{16} ,, c_4 = frac1{256} ,, ldots ,, c_k = frac1{2^{2^k}} ,, ldots$

$d_0 = 1 ,, d_1 = frac12 ,, d_2 = frac13 ,, d_3 = frac14 ,, d_4 = frac15 ,, d_5 = frac16 ,, ldots ,, d_k = frac1{k+1} ,, ldots$

The sequence {a_k} converges linearly to 0 with rate 1/2. More generally, the sequence Cμ^k converges linearly with rate μ if |μ| < 1. The sequence {b_k} also converges linearly to 0 with rate 1/2 under the extended definition, but not under the simple definition. The sequence {c_k} converges superlinearly. In fact, it is quadratically convergent. Finally, the sequence {d_k} converges sublinearly.

References

The simple definition is used in

Michelle Schatzman (2002), Numerical analysis: a mathematical introduction, Clarendon Press, Oxford. ISBN 0-19-850279-6.

The extended definition is used in

Kendell A. Atkinson (1988), An introduction to numerical analysis (2nd ed.), John Wiley and Sons. ISBN 0-471-50023-2.
Walter Gautschi (1997), Numerical analysis: an introduction, Birkhäuser, Boston.
Endre Süli and David Mayers (2003), An introduction to numerical analysis, Cambridge University Press. ISBN 0-521-00794-1.

Category: Mathematical analysis

Results from FactBites:

Rate of convergence - Wikipedia, the free encyclopedia (395 words)
	In numerical analysis (a branch of mathematics), the speed at which a convergent sequence approaches its limit is called the rate of convergence.
	The next definition is used to distinguish superlinear rates of convergence.
	Therefore, the definition of rate of convergence is sometimes extended as follows.

Talk:Rate of convergence - Wikipedia, the free encyclopedia (848 words)
	So, the rate of convergence is not a perfect indicator of how fast a sequence converges.
	This is the correct definition in the framework of root-finding algorithms (unless I'm wildly wrong): the error of the bisection method is 1/2^k, which converges linearly, and the error of Newton's method is something like 1/2^(2^k), which converges quadratically.
	We need to get the defintion of rate of convergence right, but ultimately, we discuss technicalities, the point of all numerical textbooks is that quadratic convergence is what we want, everything else is not important.

More results at FactBites »

COMMENTARY

Share your thoughts, questions and commentary here

Want to know more?
Search encyclopedia, statistics and forums:

Feeds | Contact
The Wikipedia article included on this page is licensed under the GFDL.
Images may be subject to relevant owners' copyright.
All other elements are (c) copyright NationMaster.com 2003-5. All Rights Reserved.
Usage implies agreement with terms.

Contents

The definition of rate of convergence GA_googleFillSlot("encyclopedia_square");

The extended definition of rate of convergence

Examples

References

The definition of rate of convergence