In probability theory, there exist several different notions of convergence of random variables. The convergence (in one of the senses presented below) of sequences of random variables to some limiting random variable is an important concept in probability theory, and its applications to statistics and stochastic processes. For example, if the average of n independent, identically distributed random variables Y_i, i = 1, ..., n, is given by Probability theory is the mathematical study of probability. ... In the absence of a more specific context, convergence denotes the approach toward a definite value, as time goes on; or to a definite point, a common view or opinion, or toward a fixed or equilibrium state. ... A random variable is a term used in mathematics and statistics. ... In mathematics, a sequence is a list of objects (or events) arranged in a linear fashion, such that the order of the members is well defined and significant. ... In mathematics, the concept of a limit is used to describe the behavior of a function as its argument either gets close to some point, or as it becomes larger and larger; or the behavior of a sequences elements, as their index becomes larger and larger. ... A graph of a bell curve in a normal distribution showing statistics used in educational assessment, comparing various grading methods. ... In the mathematics of probability, a stochastic process is a random function. ...

then as n goes to infinity, X_n converges in probability (see below) to the common mean, μ, of the random variables Y_i. This result is known as the weak law of large numbers. Other forms of convergence are important in other useful theorems, including the central limit theorem. In statistics, mean has two related meanings: the average in ordinary English, which is more correctly called the arithmetic mean, to distinguish it from geometric mean or harmonic mean. ... In a statistical context, laws of large numbers imply that the average of a random sample from a large population is likely to be close to the mean of the whole population. ... Central limit theorems are a set of weak-convergence results in probability theory. ...

Throughout the following, we assume that (X_n) is a sequence of random variables, and X is a random variable, and all of them are defined on the same probability space (Ω, F, P). In mathematics, a probability space or probability measure is a set S, together with a σ-algebra X on S and a measure P on that σ-algebra such that P(S) = 1. ...

Convergence in distribution

Suppose that F₁, F₂, ... is a sequence of cumulative distribution functions corresponding to random variables X₁, X₂, ..., and that F is a distribution function corresponding to a random variable X. We say that the sequence X_n converges towards X in distribution, if In mathematics, a sequence is a list of objects (or events) arranged in a linear fashion, such that the order of the members is well defined and significant. ... In probability theory, the cumulative distribution function (abbreviated cdf) completely describes the probability distribution of a real-valued random variable, X. For every real number x, the cdf is given by where the right-hand side represents the probability that the random variable X takes on a value less than...

for every real number a at which F is continuous. Since F(a) = Pr(X ≤ a), this means that the probability that the value of X is in a given range is very similar to the probability that the value of X_n is in that range, provided n is large enough. Convergence in distribution is often denoted by adding the letter over an arrow indicating convergence: In mathematics, the real numbers are intuitively defined as numbers that are in one-to-one correspondence with the points on an infinite line—the number line. ... In mathematics, a continuous function is a function for which, intuitively, small changes in the input result in small changes in the output. ...

Small d is also possible, although less common.

Convergence in distribution is the weakest form of convergence, and is sometimes called weak convergence. It does not, in general, imply any other mode of convergence. However, convergence in distribution is implied by all other modes of convergence mentioned in this article, and hence, it is the most common and often the most useful form of convergence of random variables. It is the notion of convergence used in the central limit theorem and the (weak) law of large numbers. Central limit theorems are a set of weak-convergence results in probability theory. ... In a statistical context, laws of large numbers imply that the average of a random sample from a large population is likely to be close to the mean of the whole population. ...

A useful result, which may be employed in conjunction with law of large numbers and the central limit theorem, is that if a function  g: R → R  is continuous, then if  X_n  converges in distribution to  X, then so too does  g(X_n)  converge in distribution to  g(X). (This may be proved using Skorokhod's representation theorem.) This fact could be taken as a definition for the convergence in distribution.

Convergence in distribution is also called convergence in law, since the word "law" is sometimes used as a synonym of "probability distribution."

Convergence in probability

We say that the sequence X_n converges towards X in probability if

for every ε > 0. Convergence in probability is, indeed, the (pointwise) convergence of probabilities. Pick any ε > 0 and any δ > 0. Let P_n be the probability that X_n is outside a tolerance ε of X. Then, if X_n converges in probability to X then there exists a value N such that, for all n ≥ N, P_n is itself less than δ. Suppose { fn } is a sequence of functions sharing the same domain in common (for the moment, we defer making precise the nature of the values of these functions, but the reader may take them to be real numbers if that makes anyone feel good). ...

Convergence in probability is often denoted by adding the letter 'P' over an arrow indicating convergence:

Convergence in probability is the notion of convergence used in the weak law of large numbers. Convergence in probability implies convergence in distribution. To prove it, it's convenient to prove the following, simple lemma: In a statistical context, laws of large numbers imply that the average of a random sample from a large population is likely to be close to the mean of the whole population. ...

Lemma

Let X, Y be random variables, c a real number and ε > 0; then

Proof of Lemma

since

Proof

For every ε > 0, due to the preceding lemma, we have:

So, we have

Taking the limit for , we obtain:

But is the cumulative distribution function F_X(a), which is continuous by hypothesis, that is In probability theory, the cumulative distribution function (abbreviated cdf) completely describes the probability distribution of a real-valued random variable, X. For every real number x, the cdf is given by where the right-hand side represents the probability that the random variable X takes on a value less than...

and so, taking the limit for , we obtain

Almost sure convergence

We say that the sequence X_n converges almost surely or almost everywhere or with probability 1 or strongly towards X if

This means that you are virtually guaranteed that the values of X_n approach the value of X, in the sense (see almost surely) that events for which X_n does not converge to X have probability 0. Using the probability space (Ω, F, P) and the concept of the random variable as a function from Ω to R, this is equivalent to the statement In mathematics, specifically, in probability theory, the phrase almost surely is a concise, precise way to state except on a set or event of probability measure zero. ...

Almost sure convergence implies convergence in probability, and hence implies convergence in distribution. It is the notion of convergence used in the strong law of large numbers. In a statistical context, laws of large numbers imply that the average of a random sample from a large population is likely to be close to the mean of the whole population. ...

Convergence in rth mean

We say that the sequence X_n converges in rth mean or in the L^r norm towards X, if r ≥ 1, E|X_n|^r < ∞ for all n, and In mathematics, the Lp and lp spaces are spaces of p-power integrable functions, and corresponding sequence spaces. ...

where the operator E denotes the expected value. Convergence in rth mean tells us that the expectation of the rth power of the difference between X_n and X converges to zero. In probability theory (and especially gambling), the expected value (or mathematical expectation) of a random variable is the sum of the probability of each possible outcome of the experiment multiplied by its payoff (value). Thus, it represents the average amount one expects to win per bet if bets with identical...

The most important cases of convergence in rth mean are:

• When X_n converges in rth mean to X for r = 1, we say that X_n converges in mean to X.
• When X_n converges in rth mean to X for r = 2, we say that X_n converges in mean square to X.

Convergence in rth mean, for r > 0, implies convergence in probability (by Chebyshev's inequality), while if r > s ≥ 1, convergence in rth mean implies convergence in sth mean. Hence, convergence in mean square implies convergence in mean. In probability theory, Chebyshevs inequality (also known as Tchebysheffs inequality, Chebyshevs theorem, or the Bienaymé-Chebyshev inequality), named after Pafnuty Chebyshev, who first proved it, states that in any data sample or probability distribution, nearly all the values are close to the mean value, and provides a...

Converse implications

The chain of implications between the various notions of convergence, above, are noted in their respective sections, but it is sometimes important to establish converses to these implications. No other implications other than those noted above hold in general, but a number of special cases do permit converses:

• If X_n converges in distribution to a constant c, then X_n converges in probability to c.

• If X_n converges in probability X, and if Pr(|X_n| ≤ b) = 1 for all n and some b, then X_n converges in rth mean to X for all r ≥ 1. In other words, if X_n converges in probability to X and all random variables X_n are almost surely bounded above and below, then X_n converges to X also in any rth mean.

• If for all ε > 0,

then X_n converges almost surely to X. In other words, if X_n converges in probability to X sufficiently quickly (i.e. the above sum converges for all ε > 0), then X_n also converges almost surely to X. This is a direct implication from Borel-Cantelli lemma.

• If S_n is a sum of n real independent random variables:

then S_n converges almost surely if and only if S_n converges in probability.

In probability theory, the Borel_Cantelli lemma is a theorem about sequences of events. ...

External links

• Convergence Wikibook

References

1. G.R. Grimmett and D.R. Stirzaker (1992). Probability and Random Processes, 2nd Edition. Clarendon Press, Oxford, pp 271--285. ISBN 0198536658.

1. M. Jacobsen (1992). Videregående Sandsynlighedsregning (Advanced Probability Theory) 3rd Edition. HCØ-tryk, Copenhagen, pp 18--20. ISBN 87-91180-71-6.