A central limit theorem is any of a set of weak-convergence results in probability theory. They all express the fact that any sum of many independent and identically-distributed random variables will tend to be distributed according to a particular "attractor distribution". The most important and famous result is called The Central Limit Theorem which states that if the sum of the variables has a finite variance, then it will be approximately normally distributed (i.e. following a normal or Gaussian distribution). It has been suggested that this article or section be merged into Convergence of measures. ... It has been suggested that this article or section be merged with Probability axioms. ... In probability theory, a sequence or other collection of random variables is independent and identically distributed (i. ... The normal distribution, also called Gaussian distribution by scientists (named after Carl Friedrich Gauss due to his rigorous application of the distribution to astronomical data (Havil, 2003)) is a probability distribution of great importance in many fields. ...

Since many real processes yield distributions with finite variance, this explains the ubiquity of the normal probability distribution. In probability theory and statistics, the variance of a random variable (or somewhat more precisely, of a probability distribution) is a measure of its statistical dispersion, indicating how its possible values are spread around the expected value. ...

Several generalizations for finite variance exist which do not require identical distribution but incorporate some condition which guarantees that none of the variables exert a much larger influence than the others. Two such conditions are the Lindeberg condition and the Lyapunov condition. Other generalizations even allow some "weak" dependence of the random variables. Also, a generalization due to Gnedenko and Kolmogorov states that the sum of a number of random variables with power-law tail distributions decreasing as 1/|x|^α+1 with 0 < α < 2 (and therefore having infinite variance) will tend to a symmetric stable Lévy distribution as the number of variables grows. This article will only be concerned with the central limit theorem as it applies to distributions with finite variance. Jarl Waldemar Lindeberg (1876 &#8211; 1932) was a reader of mathematics in Helsinki. ... Aleksandr Mikhailovich Lyapunov (Александр Михайлович Ляпунов) (June 6, 1857 – November 3, 1918, all new style) was a Russian mathematician, mechanician and physicist. ... Boris Vladimirovich Gnedenko was a Russian mathematician and a student of Andrey Nikolaevich Kolmogorov. ... Andrey Nikolaevich Kolmogorov (&#1040;&#1085;&#1076;&#1088;&#1077;&#769;&#1081; &#1053;&#1080;&#1082;&#1086;&#1083;&#1072;&#769;&#1077;&#1074;&#1080;&#1095; &#1050;&#1086;&#1083;&#1084;&#1086;&#1075;&#1086;&#769;&#1088;&#1086;&#1074;) (kahl-mah-GAW-raff) (April 25, 1903 in Tambov - October 20, 1987 in Moscow) was a Russian mathematician... In probability theory, a Lévy skew alpha-stable distribution or just stable distribution, developed by Paul Lévy, is a probability distribution where sums of independent identically distributed random variables have the same distribution as the original. ...

See illustration of the central limit theorem

Here is an illustration of the central limit theorem. ...

History

Tijms (2004, p.169) writes:

See Bernstein (1945) for a historical discussion focusing on the work of Pafnuty Chebyshev and his students Andrey Markov and Aleksandr Lyapunov that led to the first proofs of the C.L.T. in a general setting. Abraham de Moivre. ... To meet Wikipedias quality standards, this article or section may require cleanup. ... Aleksandr Mikhailovich Lyapunov (Александр Михайлович Ляпунов) (June 6, 1857 – November 3, 1918, all new style) was a Russian mathematician, mechanician and physicist. ... Pafnuty Lvovich Chebyshev (Russian: ) ( May 26 [O.S. May 14] 1821 – December 8 [O.S. November 26] 1894) was a Russian mathematician. ... Andrey (Andrei) Andreyevich Markov (Russian: ) (June 14, 1856 N.S. – July 20, 1922) was a Russian mathematician. ... Aleksandr Mikhailovich Lyapunov (Александр Михайлович Ляпунов) (June 6, 1857 – November 3, 1918, all new style) was a Russian mathematician, mechanician and physicist. ...

Classical central limit theorem

The theorem most often called the central limit theorem is the following. Let X₁, X₂, X₃, ... be a sequence of random variables which are defined on the same probability space, share the same probability distribution D and are independent. Assume that both the expected value μ and the standard deviation σ of D exist and are finite. In mathematics, a sequence is a list of objects (or events) arranged in a linear fashion, such that the order of the members is well defined and significant. ... In mathematics, a probability space or probability measure is a set S, together with a σ-algebra X on S and a measure P on that σ-algebra such that P(S) = 1. ... In mathematics and statistics, a probability distribution, more properly called a probability density, assigns to every interval of the real numbers a probability, so that the probability axioms are satisfied. ... In probability and statistics, the standard deviation of a probability distribution, random variable, or population or multiset of values is a measure of the spread of its values. ...

Consider the sum S_n = X₁ + ... + X_n. Then the expected value of S_n is nμ and its standard error is σ n^1/2. Furthermore, informally speaking, the distribution of S_n approaches the normal distribution N(nμ,σ²n) as n approaches ∞. The normal distribution, also called Gaussian distribution by scientists (named after Carl Friedrich Gauss due to his rigorous application of the distribution to astronomical data (Havil, 2003)) is a probability distribution of great importance in many fields. ...

In order to clarify the word "approaches" in the last sentence, we standardize S_n by setting

Then the distribution of Z_n converges towards the standard normal distribution N(0,1) as n approaches ∞ (this is convergence in distribution). This means: if Φ(z) is the cumulative distribution function of N(0,1), then for every real number z, we have In the absence of a more specific context, convergence denotes the approach toward a definite value, as time goes on; or to a definite point, a common view or opinion, or toward a fixed or equilibrium state. ... The normal distribution, also called Gaussian distribution by scientists (named after Carl Friedrich Gauss due to his rigorous application of the distribution to astronomical data (Havil, 2003)) is a probability distribution of great importance in many fields. ... In probability theory, there exist several different notions of convergence of random variables. ... In probability theory, the cumulative distribution function (abbreviated cdf) completely describes the probability distribution of a real-valued random variable, X. For every real number x, the cdf is given by where the right-hand side represents the probability that the random variable X takes on a value less than... In mathematics, the real numbers may be described informally in several different ways. ...

or, equivalently,

where

is the sample mean. In mathematics and statistics, the arithmetic mean of a set of numbers is the sum of all the members of the set divided by the number of items in the set. ...

Proof of the central limit theorem

For a theorem of such fundamental importance to statistics and applied probability, the central limit theorem has a remarkably simple proof using characteristic functions. It is similar to the proof of a (weak) law of large numbers. For any random variable, Y, with zero mean and unit variance (var(Y) = 1), the characteristic function of Y is, by Taylor's theorem, Template:Otherusescccc A graph of a bell curve in a normal distribution showing statistics used in educational assessment, comparing various grading methods. ... Much research involving probability is done under the auspices of applied probability, the application of probability theory to other scientific domains. ... In probability theory, the characteristic function of any random variable completely defines its probability distribution. ... The law of large numbers is a fundamental concept in statistics and probability that describes how the average of a randomly selected sample from a large population is likely to be close to the average of the whole population. ... In statistics, mean has two related meanings: Look up mean in Wiktionary, the free dictionary. ... In calculus, Taylors theorem, named after the mathematician Brook Taylor, who stated it in 1712, gives the approximation of a differentiable function near a point by a polynomial whose coefficients depend only on the derivatives of the function at that point. ...

where o (t² ) is "little o notation" for some function of t  that goes to zero more rapidly than t². Letting Y_i be (X_i − μ)/σ, the standardised value of X_i, it is easy to see that the standardised mean of the observations X₁, X₂, ..., X_n is just Big O notation or Big Oh notation, and also Landau notation or asymptotic notation, is a mathematical notation used to describe the asymptotic behavior of functions. ...

By simple properties of characteristic functions, the characteristic function of Z_n is

But, this limit is just the characteristic function of a standard normal distribution, N(0,1), and the central limit theorem follows from the Lévy continuity theorem, which confirms that the convergence of characteristic functions implies convergence in distribution. The Lévy continuity theorem in probability theory is the basis for one approach to the central limit theorem. ... In the absence of a more specific context, convergence denotes the approach toward a definite value, as time goes on; or to a definite point, a common view or opinion, or toward a fixed or equilibrium state. ...

Convergence to the limit

If the third central moment E((X₁ − μ)³) exists and is finite, then the above convergence is uniform and the speed of convergence is at least on the order of 1/n^½ (see Berry-Esséen theorem).-1... In the mathematical field of analysis, uniform convergence is a type of convergence stronger than pointwise convergence. ... The central limit theorem in probability theory and statistics states that under certain circumstances the sample mean, considered as a random quantity, becomes more normally distributed as the sample size is increased. ...

The convergence normal is monotonic, in the sense that the entropy of Z_n increases monotonically to that of the normal distribution, as proven by Artstein, Ball, Barthe and Naor. Entropy of a Bernoulli trial as a function of success probability, often called the binary entropy function In information theory, information entropy or Shannons entropy is a measure of the average information content associated with the outcome of a random variable. ... In mathematics, functions between ordered sets are monotonic (or monotone, or even isotone) if they preserve the given order. ...

Pictures of a distribution being "smoothed out" by summation (showing original distribution and three subsequent convolutions): Summation is the addition of a set of numbers; the result is their sum. ... In mathematics and, in particular, functional analysis, convolution is a mathematical operator which takes two functions f and g and produces a third function that in a sense represents the amount of overlap between f and a reversed and translated version of g. ...

illustration of central limit theorem, part 1 I created these images: Image:Central_limit_thm_1. ... illustration of central limit theorem, part 2 I created this image. ... illustration of central limit theorem, part 3 I created this image. ... illustration of central limit theorem, part 4 I created this image. ...

(See Illustration of the central limit theorem for further details on these images.)

An equivalent formulation of this limit theorem starts with A_n = (X₁ + ... + X_n) / n which can be interpreted as the mean of a random sample of size n. The expected value of A_n is μ and the standard deviation is σ / n^½. If we standardize A_n by setting Z_n = (A_n - μ) / (σ / n^½), we obtain the same variable Z_n as above, and it approaches a standard normal distribution. Here is an illustration of the central limit theorem. ... This article is in need of attention from an expert on the subject. ...

Note the following apparent "paradox": by adding many independent identically distributed positive variables, one gets approximately a normal distribution. But for every normally distributed variable, the probability that it is negative is non-zero! How is it possible to get negative numbers from adding only positives? The reason is simple: the theorem applies to terms centered about the mean. Without that standardization, the distribution would, as intuition suggests, escape away to infinity. Look up paradox in Wiktionary, the free dictionary. ...

The Central Limit Theorem, as an approximation for a finite number of observations, provides a reasonable approximation only when close to the peak of the normal distribution; it requires a very large number of observations to stretch into the tails.

The Central Limit theorem also applies to sums of independent and identical discrete random variables, although in this case the convergence of the sum toward a normal distribution has singular properties: namely, a sum of discrete random variables is still a discrete random variable, so that we are confronted to a series of discrete random variables whose probability distribution converges towards a probability density function corresponding to a continuous variable (namely the normal distribution). This means that if we build a histogram of the realisations of the sum of n independent identical discrete variables, the curve that joins the centers of the upper faces of the rectangles forming the histogram converges toward a gaussian curve as n approaches . The binomial distribution article details such an application of the central limit theorem in the simple case of a discrete variable taking only two possible values. In mathematics, a random variable is discrete if its probability distribution is discrete; a discrete probability distribution is one that is fully characterized by a probability mass function. ... In mathematics, a random variable is discrete if its probability distribution is discrete; a discrete probability distribution is one that is fully characterized by a probability mass function. ... In mathematics, a random variable is discrete if its probability distribution is discrete; a discrete probability distribution is one that is fully characterized by a probability mass function. ... In mathematics, a series is often represented as the sum of a sequence of terms. ... In mathematics, a random variable is discrete if its probability distribution is discrete; a discrete probability distribution is one that is fully characterized by a probability mass function. ... In mathematics, a probability density function (pdf) serves to represent a probability distribution in terms of integrals. ... The normal distribution, also called Gaussian distribution by scientists (named after Carl Friedrich Gauss due to his rigorous application of the distribution to astronomical data (Havil, 2003)) is a probability distribution of great importance in many fields. ... Example of a histogram of 100 normally distributed random values. ... In probability theory and statistics, the binomial distribution is the discrete probability distribution of the number of successes in a sequence of n independent yes/no experiments, each of which yields success with probability p. ...

Alternative statements of the theorem

Density functions

The density of the sum of two or more independent variables is the convolution of their densities (if these densities exist). Thus the central limit theorem can be interpreted as a statement about the properties of density functions under convolution: the convolution of a number of density functions tends to the normal density as the number of density functions increases without bound, under the conditions stated above. In mathematics, a probability density function (pdf) serves to represent a probability distribution in terms of integrals. ... In mathematics and, in particular, functional analysis, convolution is a mathematical operator which takes two functions f and g and produces a third function that in a sense represents the amount of overlap between f and a reversed and translated version of g. ...

Since the characteristic function of a convolution is the product of the characteristic functions of the densities involved, the central limit theorem has yet another restatement: the product of the characteristic functions of a number of density functions tends to the characteristic function of the normal density as the number of density functions increases without bound, under the conditions stated above. In probability theory, the characteristic function of any random variable completely defines its probability distribution. ...

An equivalent statement can be made about Fourier transforms, since the characteristic function is essentially a Fourier transform. In mathematics, the Fourier transform is a certain linear operator that maps functions to other functions. ...

Products of random variables

The central limit theorem tells us what to expect about the sum of independent random variables, but what about the product? Well, the logarithm of a product is simply the sum of the logs of the factors, so the log of a product of random variables tends to have a normal distribution, which makes the product itself have a log-normal distribution. Many physical quantities (especially mass or length, which are a matter of scale and cannot be negative) are the product of different random factors, so they follow a log-normal distribution. Logarithms to various bases: is to base e, is to base 10, and is to base 1. ... In probability and statistics, the log-normal distribution is the probability distribution of any random variable whose logarithm is normally distributed. ... Random redirects here. ...

Lyapunov condition

See also Lyapunov's central limit theorem. In probability theory, Lyapunovs central limit theorem is one of the variants of the central limit theorems. ...

Let X_n be a sequence of independent random variables defined on the same probability space. Assume that X_n has finite expected value μ_n and finite standard deviation σ_n. We define

Assume that the third central moments

are finite for every n, and that

(This is the Lyapunov condition). We again consider the sum S_n=X₁+...+X_n. The expected value of S_n is m_n = ∑_i=1..nμ_i and its standard deviation is s_n. If we standardize S_n by setting Aleksandr Mikhailovich Lyapunov (Александр Михайлович Ляпунов) (June 6, 1857 – November 3, 1918, all new style) was a Russian mathematician, mechanician and physicist. ...

then the distribution of Z_n converges towards the standard normal distribution N(0,1) as above.

Lindeberg condition

In the same setting and with the same notation as above, we can replace the Lyapunov condition with the following weaker one (from Lindeberg in 1920). For every ε > 0 Jarl Waldemar Lindeberg (1876 &#8211; 1932) was a reader of mathematics in Helsinki. ...

where E( U : V > c) is E( U 1{V > c}), i.e., the expectation of the random variable U 1{V > c} whose value is U if V > c and zero otherwise. Then the distribution of the standardized sum Z_n converges towards the standard normal distribution N(0,1).

Non-independent case

There are some theorems which treat the case of sums of non-independent variables, for instance the m-dependent central limit theorem, the martingale central limit theorem, the central limit theorem for mixing processes and the central limit theorem for convex bodies. In probability theory, the central limit theorem says that the sum of many independent identically distributed random variables, when scaled appropriately, converges in distribution to a standard normal distribution. ...

Examples

• The probability distribution for total distance covered in a random walk (biased or unbiased) will tend toward a normal distribution.

• Flipping a large number of coins will result in a normal distribution for the total number of heads (or equivalently total number of tails).

In mathematics, computer science, and physics, a random walk, sometimes called a drunkards walk, is a formalisation of the intuitive idea of taking successive steps, each in a random direction. ... The normal distribution, also called Gaussian distribution by scientists (named after Carl Friedrich Gauss due to his rigorous application of the distribution to astronomical data (Havil, 2003)) is a probability distribution of great importance in many fields. ...

See also

• Diversification

Diversification in finance involves spreading investments around into many types of investments, including stocks, mutual funds, bonds, and cash. ...

References

• Henk Tijms, Understanding Probability: Chance Rules in Everyday Life, Cambridge: Cambridge University Press, 2004.
• S. Artstein, K. Ball, F. Barthe and A. Naor, "Solution of Shannon's Problem on the Monotonicity of Entropy", Journal of the American Mathematical Society 17, 975-982 (2004).
• S.N.Bernstein, On the work of P.L.Chebyshev in Probability Theory, Nauchnoe Nasledie P.L.Chebysheva. Vypusk Pervyi: Matematika. (Russian) [The Scientific Legacy of P. L. Chebyshev. First Part: Mathematics] Edited by S. N. Bernstei n.] Academiya Nauk SSSR, Moscow-Leningrad, 1945. 174 pp.

External links

• Animated examples of the CLT
• Central Limit Theorem Java
• Central Limit Theorem interactive simulation to experiment with various parameters
• CLT in NetLogo (Connected Probability - ProbLab) interactive simulation w/ a variety of modifiable parameters
• General Central Limit Theorem Activity & corresponding SOCR CLT Applet (Select the Sampling Distribution CLT Experiment from the drop-down list of SOCR Experiments)
• Generate sampling distributions in Excel Specify arbitrary population, sample size, and sample statistic.
• [1] Another proof.