The study of empirical processes is a branch of mathematical statistics and a sub-area of probability theory. Mathematical statistics uses probability theory and other branches of mathematics to study statistics from a purely mathematical standpoint. ... Probability theory is the mathematical study of phenomena characterized by randomness or uncertainty. ...

The motivation for studying empirical processes is that it is often impossible to know the true underlying probability measure P. We collect observations and compute relative frequencies. We can estimate P, or a related distribution function F by means of the empirical measure or empirical distribution function, respectively. Theorems in the area of empirical processes confirm that these are uniformly good estimates or determine accuracy of the estimation. In mathematics, a probability space is a set S, together with a σ-algebra X on S and a measure P on that σ-algebra such that P(S) = 1. ...

Suppose X is a sample space of observations. X can be quite general; for example: the real line, some Euclidean space, a space of functions, a Riemannian manifold, or whatever might be of interest. Let be independent identically distributed (iid) random variables (rv's), with probability measure P on X. For a measurable set A, define In probability theory, the sample space, often denoted S, Ω or U (for universe), of an experiment or random trial is the set of all possible outcomes. ... In mathematics, the real line is simply the set of real numbers. ... In mathematics, Euclidean space is a generalization of the 2- and 3-dimensional spaces studied by Euclid. ... In Riemannian geometry, a Riemannian manifold is a real differentiable manifold in which each tangent space is equipped with an inner product in a manner which varies smoothly from point to point. ... A random variable is a mathematical function that maps outcomes of random experiments to numbers. ... In mathematics, a probability space is a set S, together with a σ-algebra X on S and a measure P on that σ-algebra such that P(S) = 1. ... In mathematics, a measure is a function that assigns a number, e. ...

If C is a collection of subsets of X, then the collection

is the empirical measure indexed by C. The empirical process B_n is defined as

and

is the empirical process indexed by C.

A special case is the empirical process G_n associated with empirical distribution functions F_n. In statistics, an empirical distribution function is a cumulative probability distribution function that concentrates probability 1/n at each of the n numbers in a sample. ...

where are real-valued random variables with distribution function F and F_n is defined by

In this case,

Major results for this special case include Kolmogorov-Smirnov statistics, the Glivenko-Cantelli theorem and Donsker's theorem. Moreover, the empirical distribution function F_n of a finite sequence of realizations of a random variable is the very essence of statistical inference. In statistics, the Kolmogorov-Smirnov test is used to determine whether two underlying probability distributions differ from each other or whether an underlying probability distribution differs from a hypothesized distribution, in either case based in finite samples. ... Suppose is a sample space of observations. ... Suppose is a sample space of observations. ... The topics below are usually included in the area of interpreting statistical data. ...

Glivenko-Cantelli theorem

By the strong law of large numbers, we know that The law of large numbers is a fundamental concept in statistics and probability that describes how the average of a randomly selected sample from a large population is likely to be close to the average of the whole population. ...

However, Glivenko and Cantelli strengthened this result.

The Glivenko-Cantelli theorem (1933):

Another way to state this is as follows: the sample paths of F_n get uniformly closer to F as n increases; hence F_n, which we observe, is almost surely a good approximation for F, which becomes better as we collect more observations. In mathematics, specifically, in probability theory, the phrase almost surely is a concise, precise way to state except on a set or event of probability measure zero. ...

Donsker's theorem

By the classical central limit theorem, it follows that A central limit theorem is any of a set of weak-convergence results in probability theory. ...

that is, G_n(x) converges in distribution to a Gaussian (normal) random variable G(x) with mean 0 and variance F(x)[1 − F(x)]. Donsker (1952) showed that the sample paths of G_n(x), as functions on the real line R, converge in distribution to a stochastic process G in the space l^∞ of all bounded functions . The function space l^∞ is used in this context to remind us that we are concerned with distributional convergence in terms of sample paths. The limit process G is a Gaussian process with zero mean and covariance given by Generally, the word gaussian pertains to Carl Friedrich Gauss and his ideas. ... Normal may refer to: Normal (behavior) Normal (mathematics), a group of mathematical concepts Surface normal, a line or vector perpendicular to a surface Normal (movie), a 2003 film directed by Jane Anderson Normal, Alabama, home to Alabama Agricultural and Mecahnical University Normal, Illinois, a town in the United States Normal... A random variable is a mathematical function that maps outcomes of random experiments to numbers. ... In mathematics, the real line is simply the set of real numbers. ... A Gaussian process is a stochastic process {Xt}t ∈T such that every finite linear combination of the Xt (or, more generally, any linear functional of the sample function Xt) is normally distributed. ...

cov[G(s), G(t)] = E[G(s)G(t)] = F[min(s, t)] − F(s)F(t).

The process G(x) can be written as B(F(x)) where B is a standard Brownian bridge on the unit interval. This article may be too technical for most readers to understand. ...

If the observations are in a more general sample space X, we seek generalizations of the Glivenko-Cantelli theorem and Donsker's theorem. Also, we seek other theorems to determine rates of convergence and accuracy of estimation.

The classical empirical distribution function for real-valued random variables is a special case of the general theory with X = R and the class of sets .

See also

• Glivenko-Cantelli class
• Donsker class

References

• P. Billingsley, Probability and Measure, John Wiley and Sons, New York, second edition, 1986.
• P. Billingsley, Probability and Measure, John Wiley and Sons, New York, third edition, 1995.

• M.D. Donsker, Justification and extension of Doob's heuristic approach to the Kolmogorov-Smirnov theorems, Annals of Mathematical Statistics, 23:277--281, 1952.
• R.M. Dudley, Central limit theorems for empirical measures, Annals of Probability, 6(6): 899–929, 1978.
• R.M. Dudley, Uniform Central Limit Theorems, Cambridge Studies in Advanced Mathematics, 63, Cambridge University Press, Cambridge, UK, 1999.
• J. Wolfowitz, Generalization of the theorem of Glivenko-Cantelli. Annals of Mathematical Statistics, 25, 131-138, 1954.

External links

• Empirical Processes: Theory and Applications, by David Pollard, a textbook available online.
• Introduction to Empirical Processes and Semiparametric Inferences, by Michael Kosorok, another textbook available online.