In statistics, the delta method is a method for deriving an approximate probability distribution for a function of an asymptotically normal statistical estimator from knowledge of the limiting variance of that estimator. More broadly, the delta method may be considered a fairly general central limit theorem. A graph of a Normal bell curve showing statistics used in educational assessment and comparing various grading methods. ... In mathematics and statistics, a probability distribution is a function of the probabilities of a mutually exclusive and exhaustive set of events. ... Graph of example function, The mathematical concept of a function expresses the intuitive idea of deterministic dependence between two quantities, one of which is viewed as primary (the independent variable, argument of the function, or its input) and the other as secondary (the value of the function, or output). A... In statistics, an estimator is a function of the observable sample data that is used to estimate an unknown population parameter; an estimate is the result from the actual application of the function to a particular set of data. ... In statistics, an estimator is a function of the observable sample data that is used to estimate an unknown population parameter; an estimate is the result from the actual application of the function to a particular set of data. ... In probability theory and statistics, the variance of a random variable (or somewhat more precisely, of a probability distribution) is a measure of its statistical dispersion, indicating how its possible values are spread around the expected value. ... A central limit theorem is any of a set of weak-convergence results in probability theory. ...

Univariate delta method

While the delta method generalizes easily to a multivariate setting, careful motivation of the technique is more easily demonstrated in univariate terms. Roughly, for some sequence of random variables X_n satisfying This is a page about mathematics. ...

where θ and σ² are finite valued constants and denotes convergence in distribution, it is the case that In probability theory, there exist several different notions of convergence of random variables. ...

for some function g satisfying the property that g'(θ) exists and is non-zero valued. (The final restriction is really only needed for purposes of clarity in argument and application. Should the first derivative evaluate to zero at θ, then the delta method may be extended via use of a second or higher order Taylor series expansion.) As the degree of the Taylor series rises, it approaches the correct function. ...

Proof in the univariate case

Demonstration of this result is fairly straightforward under the assumption that g'(θ) is continuous. To begin, we construct a first-order Taylor series expansion of g(X_n) around θ: In mathematics, a continuous function is a function for which, intuitively, small changes in the input result in small changes in the output. ...

where lies between X_n and θ. Note that since implies and g'(θ) is continuous, applying Slutsky's Theorem yields Slutskys theorem is a fundamental result in probability theory attributed to Eugen Slutsky. ...

where denotes convergence in probability. In probability theory, there exist several different notions of convergence of random variables. ...

Rearranging the terms and multiplying by gives

Since

by assumption, it follows immediately from appeal to Slutsky's Theorem that Slutskys theorem is a fundamental result in probability theory attributed to Eugen Slutsky. ...

This concludes the proof.

Motivation of multivariate delta method

By definition, a consistent estimator B converges in probability to its true value β, and often a central limit theorem can be applied to obtain asymptotic normality: In statistics, a consistent estimator is an estimator that converges in probability to the quantity being estimated as the sample size grows. ... In statistics, an estimator is a function of the observable sample data that is used to estimate an unknown population parameter; an estimate is the result from the actual application of the function to a particular set of data. ... In probability theory, there exist several different notions of convergence of random variables. ... A central limit theorem is any of a set of weak-convergence results in probability theory. ... In statistics, an estimator is a function of the observable sample data that is used to estimate an unknown population parameter; an estimate is the result from the actual application of the function to a particular set of data. ...

where n is the number of observations. Suppose we want to estimate the variance of a function h of the estimator B. Keeping only the first two terms of the Taylor series, and using vector notation for the gradient, we can estimate h(B) as As the degree of the Taylor series rises, it approaches the correct function. ... For other uses, see Gradient (disambiguation). ...

which implies the variance of h(B) is approximately

The delta method therefore implies that

or in univariate terms,

Note

The delta method is nearly identical to the formulae presented in Klein (1953, p. 258):

where h_r is the r^th element of h(B) and B_i is the i^th element of B. The only difference is that Klein stated these as identities, whereas they are actually approximations.

References

• Casella, G. and Berger, R. L. (2002), Statistical Inference, 2nd ed.
• Oehlert, G. W. (1992), A Note on the Delta Method, The American Statistician, Vol. 46, No. 1, p. 27-29.
• Greene, W. H. (2003), Econometric Analysis, 5th ed., pp. 913f.
• Klein, L. R. (1953), A Textbook of Econometrics, p. 258.
• Lecture notes from Indiana University
• More lecture notes
• Explanation from Stata software corporation