Delta method

You don't need to be Editor-In-Chief to add or edit content to WikiDoc. You can begin to add to or edit text on this WikiDoc page by clicking on the edit button at the top of this page. Next enter or edit the information that you would like to appear here. Once you are done editing, scroll down and click the Save page button at the bottom of the page.

In statistics, the delta method is a method for deriving an approximate probability distribution for a function of an asymptotically normal statistical estimator from knowledge of the limiting variance of that estimator. More broadly, the delta method may be considered a fairly general central limit theorem.

Univariate delta method

While the delta method generalizes easily to a multivariate setting, careful motivation of the technique is more easily demonstrated in univariate terms. Roughly, for some sequence of random variables X_n satisfying

where θ and σ² are finite valued constants and denotes convergence in distribution, it is the case that

for any function g satisfying the property that g'(θ) exists and is non-zero valued. (The final restriction is really only needed for purposes of clarity in argument and application. Should the first derivative evaluate to zero at θ, then the delta method may be extended via use of a second or higher order Taylor series expansion.)

Proof in the univariate case

Demonstration of this result is fairly straightforward under the assumption that g'(θ) is continuous. To begin, we construct a first-order Taylor series expansion of g(X_n) around θ:

where lies between X_n and θ. Note that since implies and g'(θ) is continuous, applying Slutsky's Theorem yields

where denotes convergence in probability.

Rearranging the terms and multiplying by gives

Since

by assumption, it follows immediately from appeal to Slutsky's Theorem that

This concludes the proof.

Motivation of multivariate delta method

By definition, a consistent estimator B converges in probability to its true value β, and often a central limit theorem can be applied to obtain asymptotic normality:

where n is the number of observations. Suppose we want to estimate the variance of a function h of the estimator B. Keeping only the first two terms of the Taylor series, and using vector notation for the gradient, we can estimate h(B) as

which implies the variance of h(B) is approximately

The delta method therefore implies that

or in univariate terms,

Example

Suppose X_n is Binomial with parameters p and n. Since

we can apply the Delta method with g(θ) = log(θ) to see

Hence, the variance of is approximately . Moreoever, if and are estimates of different group rates from independent samples of sizes n and m respectively, then the logarithm of the estimated relative risk is approximately normally distributed with variance that can be estimated by . This is useful to construct a hypothesis test or to make a confidence interval for the relative risk.

Note

The delta method is nearly identical to the formulae presented in Klein (1953, p. 258):

where h_r is the rth element of h(B) and B_iis the ith element of B. The only difference is that Klein stated these as identities, whereas they are actually approximations.

References

• Casella, G. and Berger, R. L. (2002), Statistical Inference, 2nd ed.
• Oehlert, G. W. (1992), A Note on the Delta Method, The American Statistician, Vol. 46, No. 1, p. 27-29.
• Greene, W. H. (2003), Econometric Analysis, 5th ed., pp. 913f.
• Klein, L. R. (1953), A Textbook of Econometrics, p. 258.
• Lecture notes
• More lecture notes
• Explanation from Stata software corporationit:Delta method