Efficiency (statistics)

You don't need to be Editor-In-Chief to add or edit content to WikiDoc. You can begin to add to or edit text on this WikiDoc page by clicking on the edit button at the top of this page. Next enter or edit the information that you would like to appear here. Once you are done editing, scroll down and click the Save page button at the bottom of the page.

In statistics, efficiency is one measure of desirability of an estimator. The efficiency of an unbiased statistic T is defined as

where is the Fisher information of the sample. Thus e(T) is the minimum possible variance for an unbiased estimator divided by its actual variance. The Cramér-Rao bound can be used to prove that :

Efficient estimator

If an unbiased estimator of a parameter attains e(T) = 1 for all values of the parameter, then the estimator is called efficient.

Equivalently, the estimator achieves equality on the Cramér-Rao inequality for all .

An efficient estimator is also a minimum variance unbiased estimator. This is because an efficient estimator maintains equality on the Cramér-Rao inequality for all parameter values, which means it attains the minimum variance for all parameters (the definition of an MVU estimator).

An MVU estimator is not necessarily efficient, because "minimum" does not mean equality holds on the Cramér-Rao inequality.

Asymptotic efficiency

For some estimators, they can attain efficiency asymptotically and are thus called asymptotically efficient estimators. This can be the case for some maximum likelihood estimators or for any estimators that attain equality of the Cramér-Rao bound asymptotically.

Examples

Consider a sample of size N drawn from a normal distribution of mean μ and unit variance (i.e., ).

The sample mean, , of the sample , defined as

has variance . This is equal to the reciprocal of the Fisher information from the sample (this is clear from the definition) and thus, by the Cramér-Rao inequality, the sample mean is efficient in the sense that its efficiency is unity.

Now consider the sample median. This is an unbiased and consistent estimator for μ. For large N the sample median is approximately normally distributed with mean μ and variance (i.e., ). The efficiency is thus , or about 64%. Note that this is the asymptotic efficiency — that is, the efficiency in the limit as sample size N tends to infinity. For finite values of N the efficiency is higher than this (for example, a sample size of 3 gives an efficiency of about 74%).

Many workers prefer the sample median as an estimator of the mean, holding that the loss in efficiency is more than compensated for by its enhanced robustness in terms of its insensitivity to outliers.

Relative efficiency

If T₁ and T₂ are estimators for the parameter θ, then T₁ is said to dominate T₂ if:

1. its mean squared error (MSE) is smaller for at least some value of θ
2. the MSE does not exceed that of T₂ for any value of θ.

Formally,

holds for all θ, with strict inequality holding somewhere.

The relative efficiency is defined as

Although e is in general a function of θ, in many cases the dependence drops out; if this is so, e being greater than one would indicate that T₁ is preferable, whatever the true value of θ.de:Effizienz (Statistik) it:Efficienza (statistica)