Lindley's paradox

Lindley's paradox describes a counterintuitive situation in statistics in which the Bayesian and frequentist approaches to a hypothesis testing problem give opposite results for certain choices of the prior distribution. The problem of the disagreement between the two approaches was discussed in Harold Jeffreys' textbook; it became known as Lindley's paradox after Dennis Lindley called the disagreement a paradox in a 1957 paper.

Description of the paradox

Consider a null hypothesis H₀, the result of an experiment x, and a prior distribution that favors H₀ weakly. Lindley's paradox occurs when

1. The result x is significant by a frequentist test, indicating sufficient evidence to reject H₀, say, at the 5% level, and
2. The posterior probability of H₀ given x is high, say, 95%, indicating strong evidence that H₀ is in fact true.

These results can happen at the same time when the prior distribution is the sum of a sharp peak at H₀ with probability p and a broad distribution with the rest of the probability 1-p. It is a result of the prior having a sharp feature at H₀ and no sharp features anywhere else.

Example: heavier-than air flight

To see why this might happen, consider the example of an experiment in a very well-established field of science, such as gravitation, and consider the null hypothesis to be something like, "what we currently believe about gravity is true." The prior probability for this hypothesis is of course very large.

Now, suppose that in this particular experiment, something very unusual happens, and for whatever reason the results of observations appear to be inconsistent with gravitation as we understand it (e.g., a 1,000 ton metal structure lifts off the ground and flies at near the speed of sound). The conditional probability of this occurrence is small given the null hypothesis, and so a frequentist ought to reject it.

Yet a Bayesian knows that the prior probability of the null hypothesis is so high that this experiment is not sufficient to actually conclude that the null hypothesis is false, but rather that something unusual is going on (i.e. powered air flight). In practice, even scientists who consider themselves "frequentist" would adopt this methodology, considering information outside the experiment in drawing their inferences.

References

• Shafer, Glenn (1982). "Lindley's paradox". Journal of the American Statistical Association. 77: 325–334.