Demystify Lindley’s paradox by connecting p-value and posterior probability

Abstract

In the hypothesis testing framework, p‑value is often computed to determine whether to reject the null hypothesis or not. On the other hand, Bayesian approaches typically compute the posterior probability of the null hypothesis to evaluate its plausibility. We revisit Lindley’s paradox and demystify the conflicting results between Bayesian and frequentist hypothesis testing procedures by casting a two-sided hypothesis as a combination of two one-sided hypotheses along the opposite directions. This formulation can naturally circumvent the ambiguities of assigning a point mass to the null and choices of using local or non-local prior distributions. As p‑value solely depends on the observed data without incorporating any prior information, we consider non-informative prior distributions for fair comparisons with p‑value. The equivalence of p‑value and the Bayesian posterior probability of the null hypothesis can be established to reconcile Lindley’s paradox. More complicated settings, such as multivariate cases, random effects models and non-normal data, are also explored for generalization of our results to various hypothesis tests.