Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
103,970 characters · 26 sections · 70 citation commands
Frequentist properties of Bayesian inequality tests
\onehalfspacing
Although Bayesian and frequentist properties fundamentally differ, often we can (approximately) achieve both. In other cases, Bayesian and frequentist summaries of the data differ greatly. We characterize the role of null hypothesis “shape” in determining such differences.
This paper came out of an empirical question, inspired by DeatonPaxson1998b,DeatonPaxson1998c: how can health inequality be assessed when only an ordinal health measure is observed? Specifically, the ordinal, five-category “self-reported health status” is found in many health and economics datasets, but interest is usually in the underlying (latent) health. Various relationships between two latent distributions can be “identified” by various relationships between the corresponding ordinal distributions. Although ordinal first-order stochastic dominance is relatively simple (i.e., all CDF differences are non-positive), other relationships correspond to more complex subsets of the parameter space (of ordinal CDFs). Despite such complexity, it is simple to compute Bayesian posterior probabilities for all types of relationships, even simultaneously. Indeed, often such Bayesian probabilities are straightforward to compute, whereas deriving frequentist tests for general nonlinear inequalities is notoriously difficult Wolak1991. This raises the question: do these posterior probabilities have any frequentist interpretation, like a $p$-value? Existing results comparing Bayesian and frequentist inference do not cover these more complex parameter sets; see (ref). For details of this ordinal health example, see (ref) and KaplanZhuo2019b.
The ordinal health example is a special case of testing whether a certain property holds for all possible partially identified parameter values (or, at least one) within the identified set, as discussed more generally by KlineTamer2016. The ordinal distributions can be seen as point-identified parameters that map to identified sets of latent distributions. Certain ordinal distribution pairs imply that all latent distribution pairs possess some relationship (or, that at least one does). For example, under certain conditions, latent first-order stochastic dominance implies ordinal first-order stochastic dominance. Thus, ordinal non-dominance implies latent non-dominance. The hypothesis of ordinal non-dominance can be interpreted as, “There is no latent distribution pair in the identified set that satisfies first-order stochastic dominance,” and ordinal dominance can be interpreted as, “There is at least one latent distribution pair in the identified set that satisfies first-order stochastic dominance.”
The hypotheses considered in this paper include all the types of hypotheses about the identified set described in KlineTamer2016. In addition to the hypotheses in the prior paragraph, they mention the hypothesis that a specified value is in the identified set, which translates to the point-identified parameter being in some region. They also mention specification testing, where the null hypothesis is that the identified set is nonempty. This translates to yet another region of the point-identified parameter space, and its complement (where the identified set is empty) is the null hypothesis of misspecification. If the mapping from point-identified parameter to identified set is complex, these regions generally have complex shapes not covered by the existing literature comparing Bayesian and frequentist inference; see (ref).
Inequality restrictions (often nonlinear) also feature prominently in economic theory. \footnote{Nonlinear inequalities come from other sources, too. For example, as in Kaplan2015b, $H_0 \colon \theta_1\theta_2\ge0$ can test stability of the sign of a parameter over time (or geography), the slope coefficient's sign in IV regression, or whether a treatment attenuates the effect of another regressor.} For example, inequalities involving cumulative distribution functions characterize first-order and second-order stochastic dominance, important concepts in welfare analysis and financial portfolio comparisons. Another example of nonlinear inequalities is curvature constraints on production, cost, indirect utility, and other functions. Such constraints usually result from optimization, like utility or profit maximization. Some examples are studied in detail in (ref). Additional economic examples like bifurcation in dynamic macroeconomic models are reviewed in (ref).
If (as will be shown) Bayesian and frequentist tests of the null hypothesis can disagree while credible and confidence sets coincide, why not simply report the credible or confidence set? \footnote{ \Citet{Berger2003} also notes this possible disagreement/agreement, but he writes, “The disagreement occurs primarily when testing a `precise' hypothesis” (p.\ 2), whereas we find disagreements even with inequality hypotheses. Also, CasellaBerger1987b opine, “Interval estimation is, in our opinion, superior to point null hypothesis testing,” although they do not mention composite null hypotheses like in this paper.} This makes sense if interest is only in the parameter values themselves. However, sometimes interest is in testing implications of economic theory or in specification testing. Other times, inequalities provide economically relevant summaries of a high-dimensional parameter or its properties.
Theoretically, we consider a single-draw experiment with equivalent posterior and sampling distributions. This equivalence often holds asymptotically, as in the various Bernstein--von Mises theorems in the literature. For example, after centering and scaling, a finite-dimensional parameter estimator often has an asymptotic Gaussian distribution with the same covariance matrix as the asymptotically Gaussian posterior for the same parameter. We also allow certain non-Gaussian distributions and infinite-dimensional parameters.
To quantify Bayesian--frequentist differences, we characterize the frequentist size and rejection probabilities of a particular Bayesian test. This test rejects the null hypothesis $H_0$ when the posterior probability of $H_0$ is below $\alpha$, i.e., it treats the posterior like a $p$-value. Besides being intuitive and practically salient, there are decision-theoretic reasons to examine this test; see (ref). Although size gives no insight into admissibility, it captures a practical difference between reporting Bayesian and frequentist inferences.
Specifically, we describe how the Bayesian test's size and certain rejection probabilities depend on the shape of the parameter subspace $\Theta_0$ where the null hypothesis is satisfied. If $\Theta_0$ is a half-space, then the Bayesian test has size $\alpha$, with rejection probability $\alpha$ at every point on the boundary of $\Theta_0$. If $\Theta_0$ is strictly smaller than a half-space (in a certain sense), then the Bayesian test's size is strictly above $\alpha$, with rejection probability strictly above $\alpha$ at certain boundary points of $\Theta_0$ (like support points). If $\Theta_0$ is not contained within any half-space, then the Bayesian test's size may be above, equal to, or below $\alpha$, with rejection probability strictly below $\alpha$ at certain boundary points of $\Theta_0$ (like support points of its complement).
\paragraph{Paper structure and notation}
(ref) reviews the literature. (ref) presents the setup and assumptions. (ref) contains our main results and discussion. (ref) illustrates our results with economic examples. (ref) contains proofs. (ref) contain additional details and examples. Acronyms used include those for cumulative distribution function (CDF), data generating process (DGP), negative semidefinite (NSD), posterior expected loss (PEL), probability density function (PDF), rejection probability (RP), and single crossing (SC). Notationally, $\subseteq$ is subset and $\subset$ is proper subset; $\supseteq$ is superset and $\supset$ is proper superset; $\mathcal{A}^\complement$ is the complement of set $\mathcal{A}$, and $ {}\mkern3mu\overline{\mkern-3mu\mathcal{A}}$ the closure; scalars, (column) vectors, and matrices are respectively formatted as $X$, $\boldsymbol{\mathbf{X}}$, and $\mkern3mu\underline{\mkern-3mu \boldsymbol{\mathbf{X}}\mkern-3mu}\mkern3mu$; $0(\cdot)$ denotes the zero function, i.e., $0(t)=0$ for all $t$.
Many papers compare Bayesian and frequentist inference. Here, we highlight examples of different types of conclusions (not all directly comparable to inequality testing): sometimes frequentist inference is more conservative, sometimes Bayesian, sometimes neither. Then, we discuss three more closely related papers in more detail.
Some of the literature documents cases where frequentist inference is “too conservative” from a Bayesian perspective. For testing linear inequality constraints of the form $H_0 \colon \boldsymbol{\mathbf{\theta}}\ge\boldsymbol{\mathbf{0}}$ with $\boldsymbol{\mathbf{\theta}}\in{\mathbb R}^d$, $d>1$, Kline2011 shows frequentist testing to be more conservative, especially with large $d$; see (ref) for details. As another example, under set identification, asymptotically, frequentist confidence sets for the true parameter ImbensManski2004,Stoye2009 are strictly larger than the estimated identified set, whereas Bayesian credible sets are strictly smaller when the prior is over the structural parameter of interest, as shown by MoonSchorfheide2012.\footnote{There seems to be a typo in the statement of Corollary 1(ii), switching the frequentist and Bayesian sets from their correct places seen in the Supplemental Material proof.} For testing the null of a unit root in autoregression, SimsUhlig1991 say frequentist tests “accept the null more easily” (p.\ 1592).
Other papers document cases where frequentist inference is “too aggressive” from a Bayesian perspective. Perhaps most famously, in Lindley1957's (Lindley1957) paradox, the frequentist test can reject when the Bayesian test does not. \Citet{BergerSellke1987} make a similar argument. In both cases, as noted by CasellaBerger1987b, the results follow primarily from having a large prior probability on a point (or “small interval”) null hypothesis, specifically $\operatorname{P}(H_0)=1/2$. Arguing that $\operatorname{P}(H_0)=1/2$ is “objective,” BergerSellke1987 consider even $\operatorname{P}(H_0)=0.15$ to be “blatant bias toward $H_1$.” \Citet{CasellaBerger1987b} disagree, saying $\operatorname{P}(H_0)=1/2$ is “much larger than is reasonable for most problems” (p.\ 344).
In yet other cases, Bayesian and frequentist inferences can be similar or even identical, depending on the prior. For unit root testing, SimsUhlig1991 determine sample-dependent priors that equate $p$-values and posterior probabilities. \Citet{CasellaBerger1987} compare Bayesian and frequentist one-sided testing of a location parameter, given a single draw of $X$ from an otherwise fully known density. They compare the $p$-value to the infimum of the posterior $\operatorname{P}(H_0\mid X)$ over various classes of priors. In many cases, the infimum is attained by the improper prior of Lebesgue measure on $(-\infty,\infty)$ and equals the $p$-value (p.\ 109). \Citet{BergerEtAl1994} establish an equivalence of Bayesian and conditional frequentist testing of a simple null hypothesis against a simple alternative. \Citet{GoutisEtAl1996} consider jointly testing multiple one-sided hypotheses in a single-draw Gaussian shift experiment. They propose “an impartial Bayes rule” (Section 3.2) that sets the prior $\operatorname{P}(H_0)=1/2$ to avoid “undesirable bias against $H_0$” (p.\ 1270). (This also makes the prior odds $\operatorname{P}(H_0)/\operatorname{P}(H_1)=1$, which simplifies the Bayes factor to the posterior odds, although they only examine the posterior probability of $H_0$.) If the components of $\boldsymbol{\mathbf{X}}$ are mutually independent, then taking the (improper) limit as a symmetric prior's variance goes to infinity, the posterior is proportional to one of the frequentist $p$-values they consider, but it is (weakly) smaller (p.\ 1271). This complements our setting where we do not impose independence, $\operatorname{P}(H_0)=1/2$, finite dimensionality, or restricted null hypothesis shape.
There are also many papers showing asymptotic equivalence of Bayesian credible intervals and frequentist confidence sets, due to asymptotic equivalence of posterior and sampling distributions. Our setup has such an equivalence. Besides results for point-identified parameters, asymptotically exact frequentist coverage has been established for certain Bayesian credible sets of identified sets, when the prior is (only) over some point-identified parameter and not the partially identified parameter; e.g., see KlineTamer2016. However, these are less directly related to our results on inequality testing.
\Citet{KlineTamer2016} mention many of the issues related to our paper, although they do not focus on frequentist properties of Bayesian tests. Their Sections 3 and 4 describe and interpret different statements about the identified set, but the focus is on computation and consistency of Bayesian posterior probabilities. Asymptotic approximations based on normal variables are given (Theorems 1(iii) and 3(iii,vi)) but not compared to frequentist $p$-values. In Section 5, in the context of a certain credible set, they raise the question of equality of posterior probability and $p$-value (pp.\ 348--349). They state this holds for interval-identified parameters but otherwise just refer to Kline2011.
\Citet{Kline2011} studies a special case of our setting. Similar to our setting (although we allow certain non-normal distributions), his posterior is multivariate normal (Definition 1, p.\ 3133), with covariance matrix equal to that of the multivariate normal sampling distribution. \Citet{Kline2011} motivates this equivalence from estimating a vector of means, where either the likelihood is normal and the prior is flat, or there is a Dirichlet process prior under which the asymptotic posterior is equivalent to the asymptotic sampling distribution (Sections 2.1--2.2). Most restrictive compared to our setting, the null hypothesis is that all components of the vector are non-negative.
Most relevant to our paper are Kline2011's (Kline2011) Lemmas 1 and 4 (pp.\ 3134, 3140). They show that if the null hypothesis is satisfied and at least two components of the mean vector equal zero (on the boundary of the null), then the asymptotic sampling distribution of the posterior probability of the null is first-order stochastically dominated by the standard uniform distribution. That is, if $\hat\pi$ is the posterior probability of $H_0$, then $\operatorname{P}(\hat\pi \le \alpha)>\alpha$ for all $\alpha\in(0,1)$. Consequently, the test that rejects when the posterior is below $\alpha$ has type I error rate (and thus size) above $\alpha$ in such a situation. Although not discussed in the original paper, this also implies $\operatorname{P}(\hat\pi \ge 1-\alpha)<\alpha$, so the type I error rate is below $\alpha$ if the null and alternative hypotheses are switched. \Citet[\S4]{Kline2011} also provides numerical examples and simulations showing that the magnitude of the discrepancy increases with the dimension.
\Citeauthor{Kline2011}'s (Kline2011) results are consistent with ours, but we study a more general null hypothesis. By restricting the null hypothesis, Kline2011 can more precisely quantify the Bayesian test's rejection probabilities (Lemma 1) and can propose a frequentist test using the posterior probability of $H_0$ as the test statistic, but with critical value smaller than $\alpha$ (which can be interpreted as an adjusted loss function, in light of our (ref)). Our null hypothesis is simply that the parameter lies in some subset of the parameter space, with no restrictions on the shape of the subset. Consequently, our results apply to more settings. For example, the applications in (ref) and most others mentioned in (ref) are not covered by the results of Kline2011. In particular, we allow hypotheses about identified sets without restricting the mapping from point-identified parameter to identified set.
\Citet{EfronTibshirani1998} consider the confidence level for a parameter $\boldsymbol{\mathbf{\mu}}$ being in a certain region of the parameter space. They call this the “problem of regions.” In particular, they focus on the confidence level of the region in which the point estimate $\hat{\boldsymbol{\mathbf{\mu}}}$ lies. They note (p.\ 1698) that their “first-order bootstrap” confidence value is essentially a Bayesian posterior probability given a flat prior. Although their main contribution is a bootstrap adjustment to better match the “correct” frequentist confidence level, most related to our paper is their discussion of how the Bayesian (i.e., first-order bootstrap) and frequentist measures differ depending on the curvature of the boundary between the regions. Qualitatively, they note the Bayesian posterior probability is higher than the frequentist confidence level when the boundary “curves away from $\hat{\boldsymbol{\mathbf{\mu}}}$,” whereas it is lower when the boundary “curves toward $\hat{\boldsymbol{\mathbf{\mu}}}$” (p.\ 1690). In terms of hypothesis testing, this implies the Bayesian test is more likely (than a frequentist test) to reject the null when the boundary of the null region “curves away from $\hat{\boldsymbol{\mathbf{\mu}}}$,” i.e., has a convex boundary near $\hat{\boldsymbol{\mathbf{\mu}}}$. Similarly, it implies the Bayesian test is less likely to reject if the null region's boundary is non-convex near $\hat{\boldsymbol{\mathbf{\mu}}}$. \Citet{EfronTibshirani1998} also discuss the better agreement of Bayesian and frequentist confidence when i) the boundary can be written as a level set of a smooth function of the underlying parameter and ii) the Welch--Peers prior WelchPeers1963 can be used (e.g., \S4 or Remark B on p.\ 1714).
However, the theoretical results in EfronTibshirani1998 stipulate an asymptotic approximation local to a smooth boundary of the null region. More precisely, up to smaller-order terms, the boundary is treated as quadratic, as in their (2.10). This greatly simplifies matters; e.g., the distance from $\hat{\boldsymbol{\mathbf{\mu}}}$ to the nearest boundary point has a normal distribution with mean and variance related to the quadratic coefficient matrix, as in their (2.12). Asymptotically, the smooth boundary looks nearly flat, with only $O(n^{-1/2})$ curvature (p.\ 1692 before (2.14)). Consequently, even with no adjustment, Bayesian and frequentist confidence levels differ by merely $O_p(n^{-1/2})$; see (2.24) and following text (p.\ 1694). Further, the approximate connection with Welch--Peers priors in Section 4 assumes the boundary is a level set of some scalar-valued function of $\boldsymbol{\mathbf{\mu}}$. In contrast, our results do not restrict the boundary of the null hypothesis region in any way. This is important since many null hypotheses entail non-smooth boundaries, such as joint tests of multiple inequalities Kline2011. Of course, the benefit of their stronger assumptions is that EfronTibshirani1998 can more precisely approximate the quantitative difference between frequentist confidence and Bayesian posterior probability, as in their (2.21) or (2.24).
In (ref), a specific Bayesian hypothesis test is described along with the decision-theoretic context. In (ref), the assumptions used for the results in (ref) are presented and discussed. (ref) discusses an alternative approach for this setting.
The Bayesian test rejects the null hypothesis when its posterior probability is below $\alpha$.
Besides seeming intuitive, (ref) is a generalized Bayes decision rule. Specifically, it minimizes posterior expected loss (PEL) for the loss function taking value $1-\alpha$ for type I error, $\alpha$ for type II error, and zero otherwise. Letting $H=0$ if $H_0$ is true and $H=1$ if $H_0$ is false, the loss function is
Thus, given a particular decision but unknown $H$,
To compute PEL, let $\operatorname{P}(\cdot \mid \boldsymbol{\mathbf{X}})$ denote the posterior probability given observed data $\boldsymbol{\mathbf{X}}$, and $\operatorname{E}(\cdot\mid\boldsymbol{\mathbf{X}})$ the posterior expectation. The PEL for the decision to reject $H_0$ is
i.e., the type I error loss ($1-\alpha$) times the posterior probability that $H_0$ is true and thus “reject” is a type I error. Similarly, the PEL of accepting $H_0$ is
i.e., the type II error loss ($\alpha$) times the probability that $H_0$ is false and thus “accept” is a type II error. PEL is thus minimized by rejecting $H_0$ if $\operatorname{P}(H_0 \mid \boldsymbol{\mathbf{X}}) \le \alpha$ and accepting $H_0$ otherwise; i.e., (ref) is smaller than (ref) if and only if $\operatorname{P}(H_0 \mid \boldsymbol{\mathbf{X}})\le\alpha$, with equality when $\operatorname{P}(H_0 \mid \boldsymbol{\mathbf{X}})=\alpha$.
The Bayesian test's type I error rates and size are compared to $\alpha$ (instead of some other value) for both practical and decision-theoretic reasons. Practically, the Bayesian test can be seen as treating the posterior as a (frequentist) $p$-value; we want to know if this is valid, similar to CasellaBerger1987. Decision-theoretically, given the same loss function from (ref) under which the Bayesian test in (ref) is a generalized Bayes rule, any unbiased frequentist test with size $\alpha$ is a minimax risk decision rule; see (ref).
Ideally, a single decision rule minimizes both maximum risk and PEL. However, if the Bayesian test's size is above or below $\alpha$, this is not possible.
The two main assumptions are stated and discussed in (ref).
(ref) states conditions on the sampling and posterior distributions of a functional $\phi(\cdot)$ in the single-draw experiment we consider. As usual, the sampling distribution treats the (functional of the) data $\phi(\boldsymbol{\mathbf{X}})$ as random and conditions on the parameter $\boldsymbol{\mathbf{\theta}}$, whereas the posterior treats the (functional of the) parameter $\phi(\boldsymbol{\mathbf{\theta}})$ as random and conditions on the data $\boldsymbol{\mathbf{X}}$.
The hope is that the conditions in (ref) are reasonable approximations of the conditions encountered in practice. The use of a single draw of $\boldsymbol{\mathbf{X}}$ has precedent in papers like CasellaBerger1987 and GoutisEtAl1996. Usually, $\boldsymbol{\mathbf{X}}$ in (ref) represents some estimator $\hat{\boldsymbol{\mathbf{\theta}}}$. For example, in a particular empirical setting, if the estimator (centered at the true parameter) and posterior (centered at the estimate) are both approximately Gaussian with approximately the same covariance, then our results should provide some insight. The conditions in (ref) are now further discussed.
(ref) includes the common case of a Gaussian distribution. In ${\mathbb R}^d$, continuous linear functionals are simply linear combinations $\phi(\boldsymbol{\mathbf{X}}) = \boldsymbol{\mathbf{c}}'\boldsymbol{\mathbf{X}}$ for constant vector $\boldsymbol{\mathbf{c}}\in{\mathbb R}^d$. If $\boldsymbol{\mathbf{X}}\sim\mathrm{N}(\boldsymbol{\mathbf{\theta}},\mkern3mu\underline{\mkern-3mu \boldsymbol{\mathbf{V}}\mkern-3mu}\mkern3mu)$ and $\boldsymbol{\mathbf{\theta}}\sim\mathrm{N}(\boldsymbol{\mathbf{X}},\mkern3mu\underline{\mkern-3mu \boldsymbol{\mathbf{V}}\mkern-3mu}\mkern3mu)$, then the sampling distribution of $\boldsymbol{\mathbf{c}}'\boldsymbol{\mathbf{X}}-\boldsymbol{\mathbf{c}}'\boldsymbol{\mathbf{\theta}}$ and the posterior distribution of $\boldsymbol{\mathbf{c}}'\boldsymbol{\mathbf{\theta}}-\boldsymbol{\mathbf{c}}'\boldsymbol{\mathbf{X}}$ are both $\mathrm{N}(0,\boldsymbol{\mathbf{c}}'\mkern3mu\underline{\mkern-3mu \boldsymbol{\mathbf{V}}\mkern-3mu}\mkern3mu\boldsymbol{\mathbf{c}})$, satisfying the equivalence and symmetry conditions in (ref). In infinite-dimensional spaces, if $X(\cdot)$ is a Gaussian process in some Banach space and $\phi(\cdot)$ belongs to the dual of that space, then $\phi\bigl(X(\cdot)\bigr)$ is a scalar Gaussian random variable; e.g., see Definition 2.2.1(ii) in Bogachev1998 and vanderVaartWellner1996. For example, given Gaussian process $X(\cdot)$ and $r,r_1,r_2\in{\mathbb R}$, both $\phi(X(\cdot))=X(r)$ and $\phi(X(\cdot))=X(r_2)-X(r_1)$ are Gaussian (relevant for first-order stochastic dominance and monotonicity, respectively).
The equivalence of posterior and sampling distributions in (ref) requires that the prior not influence the posterior. Implicitly, (ref) entails an improper uninformative (flat) prior. For example, take scalar $X,\theta\in{\mathbb R}$, so $\phi(X)=X$. With sampling distribution $X \mid \theta \sim \mathrm{N}\mathopen{}\mathclose\bgroup\originalleft(\theta,1\aftergroup\egroup\originalright)$ and prior $\theta\sim\mathrm{N}(m,\tau^2)$, the posterior is
The equivalence condition is satisfied (only) in the limit as $\tau^2\to\infty$, yielding the posterior $\theta\mid X \sim \mathrm{N}\mathopen{}\mathclose\bgroup\originalleft(X,1\aftergroup\egroup\originalright)$. Alternatively, the improper “flat” prior (equal to $1$ for all $\theta$) yields the same posterior. This happens because the posterior is proportional to the likelihood times the prior, so the flat prior disappears and leaves the posterior proportional to the likelihood, which in this example is $(2\pi)^{-1/2}\exp\{-(X-\theta)^2/2\}$. Interpreted as a posterior, this gives the PDF of $\theta$ given $X$ as that of a $\mathrm{N}(X,1)$ distribution. This improper prior is fine for (ref) because only posterior probabilities are used, unlike with Bayes factors that involve a ratio of prior probabilities BayarriEtAl2012.
The uninformative prior implicit in (ref) is restrictive but often relevant. It is restrictive because in practice the prior may influence the posterior greatly. Even in large samples, a sufficiently strong prior can always influence the posterior. However, to gain any general understanding, there must be some restriction on the prior, otherwise it can arbitrarily affect the results (e.g., a dogmatic prior on a point inside $\Theta_0$, or outside). Further, in practice, often the prior does not influence the posterior much. This may be due to a large sample and/or an uninformative prior on $\boldsymbol{\mathbf{\theta}}$. For example, in the application to partially identified parameters in a binary entry game, KlineTamer2016 specify “an uninformative conjugate Dirichlet prior,” similar to that in our application in (ref). Finally, our results can be seen as trying to isolate the effect of the Bayesian or frequentist framework, apart from any particular prior.
(ref) can be interpreted asymptotically. The asymptotic equivalence of posterior and sampling distributions is the conclusion of a Bernstein--von Mises theorem. Many Bernstein--von Mises theorems have been established in different parametric, semiparametric, and even nonparametric settings; see (ref). Most directly, such results could approximately justify the posterior and sampling distribution equivalence in (ref), where $\boldsymbol{\mathbf{\theta}}$ is the parameter of interest estimated by $\hat{\boldsymbol{\mathbf{\theta}}}=\boldsymbol{\mathbf{X}}$. Alternatively, if (ref) describes a limit experiment, then $\boldsymbol{\mathbf{\theta}}$ is a local mean parameter. Then, the “size” of the test is really the “local size,” i.e., the supremum type I error in a local neighborhood of some parameter value; but statements about rejection probability retain the same interpretation. Also, the shape of the null hypothesis region may change substantially when “zooming in” on a particular local neighborhood; see below for further discussion.
The following is a simple example of an asymptotic limit experiment with a local parameter. Let $X,\theta\in{\mathbb R}$ with $\phi(v)=v$. If $Y_{ni} \stackrel{\mathit{iid}}{\sim} \mathrm{N}(\mu_n,1)$, $i=1,\ldots,n$, and $\sqrt{n}\mu_n\to\theta$, then $\sqrt{n}\bar{Y}_n=n^{-1/2}\sum_{i=1}^{n}Y_{ni} \xrightarrow{d} \mathrm{N}\mathopen{}\mathclose\bgroup\originalleft(\theta,1\aftergroup\egroup\originalright)$. More generally, if $Y_{ni} \stackrel{\mathit{iid}}{\sim} \mathrm{N}\mathopen{}\mathclose\bgroup\originalleft(m+\mu_n,\sigma^2\aftergroup\egroup\originalright)$ and $\sqrt{n}\mu_n\to\theta$, then
given consistent estimator $\hat\sigma^2 \xrightarrow{p} \sigma^2$. Since $\theta$ is the local mean parameter, assuming $\theta \in {\mathbb R}$ does not require that ${\mathbb R}$ is the original parameter space (e.g., the space for $m+\mu_n$ in the example), but it does exclude boundary points. This type of result holds for a wide variety of models, estimators, and sampling assumptions. It is most commonly used for local power analysis but has been used for purposes like ours in papers like AndrewsSoares2010.
For our purpose of approximating the finite-sample frequentist size of a Bayesian test, considering a fixed DGP and drifting centering parameter can be just as helpful as considering a fixed centering parameter and drifting DGP. This allows the relevant Bernstein--von Mises theorem to hold only for fixed (not drifting) DGPs. For example, in ${\mathbb R}^d$,
where $\boldsymbol{\mathbf{\mu}}$ is the true parameter value, $\hat{\boldsymbol{\mathbf{\mu}}}$ is a $\sqrt{n}$-normal estimator (as seen), $\mkern3mu\underline{\mkern-3mu \boldsymbol{\mathbf{\Sigma}}\mkern-3mu}\mkern3mu$ is the asymptotic covariance matrix (known or consistently estimable), and $\boldsymbol{\mathbf{X}}_n$ is a statistic based on $\hat{\boldsymbol{\mathbf{\mu}}}$ and centered at the deterministic sequence $\boldsymbol{\mathbf{\mu}}_{0,n}$ that satisfies $\sqrt{n}(\boldsymbol{\mathbf{\mu}}-\boldsymbol{\mathbf{\mu}}_{0,n}) \to \boldsymbol{\mathbf{\theta}}$, the local mean parameter. This does not have a literal meaning like “we must change $\boldsymbol{\mathbf{\mu}}_0$ if our sample size increases,” just as a drifting DGP does not mean literally that “the population distribution changes as we collect more data”; rather, it is simply a way to capture the idea of $\boldsymbol{\mathbf{\mu}}_0$ being “close to” the true $\boldsymbol{\mathbf{\mu}}$ in the asymptotics. Analogously, for the posterior, letting $\boldsymbol{\mathbf{\theta}}_n = \sqrt{n}(\boldsymbol{\mathbf{\mu}}-\boldsymbol{\mathbf{\mu}}_{0,n})$ and again $\boldsymbol{\mathbf{X}}_n = \sqrt{n}(\hat{\boldsymbol{\mathbf{\mu}}}-\boldsymbol{\mathbf{\mu}}_{0,n})$, assuming a Bernstein--von Mises theorem,
(ref) is very general. The shape of $\Theta_0$ is not restricted at all. It does assume each $\boldsymbol{\mathbf{\theta}}$ satisfies either the null or alternative hypothesis (i.e., there is no third region), but this is often true.
In practice, it matters whether $\boldsymbol{\mathbf{\theta}}$ is interpreted as the original parameter or as a local parameter as in (ref) or (ref). In our results, the shape of $\Theta_0$ matters; should this be the “global shape” of the null hypothesis region, or the “local shape” in a neighborhood of the true parameter? The answer is not obvious. In practice, one idea is to construct something like a $99.9\%$ credible set or confidence set to get an idea of the “local” shape, where “local” is based on the data, similar in spirit to the idea of RomanoShaikhWolf2014.
Consider some examples in which the relevant shape of $\Theta_0$ depends on the sample size as well as the true parameter value. Let $\boldsymbol{\mathbf{X}},\boldsymbol{\mathbf{\theta}}\in{\mathbb R}^2$. First, let $H_0 \colon \theta_1\le0,\theta_2\le0$. Thus, $\Theta_0$ is the third quadrant, with a convex corner at the origin and flat sides extending down each axis to $-\infty$. If we look at a neighborhood of the origin (or equivalently scale by $\sqrt{n}$), the shape is identical; no matter how much we zoom in, the null hypothesis is the third quadrant since $(\theta_1,\theta_2)\le(0,0) \iff (\sqrt{n}\theta_1,\sqrt{n}\theta_2)\le(0,0)$. However, consider a neighborhood of $(\theta_1,\theta_2)=(-m,0)$. Perhaps with small sample size $n$, the standard deviation is much larger than $m$, so the original $\Theta_0$ shape with the corner is appropriate. But with large enough $n$, as the standard deviation shrinks to zero, a $n^{-1/2}$-neighborhood around $(-m,0)$ no longer includes the corner at the origin. Instead, the null hypothesis boundary appears flat. That is, it becomes obvious that $\theta_1<0$, so the null hypothesis reduces to $H_0 \colon \theta_2 \le 0$. This is the same idea behind moment inequality testing procedures like in RomanoShaikhWolf2014. They hope to learn which inequalities might be binding, which is equivalent to the local shape of $\Theta_0$.
Consider a variant of the above example. Remove a notch from $\Theta_0$ near the origin: if $\theta_1>-r$ and $\theta_2>-r$, then the point $(\theta_1,\theta_2)$ is excluded from $\Theta_0$ even if $(\theta_1,\theta_2)\le(0,0)$. If $r$ is very small compared to the standard deviation, then this notch's effect is negligible; $\Theta_0$ still looks like the third quadrant. However, if the sample size is large enough that $r$ is much bigger than the standard deviation, a local neighborhood of $(-r,-r)$ looks very different: the null hypothesis is satisfied in quadrants 2, 3, and 4, not just quadrant 3. Instead of a convex corner, the null region has a non-convex corner, which leads to very different behavior of the Bayesian test.
This difference between global and local shape (and its practical importance) is illustrated further in the example of (ref).
Comparing Bayes and minimax decision rules evokes the least favorable prior. Most simply, if there exists a (least favorable) prior such that the Bayes risk of the corresponding Bayes rule equals the supremum risk (over the parameter space) of the same Bayes rule, then that rule is minimax LehmannCasella1998. More generally, there is a least favorable sequence of priors, with a corresponding sequence of Bayes rules and their Bayes risks (against the prior); any decision rule whose supremum risk equals the limit of any such Bayes risk sequence is minimax LehmannCasella1998.
Deriving a least favorable prior (sequence) would be valuable, although there are two reasons we do not pursue it in depth here. First, for hypothesis testing, the (limiting) least favorable prior is never the flat prior implicit in (ref). Even if such a prior is found, it is still helpful to understand the Bayesian test's behavior under (ref). Second, for very general problems like in (ref), minimax results are notoriously difficult to obtain. For example, before discussing least favorable priors, LehmannCasella1998 remark, “We shall not be able to determine minimax estimators for large classes of problems but, rather, will treat problems individually.” (Hypothesis testing is essentially “estimation” of the population object $\operatorname{\mathds{1}}\{\boldsymbol{\mathbf{\theta}}\not\in\Theta_0\}$.)
For reference, a simple example is worked out in (ref). The null hypothesis is $H_0 \colon \theta\le0$, with scalar $X\sim\mathrm{N}(\theta,1)$. Despite the simple setup, it is more complicated than a simple estimation problem, where any sequence of priors approaching the flat (improper) prior is least favorable. Unlike with estimation, the testing decision is most difficult near $\theta=0$, so the least favorable prior sequence must (in the limit) concentrate mass near that point. Otherwise, if for example there is significant prior mass on $\theta>10$, it is obvious that the test should reject in such cases, which lowers the posterior expected loss below the minimax risk. The prior also cannot put all mass on $\theta=0$, otherwise the Bayesian test never rejects and has zero Bayes risk. One least favorable prior sequence places mass on only two points: $\theta=0$, and a strictly positive value. In the limit, the strictly positive value converges to zero, and the prior probability of $\theta=0$ converges to $\alpha$ in a particular way (depending on the value of the positive support point). We show the limit of the corresponding Bayes risk sequence equals the minimax risk $\alpha(1-\alpha)$ from (ref). Details and discussion are in (ref).
Although the example in (ref) works out intuitively, it is unclear how well this approach generalizes. Nonetheless, it is worth investigation in future work.
(ref) contains this paper's main results. Discussion and special cases follow.
Some intuition for (ref) is now provided. The focus is on the finite-dimensional case, specifically ${\mathbb R}^2$; the infinite-dimensional argument essentially finds a finite-dimensional projection in which the same logic goes through. For simplicity, here the inverse CDF $F^{-1}(\cdot)$ is assumed to exist.
The one-dimensional version of (ref) is well known but worth mentioning since it underlies all other results. The key is the symmetry of $F(\cdot)$ in (ref). Without symmetry, as seen below, the Bayesian test's size would not generally equal $\alpha$. Consider $H_0 \colon \theta\le0$. The posterior probability of $H_0$ is strictly decreasing in $X$, and it equals $\alpha$ when $X=-F^{-1}(\alpha)$, as shown below, so the Bayesian test rejects when $X>-F^{-1}(\alpha)$. To verify this critical value, since the posterior is $\theta-X\mid X\sim F$, and $\theta\le0$ can be written $\theta-X\le-X$,
The power function is increasing in $\theta$, so the type I error rate is largest at $\theta=0$. Since the sampling distribution of $X-\theta$ (which is just $X$ when $\theta=0$) also has CDF $F(\cdot)$, size is
Without symmetry, this does not equal $\alpha$. With the symmetry condition $F(-x)=1-F(x)$ in (ref), however, the Bayesian test's size simplifies to
(ref) illustrates the intuition behind (ref). Here, $\boldsymbol{\mathbf{X}},\boldsymbol{\mathbf{\theta}}\in{\mathbb R}^2$, so $\phi_2(\boldsymbol{\mathbf{\theta}})=\boldsymbol{\mathbf{\theta}}$, $\phi_3(\cdot)=\phi(\cdot)$, and $\Phi_2=\Theta_0$.
(ref) illustrates (ref). Here, $\boldsymbol{\mathbf{X}}=(X_1,X_2)$ and $\boldsymbol{\mathbf{\theta}}=(\theta_1,\theta_2)$, where $X_1$ and $\theta_1$ are on the horizontal axis while $X_2$ and $\theta_2$ are on the vertical axis. Further, $\phi(\boldsymbol{\mathbf{\theta}})=\theta_1$, and imagine $c_0=0$ for simplicity. The left vertical line shows $\phi(\boldsymbol{\mathbf{\theta}})=c_0$, i.e., $\theta_1=0$. Thus, the null is $H_0 \colon \theta_1\le0$, reducing to the one-dimensional case. (ref) says $\phi(\boldsymbol{\mathbf{X}})-\phi(\boldsymbol{\mathbf{\theta}})\mid\boldsymbol{\mathbf{\theta}}\sim F$ and $\phi(\boldsymbol{\mathbf{\theta}})-\phi(\boldsymbol{\mathbf{X}})\mid\boldsymbol{\mathbf{X}}\sim F$, which here is $X_1-\theta_1\mid\boldsymbol{\mathbf{\theta}}\sim F$ and $\theta_1-X_1\mid\boldsymbol{\mathbf{X}}\sim F$, with symmetric $F(-x)=1-F(x)$. When $X_1=-F^{-1}(\alpha)$, for any $X_2$, $\operatorname{P}(H_0 \mid \boldsymbol{\mathbf{X}})=\alpha$ exactly; this is indicated by the right vertical line in the figure. For smaller values of $X_1$ (to the left of the vertical line), $\operatorname{P}(H_0 \mid \boldsymbol{\mathbf{X}})>\alpha$, whereas for larger $X_1$ (to the right), $\operatorname{P}(H_0 \mid \boldsymbol{\mathbf{X}})<\alpha$, as indicated in the figure. For any $\boldsymbol{\mathbf{\theta}}$ with $\phi(\boldsymbol{\mathbf{\theta}})=c_0$, i.e., with $\theta_1=0$ (along the left vertical line), the rejection probability $\operatorname{P}(X_1>-F^{-1}(\alpha)\mid\boldsymbol{\mathbf{\theta}})=\alpha$ exactly, as indicated in the figure, so the test's size is also $\alpha$. This is essentially the one-dimensional result for $X_1$ and $\theta_1$, applied to all $X_2$ and $\theta_2$.
(ref) illustrates (ref). The setup is the same as for (ref) but with a different null hypothesis $H_0 \colon \boldsymbol{\mathbf{\theta}}\in\Theta_0$ (the shaded region on the left). The set $\Delta$ from (ref) is the white area between the shaded $\Theta_0$ and the vertical (dashed) line $\phi(\boldsymbol{\mathbf{\theta}})=c_0$, which again can be thought of as $\theta_1=0$. Consider any point on the right vertical line, where previously $\operatorname{P}(H_0 \mid \boldsymbol{\mathbf{X}})=\alpha$ in (ref). Since $\Delta$ has positive posterior probability given any $\boldsymbol{\mathbf{X}}$ on that line, $H_0$ must have strictly smaller posterior probability than in (ref), i.e., strictly smaller than $\alpha$. Thus, the boundary of the rejection region must shift to the left by some amount, enlarging the rejection region. Consequently, for any point on the left vertical line where previously the rejection probability was exactly $\alpha$, the rejection probability is now strictly above $\alpha$. Most such points are now outside $\Theta_0$ (so rejection probability is power, not type I error rate), but there is one such point on the boundary of $\Theta_0$.
(ref) could be drawn for any support point of $\Theta_0$, by redefining $\phi(\cdot)$. That is, consider any point on the boundary of $\Theta_0$ that is also on a supporting hyperplane (here, tangent line) $\phi(\boldsymbol{\mathbf{\theta}})=c_0$, meaning $\Theta_0$ is entirely on one side of the line. The picture will look qualitatively similar at any such point. The key is that after $\Delta$ is carved away from the supporting half-space $\phi(\boldsymbol{\mathbf{\theta}})\le c_0$, then all points with $\phi(\boldsymbol{\mathbf{\theta}})=c_0$ have rejection probability above $\alpha$, including the point on the boundary of $\Theta_0$.
(ref) illustrates (ref). The setup is again the same except the null hypothesis. Now $\Theta_0$ is larger than the original half-space, so $\Delta$ is the shaded area within $\Theta_0$ to the right of the left vertical line ($\phi(\boldsymbol{\mathbf{\theta}})=c_0$). Reversing the logic from before, this increases the posterior probability of $H_0$ for all points $\boldsymbol{\mathbf{X}}$ on the right vertical line. This in turn shifts the boundary of the rejection region to the right, making it smaller. Thus, the rejection probability for any $\boldsymbol{\mathbf{\theta}}$ on the left vertical line is smaller than in (ref), i.e., strictly below $\alpha$, including the point on the boundary of $\Theta_0$. Again, by redefining $\phi(\cdot)$, essentially the same picture could be drawn for other points on the boundary of $\Theta_0$, specifically any support points.
(ref) is partly a result of the prior $\operatorname{P}(H_0)$ being “small” when $\Theta_0$ is small. That is, since the prior over the parameter is always the same (regardless of $\Theta_0$), the implicit prior $\operatorname{P}(H_0)$ shrinks when $\Theta_0$ shrinks. (Technically, the prior in (ref) is improper, so $\operatorname{P}(H_0)$ is not well-defined, but the qualitative idea remains.) Unless $\Theta_0$ is a half-space, this differs from BergerSellke1987 and others who only consider “objective” priors with $\operatorname{P}(H_0)=1/2$. Whether placing a prior on the null (like $\operatorname{P}(H_0)=1/2$) or on the parameter is more appropriate depends on the empirical setting. Often it is easier computationally not to set a specific $\operatorname{P}(H_0)$; e.g., one may use the same posterior to compute probabilities of many different hypotheses. However, this can lead to a small $\operatorname{P}(H_0)$ and thus large type I error rate.
Although the implicitly small $\operatorname{P}(H_0)$ partially explains (ref), the shape of $\Theta_0$ also plays an important role. For example, in ${\mathbb R}^2$, let $\boldsymbol{\mathbf{\theta}}=(\theta_1,\theta_2)$ and $H_0 \colon \theta_1\theta_2 \ge 0$, so $\Theta_0$ comprises the first and third quadrants (and thus is not contained in any half-space). This $\Theta_0$ is the same “size” as the half-space $\{\boldsymbol{\mathbf{\theta}} : \theta_1 \ge 0\}$. However, with bivariate normal sampling and posterior distributions, (ref) implies the Bayesian test of the half-space has size $\alpha$, whereas the size of the Bayesian test of $H_0 \colon \theta_1\theta_2\ge0$ may be either strictly above or equal to $\alpha$, depending on the correlation. For example, let $\boldsymbol{\mathbf{X}}=(X_1,X_2)$ have a bivariate normal sampling distribution with $\operatorname{Corr}(X_1,X_2)=-1$. Then the test is equivalent to a scalar test where $H_0$ is a finite, closed interval, in which case the Bayesian test's size strictly exceeds $\alpha$ by (ref). The same holds for other strong negative correlations, although size decreases to $\alpha$ as the correlation increases; see (ref) as well as Kaplan2015b for details. Also, simulation results in (ref) show size can exceed $\alpha$ even with prior $\operatorname{P}(H_0)=1/2$.
As a special case, (ref) can apply to $\Theta_0=\{\boldsymbol{\mathbf{\theta}} : g(\boldsymbol{\mathbf{\theta}}) \le g_0\}$ when $g(\cdot)$ is directionally differentiable, as in FangSantos2018 and others. For example, all the examples in Section 4 of FangSantos2018 concern $\boldsymbol{\mathbf{\theta}}$ belonging to a convex set. They provide a frequentist resampling scheme that is consistent and corresponding hypothesis tests that control asymptotic size. Thus, in cases like this where (ref) applies, not only is the Bayesian test's size strictly above $\alpha$, but it is also strictly above the size of an available frequentist test.
Many economic examples satisfy the condition of $\Theta_0$ being a subset of a half-space as in (ref). Switching the null and alternative hypotheses then generates examples where $\Theta_0$ contains a half-space as in (ref). Often both are relevant; e.g., both the null of first-order stochastic dominance BarrettDonald2003 and the null of non-dominance DavidsonDuclos2013 have been studied. For example, the null hypothesis of satisfaction of curvature constraints on economic functions (cost, production, etc.)\ has $\Theta_0$ smaller than a half-space, and thus the null of non-satisfaction has $\Theta_0$ larger than a half-space. The ordinal distribution relationships in (ref) and KaplanZhuo2019b also have $\Theta_0$ smaller than a half-space. The examples in Section 4 of FangSantos2018 also satisfy this condition, “encompass[ing] tests of moment inequalities, shape restrictions, and the validity of random utility models” (\S4, p.\ 25), the latter referring to KitamuraStoye2018. The general moment inequality null hypothesis $H_0 \colon \operatorname{E}[\boldsymbol{\mathbf{W}}] \le \boldsymbol{\mathbf{0}}$ with $\boldsymbol{\mathbf{W}} \in {\mathbb R}^d$ as in equation (58) of FangSantos2018 covers many other applications, but it has been studied already by Kline2011 (see (ref)); and although technically not covered by his results, first-order stochastic dominance with continuous distributions is essentially an infinite-dimensional extension of this setup. Example 4.2 of FangSantos2018 considers shape restrictions on the infinite-dimensional regression quantile process, e.g., the restriction that the coefficient on the regressor of interest is monotonic in the quantile index. In dynamic macroeconomic models, certain parameter values lead to a stable equilibrium while others do not. Studies of such bifurcation find that $95\%$ confidence sets usually include values with an unstable equilibrium; more directly, equilibrium (in)stability could be tested. Even in relatively simple models, the inequalities characterizing stability involve nonlinear functions of multiple parameters. See for example BarnettDuzhak2010, BarnettEryilmaz2016, and references therein. Finally, as described in the introduction, many general hypotheses about identified sets correspond to the point-identified parameter lying in some region $\Theta_0$.
Whether a set $\Theta$ is treated as the null or alternative hypothesis affects the properties of the Bayesian test. Combining (ref), the Bayesian test of $H_0 \colon \boldsymbol{\mathbf{\theta}}\in\Theta$ may have size strictly above $\alpha$ while the Bayesian test of $H_0 \colon \boldsymbol{\mathbf{\theta}}\not\in\Theta$ has size strictly below or equal to $\alpha$. Moreover, if $\boldsymbol{\mathbf{\theta}}$ is a support point of $\Theta$, then the rejection probability of $H_0\colon\boldsymbol{\mathbf{\theta}}\in\Theta$ is strictly above $\alpha$ by (ref); but the rejection probability of $H_0\colon\boldsymbol{\mathbf{\theta}}\not\in\Theta$ is strictly below $\alpha$ by (ref).
Many nonlinear inequalities could be recast as linear inequalities, but at the expense of additional approximation error. For example, for $g \colon {\mathbb R}^d \mapsto {\mathbb R}$ and parameter $\boldsymbol{\mathbf{\mu}} \in {\mathbb R}^d$, the nonlinear $H_0 \colon g(\boldsymbol{\mathbf{\mu}}) \le 0$ could be written as $H_0 \colon \beta \le 0$ with $\beta \equiv g(\boldsymbol{\mathbf{\mu}})$. By the (first-order) delta method Hansen2020econometrics, if $g(\cdot)$ is continuously differentiable in a neighborhood of $\boldsymbol{\mathbf{\mu}}$ and $\boldsymbol{\mathbf{G}} \equiv \mathopen{}\mathclose\bgroup\originalleft. \frac{\partial }{\partial \boldsymbol{\mathbf{u}}} g(\boldsymbol{\mathbf{u}}) \aftergroup\egroup\originalright\rvert_{\boldsymbol{\mathbf{u}}=\boldsymbol{\mathbf{\mu}}}$, then $\sqrt{n}(\hat{\boldsymbol{\mathbf{\mu}}}-\boldsymbol{\mathbf{\mu}}) \xrightarrow{d} \mathrm{N}(\boldsymbol{\mathbf{0}}, \mkern3mu\underline{\mkern-3mu \boldsymbol{\mathbf{V}}\mkern-3mu}\mkern3mu)$ implies $\sqrt{n}(\hat\beta-\beta) \xrightarrow{d} \mathrm{N}(0, \boldsymbol{\mathbf{G}}'\mkern3mu\underline{\mkern-3mu \boldsymbol{\mathbf{V}}\mkern-3mu}\mkern3mu\boldsymbol{\mathbf{G}})$, where $\hat\beta \equiv g(\hat{\boldsymbol{\mathbf{\mu}}})$. However, this approximation may be poor in finite samples.
To be concrete, consider the following delta method example, continuing the same notation. Let $d=2$ and $\beta \equiv g(\boldsymbol{\mathbf{\mu}})=\mu_1^2+\mu_2^2-1$, so $\{\boldsymbol{\mathbf{u}} : g(\boldsymbol{\mathbf{u}}) \le 0\}$ is the unit disk in ${\mathbb R}^2$; the delta method gives $\sqrt{n}(g(\hat{\boldsymbol{\mathbf{\mu}}})-g(\boldsymbol{\mathbf{\mu}})) \xrightarrow{d} \mathrm{N}(0,4\boldsymbol{\mathbf{\mu}}'\mkern3mu\underline{\mkern-3mu \boldsymbol{\mathbf{V}}\mkern-3mu}\mkern3mu\boldsymbol{\mathbf{\mu}})$. Further imagine $\hat{\boldsymbol{\mathbf{\mu}}} \sim \mathrm{N}(\boldsymbol{\mathbf{\mu}}, \mkern3mu\underline{\mkern-3mu \boldsymbol{\mathbf{V}}\mkern-3mu}\mkern3mu/n)$ in a finite sample of $n$ observations, with the corresponding posterior. Then the Bayesian test's size is strictly above $\alpha$, as in (ref). Further, as the variance $\mkern3mu\underline{\mkern-3mu \boldsymbol{\mathbf{V}}\mkern-3mu}\mkern3mu/n$ grows, $\operatorname{P}(H_0\mid\hat{\boldsymbol{\mathbf{\mu}}})\to0$ given any $\hat{\boldsymbol{\mathbf{\mu}}}$, so size grows to $1$. In apparent contradiction, $H_0 \colon \beta \le 0$ suggests the Bayesian test has approximately correct size, by (ref). The “contradiction” is simply that the asymptotic approximation is less accurate due to the delta method's linear approximation of $g(\cdot)$. We avoid this layer of delta method approximation by treating nonlinear inequalities directly, providing better finite-sample insights. Another drawback of the delta method is the existence of singularity points; e.g., with $\boldsymbol{\mathbf{\mu}}=\boldsymbol{\mathbf{0}}$ in this example, the delta method's distribution of $g(\hat{\boldsymbol{\mathbf{\mu}}})$ is degenerate even if the distribution of $\hat{\boldsymbol{\mathbf{\mu}}}$ is not.
In the special case when $X,\theta \in {\mathbb R}$, similar results to (ref) are found in the literature, like in CasellaBerger1987. Less general versions of (ref) have also been given when $X,\theta \in {\mathbb R}$.
A special case of (ref) with $\boldsymbol{\mathbf{\theta}}\in{\mathbb R}^d$ and $H_0 \colon \boldsymbol{\mathbf{\theta}}\ge\boldsymbol{\mathbf{0}}$ was explored by Kline2011, as detailed in (ref).
(ref) also includes the special case of linear inequalities in ${\mathbb R}^d$. (ref) states that for a single linear inequality $H_0 \colon \boldsymbol{\mathbf{c}}'\boldsymbol{\mathbf{\theta}} \le c_0$, the Bayesian test has size $\alpha$. (ref) states that for multiple linear inequalities, the Bayesian test's size is strictly above $\alpha$, and its RP is strictly above $\alpha$ at every boundary point of $\Theta_0$.
We illustrate (ref) through two specific examples; another is in (ref).
The original motivation for this paper was inference on relationships between ordinal distributions. \Citet{KaplanZhuo2019b} connect such ordinal relationships to relationships between corresponding latent distributions. Here, we focus on one such result.
Consider the ordinal variable self-reported health status (SRHS), commonly used in health economics. Let random variables $X$ and $Y$ denote SRHS from two different populations, like different socioeconomic or demographic groups. Let $X=1$ denote “poor” health (the worst category), $X=5$ “excellent” health (the best), and $X=2,3,4$ the ordered intermediate categories (fair, good, very good), and similarly for $Y$. These numbers have no cardinal meaning; they are only for notational convenience. Imagine they are determined by an individual's underlying latent health $X^*$ or $Y^*$, where $X=1$ is reported if $X^*\le\gamma_1$, $X=2$ is reported if $\gamma_1<X^*\le\gamma_2$, etc. The ordinal CDFs are $F_X(\cdot)$ and $F_Y(\cdot)$; the latent CDFs are $F_X^*(\cdot)$ and $F_Y^*(\cdot)$.
If $Y$ values are determined from $Y^*$ using the same $\gamma_j$ thresholds as $X$, possibly shifted by a common constant ($\gamma_j+c$), then it is possible to learn about the relative dispersion of latent $X^*$ and $Y^*$ from the ordinal $X$ and $Y$. Specifically, if there is a “single crossing” of $F_Y(\cdot)$ by $F_X(\cdot)$ from below, then certain interquantile ranges of $X^*$ are smaller than those of $Y^*$. (Other interquantile ranges lack evidence for either direction.) “Single crossing” (SC) means that $F_X(\cdot)$ is below $F_Y(\cdot)$ up to a certain category, and then is above thereafter. That is, for some category $m$, $F_X(j)<F_Y(j)$ for $j<m$ and $F_X(j)>F_Y(j)$ for $j>m$.
This can also be interpreted in terms of partial identification. The two ordinal CDFs are point-identified, but the two latent CDFs are only partially identified. That is, there is a mapping from a pair of ordinal CDFs to a (very large) set of pairs of latent CDFs that are consistent with the ordinal pair. The ordinal SC implies that all pairs of latent CDFs in the identified set have certain interquantile ranges of $X^*$ smaller than those of $Y^*$.
The null hypothesis of SC (of $Y$ by $X$) can be written in terms of the parameters as follows. There are eight unknown ordinal parameters: $F_X(j)$ and $F_Y(j)$ for $j=1,2,3,4$, since $F_X(5)=F_Y(5)=1$. Define the four CDF difference parameters
SC requires $\theta_1<0$ and $\theta_4>0$, but there are multiple possibilities for $\theta_2$ and $\theta_3$. Specifically, SC rules out $\theta_2>0>\theta_3$, but the other three quadrants of $(\theta_2,\theta_3)$ satisfy SC. We ignore the difference between $<$ and $\le$ since they are statistically indistinguishable.
Writing the SC null hypothesis as $H_0 \colon \boldsymbol{\mathbf{\theta}}\in\Theta_0$, the shape of $\Theta_0$ has both convex and non-convex features. First, consider the first and fourth components. Since $\theta_1<0<\theta_4$ is a necessary (not sufficient) condition,
where $\phi(\boldsymbol{\mathbf{\theta}})=\theta_1$ is a continuous linear functional of $\boldsymbol{\mathbf{\theta}}$. That is, $\Theta_0$ is a subset of a quarter-space, which is a subset of a half-space that satisfies the conditions in (ref). Thus, the Bayesian test's size should be strictly above $\alpha$, with rejection probability above $\alpha$ for DGPs with $\boldsymbol{\mathbf{\theta}}$ on the boundary of $\Theta_0$ near the corner $(\theta_1,\theta_4)=(0,0)$.
Second, looking at only $(\theta_2,\theta_3)$ reveals local non-convexity. Imagine $\theta_1<0<\theta_4$ by a large margin, and the sample size is large enough that the posterior of $\theta_1<0<\theta_4$ is near one. Then, the test practically only relies on $(\theta_2,\theta_3)$. But $\Theta_0$ only excludes one quadrant of the $(\theta_2,\theta_3)$ subspace, the one with $\theta_2>0>\theta_3$. When the conditions on $(\theta_1,\theta_4)$ are satisfied, this leaves three quadrants of the $(\theta_2,\theta_3)$ subspace, which is more than a half-space. In this case, the Bayesian test's size probably still equals $\alpha$, but it may reject much less under certain DGPs, like with $(\theta_2,\theta_3)$ near the origin. That is, (ref) suggests asymptotic type I error rates strictly below $\alpha$ for DGPs local to $(\theta_2,\theta_3)=(0,0)$ with $\theta_1<0<\theta_4$, but since $\Theta_0$ is not larger than a half-space globally, this may not be true in small samples.
Instead of SC, the null hypothesis could be the opposite, i.e., non-SC. Then, when $(\theta_1,\theta_4)=(0,0)$, instead of (ref) implying type I error rate above $\alpha$, (ref) implies type I error rate below $\alpha$. When $(\theta_2,\theta_3)=(0,0)$ and $\theta_1<0<\theta_4$, instead of (ref) applying asymptotically, (ref) suggests asymptotic type I error rate above $\alpha$.
The following simulations show the importance of the shape of $\Theta_0$ and sometimes sample size. Independent, iid samples of $X$ and $Y$ are taken, with sample size $n$ for both. The results show the simulated rejection probability (RP) of the Bayesian test of $X$ single crossing $Y$. DGP 1 is at the convex “corner” with $(\theta_1,\theta_4)=(0,0)$, where the Bayesian test's RP should be above $\alpha$. Specifically, $F_X(1)=F_Y(1)=0.2$, $F_X(2)=0.39<0.4=F_Y(2)$, $F_X(3)=0.5<0.6=F_Y(3)$, and $F_X(4)=F_Y(4)=0.8$. DGP 2 is at the local non-convexity with $(\theta_2,\theta_3)=(0,0)$, where the RP may be below $\alpha$ with large enough $n$. Specifically, $F_X(1)=0.18<0.22=F_Y(1)$, $F_X(2)=F_Y(2)=0.4$, $F_X(3)=F_Y(3)=0.6$, and $F_X(4)=0.82>0.78=F_Y(4)$. Since both DGPs are on the boundary of SC and non-SC, the null hypothesis can be considered either SC or non-SC, and the rejection probability can be interpreted as a type I error rate. Each simulation had $1000$ replications, with $1000$ posterior draws each.
The Bayesian test is computed as follows. Let $\boldsymbol{\mathbf{p}}_X$ be the vector with entry $j$ equal to $\operatorname{P}(X=j)$, $j=1,\ldots,5$, and similarly for $\boldsymbol{\mathbf{p}}_Y$. Let $\boldsymbol{\mathbf{n}}_X$ be the vector of data counts with entry $j$ equal to the number of observations with $X_i=j$ (out of $i=1,\ldots,n$), and similarly for $\boldsymbol{\mathbf{n}}_Y$. With a uniform prior, the posterior distributions are Dirichlet; specifically, with $\boldsymbol{\mathbf{1}}\equiv(1,1,1,1,1)$,
The Bayesian test takes draws from these posteriors and computes the proportion of draws in which SC holds, rejecting the SC null hypothesis if the proportion is below $\alpha$, and rejecting the null of non-SC if the proportion is above $1-\alpha$.
Additionally, to help isolate the role of “shape” from “size” of $\Theta_0$, a test with adjusted prior is computed. It is identical except that the prior is adjusted to have $1/2$ probability on $H_0$. Here, the original probability of $H_0$ can be simulated by drawing from the uniform prior, and then the prior is normalized using this original probability. The prior remains uniform within $\Theta_0$, and uniform within $\Theta_0^\complement$, but at two different levels instead of the same level. See GoutisEtAl1996 or KaplanZhuo2019b for details.
(ref) shows the results. “RP” is the rejection probability of the first Bayesian test, and “RP (adj)” is the rejection probability of the Bayesian test with the adjusted prior. Some patterns emerge.
For DGP 1, RP is indeed well above $\alpha$ when the null hypothesis is SC, consistent with (ref). Here, the RP actually increases as $n$ increases; it is not simply a finite-sample difference. Even with the adjusted prior, RP is well above $\alpha$ and again increasing with $n$. In contrast, when the null is non-SC, the RP is near zero, consistent with (ref).
For DGP 2, the sample size plays an important role. When the sample size is small, the posterior and sampling distributions are very dispersed, so the global shape of $\Theta_0$ matters most. Globally, for the null of SC, $\Theta_0$ is a subset of a quarter-space; (ref) suggests DGPs near the boundary may have RP above $\alpha$. Indeed, RP is again well above $\alpha$ for small $n$. With the null of non-SC, instead (ref) suggests RP below $\alpha$, which is seen in the table. As $n$ increases, the local shape of $\Theta_0$ matters more, so (ref) is more appropriate for the null of SC and (ref) is more appropriate for the null of non-SC. In this case, with the largest sample size, RP is far below $\alpha$ for the SC null, opposite the result with small $n$. Similarly, the RP becomes above $\alpha$ for the non-SC null, also opposite the small $n$ results. Like with DGP 1, the RP with adjusted prior is different, but it is not always closer to $\alpha$.
One common nonlinear inequality hypothesis in economics is a curvature constraint like concavity. Such constraints come from economic theory, often the second-order condition of an optimization problem like utility maximization or cost minimization. As noted by ODonnellCoelli2005, the Bayesian approach is appealing for imposing or testing curvature constraints due to its (relative) simplicity. However, according to (ref), since curvature is usually satisfied in a parameter subspace much smaller than a half-space, Bayesian inference similar to (ref) may be much less favorable toward the curvature hypothesis than frequentist inference would be; i.e., the size of the Bayesian test in (ref) may be well above $\alpha$. This is true in (ref) below.
Our example concerns concavity of a cost function with the “translog” functional form ChristensenEtAl1973. This has been a popular way to parameterize cost, indirect utility, and production functions. The translog is more flexible than many traditional functional forms, allowing violation of certain implications of economic theory (such as curvature) without reducing such constraints to the value of a single parameter. Since Lau1978, there has been continued interest in methods to impose curvature constraints during estimation, as well as methods to test such constraints. Although “flexible,” the translog is still parametric, so violation of curvature constraints may come from misspecification of the functional form rather than violation of economic theory. \footnote{With a nonparametric model, one may more plausibly test the theory itself, although there are always other assumptions that may be violated; see DetteEtAl2016 for nonparametrically testing negative semidefiniteness of the Slutsky substitution matrix.}
Specifically, we test concavity of cost in input prices as follows.\footnote{The “translog” example on page 346 of Dufour1989 is even simpler but appears to ignore the fact that second derivatives are not invariant to log transformations.} With output $y$, input prices $\boldsymbol{\mathbf{w}}=(w_1,w_2,w_3)$, and total cost $C(y,\boldsymbol{\mathbf{w}})$, the translog model is
Standard economic assumptions imply that $C(y,\boldsymbol{\mathbf{w}})$ is concave in $\boldsymbol{\mathbf{w}}$ Kreps1990, which corresponds to the Hessian matrix (of $C$ with respect to $\boldsymbol{\mathbf{w}}$) being negative semidefinite (NSD), which in turn corresponds to all the Hessian's principal minors of order $p$ (for all $p=1,2,3$) having the same sign as $(-1)^p$ or zero.\footnote{In some cases, only leading principal minors are required; see Mandy2018.}
For simplicity, we consider local concavity at the point $(1,1,1,1)$:
This is necessary but not sufficient for global concavity; rejecting local concavity implies rejection of global concavity. In (ref), we show that even this weaker constraint corresponds to a set of parameter values much smaller than a half-space, so (ref) applies.
Our simulation DGP is as follows. To impose homogeneity of degree one in input prices, we use the normalized model (with error term $\epsilon$ added)
for both data generation and inference. \footnote{Alternatively, cost share equations may be used. Shephard's lemma implies that the demand for input $k$ is $x_k=\partial C/\partial w_k$. The cost share for input $k$ is then $s_k = x_k w_k / C = (\partial C/\partial w_k) (w_k/C) = \partial \ln(C)/\partial \ln(w_k) = b_k + a_{yk}\ln(y) + \sum_{j=1}^{3}b_{jk}\ln(w_j) $. } The parameter values are $b_1=b_2=1/3$, $b_{11}=b_{22}=2/9-\delta$ (more on $\delta$ below), and $b_{12}=-1/9$ to make some of the inequality constraints in $H_0$ close to binding, as well as $a_0=1$, $a_y=1$, $a_{yy}=0$, $a_{yk}=0$. The other parameter values follow from imposing symmetry ($b_{km}=b_{mk}$) and homogeneity. When $\delta=0$, $\mkern3mu\underline{\mkern-3mu \boldsymbol{\mathbf{H}}\mkern-3mu}\mkern3mu$ is a matrix of zeros, on the boundary of being NSD in that each principal minor equals zero. When $\delta>0$, all principal minors satisfy NSD and are non-zero (other than $\det(\mkern3mu\underline{\mkern-3mu \boldsymbol{\mathbf{H}}\mkern-3mu}\mkern3mu)=0$, which is always true under homogeneity). We set $\delta=0.001$. In each simulation replication, an iid sample is drawn, where $\ln(y)$ and all $\ln(w_k)$ are $\mathrm{N}\mathopen{}\mathclose\bgroup\originalleft(0,\sigma=0.1\aftergroup\egroup\originalright)$, $\epsilon\sim\mathrm{N}\mathopen{}\mathclose\bgroup\originalleft(0,\sigma_\epsilon\aftergroup\egroup\originalright)$, and all variables are mutually independent. There are $n=\num{100}$ observations per sample, $\num{500}$ simulation replications, and $\num{200}$ posterior draws per replication. The local monotonicity constraints $b_1,b_2,b_3\ge0$ were satisfied in $100.0\%$ of draws among all replications.
(ref) reports values from two methods. For the method denoted “Bayesian bootstrap” in the table header, the posterior probability of $H_0$ is computed by a nonparametric Bayesian method with improper Dirichlet process prior, i.e., the Bayesian bootstrap of Rubin1981 based on Ferguson1973 and more recently advocated in economics by ChamberlainImbens2003. For the method denoted “Normal,” the parameter vector is sampled from a normal distribution with mean equal to the ordinary least squares (OLS) estimate and covariance matrix equal to the corresponding (homoskedastic) asymptotic covariance matrix estimate; this is the posterior from a homoskedastic normal linear regression model with improper prior (or asymptotically). To accommodate numerical imprecision, we deem an inequality satisfied if it is within $10^{-7}$. The simulated type I error rate is the proportion of simulated samples for which the posterior probability of $H_0$ was below $\alpha$.
(ref) shows the type I error rate of the Bayesian tests of (ref) given our DGP. The values of $\alpha$ and $\sigma_\epsilon$ are varied as shown in the table. The two Bayesian tests are very similar, always within a few percentage points of each other. As a sanity check, when $\sigma_\epsilon=0$, the RP is zero since the constraints are satisfied by construction. As $\sigma_\epsilon$ increases, the RP increases well above $\alpha$, even over $50\%$. \footnote{The results with $\delta=0.01$ are similar; with $\delta=0$, RP jumps to over $80\%$ even with $\sigma_\epsilon=0.001$.} This is consistent with (ref).
We have explored the frequentist properties of Bayesian inference on general nonlinear inequality constraints, including hypotheses about identified set properties, providing formal results on the role of the shape of the null hypothesis parameter subspace. Future work could include investigation of approaches like MuellerNorets2016 applied to nonlinear inequality testing. Moreover, one could explore how to match frequentist size by adjusting the prior or loss function.
\singlespacing
\onehalfspacing