The exact contents of citations.db main_text.text for this paper — one flattened LaTeX string, title through conclusion, appendix excluded, unmodified except for removing email addresses. This is what our citation measures are computed over.
39,633 characters
Testing Monotonicity in a Finite Population
\maketitle
\begin{abstract}
We consider the extent to which we can learn from a completely randomized experiment
whether all individuals have treatment effects that are weakly of the same sign, a condition we
call \emph{monotonicity}. From a classical sampling perspective, it is well-known that monotonicity is not falsifiable. By contrast, we show from the
design-based perspective---in which the units in the population are fixed and only
treatment assignment is stochastic---that the distribution of treatment effects in the
finite population (and hence whether monotonicity holds) is formally \emph{identified}. We
argue, however, that the usual definition of identification is unnatural in the
design-based setting because it imagines knowing the distribution of outcomes over different treatment assignments for the same units. We thus evaluate the
informativeness of the data by the extent to which it enables frequentist testing and
Bayesian updating. We show that frequentist tests can have nontrivial power against some alternatives, but power is generically limited. Likewise, we show that there exist (non-degenerate) Bayesian priors that never update about whether monotonicity holds. We conclude that, despite the formal identification result, the ability to learn about monotonicity from data in practice is severely limited.
\end{abstract}
\newpage
\section{Introduction}
Let $D$ be a randomly assigned binary treatment and $Y$ a binary outcome. Researchers are
often interested in evaluating the \emph{monotonicity} assumption that everyone has a treatment effect weakly of the same sign.
\footnote{Also known as monotone treatment response \citep{manski1997monotone}.}
For
example, if $D$ is a medical intervention, we may be interested in whether some patients benefit from the treatment while others are harmed, which would indicate that outcomes could potentially be improved by better targeting.
Likewise, if $D$ is an
instrumental variable and $Y$ a treatment with imperfect compliance (more frequently
denoted $Z$ and $D$, respectively), then the monotonicity assumption is needed for
two-stage least squares to have a local-average-treatment-effect interpretation
\citep{AngristImbens(94)}.
There are two distinct approaches to statistical uncertainty in causal inference settings
that lead to different formal definitions of monotonicity. The classical perspective views
the $n$ observed units as sampled from an infinite superpopulation. Monotonicity then
imposes that there are no individuals in the superpopulation with opposite-signed
treatment effects. Formally, we assume that the potential outcomes are sampled from a
distribution $g^*_p$, $(y_i(1),y_i(0)) \sim g^*_p$, where $p =
(p_{at},p_{nt},p_{d},p_{c})$ denotes the superpopulation shares of each ``type'', where
each type corresponds to one of four possible values for $(y(1), y(0))$). Borrowing
from the instrumental-variables literature, we use $at$ to denote an ``always-taker'' with
$y(0)=y(1)=1$, and likewise use $nt$ for never-takers ($y(0)=y(1)=0$), $d$ for defiers ($y(1)=0,y(0)=1$), and $c$ for compliers
($y(1)=1,y(0)=0$). The classical definition of
monotonicity is then that $\min\{p_c,p_d\} =0$.
By contrast, the \emph{design-based} perspective views the $n$ units and their potential
outcomes as {fixed} or conditioned upon, with statistical uncertainty arising only from
the stochastic assignment of treatment. The finite population is characterized by $\theta = (\theta_{at}, \theta_{nt}, \theta_{d}, \theta_{c})$, where $\theta_t$ is the
\emph{count} of how many of the $n$ units are of type $t$. The design-based monotonicity
assumption is then that $\min\{\theta_c,\theta_d\} = 0$.
\footnote{This monotonicity assumption is distinct from the one-sided hypothesis that there are no defiers (or equivalently, no negative treatment effects), which has the (generally testable) implication that the average treatment effect is non-negative. \cite{caughey2023randomisation} employ randomization inference to construct valid test of the null of no negative treatment effects in a design-based model that also captures non-binary outcomes.}
In sum, the classical
monotonicity assumption asks whether there are any units with opposite-signed treatment
effects in the superpopulation from which the sample is drawn, while the design-based
version asks whether there are any opposite-signed treatment effects \emph{among the $n$
units at hand}.
It is well-known that the observable data are always compatible with the null that monotonicity holds in the
superpopulation. Suppose that we have a completely randomized experiment in
which $n_1$ of the $n$ units are randomly assigned to treatment. The researcher then
observes $(Y_T,Y_U) = (\sum_i D_i Y_i, \sum_i (1-D_i) Y_i)$, summarizing how many units in
each treatment group have $Y_i=1$. From the superpopulation perspective,
we have that $(Y_T,Y_U) \sim g_p$ (where $g_p$ is is the push-forward of $g_p^*$ under completely
random assignment with probability $n_1/n$). However, it is straightforward to show that
$p$ is not point-identified \citep[see, e.g.,][]{heckman_making_1997}. In particular, for any $p_1$ that violates monotonicity,
there exists a $p_0$ satisfying monotonicity such that $g_{p_1} = g_{p_0}$. Hence, from
the superpopulation perspective, the monotonicity assumption is not falsifiable.
\footnote{We note that monotonicity may be testable when combined with other assumptions. For example, there are tests of the joint assumptions of instrument monotonicity, exclusion, and independence \citep[e.g.][]{kitagawa_test_2015}.
In addition, monotonicity has testable implications for the marginal distributions of treated and control observations when the outcome has more than two levels (\citealp{angrist1995two}; see also Appendix 7.2 of \citealp{caughey2023randomisation} for an application in a design-based framework).
}
By contrast, we show that from the design-based perspective, the type counts $\theta$ are in fact identified according to the usual technical definition of identification. Let $f_\theta$ denote the distribution of the observable data $(Y_T,Y_U)$ over repeated assignments of treatment for the fixed finite population. The mapping $\theta \mapsto f_\theta$ is one-to-one, so $\theta$ is formally identified. At first blush, the typical definition of identification appears to suggest that the data \emph{are} informative about whether the design-based version of monotonicity holds.
We argue, however, that the usual notion of identification is somewhat unnatural in the
design-based setting: identification asks what we could learn if we saw repeated
realizations of the outcomes under different treatment assignments for the \emph{same}
units. In practice, however, we only observe one realized treatment assignment for each of
these units. It is therefore not clear whether identification in the usual sense implies
that we can meaningfully \emph{learn} about whether monotonicity holds from the data we
actually observe. To evaluate the extent to which one can feasibly learn about
monotonicity given a single realization of the data in the design-based setting,
we ask two questions: First, to what extent can one construct frequentist tests of the
null of monotonicity? Second, to what extent do Bayesians update about the probability
that monotonicity holds?
From the frequentist perspective, we show that there exist tests of the design-based
monotonicity assumption with non-trivial power against some alternatives, but we
generally expect the power of these tests to be poor. In particular, we show that any test of monotonicity will have trivial or near-trivial power against some alternative: when $n$ is even, any level-$\alpha$ test has power no larger than $\alpha$ against some alternative; when $n$ is odd, worst-case power is bounded above by $\alpha (1 + O(2^{-n}))$. Moreover, we show
that any test that has power against some alternative $\theta_1$ will have poor power at
alternatives ``close'' to $\theta_1$. We formalize this by deriving upper bounds on the
weighted average power (WAP) of tests in a neighborhood of $\theta_1$, where the weights
are proportional to the frequencies expected under sampling types from a superpopulation (and
thus concentrate around $\theta_1$, in terms of relative frequencies, when $n$ is
reasonably large). We show that regardless of sample size, the WAP of a size-$\alpha$
test
is no greater than $2.51 \alpha$; thus, for a 5\% test, the WAP is never more than
12.6\%. The
bound can be even tighter given a particular $n$ and a lower bound on the number of
defiers/compliers. For example, if $n=100$ and compliers and defiers are each at least
5\% of the finite population, then WAP is never greater than 5.06\%. We also derive upper bounds on power for unbiased tests (i.e. those with power at least $\alpha$ for all alternatives). This upper bound converges to $\alpha$ as $n$ grows large, indicating that with large $n$ all unbiased tests have nearly trivial power.
Taken together, our results suggest that frequentist tests provide limited learning about monotonicity.
In particular, these tests can only be useful if we are interested in testing one particular alternative and not
nearby ones. For example, with $n=30$ and $n_1=15$, the highest possible power of a 5\% test against any alternative is 31\%, which is achieved by a test targeting the alternative with 18 defiers and 12 compliers. However, this test never rejects when there are 17 defiers and 13 compliers. Our theoretical results show the low power of this test against neighboring alternatives is a generic feature of tests against monotonicity. We suspect that in most practical settings, researchers will
not be interested in ultra-specific alternatives, and thus such frequentist tests will
have relatively little use in practice.
From the Bayesian perspective, we show that there do exist Bayesians who update about
the probability that the design-based monotonicity assumption holds, but there also exist
Bayesians who never update. The existence of a Bayesian who updates follows immediately
from the fact that $\theta$ is identified. Consider a Bayesian with a two-point prior on
$\theta_0$ satisfying monotonicity and $\theta_1$ violating monotonicity. Since
$f_{\theta_0} \neq f_{\theta_1}$, the Bayesian updates about the probability that the null
is true. However, we show that there exists a non-trivial prior $\pi$ which never
updates on the probability that the null hypothesis holds, i.e. $\pi(\theta \in \Theta_0
\mid (Y_U,Y_T) ) = \pi(\theta \in \Theta_0) \in (0,1)$ ($\pi$-a.s.), where $\Theta_0$ denotes
the set of parameter values satisfying monotonicity.
\footnote{Specifically, we show such priors exist when $n$ is even. When $n$ is odd, we show that there exist priors that \emph{minimally} update, in the sense that the expected absolute difference between the prior and posterior probabilities for $\theta \in \Theta_0$ is $O(2^{-n})$; see \Cref{prop:bayesupdateodd}.}
Our results thus suggest that the data may be
informative to \emph{some} audience members, who are worried about particular violations
of monotonicity, but there will not be consensus: some audience member is always unmoved by the data on monotonicity. The existence of a Bayesian who doesn't update also implies that any classifier of whether $\theta \in \Theta_0$ does no better than random guessing at some parameter value.
\smallskip
\paragraph{\textbf{Related literature.}} Previous work has derived the likelihood of the
observed data under the design-based data-generating process we consider, in which there
is a completely randomized experiment with binary outcomes, and has shown that this can be
used for likelihood-based or Bayesian inference \citep{copas_randomization_1973,
ding_model-free_2019, christy_counting_2025}. \citet{copas_randomization_1973} and
\citet{christy_counting_2025} explicitly note that the likelihood can depend on the number
of defiers $\theta_d$. We add to this literature an explicit analysis of identification,
as well as results quantifying the extent to which a frequentist can test for monotonicity
or a Bayesian updates about monotonicity.
More broadly, an extensive previous literature has considered the different implications
of sampling-based versus model-based approaches to uncertainty for \emph{inference}
\citep[e.g.][]{li_general_2017,abadie_sampling-based_2020}. This paper highlights that
these different approaches can also have different implications for \emph{identification}.
We argue, however, that the classical notion of identification may be misleading in the
design-based setting as a criterion for whether the data is informative about a parameter,
and instead propose to evaluate the informativeness of the data through the properties of
frequentist tests and Bayesian updating. Although we focus on testing monotonicity, these
observations may prove useful in other design-based settings as well. \citet{kline2025finite} also study a design-based setting (although they do not consider monotonicity testing) and likewise find the textbook definition of identification inadequate, although they opt for defining alternative notions of identification rather than quantifying the scope for frequentist testing or Bayesian updating.
\section{Setup}
Consider a finite population of $n$ individuals subjected to a completely randomized
experiment with $n_1$ treated units, where $0 < n_1 < n$. That is:
\begin{enumerate}
\item Each individual $i$ has
potential outcomes $(y_i(1), y_i(0)) \in \br{0,1}^2$.
\item We observe $(Y_i, D_i)_
{i=1}^n$, where $Y_i = y_i(D_i)$ and $D_i \in \br{0,1}$.
\item Fixing the finite
population, randomness solely arises from
the assignment of $(D_1,\ldots, D_n)$, where \begin{align*}
&P(D_1=d_1,\ldots, D_n=d_n, Y_1=y_1,\ldots, Y_n = y_n) \\&\quad= \binom{n}{n_1}^{-1} \one
\pr{\sum_
{i=1}^n d_i = n_1; y_i = y_i(d_i) \text{ for all $i$}}.
\end{align*}
\end{enumerate}
We refer to those with $y_i(1) > y_i(0)$ as compliers, $y_i(1) = 1 = y_i(0)$ as always-takers, $y_i(1) < y_i(0)$ as defiers, and $y_i(1) = 0 = y_i(0)$ as
never-takers.
We refer to these as the \emph{type} of a unit. We observe the number of treated units with $Y_i=1$ and the number of
untreated units with $Y_i=1$:
\footnote{We note that if the researcher observes individual data, then any procedure $\delta$ that is \emph{anonymous}---i.e., that is invariant to permutations of the units, so that $\delta((Y_1,D_1),\ldots, (Y_n, D_n)) = \delta((Y_{\sigma(1)}, D_{\sigma(1)}),
\ldots, (Y_{\sigma(n)}, D_{\sigma(n)}))$ for all permutations $\sigma: [n] \to [n]$---is simply a function of the counts $(Y_T,Y_U)$. It thus suffices to restrict attention to $(Y_T,Y_U)$ for anonymous procedures.}
\[
Y := (Y_T, Y_U) = \pr{\sum_{i=1}^n D_i Y_i, \sum_{i=1}^n (1-D_i) Y_i}.
\]
The counts $Y$ depend on the potential outcomes only through
the
corresponding counts of types in the finite population, \[\theta \in \Theta := \br{(n_{at}, n_{nt}, n_d, n_c) \in (
\mathbb{N}\cup
\br{0})^4 : n_ {at} + n_{nt} + n_d + n_c
= n}.
\]
In particular,
\begin{align*}
Y_T = (\text{\# treated always takers}) + (\text{\# treated compliers}) \\
Y_U = (\text{\# untreated always takers}) + (\text{\# untreated defiers}).
\end{align*}
In a completely randomized experiment, given $\theta = (n_
{at}, n_{nt}, n_d, n_c)$, the number of treated
units of each type is distributed according to a multivariate hypergeometric
distribution with parameters $\theta$ and $n_1$. Let $f_\theta(y_T, y_U)$ denote the induced
probability mass function for $(Y_T, Y_U)$ under $\theta$.
Sometimes, we think of the finite population as being drawn from an infinite
superpopulation. To that end, let $\mathcal P = \br{(p_{at}, p_{nt}, p_d, p_c) \in [0,1]^4
: p_{at} + p_{nt} + p_d + p_c = 1}$ be the simplex. For an element $p \in \mathcal P$,
let the corresponding distribution of $(Y_T, Y_U)$ when unit types are drawn i.i.d. from
$\mathrm{Multinomial}(n, p)$ be \[
g_p(y_T, y_U) = \sum_{\theta = (n_{at}, n_{nt}, n_d, n_c) \in \Theta} f_\theta (y_T, y_U)
\frac{n!}{n_{at}! n_{nt}! n_c ! n_d!} p_{at}^{n_{at}} p_{nt}^{n_{nt}} p_d^{n_d} p_c^{n_c}.
\]
Let $\Theta_0 = \br{(n_{at}, n_{nt}, n_d, n_c) \in \Theta: \min(n_d, n_c) = 0}$ be the
set of type counts that satisfy monotonicity. Let
$\Theta_1 = \Theta \setminus \Theta_0$ be the complement. Similarly, let $\mathcal P_0 =
\br{p \in \mathcal P: \min(p_d, p_c) = 0}$ and $\mathcal P_1 = \mathcal P \setminus
\mathcal P_0$.
\begin{rem}[Practical relevance of each null]
Whether we prefer to test the superpopulation null that $p \in \mathcal{P}_0$ or the
design-based null that $\theta_0 \in \Theta_0$ will depend on the application. If the $n$
units are patients in a drug trial drawn randomly from a much larger population of
patients with the same condition, then we are likely more interested in whether there are heterogeneous responses to the drug in the super-population of patients, and thus $p \in
\mathcal{P}_0$ is more relevant than $\theta \in \Theta_0$. On the other hand, if the $n$
units are the 50 states, it may be unnatural to imagine the states as sampled from an
infinite superpopulation, rendering conceptual issues for the null $p \in \mathcal{P}_0$.
By contrast, testing $\theta \in \Theta_0$ answers the natural question as to whether any
of the 50 states have opposite-signed treatment effects. See \citet{copas_randomization_1973},
\citet{reichardt_justifying_1999}, and \citet{rambachan_design-based_2025}, among others,
for additional discussion of the relevance of design-based vs.\ superpopulation-based
estimands in general.
For our setting specifically, \cite{gelman2025russian} argues that cases in which decision loss depends on counterfactual outcomes should be analyzed through a framework in which potential outcomes are stochastic.
\end{rem}
\section{Identification}
The typical textbook definition of identification states that a parameter is \text{(point-)}identified if two
distinct values of the parameter induce different distributions of the observed data.
\footnote{For example, the Wikipedia page on \href{https://en.wikipedia.org/w/index.php?title=Identifiability&oldid=1292274414}{identifiability} states that a statistical model $P_\theta$ is ``identifiable if the mapping $\theta \mapsto P_\theta$ is one-to-one.'' \citet[][Definition 5.2]{lehmann_theory_1998} equivalently define $\theta$ to be unidentifiable if there exist $\theta_1 \neq \theta_2$ such that $P_{\theta_1} = P_{\theta_2}$.}
It
is well-known that, from the superpopulation perspective, the population shares $p$ are not
point-identified \citep{heckman_making_1997}. In particular, any type proportion that
violates monotonicity induces data that can be rationalized by some other type proportion
that obeys monotonicity. By contrast, as summarized in the following result, the type
counts $\theta$ are in fact identified, and thus monotonicity violations are likewise
distinguishable from $f_\theta$.
\begin{restatable}{prop}{propid}
\label{prop:id}
If $\theta_1 \neq \theta_0 \in \Theta$, then $f_{\theta_1} \neq f_{\theta_0}$, and hence the finite-population type counts $\theta$ are identified. On the
other hand, given any $p_1 \in \mathcal P_1$, there exists a $p_0 \in \mathcal P_0$ such
that $g_{p_1} = g_{p_0}$.
\end{restatable}
\begin{example}[Illustrative example] \label{ex:2 units}
Consider a population with 2 units, one of whom is assigned to treatment. If there is 1 always-taker and 1 never-taker ($\theta = (1,1,0,0)$), then $(Y_T,Y_U)$ is either equal to $(1,0)$ or $(0,1)$, depending on whether the always-taker is assigned to treatment or control. By contrast, if there is 1 defier and 1 complier ($\theta = (0,0,1,1)$), then $(Y_T,Y_U)$ is either equal to $(1,1)$ or $(0,0)$, depending on whether the complier is assigned to treatment or control. Thus, the distribution of the observed data differs between a population with 1 always-taker and 1 never-taker and a population with 1 complier and 1 defier, despite $E[(Y_T,Y_U)]$ being the same. By contrast, a superpopulation with half always-takers and half never-takers and a superpopulation with half compliers and half defiers would each generate the same observable data distribution for $(Y_T,Y_U)$, assigning equal probability to $(0,0), (0,1), (1,0), (1,1)$.
\end{example}
As a criterion for whether the observed data is informative of the parameter, however, the
above definition of identification falls short from the design-based perspective. Two
values of $\theta$ are distinguishable in the sense that repeated draws of $(Y_T, Y_U)$
from $f_\theta$ would have different distributions under these two values. This, however,
corresponds to knowing the distribution of outcomes from re-assigning the \emph{same}
units to \emph{different} treatment assignments. However, in the finite population, we only
observe $(Y_T, Y_U)$ once, and thus cannot use information only learned from repeated
draws. For example, knowing $f_\theta$ implies that we know $\cov(Y_T,Y_U)$, yet this is
difficult to learn from observing a {single} realization of $f_\theta$.
In the following sections, we consider two perspectives of evaluating whether a single realization of $(Y_T,Y_U)$ is useful for learning about monotonicity violations. From the frequentist perspective, we consider whether one can construct tests that have non-trivial power against the null of monotonicity. From the Bayesian perspective, we consider the extent to which a Bayesian updates their prior that monotonicity holds after seeing the data.
\begin{rem}
The discussion in previous papers sometimes suggests that $\theta$ is
not identified. For example, \citet{ding_model-free_2019} write that ``Without
monotonicity, the unknown parameters in the Science Table, $(N_{11}, N_{10}, N_{01},
N_{00})$ [$\theta$ in our notation], are no longer identifiable from the observed data.''
Likewise, \citet{rosenbaum_effects_2001} writes, ``The model of a nonnegative effect
cannot be verified or refuted by inspecting the responses of individuals, because $r_{Ti}$
and $r_{Ci}$ [$y_i(1)$ and $y_i(0)$ in our notation] are never jointly observed on the
same person.'' \Cref{prop:id} shows that according to the usual technical definition of
identification, $\theta$ is in fact identified. The underlying intuition---that violations of monotonicity are hard to detect---is consistent with our power and updating results in the following sections, however.
\end{rem}
\section{Frequentist testing}
We first consider the possibility of frequentist testing against monotonicity. First,
we show there exist frequentist tests with power against \emph{some} alternatives.
\begin{restatable}{prop}{prophaspower}
\label{prop:haspower}
\emph{(Frequentist tests exist)}
Suppose $n_0,n_1 \geq 2$. Then there exist tests for monotonicity that control size and have power against some alternatives. That is, for any $\alpha \in (0,1)$ there exists a test $\delta: \supp(Y) \to [0,1]$ such that $\sup_{\theta_0 \in \Theta_0} E_{\theta_0}[\delta(Y)] \leq \alpha$ and $E_{\theta_1}[\delta(Y)] > \alpha$ for some $\theta_1 \in \Theta_1$.\footnote{The proof shows that there is a non-randomized test satisfying the conditions of the proposition for $\alpha$ sufficiently large.}
\end{restatable}
\paragraph{\textbf{Intuition for tests.}} Observe that in \Cref{ex:2 units}, the support of $Y$ was $\{(0,1), (1,0)\}$ when there was 1 always-taker and 1 never-taker, but was $\{(0,0),(1,1)\}$ when there was 1 complier and 1 defier. This illustrates that the support of $Y$ may be different when monotonicity holds versus when it is violated. This general idea is the basis of the construction of tests for monotonicity in the proof to \Cref{prop:haspower}. We show that when $n \geq 4$, there always exists an alternative $\theta_1$ such that the support of $Y$ under $\theta_1$ does not contain the support of $Y$ under $\theta_0$ for any $\theta_0$ satisfying the null: i.e. $S^Y_{\theta_1} \not\supseteq S^Y_{\theta_0}$ for any $\theta_0 \in \Theta_0$, where $S^Y_\theta$ is the support of $Y$ under $\theta$.\footnote{Note that this is not the case in \Cref{ex:2 units}, because the support of $Y$ when there are two always-takers is $(1,1) \subseteq S^Y_{\theta_1} = \{(0,0),(1,1)\}$. There are thus no non-trivial tests of monotonicity with $n_1 = n_0 = 1$.} It follows that a test that rejects if and only if $Y \in S^Y_{\theta_1}$ has power of 1 to reject $\theta_1$, but only has size $\alpha_0 = \sup_{\theta_0 \in \Theta_0} E_{\theta_0}[ 1\{ Y \in S^Y_{\theta_1} \} ] < 1$. We can then construct a non-trivial randomized test with arbitrary size $\alpha \leq \alpha_0$ by rejecting with probability $\alpha/\alpha_0 > \alpha$ when $Y \in S^Y_{\theta_1}$.
Although non-trivial tests against monotonicity exist from the design-based perspective,
we expect their power to be poor in practice. Our next result shows that any test of
monotonicity has near-trivial power against some alternatives. Specifically, we show that if $n$ is even, there always exists an alternative for which power is no better than size ($\alpha$). For $n$ odd, we show there exists an alternative such that power is bounded above by $\alpha (1+ O(2^{-n}))$.\footnote{Surprisingly, there is a test for $n=13, n_1=6$ whose minimum power over $\Theta_1$ is ever-so-slightly larger than $\alpha$. For $\alpha=0.05$, such a test achieves power at least $0.05 + 2 \times 10^{-7}$ over all of $\Theta_1$.} This implies that there are no consistent tests of monotonicity (that is, tests for which power converges to 1 for all alternatives along a sequence of finite-populations with $n \to \infty$).
\begin{restatable}{prop}{prophasnopower}
\emph{(Near-trivial power for some alternative)}
Suppose that $\delta: \supp(Y) \to [0,1]$ is a level-$\alpha$ test, i.e. $\sup_{\theta_0
\in \Theta_0} E_{\theta_0}[\delta(Y)] \leq \alpha$ for some $\alpha \in (0,1)$. If $n$ is
even, then $\delta$ has trivial power against some alternative: there exists $\theta_1 \in
\Theta_1$ such that $E_{\theta_1}[\delta(Y)] \leq \alpha$. If $n$ is odd, then there
exists $\theta_1 \in \Theta_1$ such that $E_{\theta_1}[\delta(Y)] \leq \alpha \left(1 + \frac{1}
{2^{n-1}-1} \right)$.
\end{restatable}
We can further show that the lack of power is \emph{generic} in the following sense: if we
construct a test to have power against some alternative $\theta_1$, then there are
alternatives
``near'' $\theta_1$ such that power is low. More precisely, around any $\theta_1$,
we can define a weighting function that mimics sampling from a population with type
frequencies $p = \theta_1/n$. When $n$ is reasonably large, these weights are
``concentrated'' around $\theta_1$ in terms of type frequencies. Weighted average power (WAP) under this weighting
function turns out to never be larger than 2.51$\alpha$, uniformly over alternatives
$\theta_1$.
\begin{restatable}{prop}{propnowap}
\label{prop:nowap}
\emph{(Low WAP)} Assume $n \geq 4$. Fix any $\theta_1 \in \Theta_1$ and let $v = \frac{1}{n} \min(\theta_{1,c},
\theta_{1,d}) \ge 1/n$ be
the
size of the monotonicity violation. Let $\vartheta \sim
\mathrm{Multinomial} (n, \theta_1/n)$.
Consider the weight function $w_n(\cdot; \theta_1)$ over
$\Theta_1$ defined by the probability mass function of $\vartheta \mid (\vartheta \in
\Theta_1)$. Fix a test $\delta(\cdot)$ such that $\sup_{\theta_0 \in\Theta_0} E_{\theta_0}[\delta
(Y)] \le \alpha$. Then, the weighted average power around $\theta_1$ is bounded,
\[
\mathrm{WAP}(\delta; \theta_1) := \sum_{\tilde{\theta}_1 \in \Theta_1} E_{\tilde\theta_1}[\delta
(Y)] w_n
(\tilde{\theta}_1;\theta_1) \le \alpha \frac{1}{1-2(1-v)^n+(1-2v)^n} \le 2.51 \alpha.
\addtocounter{equation}{1}\tag{\theequation}
\label{eq:wap}
\]
\end{restatable}
\Cref{prop:nowap} implies, for example, that a 5\% test never has weighted average power
larger than 12.6\%. The bound given in \Cref{prop:nowap} becomes even tighter if one
imposes a lower bound on the fraction of the finite population that are compliers/defiers.
For example, in a population of 100, if $\min(\theta_c, \theta_d) \ge 5$, then the upper
bound becomes $5.06\%$, which is virtually the same as size, uniformly over all such
alternatives.
Interestingly, while we typically expect statistical power to increase with $n$,
\Cref{prop:nowap} implies a \emph{tighter} upper bound on WAP the \emph{larger} is $n$
(holding fixed the share of compliers and defiers). Intuitively, having a larger finite
population is more similar to having an infinite superpopulation, in which case there is
no testable content of monotonicity.
\footnote{
Similarly, the bound on WAP gets \emph{tighter} when the violation $v$ of the null hypothesis is \emph{larger}.}
\paragraph{\textbf{Numerical illustration with $n=30$.}} To illustrate these results, \cref{fig:powerhist} computes the most powerful 5\%-level
test for every possible alternative $\theta \in \Theta_1$, obtained via linear
programming, for $n = 30$ and $n_1 = 15$. Perhaps surprisingly, all 4495 alternatives are
testable, in the sense that for each alternative, there exists a test targeted to that
alternative with power more than 0.05. However, the power for these tests for the targeted
alternative tends to be modest: among these tests engineered to maximize power at a given
alternative, only 53 of 4495 alternatives have tests with power above 0.15, and all of
them have power below 0.31. These tests also tend to have very poor power for nearby
alternatives, as suggested by \Cref{prop:nowap}: Only 18 have weighted average power above the nominal threshold $0.05$, and the maximum WAP is a measly $0.0567$. As a specific example, the optimal test against 18 defiers and 12 compliers achieves the maximal power of $0.31$, but this test rejects with probability zero when there are 17 defiers and 13 compliers.
\begin{figure}[!ht]
\includegraphics[width=\textwidth]{power_wap_histogram.pdf}
\caption{Power of the most powerful test for a given alternative $\theta \in
\Theta_1$ for $n=30$, $n_1=15$}
\begin{proof}[Notes]
We construct the most powerful 5\%-test for a given alternative $\theta$, which we
obtain via linear programming: $\max_{\delta: \supp(Y) \to [0,1]} \sum_y f_{\theta}
(y) \delta (y)$ subjected
to $\sum_y f_\vartheta(y) \delta(y) \le 0.05$ for all
$\vartheta\in\Theta_0$. The
blue patches show the distribution of power at $\theta$ for each test, across all
4495 values in $\Theta_1$. For each test, we also compute its weighted average
power \eqref{eq:wap} and show its distribution in orange.
\end{proof}
\label{fig:powerhist}
\end{figure}
\paragraph{\textbf{Unbiased tests.}} We can obtain even sharper limits on power if we restrict attention to unbiased tests. Recall that a test is unbiased if its power against all alternatives is weakly greater than its size. \Cref{prop:unbiasedexists} below shows that unbiased tests for monotonicity do exist (at least when $n_1=n_0$). However, \Cref{prop:unbiased} implies that as the sample size grows large, the power of any unbiased test against any alternative becomes trivial.
\begin{restatable}{prop}{propunbiasedexists} \emph{(Unbiased tests exist)}
\label{prop:unbiasedexists}
Suppose $n_1=n_0 \geq 2$. Then there exists a non-trivial unbiased test of monotonicity: for any $\alpha \in (0,1)$, there exists $\delta: \supp(Y) \to [0,1]$ such that $\sup_{\theta_0 \in \Theta_0} E_{\theta_0}[ \delta(Y) ] \leq \alpha \leq \inf_{\theta_1 \in \Theta_1} E_{\theta_1}[ \delta(Y) ]$, with $E_{\theta_1}[ \delta(Y) ] > \alpha$ for at least one $\theta_1 \in \Theta_1$.
\end{restatable}
\begin{restatable}{prop}{propunbiased} \emph{(Unbiased tests have asymptotically trivial power)}
\label{prop:unbiased}
Fix $\epsilon > 0$ and $n \geq 4$. Fix any $\theta \in \Theta_{1}$ such that $v = \min
(\theta_{c}, \theta_{d})/n \ge \epsilon$. Let $\delta$ be any unbiased level-$\alpha$ test, i.e. a test satisfying $\sup_{\theta_0 \in \Theta_0} E_{\theta_0}[\delta(Y)] \le \alpha \le \inf_{\theta_1 \in
\Theta_1} E_{\theta_1}[\delta(Y)].$
Then we have that
\[
E_{\theta}[\delta(Y)] \le \alpha (1 + \eta_n(\epsilon))
\]
for $\eta_n(\epsilon) = 3.125 n^{1.5} (1-\epsilon)^n$.
\end{restatable}
In particular, \Cref{prop:unbiased} says that along any sequence of finite populations with $v_n = \min(\theta_{c,n}, \theta_{d,n})/n \ge \epsilon$, the power of any unbiased test is $(1+o(1)) \alpha$ as $n \rightarrow \infty$.
The proof of \Cref{prop:unbiased} casts the problem of maximizing $E_{\theta}[\delta(Y)]$
subject to unbiasedness and size control as a linear program. The dual of this
program---whereby any feasible value implies an upper bound for the power of
$\delta$---involves choosing certain weights over the null and the alternative. The weights
$w_n$ defined in \Cref{prop:nowap} turn out to enable a nontrivial upper bound of the primal value.
\section{Bayesian updating}
We next ask whether Bayesians update about whether $\theta\in\Theta_0$. The following result shows that \emph{some} Bayesians update, but there exist Bayesians who find the data totally uninformative about whether monotonicity holds, in the sense that their posterior on $\theta \in \Theta_0$ is equal to their prior almost surely.
\begin{restatable}{prop}{bayescombined}
\label{prop:bayescombined}
\emph{(Bayesian updating)}
\begin{enumerate}
\item
Some Bayesians update: there exists a prior $\pi$ with $\pi
(\theta \in \Theta_0) \in (0,1)$ such that $\pi(\theta \in \Theta_0 \mid Y) \neq \pi(\theta \in \Theta_0)$ with
positive $\pi$-probability.
\item
There exist (nontrivial) Bayesian priors over $\theta$ that never update about the probability that monotonicity holds: if $n$ is even, then for any $c \in (0,1)$, there exists a prior distribution $\pi$ over $\theta$ such that $\pi(\theta \in \Theta_0 \mid Y) = \pi(\theta \in \Theta_0) =c$, $\pi$-almost surely.
\end{enumerate}
\end{restatable}
We note that the conclusion in part 1 that \emph{some} Bayesian updates is rather weak. In fact, even in the superpopulation setting, there is \emph{some} Bayesian who updates: consider, for example, the Bayesian who believes that there are defiers if and only if the average treatment effect is larger than 0.5; this Bayesian updates about the validity of monotonicity based on the information in the data about the average treatment effect. However, the updating in the design-based setting is slightly less trivial: a Bayesian in fact updates about the relative probability of two type vectors $\theta_0, \theta_1$ that imply the same marginal distributions of $y(0)$ and $y(1)$ in the finite population, whereas this is not true in the superpopulation setting.
Part 2 of \Cref{prop:bayescombined} implies that some Bayesians do not find the data informative about whether monotonicity holds at all. Hence, the data will not be persuasive to \emph{all} audience members. This also suggests that Bayesian inference in this setting, as considered in \citet{ding_model-free_2019} and \citet{christy_counting_2025}, for example, will necessarily be sensitive to the choice of prior: for the class of priors in the second part of \Cref{prop:bayescombined}, posterior statements about monotonicity simply match the priors.
The existence of a prior that does not update also implies that any binary classifier of whether $\theta \in \Theta_0$ does no better than random guessing for some parameter values $\theta$. Specifically, given $a \in \{0,1\}$, let $\ell(a,\theta) = 1\{a \neq 1\{\theta \in \Theta_0 \} \}$ be an indicator for whether $a$ mis-classifies whether $\theta \in \Theta_0$ (i.e. zero-one classification loss). Let $c: \mathcal{Y} \to [0,1]$ be a possibly-randomized classifier that sets $a=1$ with probability $c(Y)$, and let $R(c,\theta) = E_\theta[ c(Y)\ell(1,\theta) + (1-c(Y) \ell(0,\theta)]$ be its average classification error at parameter $\theta$ (risk under 0-1 loss).
\begin{restatable}{cor}{trivialclassification}
\label{cor:trivialclassification}
Suppose $n$ is even. Then $\inf_{c(\cdot)} \sup_{\theta} R(c,\theta) = 0.5$.
\end{restatable}
\noindent Note that a trivial classifier that guesses randomly ($c(y) = 0.5,$ $\forall y$) achieves a 0.5 mis-classification rate. \Cref{cor:trivialclassification} implies that \emph{every} classifier does no better than this at some parameter value $\theta$ (when $n$ is even). This follows from Part 2 of \Cref{prop:bayescombined}: there exists a prior that never updates about whether $\theta \in \Theta_0$, and hence Bayes risk under this prior must be trivial. \citet{christy_counting_2025} consider the maximum likelihood estimator $\hat\theta$ and say that it ``provides evidence in favor of monotonicity'' if $\max\{\hat\theta_c, \hat\theta_d\} = 0$. \Cref{cor:trivialclassification} implies that this classification of whether the data support monotonicity does no better than random guessing at some parameter values.\footnote{In fact, in some cases, this classification rule does worse than random guessing. Consider \Cref{ex:2 units} above with $n=2$. Note that with 1 complier and 1 defier, the support of $Y$ is $\{(0,0),(1,1)\}$, with each support point occurring with probability $1/2$. However, $Y=(1,1)$ with probability 1 if there are 2 always-takers, and $Y=(0,0)$ with probability 1 if there are two never-takers. Thus, the MLE never corresponds to having 1 complier and 1 defier, so worst-case misclassification error is 1.} Moreover, we cannot ``fix'' this undesirable property by choosing a different classification rule.
Part 2 of \Cref{prop:bayescombined} and \Cref{cor:trivialclassification} focused on the
case where $n$ is even. In the appendix, we show that when $n$ is odd, there exist
non-degenerate priors that \emph{minimally} update in the sense that the expected change
between the prior and posterior is small: \Cref{prop:bayesupdateodd} shows that there
exists a prior such that \[ E_{Y \sim \pi_Y}[ | \pi(\theta \in \Theta_0 \mid Y) -
\pi(\theta \in \Theta_0) |] = O(2^{-n}), \] where $\pi_Y$ is the distribution of $Y$
induced by prior $\pi$. This implies \Cref{cor:classification-odd}, which states that when
$n$ is odd, a classifier of whether $\theta \in \Theta_0$ has worst-case
mis-classification error at least $0.5 - O(2^{-n})$.
\section{Conclusion}
We study what a completely randomized experiment can reveal about finite-population monotonicity. We show that from the design-based perspective, the type counts $\theta$ in the finite population are in fact identified. However, the extent to which we can feasibly learn about violations of monotonicity is severely limited: frequentist tests generically have poor power, and some Bayesians never update about whether the null is true. Thus, formal identification translates to little, if any, practical learning about monotonicity. These results highlight that conclusions about identification may differ depending on whether one adopts a sampling-based versus design-based perspective, and that studying the properties of frequentist tests and Bayesian updating may provide a more realistic assessment of the extent to which learning is possible in design-based settings. An interesting avenue for future research is to explore whether similar issues arise in other design-based causal inference problems.
\newpage