Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
103,047 characters · 13 sections · 73 citation commands
Ranking Treatment Saturations under Clustered Network Interference
\onehalfspacing {
}
{3ex}
\setcounter{page}{1}
We study how to rank a finite set of treatment saturations for a target population with clustered network interference, using data from a two-stage randomized saturation experiment. We propose an empirical success (ES) ranking rule that uses data from a two-stage randomized saturation experiment to rank saturation policies according to their estimated welfare,i.e., for each treatment saturation pair $(\pi_k,\pi_{k'})$, selects the saturation with the larger estimated welfare. We evaluate the performance of this rule within a statistical decision-theoretic framework based on additively separable regret loss. We derive finite-sample regret guarantees for the ES ranking rule, an upper-bound-minimizing two-stage experimental design, and local asymptotic admissibility and optimality results for the broader class of threshold ranking rules.
Saturation experiments are cluster-randomized designs that vary the fraction of treated individuals across clusters. They have become a canonical empirical tool for studying how intervention effects vary with treatment saturation in settings with clustered (network) interference. For example, crepon2013labor randomized 235 French employment offices across saturation levels of 0, 25, 50, 75, and 100 percent to identify the displacement effects of job-search assistance on untreated workers. egger2022general randomized 653 Kenyan villages across cash-transfer saturation levels to measure general-equilibrium effects on consumption and earnings, while banerjee2013diffusion varied the intensity of information seeding across 43 Indian villages to study the diffusion of microfinance adoption. These experiments generate evidence on the welfare consequences of alternative saturation policies. The policy question that follows is straightforward but largely unresolved: given data from a saturation experiment, how should the competing saturation levels be ranked, and which saturation policy should be selected for deployment in a target population?
Selecting the best saturation level is the primary interest in most applications. However, the full ranking of the saturation menu is just as useful for policymakers in many practical settings. First, under a slowly relaxing budget, a phased rollout requires the next-best saturation when the optimum is infeasible, and the next-best after that. Second, ex-post feasibility shocks, such as a court ruling against universal coverage, a clinic capacity bottleneck, or a donor reallocating funds across regions, may rule out the realized best option and leave the planner needing the fallback ranking. Third, a defensible ordering across the menu is required to justify the deployed choice to parliament, donors, or regulators, and to pre-empt counterfactual challenges of the form “why not the third option?”. Fourth, the gap between the top two ranked saturations is itself informative about whether the decision is sharp and whether further data collection is warranted. A ranking identifies the best saturation as a by-product, but selection alone does not produce a ranking.
When units interact within prespecified clusters, but not across clusters, clustered network interference---the phenomenon whereby a unit's outcome depends not only on its own treatment status but also on the treatment status of other units it interacts with in its cluster---is often a first-order feature of the environment. Vaccination programs generate herd-immunity benefits for untreated individuals that depend nonlinearly on the fraction vaccinated. Deworming children in one school reduces parasite transmission to nearby schools, so estimates that ignore spillovers understate the true policy benefit miguel2004worms. Conditional cash-transfer programs can alter local prices, wages, and consumption patterns in ways that substantially exceed the direct effects on recipients egger2022general. Likewise, information about new financial products diffuses through social networks at rates that depend on the intensity of treatment exposure banerjee2013diffusion. As emphasized by deaton2018understanding and duflo2004scaling, evidence from small-scale randomized trials may therefore fail to predict the consequences of large-scale implementation. The following features of clustered interference make the ranking problem difficult. First, each unit's outcome depends on the entire treatment vector inside its cluster, so observations cannot be treated as independent as in the standard manski2004statistical framework. Second, the social network that carries the spillovers is rarely observed, especially as cluster size grows, which makes parametric modeling of the spillover mechanism challenging. Third, in some designs, the experimenter assigns saturations to clusters without replacement, which couples the outcomes of every pair of clusters in the sample. The joint sampling distribution of the welfare estimates is therefore intractable.
To address these challenges, this paper develops a tractable approach to ranking treatment saturation policies in populations with clustered network interference. We formulate the problem within a statistical decision-theoretic framework and derive finite-sample guarantees for the resulting ranking rule. We make three contributions. First, we derive a finite-sample upper bound on the maximum regret of the ES ranking rule. The bound decays exponentially in the pairwise welfare gaps and depends on the within-cluster network only through a single combinatorial summary of the dependency structure. Second, we use this bound to characterize a quasi-optimal first-stage saturation distribution within the two-stage randomized saturation design of baird2018optimal, replacing their power-maximizing criterion with a regret-minimizing one. Third, we show that the ES ranking rule is asymptotically optimal in a Gaussian shift limit experiment under a rank-one structural condition, in the sense that it minimizes an upper bound on the worst-case regret within the asymptotically admissible class of threshold ranking rules.
Our analysis combines two existing technical tools tailored to the structure of this ranking problem. First, to handle the dependence among individual outcomes generated by an unobserved within-cluster network, we apply janson2004large's inequality for graph-dependent random variables. This yields exponential concentration bounds for each pairwise welfare contrast and a finite-sample upper bound on the maximum regret of the ES ranking rule that depends on the within-cluster network only through a single combinatorial summary of the dependency structure. Minimizing the bound with respect to the first-stage saturation distribution yields the quasi-optimal design within the two-stage randomized saturation design framework of baird2018optimal. Second, for the asymptotic analysis, we embed the ranking problem in a Gaussian shift limit experiment in the spirit of hirano2009asymptotics. In this limit experiment, threshold ranking rules correspond exactly to single-step procedures in the one-sided multiple-comparison literature, which lets us apply Theorem 4.1 of cohen2005decision directly to obtain admissibility of the threshold class of ranking rules. Under an additional rank-one structural condition, the minimization of an upper bound on the worst-case regret decouples across saturations and is attained at the limit version of the ES ranking rule.
The remainder of this paper is organized as follows. We conclude this section with a review of the related literature. Section (ref) introduces the framework and notation. Section (ref) formalizes the ranking problem and assesses the finite-sample performance of the ES ranking rule. Section (ref) characterizes the quasi-optimal treatment-assignment mechanism. Section (ref) establishes the asymptotic admissibility and optimality of threshold ranking rules. Section (ref) reports Monte Carlo simulations that illustrate the finite-sample bounds and confirm the result of the quasi-optimal design. Section (ref) concludes. All proofs and extensions are collected in the supplementary material.
This paper is closely related to the decision-theoretic literature on treatment choice. manski2004statistical studies the finite-sample behavior of ES ranking rules under additively separable regret loss, while hirano2009asymptotics establishes their asymptotic optimality using Le Cam's local-asymptotic-theory machinery. Furthermore, stoye2009minimax characterizes the minimax regret choices under finite samples, tetenov2012statistical analyzes an asymmetric minimax regret, and manski2016sufficient apply the framework to determine sufficient trial size for clinical practice. Subsequent work, including kitagawa2018should, athey2021policy, and mbakop2021model, develops empirical welfare maximization (EWM) procedures that select treatment rules to maximize estimated welfare. More recently, masten2023minimax and chen2025note develop minimax-regret frameworks for selecting among several treatment options. All of these papers assume individualistic responses, i.e., no interference. We differ by ranking rather than selecting and by allowing clustered interference, which precludes the direct extension of the matched-balanced and random-assignment experimental designs of chen2025note.
A recent literature extends statistical decision theory to settings with interference. zhang2023individualized, park2024minimum, and viviano2025policy develop network-based EWM rules that use covariates to guide individualized treatment assignment under various network structures. We address a complementary problem: choosing a single population-level saturation from a discrete menu without network information, not an individualized assignment rule.
On the design side, we are closest to baird2018optimal and viviano2022policy. baird2018optimal derive the two-stage randomized saturation design framework we adopt, but motivate it by maximizing statistical power for hypothesis tests; we replace their power criterion with a regret-minimizing one, holding fixed the two-stage cluster-randomization structure. viviano2022policy proposes a single-wave design for hypothesis testing and a multi-wave adaptive design for learning optimal policies in a parametric class. Our one-shot regret-minimizing design is methodologically distinct from both.
Finally, a parallel literature in econometrics develops inference for ranks and winners rather than decision rules that produce them. mogstad2024inference and bazylik2025finite construct confidence sets for the rank of a fixed population (e.g., neighborhoods by intergenerational mobility; parties by vote share). andrews2024inference corrects the winner's curse in post-selection inference for the best-performing treatment in a multi-arm trial. gu2023invidious develops an empirical-Bayes compound-decision approach to ranking populations by latent quality. These papers infer fixed rank objects under independent populations; we instead derive a decision rule that outputs a ranking of treatment saturations under a designed clustered-interference experiment, with finite-sample regret guarantees and a corresponding quasi-optimal design.
We consider a setting in which observed units in the target population can be partitioned into clusters, such as classrooms, villages, or states. Let $n_i< \infty$ denote the number of units in cluster $i\in \mathcal{I}$, where $\mathcal{I}$ represents a super population of clusters. For unit $j $ in cluster $i,$ let $Y_{ij} \in \mathcal{Y}\subset \mathbbm{R}$ represent the observed outcome and $Z_{ij} \in \{0,1\}$ denote the binary treatment assigned to this unit. We use $\mathbf{Y}_i= (Y_{i1}, \dots, Y_{in_i})^\top$ and $\mathbf{Z}_i= (Z_{i1}, \dots, Z_{in_i})^\top$ to denote the cluster-level vectors of outcomes and treatments, respectively.
To model the interference structure, we assume that interactions among units occur exclusively within clusters and are mediated by an unobserved within-cluster network. Specifically, each cluster $i$ has an undirected graph $G_i \in \mathcal{G}$ with no self-loops, where the vertices correspond to individuals and the edges capture channels of interaction. The full population network is the disjoint union $G:= \cup_{i \in \mathcal{I}} G_i$, which we interpret as a countably infinite union of internally connected but mutually disconnected subgraphs. This setting defines a clustered network interference structure, where spillovers are localized within clusters and shaped by the topology of $G_i$.
To formalize the causal framework in this clustered networked environment, we adopt the potential outcomes framework of neyman1923applications and rubin1974estimating. We allow for unrestricted interference within clusters by modeling each unit's potential outcome as a function of the treatment assignments of all individuals in its cluster. Specifically, for each unit $j$ in cluster $i$, the potential outcome is represented by the response function \[ Y_{ij} : \{0,1\}^{n_i} \to \mathcal{Y} = [0,1], \] which maps the treatment vector $\mathbf{z}_i = (z_{i1}, \dots, z_{in_i})^\top \in \{0,1\}^{n_i}$ in cluster $i$ to the potential outcome $Y_{ij}(\mathbf{z}_i)$. The observed outcome for unit $j$ in cluster $i$ is then given by $Y_{ij} = Y_{ij}(\mathbf{Z}_i)$. Note that the assumption of bounded outcomes is essential for establishing the finite-sample results, but is not required for the asymptotic analysis.
This formulation is fully nonparametric: it permits each unit's potential outcome to depend arbitrarily on the treatment assignments of all other units in the same cluster, imposing no structural restrictions on the form of interference.
Following the population perspective of manski2004statistical, we treat potential outcomes as fixed population quantities. Uncertainty enters the analysis solely through the treatment assignment mechanism and the random sampling of clusters from the super-population $\mathcal{I}$.
Let $\pi \in [0,1]$ denote the treatment saturation of a cluster. Then, for unit $j$ in cluster $i$ assigned treatment saturation $\pi$, the individual average potential outcome is
where $\gamma(\pi,\mathbf{z}_i)$ denotes the probability mass function (pmf) of the complete-randomization assignment design with treatment saturation $\pi$, given by $$ \gamma(\pi,\mathbf{z}_i) = {n_i\choose n_i\pi}^{-1}\cdot\mathbbm{1}\left\{\sum_{j=1}^{n_i} z_{ij}=n_i \pi\right\}.$$ We adopt the following joint admissibility condition on the cluster sizes $\mathcal{N}=\{n_i\}_{i\in\mathcal{I}}$ and the saturation set $\boldsymbol{\Pi}$.
Assumption (ref) is imposed only for the exact-count complete-randomization design used in the main text. It restricts the joint admissibility of cluster sizes and saturation levels so that within-cluster complete randomization at every $\pi\in\boldsymbol{\Pi}$ is well-defined as an exact-count design. When $n_i\pi$ is not an integer, complete randomization at saturation $\pi$ may be defined via a convex combination of designs with $\lfloor n_i\pi\rfloor$ and $\lceil n_i\pi\rceil$ treated units; we omit these cases to maintain focus on the paper's primary objective of ranking treatment saturations.
The pmf $\gamma(\pi,\mathbf z_i)$ specifies the distribution of treatment assignments under a hypothetical complete-randomization policy that treats exactly a fraction $\pi$ of units in cluster $i$. This policy represents the treatment assignment mechanism that the planner intends to deploy in the target population. Consequently, $\gamma(\pi,\mathbf z_i)$ is used to average potential outcomes and define the welfare associated with saturation level $\pi$. Importantly, it need not coincide with the assignment mechanism that generated the observed experimental data. Thus, $\bar Y_{ij}(\pi)$ represents the expected outcome of unit $j$ in cluster $i$ under a policy that assigns exactly a fraction $\pi$ of units in the cluster to treatment, regardless of how treatment was assigned in the experiment used to estimate this quantity hudgens2008toward.
Aggregating the individual average potential outcome across units within a cluster, the cluster-level average potential outcome at saturation $\pi$ is given by \[ \bar{Y}_i(\pi) = \frac{1}{n_i}\sum_{j=1}^{n_i}\bar{Y}_{ij}(\pi), \qquad \text{for all } i. \] Finally, we define the population-level mean potential outcome at saturation $\pi$ as \[ \bar{Y}(\pi) \;:=\; \mathbbm{E}_{\theta}\!\left[\bar{Y}_i(\pi)\right], \] where the expectation is taken with respect to the distribution of cluster-level average potential outcomes, denoted by $F_\pi(\cdot,\theta)$, which belongs to a family of distributions indexed by a common parameter $\theta \in \Theta$. We assume that, for any $\pi \neq \pi' \in [0,1]$, the distributions $F_\pi(\cdot,\theta)$ and $F_{\pi'}(\cdot,\theta)$ are indexed by the same parameter space $\Theta$, so that differences in $\bar{Y}(\pi)$ across saturation levels reflect changes in the treatment policy rather than changes in the underlying population.
We adopt a utilitarian welfare paradigm, under which social welfare is measured by the equally weighted expected outcome of the population, with higher values indicating greater welfare. Therefore, for a given treatment saturation level $\pi$ and parameter value $\theta\in\Theta$, potential welfare is defined as $$U(\pi,\theta):=\bar{Y}(\pi).$$
The choice to weight clusters equally rather than units equally is a normative commitment. Our estimand $U(\pi,\theta) = \mathbbm{E}_\theta[\bar Y_i(\pi)]$ averages cluster-level mean potential outcomes uniformly across the super-population of clusters and corresponds to a one-cluster-one-vote planner. A unit-weighted alternative, $U^{\mathrm{unit}}(\pi, \theta) = \mathbbm{E}_\theta[n_i \bar Y_i(\pi)] / \mathbbm{E}_\theta[n_i]$, would be appropriate for a planner who values each individual equally. We use the cluster-level target because the sampling and saturation-assignment units are clusters, matching the estimand to the design; this is also the convention in the saturation-experiment literature hudgens2008toward,liu2014large,baird2018optimal. In simulations with equal cluster sizes, the cluster-weighted and unit-weighted welfare targets coincide.
Next, we turn to the problem of learning about $U(\pi, \theta)$ from data. We assume the planner has access to experimental data from a random sample of $C$ clusters from the target population; the specific experimental design is described in the next subsection. In many settings, the set of allowable treatment saturations may be exogenously constrained by ethical, budgetary, equity-based, legislative, or political considerations. To reflect these constraints, we assume the following about the support of the treatment saturation.
Assumption (ref) restricts the number of possible treatment saturations under evaluation. The observed experimental data consist of units nested within sampled clusters, where each cluster is assigned to one of $K$ treatment saturation levels.
We now describe the two-stage randomized saturation design commonly used in settings where treatment may spill over within clusters hudgens2008toward, tchetgen2012causal, liu2014large, crepon2013labor, baird2018optimal. Recall that $C$ clusters are randomly sampled from the infinite population of clusters.
In the first stage, each of the \(C\) clusters is randomly assigned to one of \(K\) treatment strategies, denoted by \(\psi_1,\ldots,\psi_K\), where each \(\psi_k\) specifies the individual-level treatment assignment mechanism within a cluster and corresponds to a distinct treatment saturation level. For example, \(\psi_1\) may correspond to assigning one-third of individuals in a cluster to treatment, while \(\psi_2\) may correspond to assigning two-thirds.
Let \(\mathbf{S} = (S_1,\ldots, S_C)\) denote the resulting vector of first-stage cluster-level assignments, where \(S_i = \pi_k\) indicates that cluster \(i\) is assigned to treatment strategy \(\psi_k\). The randomization distribution of \(\mathbf{S}\) is governed by a parameter \(\nu\), which indexes the first-stage assignment mechanism. When \(\nu\) corresponds to a Bernoulli design, clusters are independently assigned to strategies according to prespecified probabilities. When \(\nu\) corresponds to complete randomization (CR, hereafter), a fixed number of clusters are assigned to each strategy, and each assignment vector \(\mathbf{S}\) satisfying these counts is equally likely.
In the second stage, treatment is assigned to individuals within each cluster according to the first-stage assignment \(S_i\). Conditional on \(S_i = \pi_k\), cluster \(i\) follows the treatment strategy \(\psi_k\), which specifies a target treatment saturation level \(\pi_k\). Under \(\psi_k\), individual treatment may be assigned via a Bernoulli design, in which each unit is independently treated with probability \(\pi_k\), or via complete randomization, in which exactly \(\pi_k\cdot n_i\) of the \(n_i\) individuals in cluster \(i\) are assigned to treatment, with all such assignments equally likely.
Following hudgens2008toward, we focus on the two-stage complete randomization assignment mechanism summarized in the following assumption. In Section (ref) of the appendix, we derive the finite-sample results that follow under the alternative two-stage Bernoulli design.
Assumption (ref) allows the cluster and treatment assignments to be dependent across clusters and across units. This negative dependence comes from the assignment design itself, rather than from the network structure $G$. Under the two-stage complete randomization design in Assumption (ref), each unit's realized outcome is correlated with that of all other units in the sample, because complete randomization constrains the number of treated units to meet the saturation level $\pi_k$ similar to sampling without replacement. From a graph-theoretic perspective, the two-stage complete randomization induces a complete dependency graph over the realized outcomes, distinct from the underlying network structure $G$. Example (ref) illustrates the outcome dependencies induced by the experimental design.
Let the total sample size be $n = \sum_{i=1}^C n_i$. Then, based on Assumptions (ref) and (ref), we define the sample space as \[ \Omega := \left( \{0,1\} \times [0,1] \times \{\pi_1,\pi_2,\dots,\pi_K\}\times \mathbbm{Z}_{>0} \right)^n, \] where each observation records the treatment indicator $Z_{ij} \in \{0,1\}$, outcome $Y_{ij} \in [0,1]$, saturation level $S_i \in \{\pi_1,\pi_2,\dots,\pi_K\}$, and cluster size $n_i\in \mathbbm{Z}_{>0}$. Recall that the within-cluster network structure $G_i$ is unobserved.
Following hudgens2008toward, under the two-stage complete randomization design of Assumption (ref), an unbiased estimator of the cluster-level average potential outcome $\bar{Y}_i(\pi_k)$, conditional on $S_i = \pi_k$, for all \( k = 1, \dots, K \), is the simple cluster mean $$\widehat{\bar{Y}}_i(\pi_k) = \frac{1}{n_i}\sum_{j=1}^{n_i}Y_{ij}.$$ Unbiasedness follows from the same randomization argument as in hudgens2008toward: conditional on $S_i = \pi_k$, the within-cluster assignment law is exactly the complete-randomization pmf $\gamma(\pi_k, \cdot)$ used to define $\bar Y_{ij}(\pi_k)$ in (ref), so the simple cluster mean targets the CR-mixture welfare $\bar Y_i(\pi_k)$. Under the alternative two-stage Bernoulli design, the conditional law is the product Bernoulli pmf $\beta(\pi_k, \cdot)$ rather than $\gamma(\pi_k, \cdot)$, so an inverse-probability-weighting adjustment is needed to recover the same CR-mixture target; see Section (ref) of the supplementary material.
The corresponding estimator of the population-level mean potential outcome at saturation $\pi_k$ is
where $C_k$ denotes the number of clusters assigned to saturation $\pi_k$.
In this section, we describe the problem of ranking treatment saturations. If Assumption (ref) holds, then the policymaker has a prespecified discrete set of treatment saturations to rank. Thus, we study the statistical ranking rules designed to compare the welfare at a finite number of treatment saturation levels. Formally, a ranking rule $\delta$ denotes the vector collecting pairwise rules $\delta^{k,k'}$ with $k<k'=1,\dots, K$, i.e., \[ \delta = \big( \delta^{k,k'} \big)_{k,k' \in \{1,\dots,K\}}, \quad \text{where} \quad \delta^{k,k'} : \Omega \rightarrow \{0,1\}, \quad \text{for all } k,k' \in \{1,\dots,K\},\, k<k'. \] Each component $\delta^{k,k'}$ encodes the rule's decision as to whether treatment saturation level $\pi_k$ is preferred to $\pi_{k'}$ based on the observed sample. Hence, the vector rule $\delta$ maps the observed data to a binary vector of length $T = K(K - 1)/2$, indicating the direction of all possible pairwise comparisons between treatment saturations. For example, consider the case where $K = 3$, so that $T = 3$. For $\omega\in\Omega$, a vector rule of $\delta(\omega) = \left(\delta^{1,2}, \delta^{1,3}, \delta^{2,3}\right) = (1, 1, 1)$ implies that the rule ranks saturation $\pi_1$ higher than both $\pi_2$ and $\pi_3$, and ranks $\pi_2$ higher than $\pi_3$.
Note that for each pair $(k,k')$ with $k,k' \in \{1,\dots,K\}$ and $k < k'$, $\delta^{k,k'}$ is a nonrandomized rule, thus the set $\{\delta^{k,k'}(\cdot):k,k' \in \{1,\dots,K\}, k<k'\}$ uniquely determines $\delta(\cdot)$ which is also a nonrandomized rule on the space $\{0,1\}^T$ cohen2005decision.
The infeasible optimal ranking rule for a given \( \theta \in \Theta \), referred to as the oracle rule, ranks all treatment saturation levels in \( \boldsymbol{\Pi} \) according to their true welfare values. Formally, the oracle rule is a nonrandomized rule defined as the vector of pairwise comparisons:
Thus, for each pair \( (\pi_k, \pi_{k'}) \in \boldsymbol{\Pi} \), the oracle rule prefers \( \pi_k \) over \( \pi_{k'} \) whenever the associated welfare under \( \theta \) is weakly greater. This rule, however, is infeasible in practice because it depends on the unknown welfare function \(U(\cdot, \theta) \).
To obtain a feasible alternative to the oracle rule, we replace the unknown population welfare parameters in the oracle ranking rule with their unbiased estimators \(\widehat{U}(\pi, \theta)\) defined in (ref). The resulting rule, the empirical success (ES) ranking rule, is defined as the vector of pairwise comparisons:
Therefore, for any pair \( (\pi_k, \pi_{k'}) \), the ES ranking rule ranks \( \pi_k \) weakly above \( \pi_{k'} \) whenever its estimated welfare is greater than or equal to that of \( \pi_{k'} \). This plug-in approach yields a practical rule that mimics the oracle ranking using observable data. Note that we adopt a tie-breaking convention under which the pairwise ES ranking rules are nonrandomized.\footnote{Using an alternative tie-breaking convention that randomly allocates some individuals to one treatment saturation and the rest to the other would not affect the subsequent analysis.}
In this subsection, we evaluate the finite-sample performance of the proposed ES ranking rule in the target population. Our analysis follows the statistical decision theoretic framework developed by wald1950statistical.
We model loss as additively separable across the \( T \) pairwise comparisons. Specifically, we adopt an additively separable regret loss function defined as
where \(\pi(\delta_{\mathrm{ES}}^{k,k'}):=\pi_k \cdot \delta_{\mathrm{ES}}^{k,k'} + \pi_{k'} \cdot (1-\delta_{\mathrm{ES}}^{k,k'}) \). Note that \( U(\pi(\delta_{\mathrm{ES}}^{k,k'}), \theta) \) denotes the welfare associated with the treatment saturation selected by the pairwise ES ranking rule \( \delta_{\mathrm{ES}}^{k,k'} \) for the \( (k,k') \) comparison. Specifically, \(U(\pi(\delta_{\mathrm{ES}}^{k,k'}), \theta)= U(\pi_k, \theta) \) if \( \delta_{\mathrm{ES}}^{k,k'} = 1 \), and \( U(\pi_{k'}, \theta) \) otherwise. This loss function naturally reflects the structure of the treatment ranking problem, where decisions are made through repeated binary comparisons. Unlike the hypothesis testing loss function in lehmann1957theory and cohen2005decision, which penalizes differently for two types of errors, the loss function here directly penalizes the welfare losses arising from incorrect pairwise rankings.
Based on the loss function in (ref), for any $\theta \in \Theta$, the risk of the empirical success rule $\delta_{\mathrm{ES}}$ is given by
The risk function measures the expected welfare loss from using the ES ranking rule instead of the oracle ranking rule. For each pair of treatment saturations $(\pi_k,\pi_{k'})$, the risk reflects the discrepancy between the welfare under the optimal (oracle) decision and that under the ES ranking rule, averaged over the sampling distribution of the data. Specifically, the first term in the last equality represents the welfare attained by always choosing the better of any two saturations based on true welfare, while the second term represents the expected welfare attained by choosing based on estimated welfare. The difference between these two quantities aggregates the pairwise regret across all treatment comparisons and provides a finite-sample measure of the ES ranking rule's performance.
However, the risk function \( R(\delta_{\mathrm{ES}}, \theta) \) is analytically intractable, as its evaluation requires full knowledge of the joint sampling distribution of the estimated welfare values \( \widehat{U}(\pi_k, \theta) \) and \( \widehat{U}(\pi_{k'}, \theta) \) across all \( (k, k') \) pairs.\footnote{Even under standard outcome models such as the normal or Bernoulli distribution, the derivation of finite-sample minimax-regret rules, as pursued in tetenov2012statistical, remains analytically intractable in the presence of dependencies. The key challenge is that, due to outcome dependencies, the estimated welfare fails to serve as a sufficient statistic for the underlying welfare parameter.} To overcome this challenge, we follow the approach of manski2004statistical and derive a finite-sample upper bound on the risk. Our analysis uses the concentration inequality of janson2004large for dependent random variables described by dependency graphs.
To state the bounds, we introduce notation for the dependency structure. For each pair $(k, k')$ with $k < k'$, let $G_{kk'}$ denote the unit-level dependency graph: its vertices are the units in sampled clusters assigned to $\pi_k$ or $\pi_{k'}$, and two units are joined by an edge when their realized outcomes are stochastically dependent. Similarly, let $G^{\text{cls}}_{kk'}$ denote the cluster-level dependency graph, whose vertices are the sampled clusters assigned to $\pi_k$ or $\pi_{k'}$ and whose edges join clusters with dependent estimated cluster-level average potential outcomes. For any graph $G$, let $\chi_f(G)$ denote its fractional chromatic number. Intuitively, $\chi_f(G)$ quantifies the complexity of the dependency structure: it equals $1$ when all variables are independent (the graph has no edges), and grows to the number of vertices when all variables are mutually dependent (the graph is complete). In the concentration inequality underlying the bounds below, $\chi_f(G)$ enters the exponent as a multiplicative penalty on the scale term, much as the design effect inflates the variance of a cluster-sample estimator relative to simple random sampling. A larger $\chi_f$ thus reflects a more entangled dependence pattern that weakens concentration and inflates the risk bound.
We collect the finite-sample risk bounds in a single theorem with three parts. Part (i) is the unit-level bound under bounded cluster sizes, part (ii) is the unit-level bound under the stronger equal-size restriction, and part (iii) is the cluster-level bound, which holds with no restriction on cluster sizes. We first state the two cluster-size restrictions used by the unit-level parts.
Assumption (ref) is the special case of Assumption (ref) with $\underline{n}=\overline{n}=n_0$; under it, the cluster sizes are non-stochastic, so the unit-level scale factor becomes exact rather than enveloped.
Under two-stage complete randomization, the two stages assign saturations and treatments without replacement, so within the two arms being compared, every pair of realized outcomes is stochastically dependent. The unit-level dependency graph $G_{kk'}$ is therefore complete on the units belonging to clusters assigned to $\pi_k$ or $\pi_{k'}$ (with $(C_k+C_{k'})n_0$ vertices under equal sizes), and the cluster-level graph $G^{\mathrm{cls}}_{kk'}$ is complete on the $C_k+C_{k'}$ such clusters. Their fractional chromatic numbers can thus be evaluated explicitly, and each bound admits a transparent closed form.
The three bounds trace out a tradeoff between the granularity of the information they exploit and the restrictions they require. A careful comparison clarifies when each bound is the relevant object.
We first compare parts (i) and (iii), which differ in how they handle cluster-size heterogeneity. The unit-level bound exploits individual-level variation within clusters. Under two-stage complete randomization, the unit-level dependency graph is complete on the units belonging to the two treatment arms. Hence, under bounded cluster sizes, $\chi_f(G_{kk'}) \leq (C_k+C_{k'})\overline n.$ Combined with the unit-level scale factor, this yields the closed form in (ref). By contrast, the cluster-level bound first aggregates outcomes within clusters. Each cluster mean lies in \([0,1]\), irrespective of cluster size, and the relevant dependency graph is the complete cluster graph, for which $\chi_f(G^{\mathrm{cls}}_{kk'}) = C_k+C_{k'} .$ This gives (ref). The two closed forms differ only through the heterogeneity ratio \(\overline n/\underline n\geq 1\) in the exponent. Therefore, the cluster-level bound is never looser than the unit-level bound, and the gap between the two is governed entirely by the dispersion of cluster sizes. This gap vanishes as \(\overline n/\underline n\to 1\). The unit-level form, however, comes at the cost of Assumption (ref), whereas the cluster-level form imposes no restriction on cluster sizes.
Next, compare parts (ii) and (iii). When cluster sizes are equal, say \(n_i=n_0\) for all \(i\), the complete unit-level graph has $\chi_f(G_{kk'}) = (C_k+C_{k'})n_0 .$ The factor \(n_0\) in this dependence penalty is exactly offset by the factor \(n_0\) in the unit-level numerator. Consequently, the unit-level bound in (ref) coincides with the cluster-level bound in (ref), as stated in Corollary (ref). Equivalently, equal cluster sizes imply \(\overline n/\underline n=1\), which is precisely the point at which the unit-level and cluster-level exponents meet. Thus, under equal cluster sizes, nothing is lost by passing to the cluster level: the two granularities deliver identical guarantees.
A direct implication is that, under two-stage complete randomization with equal cluster sizes, increasing the within-cluster sample size \(n_0\) does not tighten the risk bound. The closed-form exponent \[ -2\Delta_{kk'}^2 \frac{C_kC_{k'}}{(C_k+C_{k'})^2} \] does not depend on \(n_0\). This is a structural feature of the design, not an artifact of the bounding argument. Complete randomization makes the \(n_0\) units in a cluster fully dependent, so additional units within a cluster do not contribute independent information about the cluster-level welfare contrast. The within-cluster sample size, therefore, cancels between the dependence penalty and the scale factor.
A more striking feature is that the bounds do not contract with the number of clusters. To demonstrate this, note that under Assumption (ref), we can write \(C_k=\alpha_k C\), where $\alpha_k$ denotes the share of sampled clusters assigned to $\pi_k.$ Thus, the common exponent coefficient satisfies \[ \frac{C_kC_{k'}}{(C_k+C_{k'})^2} = \frac{\alpha_k\alpha_{k'}}{(\alpha_k+\alpha_{k'})^2}, \] which depends on the design only through the allocation shares and not on \(C\). Under balanced allocation, \(\alpha_k=\alpha_{k'}\), this coefficient equals \(1/4\), so the per-pair exponent is $-1/2\cdot\Delta_{kk'}^2$ for every value of \(C\). This is the price of the complete dependency structure induced by two-stage complete randomization. Because both stages assign saturations and treatments without replacement, every pair of outcomes in the two arms is dependent. Janson's inequality, when applied to a complete graph, cannot exploit the negative association generated by sampling without replacement, as with Hoeffding's inequality hoeffding1963probability, which is inapplicable because of the underlying within-cluster interference.\footnote{The risk bounds based on the Hoeffding inequality, as shown in our simulation results, could be valid when the underlying within-cluster interferences are weak. We defer formal arguments on this front to future research, as this would involve nontrivial covariance decomposition between the outcomes of any pair of units.} The bound, therefore, is a finite worst-case regret governed by the allocation shares rather than by the total number of clusters.\footnote{In contrast, in Section (ref) of the supplementary material, we show that when the design is Bernoulli, the resulting bounds contract in $C$ since the design does not induce coupling in saturation and treatment assignment.} The true risk may still contract with \(C\), as the simulations in Section (ref) show, but the complete-graph structure does not capture that contraction.
Among the three results, the cluster-level bound in (ref) is both the tightest and the least restrictive. It is never looser than the unit-level bound in part (i), it coincides with the unit-level bound in part (ii) under equal cluster sizes, and it requires no cluster-size assumption. We therefore adopt it as the basis for the design analysis in Section (ref), where we show that it is minimized over allocations by the balanced design.
Finally, these bounds can be contrasted with the no-interference benchmark of manski2004statistical. In the independent-sampling setting, the dependency graph is empty and \(\chi_f=1\). The exponent then carries the full sample count, so the corresponding risk bound contracts at the parametric rate. Two-stage complete randomization departs from this benchmark in two ways. First, within-cluster interference couples units within the same cluster. Second, assignment without replacement at both stages couples units and clusters across the sample. These features force the relevant dependency graphs to be complete. Hence, applying the interference-free, independence-based bound by setting \(\chi_f=1\) would understate the true upper bound on the risk of the empirical-success ranking rule. The simulations in Section (ref) confirm this point: when within-cluster dependence is strong, the Manski bound can fall below the actual risk, whereas the complete-graph bound remains valid.
In this section, we study the optimal allocation of treatment saturations across clusters in the first stage of the experimental design under clustered interference. Our objective is to design a two-stage complete randomization experiment that minimizes the finite-sample risk of the ES ranking rule. We apply the cluster-level risk bound (ref) established in Theorem (ref)(iii) to characterize optimal cluster allocations. Since the cluster-level bound requires no restriction on cluster sizes, the design results below hold for arbitrary cluster sizes. Recall that under Assumption (ref), the cluster counts $C_k=\alpha_k C$ are fixed by the design, so the right-hand side of (ref) is deterministic in the design choice $\boldsymbol{\alpha}:= (\alpha_1, \dots, \alpha_K)$.
Formally, under Assumption (ref), the two-stage experimental design randomly assigns sampled clusters to treatment saturations in \( \boldsymbol{\Pi} \) via complete randomization, followed by complete randomization of treatment assignments within each cluster in the second stage. Let \( \boldsymbol{\alpha}= (\alpha_1, \dots, \alpha_K) \in \Delta_K\) denote the probability distribution over the support \( \boldsymbol{\Pi} = \{ \pi_1, \dots, \pi_K \} \) used to assign treatment saturations to the sampled clusters, where \( \Delta_K \) is the probability simplex over \( K \) elements. Given the total number of clusters $C$, the experimental design is fully specified by the pair \( (\boldsymbol{\Pi}, \boldsymbol{\alpha}) \).
Our goal is to determine the allocation of treatment saturations, $\boldsymbol{\alpha}$, that minimizes the supremum of risk of the ES ranking rule for any saturation set $\boldsymbol{\Pi}$. Since the supremum of the risk function \( \sup_{\theta \in \Theta}R(\delta_{\mathrm{ES}}, \theta) \) is analytically intractable, we instead consider a minimax optimization over a feasible upper bound.
Maximizing the cluster-level bound (ref)---equivalently its closed form (ref)---over $\Delta_{kk'} \geq 0$, each summand $\Delta_{kk'}\exp\!\big(-2\Delta_{kk'}^2\,C_kC_{k'}/(C_k+C_{k'})^2\big)$ attains its maximum at \[ \Delta^{*}_{kk'} = \frac{C_k+C_{k'}}{2\sqrt{C_kC_{k'}}}. \] Substituting $\Delta^{*}_{kk'}$ yields a uniform bound on the risk over the entire parameter space \( \Theta \):
where the last equality holds since $C_k=\alpha_kC$ and the factor $C$ cancels. Unlike in the independence-based analysis of manski2004statistical, the right-hand side carries no $C^{-1/2}$ factor: under the complete dependency structure induced by two-stage complete randomization, the worst-case bound depends on the design only through the allocation shares $\boldsymbol{\alpha}$ and does not contract in the total number of clusters.
Formally, we aim to solve the optimization problem $ \operatorname*{arg min}_{\boldsymbol{\alpha} \in \Delta_K} \sup_{\theta \in \Theta} R(\delta_{\mathrm{ES}}, \theta)$. Since the positive constant $\tfrac{1}{2}\exp(-1/2)$ does not affect the minimizer, the quasi-optimal design problem reduces to
The following theorem establishes that the solution to this optimization problem is the balanced allocation \( \alpha_k^* = 1/K \) for all \( k \in \{1, \dots, K\} \).
Theorem (ref) establishes that, under a two-stage completely randomized treatment design, balanced allocation of clusters minimizes the upper bound on the risk. In practical terms, the result prescribes assigning an equal number of clusters to each treatment saturation level in the first stage of the experiment. At the balanced allocation, the right-hand side of (ref) simplifies to $\tfrac12 e^{-1/2}\,K(K-1)=\binom{K}{2}e^{-1/2}$, a constant that does not depend on $C$; consistent with the discussion following Corollary (ref), the worst-case bound is governed by the allocation shares rather than the number of clusters. The proof, which follows from the arithmetic--geometric mean inequality $\tfrac{\alpha_k+\alpha_{k'}}{\sqrt{\alpha_k\alpha_{k'}}}\ge 2$ with equality if and only if $\alpha_k=\alpha_{k'}$, is provided in the supplementary material.
Analogous to the discussion on optimal stratified designs in manski2004statistical, the resulting allocation is regarded as quasi-optimal for two reasons. First, the analysis is restricted to ES ranking rules; thus, the solution need not be optimal for other decision rules. Second, the objective function minimizes an upper bound on the supremum of the risk function, rather than the supremum of the risk function itself. Consequently, the allocation derived from this approximation may not attain true minimax optimality, but nevertheless offers a principled and implementable approach to the first-stage design.
While the analysis in this section echoes classical design principles, our objective departs significantly from prior work. For instance, baird2018optimal also studies optimal treatment saturation designs, but does so through the lens of minimizing standard errors (i.e., maximizing the probability of detecting a statistically significant average treatment effect). In contrast, we adopt a decision-theoretic perspective, aiming to minimize the supremum of risk associated with the ES ranking rule. This yields finite-sample welfare guarantees that match the planner's goal of identifying the treatment saturation that maximizes expected welfare. This focus on worst-case performance builds on research advocating the selection of sample sizes in clinical trials using finite-sample welfare criteria; for example, see manski2016sufficient (manski2016sufficient, manski2019trial).
A limitation of our approach, which also highlights its generality, is that the resulting quasi-optimal design is uniform: it does not depend on the specific configuration of treatment saturation levels or within-cluster sample sizes. This arises from our use of an upper bound on maximum regret, which ensures robustness but abstracts from the additional structure that knowledge of the saturation set could provide. While more refined designs may be attainable when such information is available, the ability to characterize a saturation-independent design with finite-sample guarantees offers a practical approach to experimental design in clustered settings.
Having established the finite-sample properties of the ES ranking rule and the quasi-optimal first-stage allocation, we turn in the next section to the asymptotic behavior of the ranking problem.
In this section, we establish the asymptotic admissibility and optimality of the ES ranking rule using the limit-of-experiment framework. Specifically, we characterize conditions under which threshold ranking rules\footnote{An ES ranking rule is a threshold rule with threshold values equal to zero.} are admissible (Proposition (ref)), Bayes optimal (Proposition (ref)), and minimax optimal (Theorem (ref)) in the limiting Gaussian model. Moreover, we show that the finite-sample ES ranking rule converges to the asymptotically optimal rule (Theorem (ref)).
Following hirano2009asymptotics, we investigate the asymptotic optimality of the ES ranking rule around a reference local parameter. We first restrict our attention to parametric regular models; the extension to the semiparametric class is discussed at the end of this section.
Recall that the planner observes the data $\omega\in \Omega$ which is informative about the parameter $\theta.$ We let $P_{\theta,n}$ denote the distribution of $\omega$ on the space $\Omega$, i.e., $\omega\sim P_{\theta,n}.$ In what follows, we consider a sequence of experiments $\mathcal{E}_n:=\{ P_{{\theta,n}}:{\theta} \in \Theta \subset \mathbbm{R}^d \}$ that may grow as the sample size grows, where $\Theta$ is an open subset of $\mathbbm{R}^d$.
We define a vector of welfare contrasts $g({\theta}) := \left(g_1({\theta}), \ldots, g_T({\theta}) \right)^{\top}$, where $g_t(\theta):=U(\pi_k,{\theta}) - U(\pi_{k'},{\theta})$ is the welfare contrast between some $\pi_k$ and $\pi_{k'}.$ Note that we use $t$ for a generic combination $(k,k')$, hence $t\in \{1,\ldots, T\}.$ For each $t$, we assume that $g_t(\theta)$ is continuously differentiable in $\theta$.
We consider a sequence of local parameters around $\theta_0\in \Theta$, where $g({\theta}_0) = 0$. The ranking problem at the local reference parameter $\theta_0$ is the most difficult case in the parameter space. If $g_t({\theta})\neq 0$ for a given $\theta$, one action is strictly dominated and the decision between $(k,k')$ becomes trivial asymptotically.
The form of admissible and optimal rules in the limit experiment depends on the loss function. Hereafter, we consider two component loss functions: the hypothesis testing loss function defined as $$L^H_t(\delta^{t},\theta):= \mathbbm{1}(g_t(\theta)>0)(q - \delta^{t}(1+q)) +\delta^{t}$$ for $q>0$, and the regret loss function (introduced in Section (ref)) redefined as $$L^R_t(\delta^{t},\theta):= g_t(\theta)\left[ \mathbbm{1}(g_t(\theta)>0) - \delta^{t} \right].$$ Summing each component loss function across $t$ gives us the corresponding additively separable loss functions denoted as $L^H(\delta,\theta)$ and $L^R(\delta,\theta)$, respectively. Hence, the risk function can be written as:
where $R^m_t(\delta^{t},\theta)$ denotes the component risk for a pair of saturations and $m\in \{H, R\}.$
To obtain an asymptotic approximation of the original experiment, we assume that the sequence of experiments $\mathcal{E}_n:=\{ P_{{\theta,n}}:{\theta} \in \Theta\}$ is differentiable in quadratic mean (DQM) at $\theta_0$, i.e., we assume that for each $n$, the probability measures $\{P_{\theta,n}: \theta \in \Theta\}$ are dominated by a common $\sigma$-finite measure $\mu$, with corresponding densities $p_{\theta,n} = dP_{\theta,n}/d\mu$. Then, there exists a measurable function $s:\Omega\mapsto \mathbbm{R}^d$ (called the score function associated with model $\mathcal{E}_1$) such that, as $h \to 0$,
where $s(\omega) = \frac{\partial \log p_{_{\theta,n}}(\omega)}{\partial \theta}\vert_{\theta=\theta_0}$. We assume that the Fisher information matrix $I_0=E_{\theta_0}[s(\omega)s(\omega)^\top]$ is nonsingular. Applying Le Cam's local asymptotic normality (LAN) arguments, the Gaussian shift experiment $\{ N(h, I_0^{-1}): h\in \mathbbm{R}^d\}$ serves as the “limit experiment” of the original experiment, with observation $\Delta \sim N(h, I_0^{-1})$ for some $h\in \mathbbm{R}^d$.
Hence, any converging sequence of ranking rules in the original experiment is matched by some ranking rule in the limit experiment (see Proposition 3.1 in hirano2009asymptotics). This reduction from the original complex experiment to a Gaussian shift experiment is the foundation for the optimality results that follow.
We next derive the limiting versions of the loss and risk functions. Define $\nabla_\theta g:=[\partial g_t(\theta_0)/\partial \theta_v]_{\substack{{\nu=1,\dots, d}\\{t=1,\dots,T}}}$ as a $T\times d$ Jacobian matrix of $g(\theta)$ evaluated at $\theta_0$. Then, since $g_t(\cdot)$ is continuously differentiable in $\theta$, we can show that $\sqrt{n}g_t(\theta_0+h/\sqrt{n}) \to (\nabla_{\theta} g_t)h$, the true welfare contrast at $\theta_0$ in the limit experiment. Consequently, we have
and
where $\nabla_{\theta} g_t$ is the $t$-th row of $\nabla_{\theta} g$. The corresponding component limiting risk function is
where $m\in\{H, R\}$. Note that we abuse notation slightly: we use the same $\delta^t$ for both $L^m_{t}(\delta^t,\theta)$ and $L^m_{t,\infty}(\delta^t,h)$. However, the rule in $L^m_t(\cdot, \theta)$ is defined on the sample $\omega \in \Omega$, while the rule in $L^m_{t,\infty}(\cdot, h)$ is defined on the simpler asymptotic experiment space $\Delta \in \mathbbm{R}^d$. This extends to the rules in $R^m_t(\cdot, \theta)$ and $R^m_{t,\infty}(\cdot, h)$ as well.
Aggregating the component loss functions, we obtain the asymptotic additively separable loss function
and the corresponding asymptotic additively separable risk function and its supremum,
respectively.
Following cohen2005decision, we note that the ranking problem in the limit experiment can be written as a one-sided multiple endpoints problem of the form:
Thus, the ranking problem can be viewed as a $ 2^T$-action problem in which one selects an action to accept or reject $H_{0t}$ for $t=1,\dots, T$. Thus, for any $h\in\mathbbm{R}^d$, we define the set of true $H_{0t}'\,s$ as $\mathcal{T}(h):=\{t: (\nabla_{\theta} g_t)h\leq 0\}.$
In the limit experiment, the “best” estimator of the vector of welfare contrasts $(\nabla_{\theta} g)h$ is the Gaussian estimator $(\nabla_{\theta} g) \Delta$ van2000asymptotic. It has a mean $(\nabla_{\theta} g)h$ and covariance matrix $(\nabla_{\theta} g)\, I_{0}^{-1}\, (\nabla_{\theta} g)^{\top}$. Using the hypothesis loss function and under appropriate restrictions on the covariance of the Gaussian estimator, we find ranking rules that are admissible in the limiting Gaussian model.
Proposition (ref) states that if the covariance matrix of the estimator of welfare contrasts in the limit experiment is intra-class, i.e., $(\nabla_{\theta} g)\, I_{0}^{-1}\, (\nabla_{\theta} g)^{\top}=\sigma^2[(1-\rho)I_T +\rho \mathbf{1}\mathbf{1}^\top]$, where $I_T$ is the identity matrix of dimension $T$, with correlation between individual estimators of welfare contrasts at least $-1/q$, then the ranking rule $\delta_{\kappa}(\Delta)$ is admissible under the hypothesis loss function. The two restrictions on $\rho$ in Proposition (ref) imply that $\max\{-1/(T-1), -1/q\}\leq \rho\leq 1$. We consider two special cases.
First, consider $\rho=0$, so that $(\nabla_{\theta} g)\, I_{0}^{-1}\, (\nabla_{\theta} g)^{\top}=\sigma^2I_T$. This asserts asymptotic independence of the estimator of welfare contrasts, which is possible even though the estimators are dependent by construction in finite samples. Indeed, asymptotic independence concerns the leading $1/\sqrt{n}$ terms: the shared estimated welfare parameters may be negligible at that scale or may cancel in covariance, and this is equivalent to orthogonality of the corresponding gradient directions.
Second, consider $\rho=1$, so that $(\nabla_{\theta} g)\, I_{0}^{-1}\, (\nabla_{\theta} g)^{\top}=\sigma^2\mathbf{1}\mathbf{1}^\top$. This implies that the gradients of all welfare contrasts are identical, so that $rank(\nabla_{\theta} g)=1$. Consequently, the asymptotic additively separable hypothesis loss reduces to $L^H_{\infty}(\delta,h)=\sum_{t=1}^T L^H_{t,\infty} (\delta^t, h)=T\cdot(\mathbbm{1}((\nabla_{\theta} g_1)h>0)(q - \delta^{1}(1+q)) +\delta^{1})$. Thus, in the limit experiment, the ranking problem becomes effectively one-dimensional and is straightforward to solve. In contrast, the corresponding problem in the original experiment need not be one-dimensional, even though its limit representation is. Therefore, when $\rho=1$, the ranking problem need not be trivial in the original experiment, underscoring the usefulness of the sufficient conditions of Proposition (ref).
Moreover, Proposition (ref) shows that multiple admissible ranking rules arise in the limit experiment, with the specific rule depending on the underlying welfare contrasts and covariance structure. Since the finite-sample ES ranking rule corresponds to the limiting rule where $\boldsymbol{\kappa}=\mathbf{0}$, it converges to an admissible rule when $\inf_{h \in \mathcal{H}_0} \Phi_{|\mathcal{T}(h)|} \!\left( - \mu_{\mathcal{T}(h)}(h); \Sigma_{\mathcal{T}(h),\,\mathcal{T}(h)} \right) \ge 1 - \alpha.$ Thus, the ES ranking rule is asymptotically admissible when the limiting welfare contrasts are sufficiently negative relative to their covariance.
Next, we show that, under additional restrictions beyond those in Proposition (ref), the ranking rule in Proposition (ref) is proper Bayes (Bayes optimal) in the limiting Gaussian model.
The sufficient condition for the result in Proposition (ref) holds if $(\nabla_{\theta} g_{_t})^{\top}\, I_{0}^{-1}\, (\nabla_{\theta} g_{_{t}})=1$ for all $t=1,\dots, T$ and $(\nabla_{\theta} g_{_t})^{\top}\, I_{0}^{-1}\, (\nabla_{\theta} g_{_{t'}})=0$ for $t\neq t'$. It implies a special intra-class structure on the covariance matrix $(\nabla_{\theta} g)\, I_{0}^{-1}\, (\nabla_{\theta} g)^{\top}$, where $\sigma^2=1$ and $\rho=0$. Hence, similar to the case when $\rho=0$ in Proposition (ref), this is a plausible condition that depends primarily on the Fisher information and the gradients of the welfare contrasts evaluated at $\theta_0.$ The sufficient condition in Proposition (ref) does not reduce the dimension of the ranking problem in either the original experiment or the corresponding limit experiment, underscoring the relevance of the Proposition.
Moreover, Proposition (ref) implies that the finite sample ES ranking rule is asymptotically proper Bayes (converges to the limit rule where $\boldsymbol{\kappa}=\mathbf{0}$) if $\inf_{h\in\mathcal{H_0}}\prod_{t\in\mathcal{T}(h) } \Phi\!\left(-(\nabla_{\theta} g_t)h\right)\ge 1-\alpha.$ A sufficient condition is that there exists an $h^*\in\mathcal{H}_0$ such that $\Phi\!\left(-(\nabla_{\theta} g_t)h^*\right)\ge (1-\alpha)^{1/T}\iff(\nabla_{\theta} g_t)h^\ast \leq\Phi^{-1}((1-\alpha)^{1/|T|} ) \text{ for all } t\in \mathcal{T}(h^*).$
Together, Propositions (ref) and (ref) demonstrate that, in the presence of additional structure in the limit experiment, characterizing admissible and optimal decision rules using the additively separable hypothesis loss function in the Gaussian model is generally a feasible task. In particular, such characterizations require restrictions on the joint distribution of estimators of welfare contrasts in the limit.
Motivated by Propositions (ref) and (ref), we impose a restriction that permits the characterization of admissible and optimal decision rules in the limiting Gaussian model using the same additively separable regret loss function employed in the finite-sample analysis. Specifically, we assume that the Gaussian estimator of welfare contrasts in the limit Gaussian model, $(\nabla_{\theta} g) \Delta$, has one-dimensional stochastic variation, i.e., $rank(Var((\nabla_{\theta} g) \Delta))=1,$ while allowing estimators of individual welfare contrasts to be either perfectly positively or perfectly negatively correlated. This configuration does not trivialize the original ranking problem. Even the ranking problem in the limit is not trivial under the rank 1 condition, as treatment orderings need not be transitive. Hence, although the source of randomness is one-dimensional, the decision problem remains intrinsically multidimensional. We formalize this setting through the following assumption.
Under Assumption (ref), for $t \geq 2$, we write $\nabla_{\theta} g_t = \lambda_t \nabla_{\theta} g_1$, where we set $\lambda_1 = 1$, $\nabla_{\theta} g_1$ is the first row of $\nabla_{\theta} g$, and $\lambda_t \in \mathbbm{R} \setminus \{0\}$ for every $t$ (a nonzero loading is required for the slice reparametrization in Theorem (ref), which divides by $\lambda_t$). Thus, the estimator of welfare contrasts in the limit experiment is one-dimensional in uncertainty since $$rank(Var((\nabla_{\theta} g) \Delta))=rank((\nabla_{\theta} g)\, I_{0}^{-1}\, (\nabla_{\theta} g)^{\top})=rank(((\nabla_{\theta} g_1)^{\top}\, I_{0}^{-1}\, (\nabla_{\theta} g_1))\cdot\boldsymbol{\lambda}\boldsymbol{\lambda}^{\top})=1,$$ where $\boldsymbol{\lambda}=(1, \lambda_2, \dots, \lambda_T)$. However, the ranking problem in the limit experiment is not one-dimensional in choice since $\lambda_t\in \mathbbm{R}$: $(\nabla_{\theta} g_t) \Delta\geq 0$ does not inform one about the sign of $(\nabla_{\theta} g_{t'}) \Delta$ for $t'\neq t$. In the following example, we demonstrate that for the LIM models in Example (ref), Assumption (ref) holds.
We remark on the scope of Assumption (ref). The rank-one restriction in Assumption (ref) characterizes a class of structural models in which welfare contrasts collapse in the limit to a scalar index. The Linear-in-Means specification in Example (ref) satisfies this restriction because the social multiplier $1/(1-\eta_1)$ enters every contrast multiplicatively. The assumption rules out three families of natural extensions: (i) models with heterogeneous treatment effects, in which case the rank rises with the number of heterogeneity dimensions $\dim(\eta_2,\eta_3,\dots)$; (ii) quadratic peer effects of the form $\eta_4\Pi_i^2$, which generate a curvature term that is not collinear with the linear social-multiplier term; and (iii) multi-channel spillovers in which separate spillover mechanisms (e.g., information versus reciprocity) contribute additive but non-proportional contrasts. Our admissibility and optimality results in what follows are therefore confined to ranking problems in which the limiting Gaussian experiment exhibits one-dimensional stochastic variation; extending these results to higher-rank settings is left for future work.
In the next theorem, we show that if Assumption (ref) holds, threshold rules are admissible and minimax optimal in the limiting Gaussian model. To do so, we introduce additional notation. Let $h_0$ be a vector such that $(\nabla_{\theta} g) h_0 = 0$, so that $h_0$ is orthogonal to the space spanned by the row vectors of $\nabla_{\theta} g$. For each $t,$ define a slice using the reparametrization $h_t(b_t,h_0)$ as $S_t(h_0):=\{h_t(b_t,h_0): b_t \in \mathbbm{R}\}$ where $$h_t(b_t,h_0) = h_0 + \frac{b_t}{\lambda_t(\nabla_{\theta} g_1) I_0^{-1} (\nabla_{\theta} g_1)^{\top}} I_0^{-1} (\nabla_{\theta} g_1)^{\top},$$ with $b_t=\nabla_{\theta} g_t h_t(b_t,h_0)$. As a result, any pair of slices $S_t(h_0)$ and $S_{t'}(h_0)$, $t,t'=1,\dots, T$, are equal, i.e., $\{h_1(b,h_0): b \in \mathbbm{R}\}=\{h_1(b_1,h_0): b_1 \in \mathbbm{R}\}=\{h_2(b_2,h_0): b_2 \in \mathbbm{R}\}=\dots=\{h_{T}(b_T,h_0): b_T \in \mathbbm{R}\}$ under Assumption (ref), with $$h_1(b,h_0) = h_0 + \frac{b}{(\nabla_{\theta} g_1) I_0^{-1} (\nabla_{\theta} g_1)^{\top}} I_0^{-1} (\nabla_{\theta} g_1)^{\top}.$$
Condition (ref) requires that a higher loss be assigned to any incorrect choice for each $t$, and $L^m_{t,\infty}(\delta^{t},h)$ for $m\in \{H,R\}$ satisfies this condition.
Theorem (ref)(i) implies that the threshold rule ${\delta}_{c}(\Delta)$ is admissible on the subspace $\{h_1(b,h_0):b \in \mathbbm{R}\}$. Theorem (ref)(ii) asserts that the optimal cutoff points under the minimax criterion can be obtained by solving an easier problem. In particular, the minimax rule under Assumption (ref) in the limit experiment is a cutoff rule where the cutoffs are the solution to $$\inf_{(c_1,\dots,c_T)}\max\{\sup_{b>0} f_+(b, (c_1, \dots, c_T)), \sup_{b<0} f_-(b, (c_1,\dots,c_T))\}$$ where $f_+(b, (c_1, \dots, c_T)):=b(\sum_{t:\lambda_t>0}\sigma_{g_{t}}\lambda_t\Phi(c_t-\lambda_tb)-\sum_{t:\lambda_t<0}\sigma_{g_{t}}\lambda_t\Phi(\lambda_tb-c_t))$ and \\ $f_-(b, (c_1, \dots, c_T)):=b(\sum_{t:\lambda_t<0}\sigma_{g_{t}}\lambda_t\Phi(c_t-\lambda_tb)-\sum_{t:\lambda_t>0}\sigma_{g_{t}}\lambda_t\Phi(\lambda_tb-c_t)).$ Unfortunately, there is no closed-form analytical solution to this problem.
Building on Theorem (ref)(ii), part (iii) states that if we adopt a slightly more conservative objective, namely minimizing the upper bound of the worst-case risk, the optimal cutoff points follow from a simplified optimization such that $c^{*}$ equals a $T$-dimensional vector of zeros.
To obtain the finite-sample rule that converges to the asymptotically optimal rule in Theorem (ref)(iii), we impose the following regularity conditions.
These regularity conditions are similar to those in hirano2009asymptotics. Assumption (ref)(i) ensures the existence of an efficient estimator for $\theta_0$, and (ii) ensures the existence of a consistent estimator for $\sigma_{tt}$ for each $t=1,\dots, T$.
Theorem (ref) implies that under Assumption (ref) and using the regret loss function, the ES ranking rule converges to a rule that minimizes the upper bound of the worst-case risk in the limit. This lends theoretical support in the asymptotics for the first-stage design problem studied in Section (ref), where quasi-optimal designs are obtained by minimizing a feasible upper bound on the worst-case risk of the ES ranking rule.
We have restricted our attention to the class of parametric models $\Theta$ in this section, but the results extend to the class of distributions $\mathcal{P}$ with more involved notation. Instead of repeating the same arguments with more complex notation, we refer to hirano2009asymptotics and van1991asymptotic for the extension. The main difference is that the multivariate Gaussian limit experiment is replaced by an infinite Gaussian sequence.
In this section, we conduct Monte Carlo experiments to examine the theoretical properties established in the previous sections. All simulations use the LIM model of manski1993identification as the data generating process. The outcome for individual $j$ in cluster $i$ is
where $\bar{Y}_i = n_i^{-1}\sum_{j'} Y_{ij'}$ is the cluster mean outcome, $\Pi_i = n_i^{-1}\sum_{j'} Z_{ij'}$ is the realized saturation, $Z_{ij}\in\{0,1\}$ is the treatment indicator, and $\epsilon_{ij}\sim N(0,\sigma_\epsilon^2)$ is independent of $(Z_{ij},\Pi_i)$. The outcome $Y_{ij}$ is endogenous: it depends on $\bar{Y}_i$, which in turn depends on $Y_{ij}$ itself. This simultaneity, governed by the endogenous peer effect $\eta_1$ with $|\eta_1|<1$, creates within-cluster dependence that strengthens as $\eta_1$ approaches unity through the social multiplier $1/(1-\eta_1)$. The remaining parameters $\eta_2$ and $\eta_3$ capture the exogenous peer effect and the direct treatment effect, respectively. Solving (ref) for the equilibrium cluster mean yields the reduced form
where $\bar{\epsilon}_i \sim N(0, \sigma_\epsilon^2/n_i)$. The welfare function is $U(\pi_k, \theta) = (\eta_2+\eta_3)/(1-\eta_1)\cdot \pi_k$, so the composite parameter $\eta_2+\eta_3$ determines the signal strength for ranking.
We consider $K=4$ saturation levels $\Pi = \{0.1, 0.3, 0.5, 0.7\}$, yielding $T = 6$ pairwise comparisons. Clusters and units are assigned to saturation levels and treatments, respectively, under the two-stage complete randomization (Assumption (ref)), and all results are based on $R = 10{,}000$ replications.
We first compare the Monte Carlo (MC) risk of the ES ranking rule against the risk bounds we derived in the previous section. We also illustrate that the manski2004statistical bound, which does not account for within-cluster dependence, may fall below the actual MC risk when interference is sufficiently strong.
We compare the MC risk and the finite-sample upper bounds while changing the number of clusters ($C$), the lower bound of cluster sizes ($\underline{n}$), and the size of the signal strength for ranking ($\eta_2+\eta_3$). We fix the remaining parameters to $\eta_1 =0.5$, $\sigma_\epsilon = 2.0$. We conduct the two-stage complete randomization with balanced allocation, and draw cluster sizes i.i.d.\ uniformly on $\{\underline{n},\dots,\overline{n}\}$ so that the unit-level and cluster-level closed-form bounds of Corollary (ref) differ.
We report both closed-form bounds alongside the MC risk: the unit-level bound (ref), which carries the cluster-size heterogeneity factor $\underline{n}/\overline{n}$, and the cluster-level bound (ref), which does not. By Corollary (ref) the unit-level bound is never tighter than the cluster-level bound, the two coincide as $\underline{n}/\overline{n}\to 1$, and both share the common exponent coefficient $C_kC_{k'}/(C_k+C_{k'})^2$ that equals $1/4$ under the balanced design.
Table (ref) and Figure (ref) report results across three panels. In Panel A, we fix the size support $\underline{n}=10$, $\overline{n}=40$ and $\eta_2+\eta_3=0.05$, and vary $C \in \{12, 24, 48, 96, 192\}$. Both bounds are constant in $C$ under the balanced design---the per-pair exponent coefficient $C_kC_{k'}/(C_k+C_{k'})^2$ equals $1/4$ for every $C$---so they stay near $0.20$ throughout, while the MC risk declines slowly from $0.095$ to $0.082$. At this weak signal, the unit-level and cluster-level bounds are nearly indistinguishable ($0.2000$ versus $0.1998$): the exponent is close to zero, so each bound is dominated by the sum of welfare gaps, and the heterogeneity factor $\underline{n}/\overline{n}$ has almost no effect.
In Panel B, we fix $C=48$, $\eta_2+\eta_3=0.05$, and the upper size bound $\overline{n}=100$, and raise the lower bound $\underline{n}\in\{5,10,20,50,100\}$ toward $\overline{n}$. As $\underline{n}$ increases the size distribution becomes less dispersed, so the heterogeneity factor $\underline{n}/\overline{n}$ rises toward $1$ and the unit-level bound descends monotonically toward the cluster-level bound, reaching it exactly when $\underline{n}=\overline{n}=100$ ($0.2000\to0.1998$, against a flat cluster-level $0.1998$). The cluster-level bound is unaffected by the sizes throughout, and the MC risk falls from $0.089$ to $0.081$ as the clusters grow larger on average.
In Panel C, we fix $\underline{n}=10$, $\overline{n}=40$ and $C=48$, and vary $\eta_2+\eta_3$ over $\{0.1, 0.2, \ldots, 0.9\}$. The MC risk displays a hump-shaped pattern, rising to a peak of $0.314$ at $\eta_2+\eta_3=0.4$ and then declining to $0.197$ at $\eta_2+\eta_3=0.9$. This reflects two opposing forces: as the signal grows, the cost of each misranking (proportional to $\Delta_{kk'}$) increases, but the probability of misranking (which decays exponentially in $\Delta_{kk'}^2$) decreases. For small signals, the linear cost effect dominates; for large signals, the exponential probability decay takes over. Both bounds, by contrast, increase monotonically over this range, and here they separate visibly: the cluster-level bound rises from $0.40$ to $2.73$ while the looser unit-level bound rises from $0.40$ to $3.35$. The gap between them is exactly the $\underline{n}/\overline{n}$ factor in the exponent, which is immaterial at low signal but magnified as the welfare gaps grow. In all cases, the MC risk lies strictly below both bounds.
We next examine the behavior of the manski2004statistical bound under strong interference. We search over configurations with $\eta_1 \in \{0.5, 0.8, 0.9\}$, $\sigma_\epsilon=0.3$, and $n_i \in \{20, 50, 100, 200\}$, with $C=12$ (3 clusters per arm) and $\eta_2+\eta_3=0.05$. As $\eta_1$ increases to $1$, the social multiplier $1/(1-\eta_1)$ amplifies within-cluster dependence. The Manski bound, treating each of the $C_k \cdot n_i$ unit-level observations as independent, shrinks rapidly with $n_i$, while the true risk does not shrink commensurately because the observations within each cluster are highly correlated. Figure (ref) illustrates the results. At $\eta_1 = 0.5$ (left panel), the Manski bound lies safely above the MC risk. At $\eta_1 = 0.8$ (center), the bound is violated for $n_i \geq 100$. At $\eta_1 = 0.9$ (right), the MC risk exceeds the Manski upper bound at every cluster size, with the gap widening as $n_i$ grows. The paper's bound remains valid throughout, demonstrating the necessity of accounting for interference in the upper bounds.
We confirm that balanced cluster allocation minimizes the maximum risk of the ES ranking rule, as predicted by the quasi-optimal design theory in Section (ref). We examine this by comparing the MC risk of the ES ranking rule under three designs: balanced ($\alpha_k = 1/4$ for all $k$), extreme tilt ($.50/.20/.20/.10$), and random Dirichlet$(1,1,1,1)$. The parameters are $K=4$, $C=60$, $n_i=20$, $\eta_1 = 0.3$, $\sigma_\epsilon=0.2$, and $\eta_2+\eta_3$ varies over $\{0.1, 0.2, 0.3, 0.4, 0.5\}$.
Table (ref) and Figure (ref) report the maximum MC risk across the signal grid and the envelope bound for each design. The balanced design achieves the lowest maximum risk ($0.010$), confirming the quasi-optimality result, compared to $0.014$ for the extreme tilt and $0.023$ for random Dirichlet.
We develop a decision-theoretic framework for ranking treatment saturations under clustered interference. We propose an empirical success ranking rule that is simple to implement and does not require knowledge of the within-cluster network. We derive finite-sample upper bounds on the regret of the ES ranking rule and use them to characterize a quasi-optimal first-stage saturation distribution within the two-stage randomization design of baird2018optimal. The bounds depend on the network only through a single combinatorial summary of the dependency structure.
We further study the local asymptotic behavior of the ES ranking rule and show that it belongs to a class of threshold ranking rules that is asymptotically admissible. Under a rank-one structural condition on the limiting Gaussian experiment, this class is also minimax-optimal in the sense of minimizing the worst-case risk bound under additively separable regret loss. Several extensions remain open, including ranking with covariate-adaptive policies and dynamic optimal saturation design with repeated experiments.
\onehalfspacing