Identifying Causal Effects in Experiments with Spillovers and Non-compliance

Identifying Causal Effects in Experiments with Spillovers and Non-compliance

\thispagestyle{empty}

\onehalfspacing

Introduction

Random saturation experiments provide a powerful tool for estimating causal effects in the presence of spillovers---also known as interference---by generating exogenous variation in both individuals' own treatment offers and the fraction of their peers who are offered treatment hudgens2008. These two sources of variation allow researchers to study both direct causal effects---the effect of Alice's treatment on her own outcome---and indirect causal effects---the effect of Bob's treatment on Alice's outcome. A complete understanding of both direct and indirect effects is crucial for program evaluation in settings with spillovers. When considering a national job placement program, for example, policymakers may worry that the indirect effects of the program could completely offset the direct effects: in a slack labor market, job placement could merely change who is employed without affecting the overall employment rate crepon2013.

In this paper we provide methods that use data from a randomized saturation design to identify and estimate direct and indirect causal effects in the presence of spillovers and one-sided non-compliance. In real-world experiments non-compliance is the norm rather than the exception. In their study of the French labor market, crepon2013 found that only 35% of workers offered job placement services took them up. Despite pervasive non-compliance in practice, most of the existing literature on randomized saturation designs either assumes perfect compliance---all subjects adhere to their experimentally-assigned treatment allocation---or identifies only intent-to-treat-effects---the effect of being offered treatment. Intent-to-treat effects are generally insufficient for policy analysis: comparing costs and benefits requires an estimate of the average effect of treatment on those who experience it. For this reason, we go beyond intent-to-treat effects. In particular, we use the randomized saturation design as a source of instrumental variables to estimate treatment-on-the-treated and treatment-on-the-untreated effects when subjects endogenously select into treatment on the basis of their experimental offers.

In a world of homogeneous treatment effects, a simple instrumental variables (IV) regression using individual treatment offers and group saturations as instruments would identify both direct and indirect effects. In most if not all real-world settings, however, treatment effects vary across individuals. In the presence of heterogeneity, this “na\"{i}ve” IV approach will not in general recover interpretable causal effects. To allow for realistic patterns of heterogeneity in a tractable framework, we study a flexible random coefficients model in which causal effects may depend on an individual's treatment take-up as well as that of her peers.\footnote{As we discuss further below, the random coefficients model is not itself restrictive, but the flexibility of the models that one can identify in practice is constrained by the design of the experiment. See Appendix (ref) for a detailed discussion of this point.}

Our approach relies on four key assumptions. First is partial interference: we assume that each subject belongs to a single, known group and that spillovers occur only within groups. This is reasonable in many experimental settings where, for example, groups correspond to villages, and spillovers across them are negligible. Second is anonymous interactions: we assume that individuals' potential outcome functions depend on their peers' treatment take-up only through the average take-up in their group. Under this assumption only the number of treated neighbors matters, not their identities Manski2013. In the absence of detailed network data, the assumption of anonymous interactions is a natural starting point and is likely to be reasonable in settings such as the labor market example described above. Third is one-sided non-compliance: we assume that the only individuals who can take up treatment are those to whom treatment was offered via the experimental design. One-sided non-compliance is relatively common in practice, for example when an “encouragement design” is used to introduce a new program, product or technology that is otherwise unavailable miguel2004, crepon2013.

We refer to our fourth key assumption as individualized offer response, or IOR for short. IOR requires that each subject's treatment take-up decision is invariant to the realized treatment offers made to her peers. While IOR is a strong assumption, it is a priori reasonable in many contexts, for example in online settings where other subjects' treatment offers are unobserved by others bond2012,anderson2014,eckles2016 confidential yi2015, or observed with a delay. IOR limits but does not rule out strategic behavior. For example, it holds when agents act strategically on their own beliefs about others' actions provided that they are unaware of their peers' offers when making their own take-up decisions. (bhattacharya2021demand call this an “incomplete information equilibrium.”) Most importantly, IOR has testable implications and we find no evidence against it in our empirical example.\footnote{See (ref) for details.}

When combined with one-sided non-compliance, IOR allows us to divide the population into never-takers and compliers, two of the traditional LATE strata.\footnote{One-sided non-compliance rules out always-takers and defiers.} Under the randomized saturation design and a standard exclusion restriction, we show how to construct valid and relevant instruments that identify the average causal effects of interest. The key to our approach is a result showing that conditioning on group size $n$ and the share of compliers $\bar{c}$ in a group breaks any dependence between peers' average take-up and an individual's random coefficients.\footnote{Our identification approach relates to a large literature on random coefficients models, e.g.\ wooldridge2004, masten2016, and graham2022semiparametrically, and to the literature that uses control functions to identify structural effects altonji2005cross,imbens2009identification.} Under the randomized saturation design, the share of Alice's neighbors who are offered treatment is exogenous. Under IOR, their average take-up depends only on how many of them are compliers and whether they are offered treatment. Thus, conditional on $n$ and $\bar{c}$, any residual variation in the take-up of Alice's neighbors comes solely from the experimental design. Although group size is observed, the share of compliers in a given group is not. In a large group, however, the rate of take-up among those offered treatment, call it $\widehat{c}$, closely approximates $\bar{c}$. Using this insight, we provide feasible estimators of direct and indirect causal effects that are consistent and asymptotically normal in the limit as group size grows at an appropriate rate relative to the number of groups. After constructing the appropriate instruments, our estimators can be implemented as simple IV regressions without the need for non-parametric estimation. In a series of simulations we demonstrate that our estimator works well at reasonable sample sizes.

We apply our methods to experimental data from crepon2013, a large-scale randomized saturation experiment carried out across French labor markets that offered job-placement services to young adults. In particular, we estimate direct and indirect treatment effects of program take-up for compliers (the treated) and spillovers for never-takers (the untreated). We find large negative indirect effects for compliers, a more vulnerable sub-population than never-takers based on their observed characteristics at baseline. Take-up of the program by these individuals, however, shields them from the negative spillovers induced by the increased take-up of job-placement services by others in their city. The never-taker sub-population, in contrast, is unaffected by such negative spillover effects. Our results go beyond the intent-to-treat effects estimated by crepon2013. Whereas they estimate the spillovers from offering job placement services, we estimate the labor market displacement effects of providing them. While we do not consider additional applications here, we believe that the methods developed in this paper can be usefully applied in a variety of other settings.\footnote{In (ref) we discuss a number of recent studies with non-compliance that appear to satisfy the assumptions and data requirements of our estimators.}

This paper relates most closely to recent work by Kang2016 and Imai2018, who also study randomized saturation experiments with social interactions under non-compliance. Imai2018 identify a “complier average direct effect” (CADE), in essence a Wald estimand calculated for all groups with the same share of offers (saturation). While it is identified under a weaker condition than IOR, the CADE is a hybrid of direct and indirect effects unless one is willing to impose IOR. Under IOR, the CADE quantifies the effect of an individual's own treatment take-up, given that her group has been assigned a particular saturation. In contrast, the direct effects that we recover below quantify the effect of an individual's own treatment take-up given that a certain share of her neighbors have taken up treatment. Kang2016 identify effects similar to those of Imai2018 using an assumption they call “personalized encouragement,” the equivalent of our IOR assumption. Both Kang2016 and Imai2018 identify well-defined effects while placing limited structure on the potential outcome functions. The cost of this generality is that the effects they recover have a “reduced form” flavor, and are only defined relative to the specific saturations used in the experiment. While our assumption of anonymous interactions places more restrictions on the potential outcome functions, we recover “fully structural” causal effects that are not specific to the design of the experiment.

In a recently and closely related paper, VazquezBare uses instrumental variables to identify spillovers without relying on a particular experimental design. VazquezBare focuses on settings with pairs of people, for example roommates or couples, and considers spillovers both in outcomes and take-up. Under one-sided non-compliance and a novel monotonicity restriction, he identifies two causal effects without invoking the IOR assumption: a direct effect for compliers whose partner is untreated, and an indirect effect for untreated individuals whose partner is a complier. This identification result does not extend to groups of more than two people. In larger groups, VazquezBare identifies average potential outcomes under anonymous interactions without IOR, instead assuming that individuals' potential outcomes are independent of their peers' compliance types. While our results rely on IOR, we do not invoke his latter assumption because in many applied settings a person's potential outcomes may be related to the characteristics of her peers.

Our paper also relates to the applied literature that estimates spillover effects, including “partial population” studies in which a subset of subjects in the treatment group are left untreated and their outcomes are compared to those of subjects in a control group duflo2003,bobonis2009,angelucci2009,barrera2011,haushofer2016. It also includes cluster-randomized trials where groups are defined by a spatial radius within which spillovers may arise miguel2004,bobba2014. In general, the applied literature focused on spillovers estimates intent-to-treat (ITT) effects. Two notable exceptions are crepon2013 and Akram2018 who estimate effects that are similar in spirit to the CADE of Imai2018.

The remainder of the paper is organized as follows. Section (ref) details our notation and assumptions, (ref) presents our identification results, and (ref) provides consistent and asymptotically normal estimators of the effects identified in (ref). In (ref) we implement our estimator on data from a well-known labor market experiment, and discuss our findings. In (ref) we present a brief simulation study illustrating the behavior of our estimator. Section (ref) concludes. Proofs and additional results appear in the appendix.

Notation and Assumptions

We observe $N$ individuals divided between $G$ groups. We assume throughout the paper that each group has at least two members so there is scope for spillovers. Let $g =1, \dots, G$ index groups and $i = 1, \dots, N_g$ index individuals within a given group $g$. Using this notation, $N = \sum_g N_g$. For each individual $(i,g)$ we observe a binary treatment offer $Z_{ig}$, an indicator of treatment take-up $D_{ig}$, and an outcome $Y_{ig}$. For each group $g$ we observe a saturation $S_g \in [0,1]$ that determines the fraction of individuals offered treatment in that group. A bold letter indicates a vector and a $g$-subscript shows that this vector is restricted to members of a particular group. For example $\boldsymbol{Z}$ is the $N$-vector of all treatment offers $Z_{ig}$ while $\boldsymbol{Z}_g$ is the $N_g$-vector obtained by restricting $\boldsymbol{Z}$ to group $g$. Define $\boldsymbol{D}$ and $\boldsymbol{D}_g$ analogously and let $\boldsymbol{S}$ denote the $G$-vector of all $S_g$. At various points in our discussion we will need to refer to the average value of a variable for everyone in a group besides person $(i,g)$. As shorthand, we refer to these other individuals as person $(i,g)$'s neighbors. To indicate such an average, we use a bar along with an $(i,g)$ subscript. For instance, $\bar{D}_{ig}$ denotes the treatment take-up rate in group $g$ excluding $(i,g)$, while $\bar{Z}_{ig}$ is the analogous treatment offer rate:

Under this definition, $\bar{D}_{ig}$ and $\bar{Z}_{ig}$ vary across individuals in the same group depending on their values of $D_{ig}$ or $Z_{ig}$. For example in a group of eleven people, of whom five take up treatment, $\bar{D}_{ig}=0.5$ if $D_{ig} = 0$ and $0.4$ if $D_{ig} = 1$. We now introduce our basic assumptions, beginning with the experimental design.

(ref) details the first stage of the randomized saturation design. In this stage, each group $g$ is assigned a saturation $S_g$ drawn completely at random from a set $\mathcal{S}$. In the example from (ref), fifty groups (balls) are divided equally between five saturations (urns), namely $\mathcal{S} = \{0, 0.25, 0.5, 0.75, 1\}$. The saturation drawn in this first stage determines the fraction of individuals in the group that will be offered treatment in the second stage. (ref), for example, depicts a group of eight individuals that has been assigned to the 25% saturation: two are offered treatment and six are not. For simplicity we assume that treatment offers in the second stage follow a Bernoulli design, in which $S_g$ determines the probability of treatment rather than the number of treatment offers. With minor modifications, our results can be extended to a completely randomized design, in which the number of treatment offers made to a given group is fixed conditional on the saturation.\footnote{For details see (ref).}

The randomized saturation design creates exogenous variation at the individual and group levels. Within a group some individuals are offered while others are not. Between groups, some have a large number of individuals offered treatment---a high saturation---while others do not. Many randomized saturation experiments, like the illustration in (ref), feature a 0% saturation or even a 100% saturation. We refer to 0% and 100% saturations as corner saturations to distinguish them from all other saturations, which we call interior. There is no variation in treatment offers between individuals in a group assigned a corner saturation. For this reason, as we discuss in (ref) below, the number of interior saturations in the design will determine the flexibility with which we can model potential outcome functions.

Assumptions (ref)--(ref) concern the design of the experiment. Our remaining assumptions, in contrast, concern the potential outcome and treatment functions. Without imposing any restrictions, an individual's potential outcome function $Y_{ig}(\cdot)$ could in principle depend on the treatment take-up of all individuals in the sample. We denote this unrestricted potential outcome function by $Y_{ig}(\boldsymbol{D})$. (ref) restricts $Y_{ig}(\cdot)$ to depend only on $D_{ig}$ and $\bar{D}_{ig}$ via a random coefficients model.

The first equality in (ref) is the so-called partial interference assumption, used widely in the literature on spillover effects. This assumption states that there are no spillovers between people in different groups: only the treatment take-up of individuals in group $g$ affects the potential outcome of person $(i,g)$. The second equality in (ref) states that person $(i,g)$'s potential outcome is only affected by the treatment take-up the others in her group through the aggregate $\bar{D}_{ig}$.\footnote{Recall that $\bar{D}_{ig}$ is defined to exclude person $(i,g)$.} This is effectively identical to the anonymous interactions assumption from the network literature Manski2013.\footnote{In particular, because our treatment is binary, assuming that $\boldsymbol{D}_g$ only affects $Y_{ig}$ through $\bar{D}_{ig}$ is equivalent to assuming exchangeability: only the number of $(i,g)$'s neighbors who take up treatment matters for her outcome; their identities are irrelevant. When researchers do not observe the social network within groups, as in crepon2013, exchangeability is a natural assumption.} While we maintain this assumption throughout, (ref) discusses some potential ways of relaxing it.

The third equality in (ref) posits a finite basis function expansion for the potential outcome functions $Y_{ig}(0,\bar{D}_{ig})$ and $Y_{ig}(1,\bar{D}_{ig})$, namely \[ Y_{ig}(0, \bar{D}_{ig}) = \sum_{k=1}^K \theta_{ig}^{(k)} f_k(\bar{D}_{ig}), \quad Y_{ig}(1, \bar{D}_{ig}) = \sum_{k=1}^K \psi_{ig}^{(k)} f_k(\bar{D}_{ig}) \quad \] or, written more compactly in matrix form,

where the coefficient vectors $\boldsymbol{\theta}_{ig}$ and $\boldsymbol{\psi}_{ig}$, and hence $\mathbf{B}_{ig}$, are allowed to vary arbitrarily across groups and individuals. If, for example, person $(i,g)$ has some prior knowledge of her potential outcome function $Y_{ig}(\cdot,\cdot)$, her take-up decision may depend on $\boldsymbol{\theta}_{ig}$ and $\boldsymbol{\psi}_{ig}$. More generally, the same unobserved characteristics that determine a person's decision to take up treatment could affect her potential outcomes. To account for these possibilities, we allow arbitrary statistical dependence between $(D_{ig},\bar{D}_{ig})$ and $\mathbf{B}_{ig}$. Our assumption of a random coefficients model is not in itself restrictive. In principle one could even consider adapting the choice of $K$ to the data at hand using non-parametric series methods. In practice, however, the design of the randomized saturation experiment limits the number of basis functions that can be used in practice. To satisfy the rank condition introduced below, $K$ should not exceed the number of saturations.\footnote{See Appendix (ref) for details.} For this reason, we treat $K$ as fixed throughout.

Ideally, our goal would be to identify the average direct and indirect causal effects of the binary treatment $D_{ig}$. Under (ref), we define these as follows, building on the definitions of hudgens2008. The direct treatment effect, $\text{DE}$, gives the average effect of exogenously changing an individual's own treatment $D_{ig}$ from 0 to 1 while holding the share of her treated neighbors $\bar{D}_{ig}$ fixed at $\bar{d}$, namely

where the expectations are taken over all individuals in the population from which our experimental subjects were drawn. Recall that $\bar{D}_{ig}$ excludes person $(i,g)$, ensuring that $\text{DE}(\bar{d})$ is well-defined. An indirect treatment effect, in contrast, gives the average effect of exogenously increasing a person's share of treated neighbors $\bar{D}_{ig}$ from $\bar{d}$ to $\bar{d} + \Delta$ while holding her own treatment $D_{ig}$ fixed at $d$, in other words,

where $\Delta$ is a positive increment. There are two indirect treatment effect functions, $\text{IE}_0$ and $\text{IE}_1$, corresponding to the two possible values at which we could hold $D_{ig}$ fixed: a spillover on the untreated, and a spillover on the treated. Because the direct and indirect causal effects are fully determined by $\mathbbm{E}[\mathbf{B}_{ig}]$ under (ref), this is our object of interest below. For example, if $\mathbf{f}(x)' = \left[1 \quad x\right]$ we obtain a linear model of the form

In this case the direct effect is $\text{DE}(\bar{d}) = \mathbbm{E}[\beta_{ig}] + \mathbbm{E}[\delta_{ig}] \bar{d}$ while the indirect effects are \[ \text{IE}_0(\bar{d},\Delta) = \Delta \times \mathbbm{E}[\gamma_{ig}], \quad \text{IE}_1(\bar{d},\Delta) = \Delta \times \mathbbm{E}[\gamma_{ig} + \delta_{ig}]. \] Notice that in the linear model, $\text{IE}_0$ and $\text{IE}_1$ do not depend on $\bar{d}$. While all of our theoretical results apply to arbitrary random coefficients models, we focus on the linear model from (ref) in our empirical example and simulation study below.

(ref) presents a hypothetical example of (ref) in a setting with employment displacement effects. Suppose that $Y_{ig}$ is Alice's probability of long-term employment. Both $Y_{ig}(1, \bar{d})$ and $Y_{ig}(0, \bar{d})$ have a negative slope. This means that Alice's probability of long-term employment decreases if more of her neighbors obtain job placement services. But since $\delta_{ig}$ is positive, the spillover is more harmful if Alice is untreated. Alice's direct effect of treatment $Y_{ig}(1, \bar{d}) - Y_{ig}(0, \bar{d})$ is positive for all $\bar{d}$ in this example and increases as $\bar{d}$ does: job placement services are more valuable to Alice when more of her neighbors obtain them. By averaging these effects for everyone in the population, we obtain $\text{IE}_0, \text{IE}_1$, and $\text{DE}$.

Under perfect compliance $D_{ig}$ would simply equal $Z_{ig}$, making both $D_{ig}$ and $\bar{D}_{ig}$ exogenous. In this case a sample analogue of $\mathbbm{E}[Y_{ig}(d,\bar{d})]$ could be used to recover all of the treatment effects discussed above, at least at values of $\bar{d}$ that arise in the experimental design. Unfortunately non-compliance is pervasive in real-world experiments, greatly complicating the identification of causal effects. In a large-scale experiment carried out in France, for example, only 35% of unemployed workers offered job placement services took them up crepon2013. Those who did take up treatment likely differ in myriad ways from those who did not: they may, for example, be more conscientious. One way to to avoid this problem of self-selection is to carry out an intent-to-treat (ITT) analysis, conditioning on $Z_{ig}$ and $S_g$ rather than $D_{ig}$ and $\bar{D}_{ig}$. But with take-up rates as low as 35%, ITT estimates could be very far from the causal effects of interest. In this paper we adopt a different approach. Following the tradition in the local average treatment effect (LATE) literature, we provide conditions under which direct and indirect causal effects--rather than ITT effects--can be identified for well-defined sub-populations of individuals. We focus on the case of one-sided noncompliance, in which only those offered treatment can take it up. One-sided non-compliance is common in practice and simplifies the analysis.\footnote{We suggest an avenue for extending our results to two-sided non-compliance in (ref).}

To account for endogenous treatment take-up, we define potential treatment functions $D_{ig}(\cdot)$. In principle these could depend on the treatment offers of every individual, $\boldsymbol{Z}$ in the experiment. The following assumption restricts $D_{ig}(\cdot)$ to permit identification of the direct and indirect causal effects described above.

The first equality of (ref) is a partial interference assumption: it requires that person $(i,g)$'s take-up decision is invariant to the realized treatment offers made to people in different groups. The second equality of (ref) states that person $(i,g)$'s take-up decision depends on the realized treatment offers of others in her group only through the fraction $\bar{Z}_{ig}$ of treatment offers made to the others in her group. These first two equalities are not in general sufficient to point identify direct and indirect causal effects. The third equality, which we call individualistic offer response or IOR for short, imposes the further restriction that each person's take-up decision is invariant to the realized offers made to her peers. Assumptions analogous or equivalent to IOR have appeared in the existing literature. Kang2016, for example, employ an assumption equivalent to IOR, which they call “personalized encouragement.” While Imai2018 derive their “complier average direct effect (CADE)” under a weaker condition, this effect is a hybrid of direct and indirect effects unless one is willing to impose IOR.

IOR is a reasonable assumption in some but not all applications. In settings where participants observe neither the saturation assigned to their group nor the treatment offers made to their neighbors, for example, IOR clearly holds. IOR restricts but does not rule out strategic take-up. For example, it also holds when agents act strategically on their beliefs about others' actions, provided that they are unaware of their peers' offers when making their own take-up decisions. In the introduction we list a number of recent randomized saturation experiments in which we consider IOR to be a reasonable assumption. Moreover, as we discuss further in (ref), IOR has testable implications in a randomized saturation experiment. If the take-up rate among individuals who are offered treatment varies with saturation, this indicates a violation of IOR.

Under IOR and one-sided non-compliance (Assumptions (ref) and (ref)), we can divide individuals into never-takers and compliers, two of the principal strata from the LATE literature. Never-takers are defined as those for whom $D_{ig}(0)=D_{ig}(1) = 0$, while compliers are those for whom $D_{ig}(z) = z$ for all $z$.\footnote{Under one-sided non-compliance, (ref), there are no always-takers.} Defining $C_{ig}$ to be the indicator that person $(i,g)$ is a complier, Assumptions (ref)--(ref) imply that

By analogy to $\bar{Z}_{ig}$ and $\bar{D}_{ig}$, we define $\bar{C}_{ig}$ to be the share of compliers among person $(i,g)$'s neighbors in group $g$, namely

Note that $\bar{C}_{ig}$ varies across individuals in the same group, depending on their values of $C_{ig}$. Finally, let $\boldsymbol{C}_g$ denote the vector of $C_{ig}$ for all individuals in group $g$.

Our final assumption is an exclusion restriction for the treatment offers $\boldsymbol{Z}_g$ and saturation $S_g$. To state it we require two additional pieces of notation. First, let $\mathbf{B}_g$ denote the vector that stacks $\mathbf{B}_{ig}$ for all individuals in group $g$. Second, following Dawid1979, let “$\indep$” denote (conditional) independence so that $X \indep Y$ indicates that $X$ is statistically independent of $Y$ while $X \indep Y|Z$ indicates that $X$ is conditionally independent of $Y$ given $Z$. Using this notation, the exclusion restriction is as follows.

Intuitively, (ref) states that $(\boldsymbol{C}_g, \mathbf{B}_g, N_g)$ are “predetermined” with respect to the treatment offers and saturations. In a traditional LATE setting, its counterparts are the “unconfounded type” assumption and the independence of potential outcomes and treatment offers. (ref) could be violated in a number of ways. If, for example, individuals chose their group membership based on knowledge of their group's saturation, $N_g$ would not be independent of $S_g$. Similarly, if some individuals decided to comply with their treatment offers only because their group was assigned a high saturation, $\boldsymbol{C}_g$ would not be independent of $S_g$. This latter possibility illustrates that (ref) partially embeds IOR by ruling out “selection into compliance.” As discussed in (ref), it also yields testable implications of the IOR assumption. More prosaically, (ref) would be violated if either $S_g$ or $Z_{ig}$ had a direct effect on the random coefficients $\mathbf{B}_g$. Notice that part (ii) of (ref) conditions on $(S_g, N_g)$. This is because the second stage of the randomized saturation experiment assigns $\boldsymbol{Z}_g$ conditional on this information: see (ref).

Identification

Conditioning on the Share of Compliers

Under (ref), the functional form of the random coefficients model is known. So why not simply use $(Z_{ig},S_g)$ as instrumental variables for $D_{ig}$ and $\mathbf{f}(\bar{D}_{ig})$? As shown in a number of papers from the literature on random coefficients models Wooldridge1997,heckman1998,Wooldridge2003,Wooldridge2016, two-stage least squares identifies average effects when the causal effect of the instruments on the endogenous regressors is homogeneous. In our setting, however, this result does not apply because the conditional distribution of $\bar{D}_{ig}$ given $S_g$ varies with $(\bar{C}_{ig},N_g)$, as the following lemma shows.

Intuitively, the problem presented by (ref) is as follows. Although $S_g$ is randomly assigned, the variation that it induces in $\bar{D}_{ig}$ is mediated by the share of compliers $\bar{C}_{ig}$. Accordingly if $\bar{C}_{ig}$---a source of first-stage heterogeneity---is correlated with the random coefficients in the second stage, the IV estimator will not identify the effects of interest. To make this problem more concrete, consider the linear potential outcomes model from (ref) and let $\boldsymbol{\vartheta}_{\text{IV}}$ be the IV estimand using instruments $(1, Z_{ig}, S_g, Z_{ig} S_g)$. Throughout, we will refer to it as the “na\"ive IV". In this example $\boldsymbol{\vartheta}_{\text{IV}}$ takes a particularly simple form, as shown in the following lemma.

As we see from (ref), IV identifies the population average of $\alpha_{ig}$, along with the population average of $\beta_{ig}$ for the subset of individuals who select into treatment. Neither of these, however, is itself a causal effect. In general, IV recovers neither direct nor indirect causal effects for any well-defined group of individuals. Specializing (ref) to the linear model from (ref) gives $\text{IE}_0(\bar{d}, \Delta) = \mathbbm{E}[\gamma_{ig}] \Delta$. In other words, $\mathbbm{E}[\gamma_{ig}]$ is an average spillover. (ref) shows that IV fails to identify this quantity unless the individual-specific spillovers $\gamma_{ig}$ are uncorrelated with the share of compliers $\bar{C}_{ig}$. This condition could easily fail in practice. In the labor market example from the introduction, cities with a particularly depressed labor market might be expected to contain a large share of compliers. If negative spillovers are more intense in such cities, IV will not recover the average indirect effect. A similar problem hampers the interpretation of $\delta_{\text{IV}}$. Under (ref) the average direct effect for compliers, as a function of $\bar{d}$, is given by $\mathbbm{E}[\beta_{ig}|C_{ig}=1] + \mathbbm{E}[\delta_{ig}|C_{ig}=1] \bar{d}$. While IV identifies the intercept of this function, it only identifies the slope if $\delta_{ig}$ is uncorrelated with $\bar{C}_{ig}$ for compliers.

As this example illustrates, identifying direct and indirect causal effects requires us to correct for possible dependence between individual-specific coefficients and group-level take-up that arises from the first-stage relationship in (ref). The key to our approach, as shown in the following theorem, is to condition on $\bar{C}_{ig}$ and $N_g$.

(ref) implies that conditioning on $(\bar{C}_{ig}, N_g)$ is sufficient to break any dependence between $\mathbf{f}(\bar{D}_{ig})$ and $(\mathbf{B}_{ig},C_{ig})$ that may be present. The intuition for this result is as follows. Conditional on $\bar{C}_{ig}$ and $N_g$, we know precisely how many of $(i,g)$'s neighbors are compliers. Given this information, IOR implies that all remaining variation in $\bar{D}_{ig}$ is arises solely from experimental variation in the saturation $S_g$ assigned to different groups, and the share of compliers offered treatment across groups assigned the same saturation. So long as $Z_{ig}$ and $S_g$ do not affect $(\mathbf{B}_{ig},C_g)$, (ref), it follows that $(Z_{ig}, \bar{D}_{ig},S_g)$ are exogenous given $(\bar{C}_{ig}, N_g)$, even when individuals decide whether or not to take up treatment based on knowledge of their potential outcome functions. In effect, our identification approach is a combination of instrumental variables and control function methods. First $(\bar{C}_{ig}, N_{g})$ serves as a control function for the endogenous regressor $\bar{D}_{ig}$, similar to masten2016. Second, $Z_{ig}$ serves as an instrument for $D_{ig}$, because this regressor remains endogenous even conditional on $(\bar{C}_{ig}, N_g)$.

An Inverse-Weighting Instrumental Variables Approach

Before stating our identification results, we require some additional notation and one further assumption. Define the vector $\mathbf{W}_{ig}$ and matrix-valued functions $\mathbf{Q}, \mathbf{Q}_0, \mathbf{Q}_1$ as follows:

These functions depend only on the distribution of $\bar{D}_{ig}|(Z_{ig}, \bar{C}_{ig}, N_g)$, which can be calculated from (ref), and the distribution of $Z_{ig}|(\bar{C}_{ig},N_g)$, which coincides with its unconditional distribution by (ref). As such, under our assumptions $\mathbf{Q},\mathbf{Q}_0, \mathbf{Q}_1$ are completely determined by the design of the randomized saturation experiment. We can always calculate them by simulating the experimental design. Depending on the choice of $\mathbf{f}$, analytical expressions may also be available, as shown in (ref) for the linear potential outcomes model from (ref).

We use $\mathbf{Q}, \mathbf{Q}_0, \mathbf{Q}_1$ to construct valid instrumental variables by inverse-weighting. Rather than using the randomly assigned saturation $S_g$ as a source of instruments for $\mathbf{f}(\bar{D}_{ig})$ we transform the endogenous regressors $\mathbf{W}_{ig}$ into a set of exogenous instruments using $\mathbf{Q}_0(\bar{C}_{ig}, N_g)^{-1}$ and $\mathbf{Q}_{1}(\bar{C}_{ig}, N_g)^{-1}$, in particular

Constructing these instruments requires us to evaluate $\mathbf{Q}_0$ and $\mathbf{Q}_1$ at $(\bar{C}_{ig}, N_g)$.\footnote{The function $\mathbf{Q}$ can be constructed from $\mathbf{Q}_0$ and $\mathbf{Q}_1$, as shown in (ref).} The group size $N_g$ is observed, while the share of compliers $\bar{C}_{ig}$ is not. In large groups, $\bar{C}_{ig}$ can be precisely estimated by calculating the rate of treatment take-up among the neighbors of $(i,g)$ who are offered treatment. We formally establish the rates of convergence of IV estimators that plug-in a proxy for $\bar{C}_{ig}$ in Section (ref). For the remainder of this section, however, we consider identification conditional on knowledge of $\bar{C}_{ig}$.

To understand the intuition behind $\boldsymbol{\mathcal{Z}}_{ig}^W$, $\boldsymbol{\mathcal{Z}}_{ig}^0$, and $\boldsymbol{\mathcal{Z}}_{ig}^1$, consider the linear potential outcomes example from (ref) above. Here we have $\mathbf{f}(x) = (1, x)'$ and thus \[ \mathbf{Q}_z(\bar{C}_{ig}, N_g) = \mathbbm{P}(Z_{ig} = z)\mathbbm{E}\left[ \left.

\right|\bar{C}_{ig}, N_g, Z_{ig} = z\right], \quad z \in \left\{ 0, 1 \right\} \] using the fact that $Z_{ig} \indep (\bar{C}_{ig}, N_g)$ by (ref). It follows after a few steps of algebra that \[ \mathbf{Q}_z(\bar{C}_{ig}, N_g)^{-1} \mathbf{f}(\bar{D}_{ig}) = \frac{1}{\mathbbm{P}(Z_{ig} = z)}

. \] While $\bar{D}_{ig}$ is endogenous, the scaled difference between $\bar{D}_{ig}$ and its conditional expectation is a valid instrument under the linear potential outcomes model. Intuitively, this transformation adjusts for the first-stage heterogeneity discussed in (ref): after controlling for differences in $(\bar{C}_{ig}, N_g)$, the remaining variation in $\bar{D}_{ig}$ arises only from the experimentally--assigned saturations. Thus, rather than using $S_g$ as an instrument directly, we use it indirectly to generate variation in $\bar{D}_{ig}$ given $(\bar{C}_{ig}, N_g)$. The final ingredient that we require is a rank condition.

Part (i) of (ref) asserts that the population contains at least some never-takers, $\mathbbm{E}(C_{ig}) < 1$, and at least some compliers, $\mathbbm{E}(C_{ig}) > 0$.\footnote{Note that this condition applies to the entire experiment rather than any particular group. If $\mathbbm{E}(C_{ig})=1$, then there is perfect compliance in every group and no need for our methods. If $\mathbbm{E}(C_{ig}) = 0$, then no one in any group takes the treatment so it is impossible to identify treatment effects.} Part (ii) requires that the matrix-valued function $\mathbf{Q}$ defined in (ref) is full rank when evaluated at any share of compliers $\bar{c}$ and group size $n$ that occur in the population. (ref) does not explicitly restrict $\mathbf{Q}_0$ or $\mathbf{Q}_1$. By the linearity of conditional expectation, however,

so (ref)(ii) could equivalently be stated in terms of $\mathbf{Q}_0$ and $\mathbf{Q}_1$.

We discuss low-level conditions for the invertibility of $(\mathbf{Q}_0, \mathbf{Q}_1)$, and hence $\mathbf{Q}$, below in (ref) and Appendix (ref). As a preview: researchers should not include more basis functions $K$ than there are saturations in the experimental design. Having assumed the necessary rank condition, we can now state our main identification results. The following theorem shows how $\mathbf{Q}_0(\bar{C}_{ig}, N_g)$ and $\mathbf{Q}_1(\bar{C}_{ig}, N_g)$ can be used to construct instrumental variables that identify average values of the random coefficients for well-defined groups of individuals.

The first part of (ref) identifies the average effects that the na\"{i}ve IV approach from (ref) in general fails to. Parts (ii) and (iii) use a similar approach to obtain moment equations for the average value of $\boldsymbol{\psi}_{ig}$ for compliers and $\boldsymbol{\theta}_{ig}$ for never-takers. Given part (i), part (iv) is technically redundant, but it is convenient to have an expression for $\mathbbm{E}(\boldsymbol{\theta}_{ig})$ in isolation. As discussed below in Section (ref), having sufficient variation in the saturations is crucial for part (ii) of (ref).

Notice that (ref) does not explicitly invoke the randomized saturation design, Assumptions (ref)--(ref), or the exclusion restriction, (ref). Using this result for identification, however, requires us to satisfy $(Z_{ig}, \bar{D}_{ig})\indep (\mathbf{B}_{ig}, C_{ig})|(\bar{C}_{ig}, N_g)$. As shown in (ref) above, the randomized saturation design and exclusion restriction are sufficient for this condition to hold under one-sided non-compliance and IOR, Assumptions (ref) and (ref). The following result catalogues the full set of causal effects that are identified under our assumptions.

Part (i) of (ref) is a population average indirect treatment effect, as defined in (ref) above. It measures the causal impact of increasing the treatment take-up rate among Alice's neighbors from $\bar{d}$ to $(\bar{d} + \Delta)$ when Alice's own treatment is held fixed at zero. In the crepon2013 experiment discussed in our empirical example below, this corresponds to the average labor market displacement effect. Whereas part (i) is an average treatment effect, parts (ii)--(iv) are the effects of treatment-on-the-treated.\footnote{Because this is a setting with one-sided non-compliance, any participant with $D_{ig} = 1$ must be a complier.} Part (ii) gives the direct effect of treating Alice while holding the treatment take-up rate of her neighbors fixed at $\bar{d}$, while (iii) and (iv) give the indirect effect of increasing her neighbors' treatment take-up from $\bar{d}$ to $\bar{d} + \Delta$ while holding Alice's treatment fixed at either zero, part (iii), or one, part (iv). Part (v) is a treatment-on-the-untreated version of (ref): it gives the indirect effect for never-takers, holding their treatment fixed at zero. While we identify the full set of direct and indirect effects for the treated sub-population, we only identify a subset of these effects for other groups. By definition, never-takers cannot be observed with $D_{ig} = 1$. As such, we cannot identify direct treatment effects for this group or indirect treatment effects when $D_{ig}$ is held fixed at one. This in turn implies that we cannot identify the average direct effect for the population as a whole, $\text{DE}(\bar{d})$, or the average indirect effect when $D_{ig}$ is held fixed at one, $\text{IE}_1(\bar{d}, \Delta)$.

The treatment effects identified in (ref) provide information that should be of interest to policymakers who are concerned about the distributional consequences of policies that may generate spillovers. For example, comparing $\text{IE}_0$ to $\text{IE}_1$ allows policymakers to determine whether the treated mainly create spillovers on the untreated, or whether they create spillovers on each other. Similarly, $\text{DE}_1$ gives the average effect of program participation for those who are willing to participate, allowing policymakers to determine whether the treatment is on net beneficial to those who receive it. Intent-to-treat effects alone do not provide this information.

Identification in Practice

Given that $\mathbf{Q}_0$ and $\mathbf{Q}_1$ are completely determined by the experimental design, we can directly check part (ii) of (ref) for any choice of basis functions $\mathbf{f}$ and probability distribution over saturations. Consider again the linear potential outcomes model from (ref). In this example $\mathbf{f}\left(x \right) = (1, x)'$ and thus,

by Bayes' Theorem, the Law of Total Probability, and Lemmas (ref) and (ref). Suppose first that there is a single saturation $s$. Then (ref) and (ref) simplify to yield \[ \left|\mathbf{Q}_0(\bar{c},n)\right| = \frac{\bar{c}s(1-s)^3}{n-1}, \quad \left|\mathbf{Q}_1(\bar{c},n)\right| = \frac{\bar{c}s^3(1-s)}{n-1}. \] so that $\mathbf{Q}_0(\bar{c}, n)$ and $\mathbf{Q}_1(\bar{c}, n)$ are both invertible for any $n$ and all $\bar{c}$ greater than zero provided that $0 < s < 1$. The identifying power of this “degenerate” randomized saturation design, however, is weak: $\mathbf{Q}_0, \mathbf{Q}_1$ are arbitrarily close to being singular for any $\bar{c}$ if $n$ is sufficiently large. Consider next a so-called “cluster randomized” experiment in which there are two saturations, 0 and 1, and $\mathbbm{P}(S_g = 1) = p$. Calculating the expectations in (ref) and (ref), \[ \mathbf{Q}_0(\bar{c},n) =

, \quad \mathbf{Q}_1(\bar{c},n) =

. \] In this case neither $\mathbf{Q}_0$ nor $\mathbf{Q}_1$ is invertible for any values of $n$ and $\bar{c}$. Finally, consider a design with two distinct, equally likely saturations $s_L < s_H$. For this design, straightforward but tedious algebra gives

So long as neither $s_L$ nor $s_H$ equals zero or one, both terms in each expression are strictly positive for any $\bar{c} > 0$, so that $\mathbf{Q}_0$ and $\mathbf{Q}_1$ are invertible. Moreover, in contrast to the single saturation design discussed above, this design does not suffer from a weak identification problem. While the second term in each of the preceding equalities vanishes for large $n$, the first term does not. Thus, two interior saturations are sufficient to strongly identify the linear potential outcomes model that we use in our empirical example and simulation study below.

As the preceding examples show, two distinct sources of experimental variation determine the rank of $\mathbf{Q}_0(\bar{c},n)$ and $\mathbf{Q}_1(\bar{c},n)$: “between” saturation variation, and “within” saturation variation. Our first example lacks “between” variation because each group is assigned the same saturation, $S_g = s$. Yet even with a single saturation, there is still “within” variation under (ref), because the number of offers made to a given group is random. This “within” variation, however, is negligible when $n$ is large. In our second example, the cluster randomized experiment, the situation is reversed. Because everyone in a given group is either offered ($S_g = 0$) or unoffered ($S_g = 1)$, this design generates no “within” variation. While a cluster randomized design does generate some “between” variation, it is too coarse to identify our effects of interest: under our assumptions $\bar{D}_{ig}$ equals zero when $S_g = 0$ and $\bar{C}_{ig}$ when $S_g = 1$. Our third example, with two saturations $0 < s_L < s_H < 1$, features sufficient “between” variation to identify the effects of interest even when $n$ is so large that “within” variation becomes negligible.

In general, sufficient conditions for (ref)(ii) will depend on the specific choice of basis functions $\mathbf{f}$. For large $n$, however, a necessary condition is that the design contains at least as many distinct interior saturations as there are elements in $\mathbf{f}$. Appendix (ref) provides a detailed explanation of this result.

Estimation and Inference

If $\bar{C}_{ig}$ were observed, a handful of just-identified IV regressions would suffice to estimate the causal effects from (ref). While $\bar{C}_{ig}$ is unobserved in practice, fortunately we can estimate it under one-sided non-compliance by comparing treatment take-up to the share of treatment offers, i.e.\

where we arbitrarily define $\widehat{C}_{ig} = 0$ if none of $(i,g)$'s neighbors are offered treatment.\footnote{Under (ref) it is possible, although unlikely, that $\bar{Z}_{ig}$ could be zero even if $S_g>0$.} In this section we use (ref) to derive feasible, consistent, and asymptotically normal estimators of the direct and indirect causal effects identified in (ref). (ref) provides full implementation details specialized to the linear outcome model from (ref). For simplicity, we assume throughout that the random saturation $S_g$ is bounded below by $\underline{s} > 0$. Because we cannot estimate $\bar{C}_{ig}$ when $S_g = 0$, experiments that include a 0% saturation require a slightly different approach. We explain these differences in Appendix (ref).

In the interest of brevity, we introduce shorthand notation and high-level regularity conditions that apply to all four of our sample analogue estimators. These take the form

where $Y_{ig}$ is the outcome variable from (ref), and $\mathbf{M}^+$ denotes the Moore-Penrose inverse of a square matrix $\mathbf{M}$. (ref) gives the definitions of $\boldsymbol{X}_{ig}, \mathbf{R}$, and $\boldsymbol{W}_{ig}$ corresponding to each part of (ref). The “estimated” instrument $\widehat{\boldsymbol{\mathcal{Z}}}_{ig}$ is a stand-in for the unobserved “true” instrument $\boldsymbol{\mathcal{Z}}_{ig} \equiv \mathbf{R}(\bar{C}_{ig}, N_g)^{-1} \boldsymbol{W}_{ig}$. While $\mathbf{R}(\bar{C}_{ig}, N_g)$ is invertible under (ref), $\mathbf{R}(\widehat{C}_{ig}, N_g)$ may not be so, since $\widehat{C}_{ig}$ could fall outside the support set of $\bar{C}_{ig}$ or even equal zero. For this reason we define $\widehat{\boldsymbol{\mathcal{Z}}}_{ig}$ using the Moore-Penrose inverse, which always exists and coincides with the ordinary matrix inverse when $\mathbf{R}(\widehat{C}_{ig}, N_g)$ is indeed invertible.

As $G$ grows, so does the number of unknown values $\bar{C}_{ig}$ that we must estimate to construct the instrument vectors $\widehat{\boldsymbol{\mathcal{Z}}}_{ig}$.\footnote{While $\bar{C}_{ig}$ can vary across individuals in the same group, it takes on at most two distinct values for fixed $g$. If a group contains $T$ total individuals, of whom $c$ are compliers and $n$ never-takers, then the share of compliers among a given person's neighbors is either $(c-1)/(T-1)$ if she is a complier or $c/(T-1)$ if she is a never-taker. Thus, the number of incidental parameters is $2G$.} For this reason, we consider an asymptotic sequence in which the minimum group size $\underline{n}$ grows along with the number of groups $G$. Under appropriate assumptions, this implies that the limit behavior of $\widehat{\boldsymbol{\vartheta}}$, which we refer to as the “random saturation IV" (RS-IV), coincides with that of the infeasible estimator that uses the true instrument vector $\boldsymbol{\mathcal{Z}}_{ig}$ instead of its estimate $\widehat{\boldsymbol{\mathcal{Z}}}_{ig}$.

Like baird2018, we take an infinite population approach to inference, assuming that the researcher observes a random sample of size $G$ from a population of groups. Unlike baird2018, we allow these groups to differ in size. Upon drawing a group $g$ from the population, we observe the group-level random variables $(S_g, N_g)$ along with the individual-level random variables $(Y_{ig}, D_{ig}, Z_{ig})$ for each member of the group: $1 \leq i \leq N_g$. We further assume that observations are identically distributed, but not independent, within groups.\footnote{The assumption that observations are identically distributed within group amounts to stipulating that the indices $1 \leq i \leq N_g$ are assigned at random.}

Groups are only observed as a unit: either everyone from the group appears in the sample or no one does. For this reason, some care is needed in defining random variables to represent our sampling procedure and expectations to represent the population averages that define our causal effects of interest. The expectations in Theorems (ref)--(ref) are averages that give equal weight to each individual in the population, or sub-population if we condition on $C_{ig}$. Analogously, the estimator in (ref) is an average that gives equal weight to each individual in the sample. Both of these are precisely what we want, as our goal is to identify and estimate average causal effects for individuals. Under iid sampling of groups, however, $(Y_{ig}, D_{ig}, Z_{ig}, \bar{D}_{ig})$ represent a single person chosen at random from a randomly-selected group. If all groups were the same size, this would be equivalent to choosing a person uniformly at random from the population of individuals. When groups vary in size, however, the equivalence no longer holds.\footnote{Consider a population of 100 groups, half of which have 5 members and the rest of which have 15 members so that 250 of the 1000 people in the population belong to a small group and the remaining 750 belong to a large group. Suppose first that we choose a single group at random and then a single person within the selected group. Then someone from a small group has probability 1/500 of being selected while someone from a large group has probability 1/1500 of being selected.} This creates the possibility for ambiguity when taking the expectation of an individual-level random variable, such as $Y_{ig}$, without conditioning on group size.

Fortunately this is only a question of defining appropriate notation. Our group sampling procedure unambiguously gives equal weight to each individual in the population because we observe not isolated individuals but whole groups. While small groups are just as likely to be drawn as large groups, large groups make a greater contribution to the sample averages from (ref) because they contain more people.\footnote{Continuing from the example in the preceding footnote: suppose we randomly sample $10$ groups and observe everyone in them. Then, on average, our sample will contain 5 small groups and 5 large groups. While the total sample size is random, on average we will observe 100 people, of whom 25 come from small groups and the rest from large groups, matching the shares of each kind of person in the population.} The question is merely how to represent this mathematically. Let $\rho_g \equiv N_g / \mathbbm{E}(N_g)$ denote the relative size of group $g$. We write $\mathbbm{E}[Y_{ig}]$ to denote the average that gives equal weight to groups---choosing one person at random from a randomly-chosen group---and $\mathbbm{E}[\rho_g Y_{ig}]$ to denote the average that gives equal weight to individuals---observing an entire group chosen at random. It is the latter expectation that appears in our results below, as it denotes the population equivalent of the double sums from (ref). While this is a slight abuse of notation, expectations from (ref) that involve individual-level random variables but do not condition on group size should be interpreted as (implicitly) weighting by relative group size. Using the notation and sampling scheme defined above, we now state high-level sufficient conditions for the consistency of $\widehat{\boldsymbol{\vartheta}}$ in (ref).

Condition (i) of (ref) simply restates our group sampling assumption. Conditions (ii) and (iii) hold under the assumptions of (ref), as shown in the proof of that result: for each average effect $\boldsymbol{\vartheta}$ from the theorem, we can define an appropriate error term $U_{ig}$, vector of regressors $\boldsymbol{X}_{ig}$, and vector of instruments $\boldsymbol{\mathcal{Z}}_{ig}$ such that $Y_{ig} = \boldsymbol{X}_{ig}' \boldsymbol{\vartheta} + U_{ig}$ where $\boldsymbol{\mathcal{Z}}_{ig}$ is an exogenous and relevant instrument. Moreover, for each part of (ref), $\mathbbm{E}(\rho_g \boldsymbol{\mathcal{Z}}_{ig} \boldsymbol{X}_{ig}')$ equals the identity matrix.\footnote{For effects that condition on $C_{ig}=c$, e.g.\ those from parts (ii) and (iii) of (ref), the appropriate definition of $\rho_g$ becomes $N_g \mathbbm{E}[\mathbbm{1}(C_{ig}=c)]/\mathbbm{E}[N_g \mathbbm{1}(C_{ig} = c)]$.}\textsuperscript{,}\footnote{Given that $\mathbbm{E}(\rho_g \boldsymbol{\mathcal{Z}}_{ig} \boldsymbol{X}_{ig}') = \mathbbm{I}$, we could have defined our estimator to be $\frac{1}{N} \sum_{g=1}^G \sum_{i=1}^{N_g} \widehat{\boldsymbol{\mathcal{Z}}}_{ig} Y_{ig}$ rather than $\widehat{\boldsymbol{\vartheta}}$. It is more convenient both for our asymptotic derivations and practical implementation, however, to work with an IV estimator.} Conditions (iv) and (v) of (ref) would be implied by requiring that the second moments of $\rho_g \boldsymbol{\mathcal{Z}}_{ig} \mathbf{X}_{ig}'$ and $\rho_g \boldsymbol{\mathcal{Z}}_{ig} U_{ig}$ exist and are bounded. We choose to state these conditions in a slightly weaker form because the distribution of $\rho_g$ necessarily changes with $G$ if we consider an asymptotic sequence in which the minimum group size $\underline{n}$ increases with the number of groups, as we will assume below. Requiring the relevant expectations to be $o(G)$ in principle allows the variance of relative group size $\rho_g$ to grow along with the number of groups, provided that it does not grow too quickly. Conditions (i)--(v) together are sufficient for the consistency of

an infeasible estimator that uses the true instrument vector $\boldsymbol{\mathcal{Z}}_{ig}$ instead of its estimate $\widehat{\boldsymbol{\mathcal{Z}}}_{ig}$. The final two conditions of (ref) assume that $\widehat{\boldsymbol{\mathcal{Z}}}_{ig}$ is a sufficiently accurate estimator of $\boldsymbol{\mathcal{Z}}_{ig}$ to ensure that $\widehat{\boldsymbol{\vartheta}} = \widetilde{\boldsymbol{\vartheta}} + o_{\mathbbm{P}}(1)$. In the setting we consider here, this will require a condition on how quickly the minimum group size $\underline{n}$ grows relative to $G$, as we discuss in detail below. Strengthening conditions (v) and (vii) and adding one further assumption implies that $\widehat{\boldsymbol{\vartheta}}$ is asymptotically normal.

Combined with the first four conditions of (ref), (i) and (ii) from (ref) are sufficient for the asymptotic normality of $\widetilde{\boldsymbol{\vartheta}}$, the infeasible estimator defined in (ref). Condition (i) implies that the rate of convergence of $\widetilde{\boldsymbol{\vartheta}}$ is $G^{-1/2}$. Obtaining a rate of convergence that depends on the total number of individuals rather than groups in the sample would require assumptions that are implausible in typical applications of the randomized saturation design.\footnote{Obtaining the faster rate of convergence would require $\text{Var}\left( \frac{1}{N_g} \sum_{i=1}^{N_g} \rho_g \boldsymbol{\mathcal{Z}}_{ig} U_{ig} \right) \rightarrow \mathbf{0}$ as $G \rightarrow \infty$. Because we consider an asymptotic sequence in which the minimum group size grows with $G$, this is technically possible. It would, however, require us to assume that both heterogeneity between groups and dependence within groups vanish in the limit.} Conditions (ii) and (iii) strengthen (v) and (vii), respectively, from (ref): (ii) is sufficient for the Lindeberg condition, which we use to establish a central limit theorem, while (iii) ensures that the limit distribution of the feasible estimator $\widehat{\boldsymbol{\vartheta}}$ coincides with that of the infeasible estimator $\widetilde{\boldsymbol{\vartheta}}$.

Conditions (vi)--(vii) of (ref), along with condition (iii) of (ref), require the difference $(\widehat{\boldsymbol{\mathcal{Z}}}_{ig} - \boldsymbol{\mathcal{Z}}_{ig})$ to be sufficiently small on average that the limiting behavior of $\widehat{\boldsymbol{\vartheta}}$ coincides with that of the infeasible estimator. We now provide low-level sufficient conditions for this to obtain. By definition,

Accordingly, so long as $\mathbf{R}$ is a sufficiently well-behaved function, $(\widehat{\boldsymbol{\mathcal{Z}}}_{ig} - \boldsymbol{\mathcal{Z}}_{ig})$ will be small if $|\widehat{C}_{ig} - \bar{C}_{ig}|$ is. As shown in the following lemma, a sufficient condition for this difference to vanish uniformly over $(i,g)$ is for the minimum group size $\underline{n}$ to be large relative to $\log G$.

The following regularity conditions are sufficient for $\mathbf{R}(\widehat{C}_{ig}, N_g)^+ - \mathbf{R}(\bar{C}_{ig}, N_g)^{-1}$ to inherit the asymptotic behavior of $(\widehat{C}_{ig} - \bar{C}_{ig})$.

Parts (i) and (ii) of (ref) require that $\mathbf{R}$ is well-defined and uniformly invertible over a range of values for $\bar{c}$ that includes the support of $\bar{C}_{ig}$ and excludes zero. Part (iii) is a variant of Lipschitz continuity that holds in the limit as $n$ grows. These conditions are mild: they amount to a slight strengthening of the rank condition from (ref). In the linear basis function example from (ref) and (ref), for instance, (ref) holds whenever $\bar{C}_{ig}$ is bounded away from zero and $S_g$ takes on at least two distinct values between zero and one.\footnote{See the discussion in (ref) immediately following (ref) for details.} More generally, provided that (ref) holds, whenever $\bar{C}_{ig}$ is bounded away from zero and the basis functions $\mathbf{f}$ are well-behaved, we can always extend the definitions of $\mathbf{Q}_0, \mathbf{Q}_1$ from (ref)--(ref) to ensure that (ref) holds. See (ref) for full details. Under this assumption, we can derive sufficient conditions on the rates at which $G$ and $\underline{n}$ approach infinity to ensure that the difference between $\widehat{\boldsymbol{\mathcal{Z}}}_{ig}$ and $\boldsymbol{\mathcal{Z}}_{ig}$ is negligible.

Taken together, Theorems (ref)--(ref) establish that $\widehat{\boldsymbol{\vartheta}}$ from (ref) is consistent, and asymptotically normal in the limit as $G$ and $\underline{n}$ grow at an appropriate rate. In practical terms, our estimators are appropriate for settings with many large groups such as the experiment of crepon2013. To implement them in practice, all that is required is to calculate the estimated instrument $\widehat{\boldsymbol{\mathcal{Z}}}_{ig}$ and then run the appropriate just-identified IV regression from (ref) with standard errors clustered by group. (ref) provides full implementation details for the linear outcome model from (ref).

Application: Job Placement Program in the French Labor Market

In this section we illustrate our methods using data from crepon2013, who implemented a large-scale randomized saturation experiment across French cities, offering job placement program services to young workers seeking employment. In doing so, we uncover patterns of spillovers that could prove relevant for the design of similar labor market programs. The intervention included 235 cities (labor markets), covering a sample of \num[group-separator={,}]{21431} workers of whom \num[group-separator={,}]{11806} were unemployed at the time of randomization.\footnote{The formal criteria for eligibility included “aged below 30, with at least a two-year college degree, and having spent either 12 out of the last 18 months or 6 months continuously unemployed or underemployed” crepon2013.} Two questions of interest arise in this setting. First, the presence of direct effects: whether receiving job placement services impacts subsequent labor market outcomes of participants, in particular the likelihood of being employed. Second, the presence of indirect (spillover) effects: whether the receipt of job placement services by others in the same labor market impacts subsequent labor market outcomes of participants. For example, in such a large-scale experiment one may worry that increasing some workers' likelihood of obtaining a job may hurt the labor market prospects of other workers.

Cities were initially randomly assigned to five saturation bins $\mathcal{S} = \{0, 0.25, 0.5, 0.75, 1 \}$. For reasons outside of the experiment, 43 of the 47 cities initially assigned to the $25\%$ saturation bin in fact received a $50\%$ saturation, and 12 of the 47 cities initially assigned to the $75\%$ saturation bin received a $100\%$ saturation.\footnote{The reassignment of cities across bins is not a problem for the analysis in crepon2013, because the main results in that study make only a binary comparison between the cities assigned to the $0\%$ saturation bin and the pooled group of cities assigned to positive saturation bins.} For this reason all of the results we present below restrict attention to the subset of cities that received their initially assigned saturation.\footnote{Naturally, the validity of this restriction relies on the assumption that the reassignment of cities across saturation bins was unrelated to their underlying characteristics.} Thus, our estimation sample consists of 47 cities in the $0\%$ saturation bin, 4 cities in the $25\%$ saturation bin, 47 cities in the $50\%$ saturation bin, 35 cities in the $75\%$ saturation bin, and 47 cities in the $100\%$ saturation bin.

Eligible workers in each city then received offers with a probability equal to the saturation assigned to their city. As mentioned in the introduction, the overall take-up rate of job placement services was 35%. Only workers who were assigned to treatment could receive it, so (ref) (one-sided non-compliance) holds. In addition, (ref) (IOR) is reasonable in this setting: using a simple regression-based test, (ref) shows that an individual's probability of treatment take-up is statistically unrelated to her group's randomly assigned saturation.\footnote{As far as we are aware, subjects in the experiment of crepon2013 were not informed of their groups' saturations, making IOR a priori plausible as well.} Researchers collected data on labor market outcomes in a follow-up 8 months after treatment receipt. Here we present results for two outcome variables: long-term employment (indefinite contract or fixed-term contract longer than 6 months) and any employment. We estimate a linear outcome model, $\mathbf{f}(\bar{d}) = (1, \bar{d})$, so that

The linear specification is simple to implement and easy to interpret. For full implementation details, see Appendices (ref) and (ref). As discussed in Appendix (ref), the number of basis functions that can be included in practice is limited by the number of saturations. Given the re-assignment of 25% saturations in this experiment, we “effectively” have only three interior saturations, one more than the minimum needed to identify a linear outcome model. (See (ref).) Given the limits imposed by the design, we limit attention to the linear model throughout this section. Our simulation study, discussed in (ref) suggests that the sample size of crepon2013 is sufficient to permit reasonably precise estimation of the linear specification.

Recall that our RS-IV estimator recovers average coefficients for compliers $(\alpha^c,\beta^c,\gamma^c,\delta^c)$, for never-takers $(\alpha^n,\gamma^n)$, and for the whole population $(\alpha,\gamma)$.\footnote{In the more general notation in section (ref), $\boldsymbol{\theta}_{ig}=(\alpha_{ig},\gamma_{ig})$ and $(\boldsymbol{\psi}_{ig} - \boldsymbol{\theta}_{ig}) = (\beta_{ig},\delta_{ig})$.} Using these, we can reconstruct the average potential outcome functions for treated and untreated compliers, for untreated never-takers, and for the whole population.\footnote{Because non-compliance is one-sided, compliers are synonymous with “the treated” and never-takers with “the untreated.”}

(ref) presents estimates and standard errors (clustered at the city level) of the average effects for the whole population, for never-takers, and for compliers using long-term employment and any employment as outcome variables.\footnote{We include observations from the 0% saturation cities as described in (ref).} We estimate large negative spillovers ($\gamma^c = -0.51$) for untreated compliers, and effectively no spillovers ($\gamma^c + \delta^c = 0.62 - 0.51 = 0.11$) for treated compliers. For the average untreated complier, increasing the treated share among his neighbors from 10 percent to 50 percent would decrease his likelihood of employment by 20 percentage points. This is a considerable negative indirect effect of the policy intervention. However, this negative spillover effect is nullified --and possibly reversed-- when compliers are assigned to, and therefore receive, the treatment.

For completeness, (ref) depicts the implied average potential outcome functions for untreated and treated compliers, using long-term employment as the outcome variable. (ref) depicts the corresponding functions using any employment as the outcome variable instead. We report average functions as bold lines, and corresponding (pointwise) 95% confidence intervals as dashed curves. The downward sloping functions on the left of both figures illustrate the negative estimated spillover for untreated compliers: employment prospects for those who would have taken up treatment if offered worsen rapidly as more job seekers in their city take up the job placement program. The flat curves on the right, in contrast, reveal that employment prospects for those who take up treatment are unaffected by the average city-level treatment take up. These patterns are consistent with the idea that compliers who did not receive job placement assistance are hurt by competition in the labor market, while job placement assistance shields those who take it up from these negative spillovers.

Thus, among those willing to receive job placement services, more widespread take-up of the program, possibly via increased labor market competition, has a differential impact across those who do receive and those who do not receive treatment. This difference is driven by the direct treatment effects on compliers, which we plot in (ref). The estimated direct effect increases with $\bar{D}_{ig}$ and is positive for most values of $\bar{D}_{ig}$ observed in the data, although the 95% confidence interval contains an effect size of zero for most observations. Finally, although we cannot recover full treatment effects for never-takers or for the population as a whole, (ref) also illustrates that the average spillover $\gamma^n$ for never takers is positive albeit statistically insignificant. The resulting average spillover for the population as a whole, $\gamma$, although much smaller in magnitude compared to the one for compliers, is negative and marginally significant for any employment ($\gamma = -0.11$).

In settings with potential non-compliance such as this one, participants' take-up decisions may be driven by the expected gains from participation. Our findings are consistent with such behavior: those who decline participation may do so precisely if they expect they will not suffer negative spillovers from others receiving the program. In turn, compliance may in part be driven by the knowledge that, in the absence of treatment, program receipt by others hurts own labor market prospects.\footnote{Notice that these forms of `selection on gains' are compatible with the IOR assumption holding.} Indeed, in (ref) we report results from a regression of compliance indicators on pre-treatment characteristics for the sub-sample of offered individuals. Compared to never-takers, compliers appear to be a more vulnerable sub-population: at baseline they are less likely to cohabit, less educated, less likely to be employed or to have a stable labor contract, and are more likely to receive unemployment insurance.\footnote{Compliers are also less likely to have young children, which may indicate that never takers are less able to participate in the program and possibly in the labor market.} Knowledge of this pattern of effects may prove valuable for the design of other similar large-scale labor market programs.

Simulation study

We now present the results of a simulation study to demonstrate the performance of our estimator in a setting similar to that of our empirical example. As in (ref), we work with the linear outcome model $\mathbf{f}(x)' = (1, x)'$ from ((ref)); for full implementation details, see (ref). We compare the results of our estimator to those of a `na\"{i}ve' IV regression of $Y_{ig}$ on $\mathbf{X}_{ig} \equiv (1, D_{ig}, \bar{D}_{ig}, D_{ig} \bar{D}_{ig})'$ with instruments $\boldsymbol{\mathcal{Z}}_{ig} \equiv (1, Z_{ig}, S_g, Z_{ig}S_g)'$. As detailed in (ref), this estimator yields consistent estimates of $\alpha$ and $\beta^c$, but inconsistent estimates of $\gamma$ and $\delta^c$ when the random coefficients are correlated with the share of compliers.

Our simulation design broadly follows the sampling and experimental design of crepon2013, employing a simple data generating process that allows for correlation between the random coefficients and the share of compliers in a city, $\bar{C}_{ig}$. We present results from three simulation studies with different numbers of groups, $G$. Our main simulations set $G=235$ to match the experimental design in crepon2013; comparison exercises 150 and 500 groups. For simplicity we consider groups of equal size, 116 individuals each, to match the average group size from crepon2013. We randomly assign exactly $1/5$ of groups to each of five saturations, $S_g \in \{0, 0.25, 0.5, 0.75, 1\}$, then draw individual Bernoulli offers at the assigned saturation.

We randomly assign to each group $g$ a share of compliers $\bar{C}_g \in \{0.1,0.2,0.3,0.4,0.5\}$ with equal probability. Individuals are assigned a compliance status in the corresponding proportion. To generate the random coefficients, we first set values for the unconditional average parameters, $(\alpha,\beta,\gamma,\delta) = (0.5,0.2,-0.7,0.8)$. For $\theta \in \{\alpha, \beta, \gamma, \delta\}$, and $\theta_{ig} \in \{\alpha_{ig}, \beta_{ig}, \gamma_{ig}, \delta_{ig}\}$, we then draw the individual random coefficients according to \[ \theta_{ig} = \theta + \left[\frac{\bar{C}_{ig} - \mathbb{E}[\bar{C}_{ig}]}{\text{SD}(\bar{C}_{ig})} \frac{\kappa_\theta}{\sqrt{\kappa_\theta^2+1}} + \frac{u_{ig}}{\sqrt{\kappa_\theta^2+1}}\right] \sigma_\theta, \quad u_{ig} \overset{iid}{\sim} \mathcal{N}(0,1) \] where $\kappa_\theta$ controls the strength of correlation between $\bar{C}_{ig}$ and a given random coefficient so that $\text{Corr}(\theta_{ig},\bar{C}_{ig}) = \kappa_\theta / \sqrt{\kappa_\theta^2 + 1}$. We normalize the random coefficients so that their means are given by the unconditional parameters, $(\alpha,\beta,\gamma,\delta)$, and their standard deviations are given by $\boldsymbol{\sigma} = (\sigma_\alpha,\sigma_\beta,\sigma_\gamma,\sigma_\delta)$. In the simulations below, we set $\boldsymbol{\kappa}=(0,0,1.2,1.5)$ and $\boldsymbol{\sigma}= (0.3,0.3,0.2,0.4)$, which gives $\text{Corr}(\gamma_{ig},\bar{C}_{ig})\approx0.77$ and $\text{Corr}(\delta_{ig},\bar{C}_{ig})\approx0.83$.

(ref) presents means and standard deviations of estimated coefficients along with the actual coverage of the associated nominal 95% confidence intervals for both our estimator and the “na\"{i}ve” IV estimator, based on 5000 simulation replications.\footnote{In principle, one could estimate $(\alpha^n,\gamma^n)$ by estimating a na\"{i}ve IV regression of $Y_{ig}$ on a constant and $\bar{D}_{ig}$ on a subset of the data with $(Z_{ig}=1,D_{ig}=0)$, using $S_g$ as an instrument for $\bar{D}_{ig}$. Similarly, one could estimate $(\alpha^c+\beta^c,\gamma^c+\delta^c)$ by estimating the same regression on a subset of the data with $(Z_{ig}=1,D_{ig}=1)$. However, both sets of estimated parameters would be biased if $\gamma$ is correlated with $\bar{C}_{ig}$, and the second set of estimates would be biased if $\delta$ is correlated with $\bar{C}_{ig}$.} The second panel presents the results for a sample size similar to the experimental design in crepon2013: 235 groups. Our estimator performs well at this sample size---the average coefficients are very close to the true values and the coverage is close to its nominal level for all eight parameter values---and its performance improves in larger samples, as expected. In contrast, the na\"{i}ve IV estimates of $\gamma$ and $\delta^c$ are substantially biased, as predicted by (ref). The performance of the na\"{i}ve IV estimator does not improve as we increase the sample size---the average coefficients remain unchanged and the coverage worsens as the standard errors shrink. (ref) shows the empirical distribution of the point estimates for our estimator in the simulations with 235 groups and compares this to the na\"{i}ve IV for $\gamma$ and $\delta^c$. Again, our estimator performs well and the bias of the na\"{i}ve IV is clearly visible, as is the mean-variance tradeoff between the two estimators. Appendix (ref) presents similar figures for simulations with 150 and 500 groups.

Conclusion

In this paper we have proposed methods to identify and estimate direct and indirect causal effects under one-sided non-compliance, using data from a randomized saturation experiment. Under appropriate assumptions, we show that the key source of unobserved heterogeneity is the share of compliers within a given group. In a setting with many large groups, this quantity can be estimated and yields a simple IV estimator that is consistent and asymptotically normal in the limit as group size and the number of groups grow. We have also illustrated the applicability of our methods using data from a large-scale job-placement program randomized saturation experiment. In this setting, we find negative spillover effects on the sub-population willing to take up the program. The direct effects, however, shield those who take up treatment from these negative indirect effects. These findings illustrate how our methods allow researchers to go beyond intent-to-treat effects and reveal important information that may be relevant for the design of real-world policies.

A possible extension of the methods described above would be to consider settings with two-sided non-compliance. In this case our identification approach would condition on the share of always-takers in addition to the share of compliers. A related idea would be to relax the assumption of anonymous interactions by allowing individuals' potential outcome functions to depend on the take-up rates of different sub-groups within their group. For example, male students may experience stronger spillovers from their male classmates, and female students from their female classmates. In this case, one would need to condition on the compliance rate in each sub-group. A more challenging extension would consider relaxing IOR. Without this assumption, the estimands identified in this paper lack a straightforward causal interpretation. However it may be possible to identify, or at least partially identify, well-defined causal effects under somewhat weaker restrictions on treatment take-up behavior. While a full analysis of policy relevant treatment effects in the presence of spillovers is beyond the scope or this paper, we think there is ample scope for future research in this direction. It could be interesting, for example, to consider applying the marginal treatment effects approach to settings with spillovers and non-compliance. We leave this possibility for future research.

\singlespacing