Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
45,878 characters · 7 sections · 21 citation commands
Policy relevance of causal quantities in networks
Keywords: {causal inference, interference, social networks, social spillovers, counterfactual policy evaluation}
In the case of evaluating the effects of a binary treatment, such as through a randomized experiment, the average treatment effect (ATE) is often the primary quantity of interest (or estimand). It is sometimes sufficient for decision making. Consider a setting where the outcome of each unit $i\in[n]$ depends on its own treatment $Z_i$, so that we observe $Y_i \triangleq y_i(Z_i)$; that is, there is “no interference” cox58.\footnote{We use upper case letters to denote (observed) random variables and boldface letters to denote vectors.} Say that a decision maker aims to maximize utilitarian welfare, $W(\boldsymbol{z}) \triangleq \left\langley_i(\boldsymbol{Z})\right\rangle_{i\in [n]}$, where throughout we use $\left\langleh_i\right\rangle_{i\in [n]}$ to denote averages over a population of units. Then, when deciding whether to apply a costless treatment to an entire population or not, the decision maker can simply look at the sign of the $\text{ATE} \triangleq \left\langley_i(1) - y_i(0)\right\rangle_{i\in [n]}$. Even with a costly treatment, the ATE may remain a sufficient summary of the potential outcomes for decision-making, such as choosing among policies governing randomized allocation of a treatment. Given known costs of treatment (which may be nonlinear in how many units are treated), the ATE tells the decision maker what proportion of the population to treat at random in order to maximize expected welfare.\footnote{Of course, we typically do not know the ATE, but rather have sample data, such as that arising from a randomized experiment or observational data. Then we have to decide how to make choices under uncertainty and/or ambiguity; see, e.g., manski2004statistical. In this article, we focus solely on what quantities are of interest, neglecting estimation and inference.} More generally, with an ordinal or continuous treatment, averages of potential outcomes under each possible “dose” constitute a dose--response curve and can allow selecting the optimal dose or distribution of doses.
By contrast, in settings where the units' outcomes are affected by others' treatments, how to quantify effects has been less clear. This includes how to study important questions about how changes to communication technologies affect the spread of information, how environmental policy affects public health, and how recommendation and matching algorithms affect market outcomes.
Here we focus on the case of interventions in a network, which, while most obviously applicable to social networks, can also be used to represent interference in other settings (e.g., bipartite networks representing which regions are potentially exposed to pollution from which power plants). In a social network where the outcome for one unit might depend on the treatment of all other units, there are many possible comparisons to make: with a binary treatment, the table of all potential outcomes is $n \times 2^n$ (Figure (ref), top left), and there are many ways to summarize and contrast these entries. Naturally, there has been a proliferation of proposed estimands. Many of these are intended to capture various notions of not only direct effects (effects of a unit's own treatment on itself), but also spillover effects. These spillover effects include various versions of how units are affected by other units' treatments or, to put it the other way around, how some units' treatments might affect other units.
To guide what quantities we should develop estimators for and what empirical researchers should report, it may be helpful to consider two properties we would like estimands to have --- properties that the ATE and dose--response curves satisfy in the absence of interference. First, we often want estimands to be summaries of unit-level causal effects. To spell out and generalize this idea, we say an estimand is unit-level causal with respect to some policy space if it can be represented as a convex combination of unit-level contrasts of each unit's outcome under different, possibly unit-specific, policies in that space. This thus includes quantities like the ATE, but also effects of stochastic interventions munoz2012population,kennedy2019nonparametric,chin2022evaluating. For example, we may want a quantity that is an average across units of the effect of some increase in exposure for each unit. Second, we often want estimands that can inform how to assign treatments. More specifically, we may want estimands that are sufficient for policy choice in that sense that they can be used to choose a policy, from some policy class, that assigns treatments to units so as to maximize expected welfare.\footnote{Throughout the main text, we focus on utilitarian welfare with possibly costly treatments.}
In the simple “no interference” case considered before, there is a single estimand (ATE; or, with a many-valued treatment, a set of estimands) that is both (a) a summary of unit-level causal effects and (b) sufficient for optimally choosing a policy that assigns treatment homogeneously. With interference, many appealing estimands that have one property will lack the other. Figure (ref) shows different kinds of dose--response curves available here. One (Figure (ref)a) characterizes units' average outcomes as function of their local exposure to treatment. Another (Figure (ref)c) instead gives average outcomes as a function of a parameter indexing different policies (e.g., Bernoulli probability of treatment). And variations on this latter kind of curve (Figure (ref)d) characterize how the average outcomes of units with different relationships to treated units vary with the policy parameter. In what follows, we elaborate on the properties and relationship between these two types of estimands. These estimands can be seen as involving one of two orders of averaging over units and possible treatment assignments. Here we will describe these two ways of averaging, noting that one often results in quantities that are insufficient for optimal choice of policies. The other often results in quantities that are not unit-level causal, but we argue that additional attention is due, as even then they may be applicable for policy choice. In particular, both ways of averaging generally coincide for one estimand --- namely the expected average outcome --- that we argue is both policy-relevant and causally interpretable.
When statisticians and empirical researchers recognize that interference may be present, it is common to try to characterize how units respond not only to their own treatment but also to neighbors' treatments. This idea has been formalized by introducing an exposure map aronow2017interference,manski2013identification. An exposure map $d$ maps all possible treatment assignments $\{0, 1\}^n$ into exposures for each unit. For example, we might let the exposures for unit $i$ be all possible combinations of whether $i$ is treated and the number of $i$ neighbors who are treated. Let a focal map be the special case of a binary exposure map that indicates which units have a particular exposure (e.g., untreated units with exactly one treated neighbor). (We leave formal definitions and results to the Appendix, but reference specific results in the main text.)
Given such an exposure map, it is natural to summarize outcomes under these different exposures. Among such summaries, most common are estimands that can be represented as first averaging over treatment assignments within units and then over units. Considering the case of something we might label a spillover effect, we can represent the first averaging step as marginalizing over $i$'s potential outcomes according to a distribution over the treatment of all units conditional on $i$ having different number of treated neighbors (Figure (ref), top right).\footnote{ In some cases, it is straightforward to instead define a variation on this where this distribution is not conditional on the exposure. For example, if exposures only consist of a unit's own treatment (i.e. $d(\boldsymbol{Z}) = z_i$), then we can instead marginalize over the unconditional distribution of $\boldsymbol{Z}_{-i}$; this defines what savje2021ateunknown call an expected ATE. If treatment assignment is independent across units, then this coincides with the AFEOs discussed here. Related quantities appear in many other papers hudgens2008toward,vanderweele2011effect,forastiere2021identification. } If we do this for each exposure, the resulting estimands may constitute a natural dose--response curve. Call the average outcome over such focal units an average focal expected outcome (AFEO), $$\operatorname{AFEO}(\pi, f, y)\triangleq\left\langle\mathbb{E}_{\pi}\left[y_i(\boldsymbol{Z})\,\middle\vert\,f_i(\boldsymbol{Z})=1\right]\right\rangle_{i\in \operatorname{supp}_f(\pi)},$$ where $\left\langle\cdot\right\rangle_{i\in \operatorname{supp}_f(\pi)}$ denotes the conditional average over those units that have positive probability of being focal (a “unit-averaging”) and $\mathbb{E}_{\pi}\left[\cdot\,\middle\vert\,\cdot\right]$ denotes the conditional expectation under policy $\pi$ (an “assignment-averaging”). This name reflects that this estimand has an inner expectation over treatment assignments, and then an outer average over units. Call differences between AFEOs under different focal maps are focal contrasts (e.g., a contrast in average outcomes between units with zero or one treated neighbor). Thus, comparisons between points in Figure (ref)a are focal contrasts.
Do these AFEOs and focal contrasts satisfy the two desiderata we have proposed?
To start with a favorable case, assume that the exposure map $d$ is correctly specified in the sense that it gives level sets in the potential outcome function; that is, if two treatment vectors $\boldsymbol{z} \neq \boldsymbol{z'}$ correspond to the same exposure, $d_i(\boldsymbol{z}) = d_i(\boldsymbol{z'})$, then they result in the same outcome for that unit, $y_i(\boldsymbol{z}) = y_i(\boldsymbol{z'})$. Under this assumption, AFEOs defined by each unique exposure only have one meaningful averaging step --- averaging over units --- as all the potential outcomes pooled in the inner expectation are identical.
\paragraph*{As averages of unit-level causal effects.} With a correctly specified exposure map, focal contrasts are readily interpretable as averages of unit-level causal effects. In particular, with technical conditions about units having positive probability of being focal, focal contrasts are unit-level causal in any policy space that includes all deterministic interventions (i.e. all possible treatment assignments; Corollary (ref)). That means that we can interpret them as averages of the effect of assigning each unit to some exposure rather than some other exposure. Whether these focal contrasts are particularly meaningful nonetheless depends on the choice of the exposure map, as arbitrary unit-specific garblings of a correctly specified exposure map are likewise correctly specified.
\paragraph*{Insufficiency for policy choice.} We might expect that correctly specified exposure maps also make AFEOs useful for policy choice. Empirical researchers often estimate AFEOs and focal contrasts and then draw intuitive conclusions about policies for assigning treatment based on those quantities. For example, cai2015insurance use analogs of AFEOs estimated via regression to suggest the utility of social norms interventions, and forastiere2024causal examine the dose--response curves given by AFEOs to suggest what the optimal agricultural subsidy regime is accounting for spillovers. However, as we show here, they are often not sufficient for policy choice; that is, this step from AFEOs to policy choices, while intuitive, is typically not formally justified.
To begin with a simple example, consider a setting where each unit's outcome only depends on whether 0, 1, or at least 2 of its neighbors are treated; it is unaffected by its own treatment. Thus, the true exposure mapping for each unit just has three levels, $d_i: \{0,1\}^n \rightarrow \{0, 1, 2+\}$. The AFEOs are then $\overline{y}(0), \overline{y}(1),$ and $ \overline{y}(2+))$, where $ \overline{y}(e) := \left\langley_i(e)\right\rangle_{i\in[n]} $ and we abuse the notation to denote potential outcomes under exposure $e$. Differences between these can be seen as spillover effect (i.e. exogeneous peer effects). Let us imagine that average outcomes are maximized with a single treated neighbor, i.e. $\overline{y}(1) > \max(\overline{y}(0), \overline{y}(2+))$.
These quantities are a limited guide to policy. First, simply observing that the AFEOs are maximized at exactly one treated neighbor does not provide a way to achieve that average outcome. We typically cannot select a policy that directly corresponds to this average potential outcome: for many graphs, it will not be possible to assign treatments so that all units have exactly one treated neighbor. Thus, if these AFEOs are to be relevant to policy choice, only some of them will be relevant (i.e. for the choice of all-or-none-treated policy, $\overline{y}(0)$ and $\overline{y}(2+)$ are relevant), or they will need to be combined to choose among policies that treat some units at random --- though their usefulness here may also be limited, as we now describe.
Consider a network given by Figure (ref), or a network consisting of many copies of this graph. Say we are choosing between two (homogeneous Bernoulli) policies for treating units: one policy treats each unit with probability $\pi$, while the other treats each unit with probability $\pi' > \pi$. Can we use the AFEOs to make the optimal choice? No. These estimands have “compressed” the potential outcomes in a way that is not very useful for this purpose. The same three averages can arise from tables of potential outcomes that have very different implications for policy choice.\footnote{For example, say all outcomes are zero, except in some cases when a unit has exactly one treated neighbor, it may have a positive outcome (i.e. $y_i(e) = 0$ for all $e \neq 1$). In one setting, it is the higher-degree units who are helped by having one treated neighbors; in another, it is the lower-degree units who are so helped. Both of these settings can yield identical AFEOs. However, for a decision maker aiming to maximize utilitarian welfare, the optimal choice of $\pi$ or $\pi'$ is different across these settings. } Attempts to use these averages for policy choice, such as through a plug-in estimator of the expected average outcome under each policy, will be misleading. Here one problem is that we have discarded information about how likely different units are to receive different exposures under relevant policy changes. In the biclique example of Figure (ref), there is no Bernoulli policy with heterogeneous probabilities for units with different degrees (other than treating no or all units) that induces homogeneous exposure; see Figure (ref).
More generally, the expected utilitarian welfare under a policy $\pi$ is identified by AFEOs of a correctly specified exposure map if and only if the exposure distribution under $\pi$ is unit-homogeneous --- that is, all units have the same probability distribution over exposures, $\mathbb{P}_{\pi}\left(d_i(\boldsymbol{Z}) = e\right)$ independent of $i$ (Proposition (ref) and Corollary (ref)). On irregular graphs, many policies of typical interest will not produce such homogeneous exposure; in fact, no such non-trivial policy may exist. When the exposure distribution is not unit-homogeneous, the welfare is only partially identified from the AFEOs --- and these bounds can be wide. To illustrate this, we reanalyze insurance adoption decisions in the field experiment of cai2015insurance. Figure (ref) shows both estimated AFEOs by number of treated neighbors and the bounds these imply for the welfare under Bernoulli policies with different probabilities of treatment $\pi$. These bounds collapse to a single point when $\pi = 0$, since this policy induces homogeneous exposure (i.e., all units have zero treated neighbors), but they are wide for other values of $\pi$. In this case, because some units do not have 2 or more neighbors, the bounds do not collapse to a point when $\pi = 1$, as this still induces a heterogeneous exposure distribution.
Another view of the challenges here is available by considering the causal relationships between policies, treatments, exposures, and outcomes when the exposure mapping is correctly specified --- represented by the directed acyclic graph in Figure (ref). AFEOs summarize how outcomes vary with exposures $D_i$, but a decision maker chooses a policy $\pi$, which only affects $D_i$ indirectly through $\boldsymbol{Z}$. It can be easy to end up considering interventions on the exposures $D_i$ that cannot be induced by any distribution for $\boldsymbol{Z}$ (that is, any choice of policy $\pi$). Or, even if such a policy exists that induces a particular distribution on $D_i$, that policy may not be of interest at all (e.g., only simple policies may be feasible).\footnote{This is analogous to arguments, in the literature on mediation, that estimands involving cross-world counterfactuals are not relevant to policy naimi2014mediation.}
\paragraph*{With a misspecified exposure map.} Researchers have been concerned about the consequences of misspecifying the exposure map, so it is worth understanding how both desiderata fare in this case.\footnote{One concern is that if a unit's outcomes are affected by their own treatment and neighbors' outcomes, this leads to potentially global dependence manski2013identification,leung_discussion_2024, eckles2017design.} The present analysis is readily applicable to this case, as the inner averaging step for AFEOs is an expectation over potential outcomes grouped together by the exposure map, whether or not they form a level set.
Focal contrasts defined by misspecified exposure maps will generally remain averages of unit-level causal effects, though it can be difficult to understand exactly what they are effects of savje2023exposure. That they are causally interpretable has been suggested by work labeling them “exposure effects” savje2023exposure or “average distributional shift effects” hudgens2008toward. Focal contrasts are generally unit-level causal in some policy space (Proposition (ref)), but this policy space may not be interpretable or have a straightforward relationship to how exposures are defined and labeled. This adds to prior characterizations of specific cases where focal contrasts remain interpretable even with misspecification savje2021ateunknown.
Regarding policy choice, if AFEOs are insufficient even when the exposure map is correctly specified, misspecification only compounds the problem.\footnote{In noting that misspecified exposure mappings can be irrelevant to policy choice, auerbach_discussion_2024 use an example in which the correctly-specified mapping is sufficient, but this depends on other characteristics of that example.} Avoiding misspecification can motivate researchers to use a more granular exposure map; however, in addition to statistical estimation and inference challenges, making an already correctly specified exposure map more granular can sometimes result in making an exposure distribution that was homogeneous under a policy $\pi$ inhomogeneous, and thus making the welfare no longer point identified (Corollary (ref)).\footnote{ One specific example is that exposure maps giving the fraction of neighbors who are treated (i.e. $d_i(\boldsymbol{Z}) = \left\langleZ_j\right\rangle_{j: A_{ij} = 1}$, where $A$ is the adjacency matrix) yield homogeneous exposure under global treatment and global control. On the other hand, in irregular graphs, this is not true under global treatment for an exposure map giving the number of treated neighbors. }
To summarize, these widely-used assignments-then-units averages are often interpretable as averages of unit-level effects, though some of the clarity here depends on correctly specifying an exposure model. But, even in some of the best cases --- that is, even when the exposure map is correctly specified and these quantities have a clear causal interpretation --- they are often a very limited guide to policy choice. To be clear, the methodological literature working with AFEOs has typically not promised policy relevance, but empirical researchers have nonetheless often interpreted these quantities as relevant to policy, as is deceptively intuitive.
We can also represent some estimands as first averaging over units and then over possible treatment assignments. In the simplest case, we can, for each possible treatment assignment, average the potential outcomes of all units; then we marginalize over those average potential outcomes according to the probability of each under a policy of interest. This expected average outcome (EAO) directly addresses the question of what happens in expectation under that policy. And, if the decision maker's utility is linear in the average outcome, it is sufficient for policy choice. In the context of causal inference in networks, the EAO (or the difference in EAO under two policies) has been studied in some recent work chin2022evaluating,viviano2025policy. One observation is simply that EAOs under policies of interest should continue to receive attention from methodologists, and empirical researchers interested in informing policy may wish to estimate EAOs.
It is also possible to define other quantities less familiar in the causal inference literature. To again consider something like a spillover effect, we can, for each possible treatment assignment $\boldsymbol{z} \in \{0, 1\}^n$, summarize the potential outcomes of all units that had different number of treated neighbors in that assignment (Figure (ref), lower left). Then we can average these entries to produce average outcomes for units with 0, 1, or at least 2 treated neighbors under some distribution for $\boldsymbol{Z}$, some policy of interest $\pi$. We call these expected focal average outcomes (EFAOs), $$ \operatorname{EFAO}(\pi, f, y)\triangleq\mathbb{E}_{\pi}\left[\left\langley_i(\boldsymbol{Z})\right\rangle_{f_i(\boldsymbol{Z})=1}\,\middle\vert\,f(\boldsymbol{Z})\ne\boldsymbol{0}\right], $$ where $\mathbb{E}_{\pi}\left[\cdot\,\middle\vert\,\cdot\right]$ denotes the conditional expectation over $\boldsymbol{Z}$ w.r.t. policy $\pi$ (an “assignment-averaging”) and $\left\langle\cdot\right\rangle_{f_i(\boldsymbol{Z})=1}$ denotes the conditional average over focal units, a “unit-averaging”. EFAOs and AFEOs are thus named with the same words in different orders, reflecting that they perform the same two averaging steps (over units and over treatment assignments) but in the opposite order. Such EFAOs --- defined by the number of treated neighbors --- would then answer the question of what the average outcome of units with, e.g., exactly one treated neighbor is under a policy $\pi$.
Do these EFAOs and contrasts between them for different policies or different focal maps satisfy the two desiderata we have proposed?
\paragraph*{Generally not unit-level causal.} Contrasts of EFAOs for two different policies, $\operatorname{EFAO}(\pi, f, y) - \operatorname{EFAO}(\pi', f, y)$, do not generally have an interpretation as an average of unit-level causal effects. There are some special cases where an EFAO contrast coincides, exactly or asymptotically, with an intuitively related quantity represented via the other route (Proposition (ref)). For example, this comparison of average outcomes for treated and untreated units is the focal contrast of the unit's own treatment when $\pi$ is a completely randomized (i.e. $m$-out-of-$n$) design, and it is the global average treatment effect ugander2013graph when $\pi$ is an all-or-none-treated policy. These are all cases where the policy induces homogeneous probabilities of each exposure.
Outside of such special cases, EFAO contrasts are not unit-level causal. Under a heterogeneous Bernoulli policy (e.g., treating high degree nodes with higher probability), for example, this kind of one-treated-neighbors vs. no-treated-neighbors comparison can be large even with no treatment effects whatsoever. More generally, EFAO contrasts are only unit-level causal in all policy spaces if the focal units are selected determistically (i.e. the focal map is invariant in $\boldsymbol{z}$; see Proposition (ref)). That is, they only have a unit-level causal interpretation when they are not really summaries of outcomes for units with different exposures, but rather just summaries of outcomes for the whole population (i.e. the EAO) or a fixed subset of units.
\paragraph*{Other causal interpretations.} If these quantities are often not unit-level causal, are they useful for causal reasoning? We contend that these quantities are often nonetheless causally interpretable, but in a different way. First, a contrast of these quantities between two policies $\pi$ and $\pi'$ is still a causal effect of policy on aggregate outcomes. Perhaps $\pi$, compared with $\pi'$, increases what treated units' neighbors' outcomes are on average. These are causal quantities, describing how the policy chosen affects aggregate outcomes (e.g., what will outcomes of neighbors of treated units look like?), even if they are not averages of unit-level effects.
\paragraph*{Sufficiency for policy choice.} Even if these quantities are not unit-level causal, they may still be sufficient for policy choice. First, if a decision maker is maximizing utilitarian welfare, then the EFAO with all units as focal (i.e. EAO) is sufficient for policy choice. Second, decision makers are often evaluated, at least in part, based on myopic summaries (e.g., only looking at treated units) or naive comparisons (e.g., of treated and untreated units). Thus, rational (if perhaps cynical) decision makers will have preferences that separately weigh outcomes for treated and untreated units. They may prefer treated units to have good outcomes; or they may prefer treated units to not have outcomes that are too good, in the case of social programs that they want to appear to be ex post targeted to the less well-off. Furthermore, in the context of causal inference in networks, they may have preferences about the outcomes of units with various exposures to treatment. In a social network, a decision maker may have preferences about the outcomes of people within one hop of units selected for an intensive marketing intervention cai2015insurance. While some of these preferences might arise in settings without interference, positing interference often means that we expect that units will continue to interact, such that unit $i$'s intermediate outcome should be interpreted in the context of aggregate outcomes as they will, e.g., be competing for scarce resources using the income it represents.
Thus, particularly in the presence of interference, it will be natural for decision makers to have complex aggregations of outcomes in mind when choosing among policies. This is represented in Figure (ref), where exposures are potential inputs to welfare.
How then should we quantify outcomes and effects of interventions in networks? Decision makers need quantities that are relevant for policy choice. And the causal inference literature typically prefers quantities that are summaries of unit-level causal effects. The expected average outcome (EAO) is a quantity that coincides under both kinds of averaging we have described, under any treatment policy, regardless of whether it induces different exposures homogeneously or not.\footnote{Thus we could equally call this the average expected outcome, as either ordering of unit-averaging and assignment-averaging yields the same quantity.} Contrasts of EAOs are thus both policy-relevant --- since we marginalize over the treatment of all units under the same policy and therefore can compare this quantity across different policies --- and are readily interpretable as summaries of unit-level causal effects. Rather than fixating on specific unit-level causal contrasts that are informative only about policies that are homogeneous in the exposures they induce, we argue that estimating EAOs --- for all policies within a larger relevant policy space --- will be more directly decision-relevant. The EAO is the unique estimand for which both averaging routes coincide universally (Proposition (ref)), and that it is necessary and sufficient for policy choice under utilitarian welfare with arbitrary costs (Proposition (ref)).
We are not arguing for neglecting other estimands altogether. Researchers may often also be interested in AFEOs like those in Figure (ref)a as part of basic questions in behavioral science. These quantities may play a central role in tests of theory-driven choices of exposure maps. But, to the extent that researchers have more applied goals as well, these should be supplemented with other quantities that are sufficient for policy choice.
Some other estimands in the literature, which take forms not readily included in our characterization here, may be able to be combined to yield a difference in EAOs for pairs of policies while also each being interpretable as summaries of unit-level causal effects. For example, for comparing policies that treat $m$ versus $m-1$ units, we could decompose this difference in EAOs into a direct effect and an average (or total) effect of treating a random unit on all other units hu2022indirecteffects. We encourage further attention to decompositions of EAOs and differences in EAOs that may provide additional insight into the mechanisms underlying aggregate consequences of a treatment policy.
The authors have no conflicts of interest to declare.
SL was supported by a Schmidt Science Fellowship. Some ideas in this paper developed in part in response to the Symposium on Causality in Florence and the workshop on Causal Inference and Prediction for Network Data at the Banff International Research Station in 2024. We are grateful for comments from Daniel Nevo, Davide Viviano, and seminar participants at the Harvard Institute for Quantitative Social Science, Harvard Business School, and the Yale Institute for Foundations of Data Science.
SL and DE conducted the research and wrote the article.