Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
95,372 characters · 14 sections · 72 citation commands
Treatment Effects in Market Equilibrium footnote-1
}
\onehalfspacing
The standard analysis of treatment effects in a randomized trial relies on an assumption that the treatment status of one individual does not affect the outcomes of other individuals fisher1935design, banerjee2011poor,imbens2015causal. When an intervention affects supply or demand and market prices affect the outcome of interest, then general equilibrium effects lead to spillover effects, and typical estimators using data from randomized trials do not correspond to a meaningful measure of policy impact heckman1998general.
In practice, the most common approach to address interference of this type is clustering units at a higher-level at which there are no spillover effects baird2018optimal, hudgens2008toward. Examples include the evaluation of general equilibrium impacts of cash transfers in banerjee2021food, egger2022general and filmer2023, and incentives for suppliers or consumers in online two-sided markets brennan2022cluster, holtz2024reducing. However, if a market is tightly integrated, splitting the market into effectively isolated sub-markets may not be feasible. This highlights the need for spillover-aware methods for treatment effect estimation that can be used on data from an experiment run on a single interconnected market.
In this paper, we introduce a framework for treatment effect estimation in a market equilibrium setting without relying on parametric, distributional, or homogeneity assumptions. Rather than relying on market splitting, we leverage a structural assumption whereby all spillovers are mediated through the (observed) equilibrium prices of a finite number of traded goods. Our main finding is that integrating such price equilibrium effects into the potential-outcomes model for causal inference enables us to estimate spillover-aware treatment effects using unit-level experiments. In particular, we show how our approach yields asymptotic characterizations of policy-relevant average and conditional average treatment effects, and motivates the design of new forms of randomized trials and targeting rules.
We frame our model in terms of a potential outcomes specification that allows for cross-unit interference hudgens2008toward, manski2013identification, aronow2017estimating. When equilibrium effects are removed, the model reduces to the standard Neyman-Rubin potential outcomes framework imbens2015causal. In a market with $n$ participants, an intervention is assigned randomly conditional on pre-treatment covariates. Each individual's production and consumption choices are determined by latent supply and demand functions that may be shifted by the intervention angrist2000interpretation,heckman2005structural. Units interact via a market modeled as a multiple goods economy in general equilibrium, where individuals' production and consumption decisions depend only on their own type, their own treatment, and their expectation of the market price walras1900elements, mas1995microeconomic. In this setup, equilibrium prices induce an exposure mapping in the sense of manski2013identification and aronow2017estimating: One unit's treatment impacts another's outcomes only through the treatment's impact on the equilibrium price, which rules out peer effects or other forms of network-type interference. The price ensures that average net demand (demand minus supply) is equal to zero. The validity of this restriction on spillovers is context-specific, and depends both on the nature of the intervention and the type of market.
Our analysis begins by considering the marginal effect of increasing treatment probabilities for all market participants; following carneiro2010evaluating, we refer to this quantity as a marginal policy effect. As shown in hu2021average, the marginal policy effect can be decomposed into a direct and indirect treatment effect, where the indirect effect is a general measure of spillover effects. We then analyze the sample average direct and indirect effects of a binary treatment in equilibrium halloran1995causal, hu2021average, savje2021average. In finite samples, these estimands are difficult to characterize statistically because the price equilibrium induces a complex form of data dependence. We show, however, that under an assumption that the $n$ participants in the market are independently sampled from a population distribution, we can use techniques based on empirical-process theory to derive simple large-sample limiting expressions for the direct and indirect effects. These limiting expressions can then be used to guide estimation and inference for both sample and population treatment effects.
In our setting, the average direct effect can be consistently estimated via a simple difference in means estimator that ignores spillovers. Consistent estimation of the average indirect effect---and construction of confidence intervals for either estimand---also requires estimates of price elasticities. We consider augmenting standard Bernoulli-randomized experiments with small, random price perturbations that do not disrupt the overall market equilibrium. The use of such random price perturbations is in line with the documented practice of online marketplaces using randomly assigned coupons, discounts or bonuses to learn about price elasticities castillo2023. We then show how such augmented experiments can be used to provide valid large-sample inference for both the average direct and indirect effects using data from a single market.
Our model allows for unrestricted heterogeneity across agents, and thus also enables us to estimate how treatment effects vary with pre-treatment characteristics---and to exploit this heterogeneity to learn improved policies that can be deployed in a market similar to the one observed. We propose versions of the sample direct effect and indirect effect that are conditional on pre-treatment covariates. These estimands depend both on the population distribution and the size of the market. They are related to the large literature on targeting without spillovers manski2004statistical, athey2016recursive, wager2018estimation, vanderweele2019selecting, in that the conditional average direct effect is equal to the conditional average treatment effect when interference is removed. We show that as the market size grows large, the conditional estimands converge to population estimands that are simple combinations of price elasticities and conditional expectations of the direct effect of the treatment on outcomes and net demand. Furthermore, we show that it is possible to use data from a standard randomized experiment (without price perturbations) to estimate the outcome-maximizing treatment rule within the class of all treatment rules that do not move the observed market equilibrium. This optimal rule takes the form of a linear thresholding rule in the space of conditional average direct effects on outcomes and net demand.
filmer2023 use a village-level cluster randomized experiment to demonstrate that the Pantawid conditional cash transfer program in the Philippines affects the equilibrium price of perishable protein sources, such as eggs, which leads to spillover effects on health outcomes for children in non-eligible families. In Section (ref), we argue that this application fits the assumptions of our paper. We then use the data from this setting to estimate an equilibrium model of children's health outcomes and the market for eggs in a small village in the Philippines, and demonstrate the consistency and coverage properties of our treatment effect and variance estimators based on an augmented individual-level randomized experiment generated by simulating data from this model. We also augment the model to include heterogeneous responses to the cash transfer and describe the properties of the equilibrium-stable targeting rule on samples drawn from this model.
Much of the existing work on treatment effect estimation with spillovers focuses on setting where the Stable Unit Treatment Value Assumption (SUTVA) holds for higher-level clusters of units baird2018optimal, basse2019randomization, hudgens2008toward, karrer2021network, liu2014large, tchetgen2012causal. Another related approach is the network interference model, which posits that interference operates along a network and the connections between units are sparse athey2018exact,leung2020treatment,savje2021average. The sparsity of connections then enables randomization-based methods for studying cluster-randomized experiments to be extended to this setting. In our setting, however, the interference pattern produced by marketplace price effects is dense and simultaneously affects all units, so neither cluster nor sparsity-based methods are applicable.
Our use of a random sampling model presents a departure from the literature cited above, where typically the sample is held fixed and inference is only driven by random treatment assignment. This random sampling model, however, plays a key role in enabling our analysis, as it allows us to leverage tools from empirical process theory van1996weak to address difficulties around potentially unstable and non-unique equilibrium formation in finite samples.\footnote{As discussed in detail in the following section, we assume that there is a unique market-clearing price in the population; however, we allow each market participant to respond discontinuously to market prices and this may lead to non-uniqueness of possible equilibria in finite samples.} In doing so, our paper adds to a handful of recent papers that use a stochastic model in a causal inference setting. johari2020experimental use a mean-field model to examine the benefits of a two-sided randomization design in two-sided market platforms and quantify the bias due to interference between market participants. li2022random study large sample treatment effect estimation under network interference, where the exposure graph is randomly generated from a graphon model. Finally, wager2019experimenting consider a model where exogenous demand is matched with endogenous supply, with interference via supply cannibalization. They propose using zero-mean perturbations to estimate a revenue gradient and optimize a continuous decision variable dynamically. Our work also uses zero-mean perturbations as part of a unit-level experiment, but is otherwise distinct, given our focus is on treatment effects of a binary intervention in a general price equilibrium.
Our model combines a potential-outcomes model with some limited structural assumptions on the price equilibrium. The insight that structural modeling can be used to address endogeneity issues in causal inference has a long tradition in economics heckman1979sample,heckman1998general,roy1951some, and there is a large literature that uses structural modeling to develop econometric methods for causal inference in models with endogenous treatment selection angrist2000interpretation,carneiro2011estimating,heckman2005structural. To date, however, there has been less emphasis on methods for estimating equilibrium effects in systems with randomized trials, where treatment selection is not an issue.
We do note a growing literature that combines data from a randomized trials with parametric structural models to estimate both partial and general equilibrium policy effects. For example, allende2019approximating uses a randomized control trial to estimate the direct effect of information provision on school choice but simulates a fully parametric structural model to evaluate equilibrium effects of the treatment. In contrast, our approach uses a more complex unit-level experiment and the structure of the price equilibrium to analyze equilibrium effects without imposing a parametric model. To accomplish this, we rely on a representation of population estimands as a combination of price elasticities and expectations of the outcome and demand distribution. This is related to the sufficient statistics approach, as described in chetty2009sufficient, which also expresses policy effects in terms of combinations of estimable elasticities without imposing a specific parametric model, although this is usually in a partial equilibrium setting. In the macroeconomics literature, wolf2019missing uses a semi-structural approach to decompose the effect of economic shocks into partial equilibrium effects and general equilibrium effects (e.g., price effects), but uses time series methods to estimate the price effects, in contrast to the unit-level approach presented in this paper.
The literature on targeting treatments under interference is quite limited. viviano2019policy estimates treatment allocation rules in a network interference model, where his results depend on sparsity of the network. sahoo2022policy study policy learning with strategic agents, where the policymaker has a budget constraint, which leads to a type of interference. Both approaches rely on a form of direct empirical welfare maximization and do not define spillover-aware measures of treatment heterogeneity as we do in this paper.
Finally, the price perturbations that we use to estimate average price elasticities can be thought of as generating an ideal instrument for our setting. By construction, everyone in the sample is a complier, and the size of the perturbations shrink so that local elasticities can be estimated without imposing parametric assumptions on demand or supply. In empirical work with observational data, instrumental variables approaches have been used in both parametric and non-parametric settings to estimate price elasticities or a weighted average of elasticities angrist1996identification, berry2021foundations. angrist2001instrumental reviews IV approaches and their relation to causal inference for observational data. In settings where it is not possible to randomize individual-level fees, under additional structural assumptions, a traditional IV-based approach could be used to estimate relevant elasticities from observational data.
We first introduce a potential-outcomes model of a market with $n$ participants, where each participant is drawn independently from a population. A binary intervention affects individual supply, demand and outcomes in the market. We use this model to define the estimands of interest for our paper, both at the sample level and at the population level. We use “sample" to refer to a finite-sized market with $J$ goods and $n$ participants. Throughout this section, we will refer to two example interventions: The first is a supplier-level subsidy in an online market setting; the second is a household-level cash transfer that affects economic choices in a village economy.
We assume that for all $n$ market participants,\footnote{In Section (ref), our inference approach will also allow for settings where we can only observe outcomes for a sub-sample of the $n$-sized market (but equilibrium effects are still determined by the full size-$n$ sample).} we observe an outcome $Y_i \in \RR$, treatment $W_i \in \cb{0, \, 1}$, net demand (i.e., demand minus supply) for the $J$ traded goods $Z_i \in \RR^J$, along with covariates $X_i \in \mathcal X$. Following the potential-outcomes model with spillovers, we write $Y_i(\bm w)$ and $Z_i(\bm w)$ for the outcome and net demand we would have obtained for $i$-th unit under treatment market-level treatment assignment $\bm w \in \cb{0, \, 1}^n$.\footnote{We use bold-face notation to denote a vector-valued quantity collected across all market participants.} We assume that spillovers are mediated by endogenous prices $P_n \in \RR^J$ that match supply and demand for the $J$ traded goods, as detailed below. We also allow for an exogenous individual-level price shifter $U_i \in \mathbb R^J$. For our analysis of classical randomized trials, $U_i = 0$, but we will also consider a class of experiments that randomize discounts or fees in a market, where $U_i \neq 0$ and is also exogenous.
In Section (ref), we will use observations of net demand, outcomes, treatments and price-shifters in a single market of size $n$ to estimate policy-relevant treatment effects. Assumption (ref) is the main structural assumption in the paper that makes this possible. Treatment and market prices directly impact outcomes and net demand for each individual. Prices are endogenous, and are determined by a sample equilibrium condition on net demand. Because of this endogeneity, there are spillovers from individual $i$ to individual $j$ through their effect on the equilibrium that the market reaches. Although our model allows for equilibrium spillovers, it rules out other types of spillovers, such as through peer effects or some other network mechanism. The validity of this restriction on interference primarily depends on the nature of the intervention and the context of the market. A subsidy allocated to suppliers in an online market may, by design, be impossible to share with those in the control group, so spillovers occur only through market prices. On the other hand, an intervention that affects consumption in a village economy may have spillovers both through social networks in the village and through the market equilibrium.
The second part of the assumption characterizes how prices form in finite samples. The finite market price does not need to exactly clear the market; instead, we impose a restriction on the rate at which the error in the market-clearing condition shrinks with the sample size. Furthermore, it does not need to be the unique price meeting this condition, although we will require uniqueness at the population level, i.e., in the limit as the sample size grows large (Assumption (ref)).\footnote{Proposition (ref) in Appendix (ref) provides explicit sufficient conditions for Assumption (ref) to hold in a single good market without price shifters and with discontinuous net demand functions.}
We will start by considering results that hold under a standard Bernoulli-randomized RCT (Design (ref)). In Section (ref), we will show how augmenting Bernoulli-randomized trial with random individual-level price shifters (Design (ref)) enables further estimation and inferential results. Design (ref) will discussed in more detail in Section (ref).
Given this causal model, there are many treatment effects one can seek to study. The total treatment effect is the average difference in outcomes from treating everyone versus no one in the market,
where $\mathbf{1}_n$ and $\mathbf{0}_n$ are $n$-length vectors of 1s and 0s. Total treatment effects can readily be estimated across disconnected markets using cluster-randomized experiments clustered at the market level baird2018optimal. However, without further assumptions, estimating $\tau_{\text{TOT}}$ from a single (connected) market is impossible, as it requires predicting behavior at prices that are far from anything seen in the experiment. A more tractable---and still fruitful---approach is to focus on treatment effects that, in the sense of carneiro2010evaluating, are marginal to the treatment-assignment policy used to collect the data. For any $n$-length vector of Bernoulli randomization probabilities $\pivec$, write
where we use the notation $\EE[\pivec]{\cdot} = \EE{\cdot \cond \cb{Y_i(w, \, p), \, Z_i(w, \, p), U_i}_{i=1}^n}$ to denote expectations over the Bernoulli treatment assignment while holding the sample (i.e., the potential outcomes) fixed. The marginal policy effect in the sample, $\tau_{\text{MPE}}$, is a local approximation to the total treatment effect defined in (ref). It measures the average effect of increasing treatment probabilities for everyone:
hu2021average show that the marginal policy effect can be decomposed into the sum of two sample average treatment effects, which are the first set of target estimands in our paper. Here, the notation $(W_i = a ; \bm W_{-i}) $ represents the random vector we obtain by first sampling $ \bm W $, and then setting its $ i $-th coordinate to a fixed value $ a $:
$\tau_{\text{ADE}}$ is the average effect on an individual's outcome of changing their own treatment in a market with $n$ participants, and is a standard quantity in the causal inference literature halloran1995causal,savje2021average. It is a measure of the direct effect of the treatment while holding the equilibrium fixed. When SUTVA holds, then $\tau_{\text{ADE}}$ is equal to the sample Average Treatment Effect (SATE) and $\tau_{\text{AIE}}$ is 0. The average indirect effect in the sample is the average effect on everyone else's outcomes of changing an individual's treatment. It is a general measure of spillover effects.
The estimands $\tau_{\text{ADE}}$ and $\tau_{\text{AIE}}$ are defined at the sample level. We also find it useful to introduce population versions of our target estimands, and define the following whenever the limits below exist. $\mathbb E[\cdot]$ reflects expectations over all sources of randomness in our model: the sampling from the population distribution in Assumption (ref), the price-shifter assignment process, and the treatment assignment process.
In the next section, we will show that under our model and an additional set of regularity conditions, these limits in fact exist and the population estimands have a tractable representations. These representations will then be useful for designing estimators that are consistent for both sample and population-level targets. In addition, just as in the literature on causal inference under SUTVA, in some settings the population estimand is the more relevant target. For example, we may have data from a small market and are interested in doing inference on a similar but larger market.
Our asymptotic characterization of the average direct and indirect effect requires some additional regularity conditions. It is first helpful to introduce some additional notation for moments of the population distribution. Let $y(p) = \mathbb E[Y_i(W_i, p)]$ and $z(p) = \mathbb E[Z_i(W_i, p)]$. For $w \in \{0, 1\}$, let $y(w, p) = \mathbb E [ Y_i(w, p)]$ and $ z(w, p) = \mathbb E[ Z_i(w, p)]$. Define $ p^*_{\pi}$ to be the price that clears the population market when the treatment is allocated according to $\pi(\cdot)$, i.e., $p^*_{\pi} = \{ p: z( p) = 0 \}$.
We start by making regularity assumptions on the expected net demand, and in particular assume that there is a unique equilibrium price in the population. Market prices will be in a compact set as long as all net demand functions are weakly negative for prices above an upper bound and weakly positive for prices smaller than some lower bound. In a single good market, uniqueness of $p^*_{\pi}$ only requires continuity and strict monotonicity of $z(p)$ in $p$. In a market with multiple goods, uniqueness and existence of the equilibrium price can be shown by ensuring the expected net demand function is a contraction, as is common in the literature on models of strategic behavior cornes1999equilibrium, van2000existence. Other approaches that rely on more primitive assumptions are also possible; for example, under a zero-degree homogeneity assumption in net demand a gross-substitutes condition ensures uniqueness arrow1971general.
Next, we make regularity assumptions on the unit-level net demand and outcome functions. To ensure that a variety of random processes studied in the paper concentrate, Assumption (ref) and Assumption (ref) impose some constraints on outcomes and net demand at an individual level. These restrictions have a simple economic interpretation and are general enough to encompass a wide variety of possible data-generating processes that underlie market behavior. Assumption (ref) is our main assumption on the net demand, and is a generalization of the familiar assumption in a single-good setting that net demand be non-increasing in price. In the multiple good case, we also require that every unit $i$ will decrease their net demand of good $j$ more in response to a decrease in the price of good $j$ than in response to a relatively small change in the price of other goods. This assumption restricts the magnitudes of cross-price elasticities compared to own-price elasticities.
Under Assumption (ref), the outcome function is the sum of an individual-specific random function that is Lipschitz in prices and an additional term that is Lipschitz in net demand. This allows for outcomes that are discontinuous in prices through net demand. Supplier profit, for example, is a special case of Assumption (ref), where in the single-good case, profit is $Y_i(w, p) = -(p - \Gamma_i(w) ) Z_i(w, \, p) \mathbbm{1}(Z_i(w, \, p) < 0)$ and $\Gamma_i(w)$ represents a firm's production costs under treatment $w$. In Section (ref), outcomes are a measurement of height for young children, and the treatment is a household-level cash transfer. Assumption (ref) allows for a variety of flexible data-generating processes for height; one example is a random coefficients model of height that is polynomial in consumption, where $\Gamma_i(w)$ are the random coefficients and noise term in the model.
Finally, in Assumption (ref) we list some additional regularity assumptions, which are standard in the literature on asymptotic statistics. Although individual net demand and outcome functions may be discontinuous, we require their expectation to vary smoothly in prices. The weak continuity assumption in Part 2 also limits individual-level discontinuity by requiring that discontinuity points cannot concentrate at specific values of $p$. In our profit example, these two smoothness assumptions require the distribution of costs at a firm-level to be sufficiently smooth, even if production is discontinuous in prices.
Our first result is that, given these assumptions, the sample equilibrium price concentrates on $p^*_\pi$ as the sample size grows, and has an asymptotically linear representation (i.e., to first order, random price fluctuations can be written as a sum of additive contributions from each unit). We can write $P_n(\bm W)$ as a method-of-moments estimator, so its asymptotic representation has the usual form of Z-estimators van1996weak. All proofs are given in the appendix.
This result is a crucial building block towards the rest of the theory in the paper, since it provides a representation of $P_n(\bm W)$ in terms of the fixed price $p^*_{\pi}$ and an average of IID terms. This asymptotic representation is valid whether or not the Bernoulli-randomized trial is augmented with price perturbations. It will be helpful in characterizing the asymptotic variance of various random variables in our model. Before introducing our estimators in Section (ref), we next apply this result to characterizing the relationship between the sample and population-level average direct and indirect effect. To this end, we first show that $\tau^*_{\text{ADE}}$ and $\tau^*_{\text{AIE}}$ exist and have a simple representation in terms of moments of the population distribution. Our technical results rely heavily on concentration results from empirical process theory as described in van1996weak.
The population direct effect is the difference in expected outcomes at $p^*_{\pi}$ evaluated at $W_i = 1$ and $W_i = 0$. In the next section, we show that this can be estimated with a differences in means estimate. This finding is in line with savje2021average, who show that estimators that target the average treatment effect under no-interference settings generally recover the average direct effect under interference. However, our results will go beyond those of savje2021average, since we will also provide a central limit theorem for estimators of $\tau^*_{\text{ADE}}$.
The indirect effect, which is a key component of the policy counterfactual $\tau_{\text{MPE}}$, is not estimable using variation in treatment only. The population indirect effect depends on the product of three terms: how the treatment affects net demand, how net demand is affected by prices ($\xi_z$), and how changes in prices affect outcomes ($\xi_y$). In settings where the treatment impacts market-clearing prices through net demand, and outcomes are sensitive to market prices, then spillover effects and the indirect effect are stronger. In Section (ref) we will show how each component of $\tau^*_{\text{ADE}}$ and $\tau^*_{\text{AIE}}$ can be estimated using unit-level experiments.
Finally, we connect the limiting estimands $\tau^*_{\text{ADE}}$ and $\tau^*_{\text{AIE}}$ to their finite-sample counterparts $\tau_{\text{ADE}}$ and $\tau_{\text{AIE}}$. In the case of the direct effect, Theorem (ref) provides an asymptotically linear expansion for $\tau_{\text{ADE}}$ around $\tau^*_{\text{ADE}}$, thus allowing us to disambiguate between the sample and population estimands when conducting inference. This mirrors the well-known relationship between the sample- and population-average treatment effects in the no-interference setting imbens2004nonparametric.
Relative to the no-interference setting, the expansion in Theorem (ref) includes additional $\Delta_i$ terms that depend on individual-level net demand. In a finite market where treatment affects demand and supply, there is variation in realized market prices that then impacts $\tau_{\text{ADE}}$. The $\Delta_i$ terms capture this additional source of variation. We also note that, as in the no-interference setting, the variance of $ \tau_{\text{ADE}}$ is generally not identified, as it depends on the covariance of individual treated and control potential outcomes; however, as shown in Proposition (ref), it will still be possible to provide asymptotically conservative inference (i.e., construct confidence intervals with potentially greater-than-nominal coverage).
Meanwhile, for the indirect effect, $\tau_{\text{AIE}}$ converges to $\tau_{\text{AIE}}^*$, but Theorem (ref) does not provide a rate of convergence. We conjecture that establishing useful rates of convergence here would require replacing Assumptions (ref) and (ref) with stronger assumptions that are more explicit about finite-sample price formation. Under our current assumptions, the convergence rate of $\tau_{\text{AIE}}$ can be slower than $\sqrt n$; see Appendix (ref) for additional explanation in the context of a simple example. Since our estimator in Section (ref) targeting $\tau_{\text{AIE}}^*$ also converges at a slower than $\sqrt n$ rate, we conjecture that confidence intervals built for $\tau^*_{\text{AIE}}$ will also generally cover $\tau_{\text{AIE}}$; we also note that this holds in our numerical experiments.
We now move to estimation of the quantities defined in the previous section using data from unit-level randomized experiments. We first derive the limiting distribution of a differences-in-means estimator that is valid when data is generated from a standard RCT that is run on an entire market. This estimator is consistent for the average direct effect, but its asymptotic variance depends on price-elasticity terms that cannot be estimated using data from an RCT that randomizes treatment only. We show that if confidence intervals are constructed using a variance estimator based on the asymptotic variance without price interference, then coverage for the population direct effect will not be asymptotically exact.
In order to perform inference for the direct effect and to perform estimation and inference for the indirect effect, we need non-zero price shifters. Algorithmically, the estimator for the indirect effect looks like a combination of differences-in-means estimator and an instrumental variables estimator. Under an augmented randomized experiment, we show how to construct asymptotically valid confidence intervals for population versions of both effects. In this section, we assume that experiments are run on the entire market of $n$ participants. However, as described in more detail in Appendix (ref), the extension to settings where only a sub-sample of the market is observed is straightforward.
Following a number of recent papers, including savje2021average and li2022random, we consider estimating the direct effect using a differences-in-means estimator
Our first result is that---even under interference that occurs through market prices---the standard differences-in-means estimator converges to the population direct effect at a $\sqrt n$ rate and is asymptotically normal. The variance of the difference-in-means estimator depends both on the variance of the potential outcomes---as in a standard no-interference setting---as well as an additional term ($\Delta_i(w, p^*_{\pi})$) that is a function of net demand and price sensitivity of outcomes and net demand. \footnote{Our finding that the difference in means is consistent for the ADE mirrors general findings in savje2021average; however, the fact that we get a $\sqrt n$ rate of convergence and a central limit theorem depends on our marketplace model.}
Although we have shown that the familiar difference-in-means is asymptotically normal around the ADE, the above result also implies that the usual asymptotic variance derived under the no-interference setting does not match $\sigma^2_D$ in our market equilibrium model, unless $\Delta_i(W_i, p^*_{\pi}) = 0$. This term is zero if there is homogeneity in price derivatives under treatment and control, so that $\nabla_p[y(1, p) - y(0, p)] = 0$. In general, to construct confidence intervals that are asymptotically exact for the population direct effect, we need estimates of the price elasticities that appear in the $\Delta_i(w, p^*_{\pi})$ terms. In the next section, we will show how non-zero price perturbations under Design (ref) allows us to estimate these price sensitivities as well as the AIE, which is not estimable with treatment randomization only.
Finally, we note that Theorem (ref) quantifies the errors of $\hat \tau_{\text{ADE}}$ as an estimator of the population estimand $\tau^*_{\text{ADE}}$. In many settings, however, it is of primary interest to provide inference about the sample direct effect $\tau_{\text{ADE}}$, which is defined in a finite-sized market. Combining Theorems (ref) and (ref), we can verify that
and furthermore that
The asymptotic variance here still depends on the market equilibrium effects; furthermore, it depends on the correlation of $\varepsilon_i(1)$ and $\varepsilon_i(0)$ and so is generally not identified. However, the following result shows that, in an extension of the classic result of neyman1923applications, $\sigma_D$ is a conservative upper bound for $\bsigma_D$ and so confidence intervals built using Theorem (ref) that are exact for $\tau^*_{\text{ADE}}$ will also be conservative for $\tau_{\text{ADE}}$.\footnote{If $\tau_{\text{ADE}}$ is the only estimand of interest, we can adapt results from aronow2014sharp to derive a tighter bound for $\bsigma^2_D$ that is estimable using price perturbations as discussed in the following section; see Appendix (ref) for details.}
We now consider use of randomized individual-level price-shifters as defined in Design (ref) to accomplish further estimation and inferential tasks. These price perturbations can be interpreted as experimenter-created instruments for estimating price derivatives locally. They shift demand and supply in the neighborhood of the market-clearing price, and since they are randomized, they are independent of other variables that determine demand or outcomes.\footnote{The fact that we only consider small, unobstrusive perturbations to the market also means we do not get to observe behaviors at prices far from the large-sample equilibrium $p^*_\pi$, and so cannot identify or consistently estimate full demand curves.} In a typical two-sided market, implementing the experiment requires introducing a small random discount, fee, or subsidy to both producers and consumers for any products where the treatment is expected to significantly affect supply or demand.\footnote{Our theory requires that the size of the price perturbation decreases with sample size. For a fixed market size, as the perturbation size increases, the variance of $\hat \tau_{\text{AIE}}$ decreases, but large price perturbations distort production and would be likely be viewed unfavorably by various market stakeholders. The simplest guidance for choosing the size of the price perturbation is to choose a value of $c$ so that for the sample size of the experiment, the size of the price perturbation is noticeable to market participants but limited to a small percentage of the current market price (e.g. 1-2%). For markets where it is not possible for reputational or legal reasons to introduce any individual-level price variation, the components of $\tau^*_{\text{AIE}}$ can be estimated using price variation across products or time instead, at the cost of introducing additional structural assumptions on the environment.}
In Theorem (ref), we found that $\tau^*_{\text{AIE}}$ can be expressed in terms of certain price elasticities and the direct effect of treatment on net demand. Our first result using randomized price-shifters is that, using data collected under Design (ref), we can produce a simple and consistent plug-in estimator for the indirect effect based on the functional form for $\tau^*_{\text{AIE}}$ derived in Theorem (ref). Let $\bm Y$ be the $n$-length vector of observed outcomes, where $Y_i = Y_i(W_i, P_n(\bm W), + U_i),$ $\bm U$ is the $n \times J$ matrix of price perturbations, and $\bm Z$ is the $n \times J$ matrix of observed net demand, where $Z_i = Z_i(W_i, \bm P_n(\bm W) + U_i)$. The estimator is
where $\hat {\gamma}$ is a $J \times 1$ vector that estimates $[\xi^{\top}_z]^{-1} \xi_y $. The direct effect of the treatment on net demand is estimated via a difference-in-means estimator,
We provide a central limit theorem for our indirect effect estimator below. Its rate of convergence depends on the magnitude of the price perturbations $h_n$, and is always slower than the $\sqrt{n}$-rate obtained for the direct effect. In deriving this result, it is helpful to note that algorithmically $\hat \tau_{\text{AIE}}$ is the product of a rescaled instrumental variables estimator and a differences in mean estimator. We can then use standard results on the asymptotic behavior of IV estimators to guide our analysis.
Under general patterns of interference, inference on the indirect effect is challenging savje2021average, li2022random, and consistent estimators for the variance of the indirect effect are generally not available. There are two reasons why our paper overcomes this difficulty. Although our interference pattern is dense, it is structured in that all interference happens through the market price, and the market price forms by satisfying a score condition. This structure leads to an analytical functional form for the variance of the indirect effect, as given above. Second, with data from a richer randomized experiment that includes small price perturbations, we are able to estimate each component of this variance, and the variance for the indirect effect.
We estimate $\sigma_D^2$ and $\sigma_I^2$ via natural plug-in estimators. Let $n_w$ denote the number of units with $W_i = w$. The variance of the direct effect can be estimated as
where, for $w \in \{0, 1\}$, $\hat {\xi}_{yw}$ is a $J \times 1$ vector estimated from regressions of $Y_i$ on $U_i$ using only observations with indices in the set $\{ i: W_i = w \}$, and $\hat {\xi}_z $ is a $J \times J$ matrix which is computed via regressions of net demand $Z_{ij}$ on price perturbations $U_i$ for $j \in \{1, \, \ldots, J\}$ and $\hat {\xi}_y$ is a $J \times 1$ vector computed via a regression of $Y_i$ on $U_i$. Meanwhile, for the variance of the indirect effect, the natural plug-in estimator is\footnote{In Appendix (ref), we also describe a second order adjustment to $\hat \sigma^2_I$ that is asymptotically negligible but yields better coverage under in our experiments when the sample size is moderate.}
These variance estimates can then be paired with our asymptotic normal approximation to build confidence intervals for the direct and indirect estimates; for example, for a confidence level of 95%, the confidence intervals are constructed as
The following result verifies validity of these variance estimators.
So far, we defined a potential outcomes model that captured equilibrium interference but reduced to the Neyman-Rubin model without interference. We then used this model to define average direct and indirect treatment effects under general treatment allocation rules. In the previous section, we proposed estimators for these effects that relied on data generated from randomized trials where the treatment was assigned with constant probability. In this section, we return to general treatment rules and discuss heterogeneous effects and optimal targeting when there is interference through an equilibrium statistic.
The planner controls the (potentially randomized) treatment allocation function $\pi(\cdot): \mathcal X \rightarrow [0, 1]$ where $\pi(x)= \PP{W_i = 1 \cond X_i = x}$. The conditional expectation functions are defined as $y(w, p, x) = \mathbb E[ Y_i(w, p) \cond X_i = x]$ and $z(w, p, x) = \mathbb E[Z_i(w, p) \cond X_i = x]$. We are interested both in quantifying how relevant treatment effects vary with $x$, and how this information can be used to guide choices of $\pi(\cdot)$ that achieve better outcomes.
Under SUTVA, the conditional average treatment effect (CATE) is defined as $\tau(x) = \mathbb E[Y_i(1) - Y_i(0) | X_i = x]$ imbens2004nonparametric. Without interference, the optimal unconstrained targeting rule allocates treatments only to those with a positive CATE manski2004statistical, or potentially to those whose CATE exceeds a budget-informed threshold bhattacharya2012inferring. Such treatment assignment rules, however, are no longer optimal under interference---and the CATE is no longer even well defined.
We begin this section by proposing two definitions of conditional estimands that play a similar role to the average direct and indirect effect, but for targeted treatments: The Conditional Average Direct Effect (CADE) and the Conditional Average Indirect Effect (CAIE). The CADE
is the expected effect of treating an individual with covariate value $x$ on their own outcomes in a system of $n$ individuals. The CAIE
is the expected effect of treating an individual with covariate value $x$ on everyone else's outcomes in a system of $n$ individuals. These estimands are non-random quantities that depend both on the population distribution and on the market size. They are policy-relevant for settings where we expect to deploy a policy in a market of similar size and composition to the observed market. When interference is removed, the CADE is equal to the CATE, and the CAIE is zero.
We next connect these definitions of heterogeneous effects to conditional marginal effects; this result is valid under general patterns of interference. In Section (ref), we reported the results of hu2021average, showing the sum of $\tau_{\text{ADE}}$ and $\tau_{\text{AIE}}$ is equal to the effect on average outcomes of a marginal increase in each individual's treatment probability. Proposition (ref) extends this result to our proposed heterogeneity measures.
A policymaker may be interested in taking advantage of heterogeneous responses to treatment in the population by implementing a targeting rule, rather than a treatment rule with uniform probability. Proposition (ref) shows that the sum of the CADE and the CAIE is relevant for making the decision on which group's treatment probability to increase, and which to decrease, when the objective is maximizing expected outcomes.
The next step, in Theorem (ref), is to derive the population versions of the CADE and CAIE. These are purely population estimands, defined as the limit of $\bar \tau_{\text{CADE}}$ and $\bar \tau_{\text{CAIE}}$ as the market size grows to infinity. We will show below that these quantities are relevant to treatment targeting in a market that is larger but otherwise similar to the observed market. The population estimands are also useful for deriving estimators that are consistent for both the large-market and finite-market estimands. Theorem (ref) shows that the population estimands have a simple and interpretable form and implies that estimators for one class of estimands are consistent for the other.
The population estimands have a simple form that suggests estimation strategies for the estimands $\bar \tau_{\text{CADE}}(x)$ and $\bar \tau_{\text{CAIE}}(x)$. The limit of the CADE is the direct treatment effect on outcomes conditional on $x$, holding the equilibrium price fixed. The limit of the CAIE is the direct treatment effect on net demand conditional on $x$, multiplied by an elasticity correction that does not depend on $x$. The augmented randomized experiment from Design (ref) can be used to estimate the elasticity corrections and the conditional average treatment effects required to estimate $ \bar \tau_{\text{CADE}}(x)$ and $\bar \tau_{\text{CAIE}}(x)$.
For the elasticity corrections, the market price is an aggregate statistic, so individuals with different covariates all respond to the same market prices. A change in net demand of a given size always has the same impact on the market price. Although individuals' responses to the treatment through outcomes or net demand are heterogeneous, the elasticity correction that transforms the $\tau_{\text{CADE}}^{*,z}(x)$ to $\tau_{\text{CAIE}}^*(x)$ is unconditional. This implies that a group of individuals' effect on the system depends on their covariates only through conditional direct effects. Estimators for $\xi_y$ and $\xi_z$ from the previous section of the paper apply directly.
Below, we show that the $k$-nearest neighbor estimator is consistent for $\tau_{\text{CADE}}^{*}(x)$; the same result also immediately holds for $\tau_{\text{CADE}}^{*,z}(x)$. The salient fact in establishing this result is that the $k$-nearest neighbor estimator is a difference-in-means estimator that has been localized in $X$-space; and the proof suggests that other standard CATE estimators that are effectively localized difference-in-means estimators, such as the causal trees of athey2016recursive, are also consistent for the CADE. In our experiments, we use causal forests as implemented in the grf package of athey2019generalized to estimate the CADE.
The results of the previous section imply that estimates of $\tau_{\text{CADE}}^*(x)$ and $\tau_{\text{CAIE}}^*(x)$ could be used to (locally) optimize an unconstrained targeting policy. There are a variety of constrained targeting rules that may be of interest. Here, we focus on one specific question of this type, namely what is the optimal treatment assignment policy that does not move equilibrium prices relative to those seen in the experiment? There are two reasons to consider this class of targeting rules. First, a conservative policymaker may be reluctant to significantly modify the equilibrium---even if it is beneficial on average to individuals---and so they may want to know how much they can improve outcomes without changing the equilibrium. Second, from a practical point of view, we find that the answer to this question admits a simple econometric strategy, and can be answered without needing to estimate price elasticities and without recourse to an augmented experimental design. Instead, the optimal equilibrium-stable targeting rule (and its performance) can be estimated using data from a baseline RCT following Design (ref), as long as the RCT collects outcome and relevant supply and demand data at an individual level.
The optimization problem for the equilibrium-stable policy is to find a new policy $\nu(\cdot)$ that maximizes expected outcomes in the population, while maintaining the population equilibrium price obtained under the policy $\pi(\cdot)$, i.e., $p^*_{\pi}$. The value of a given policy $\nu(\cdot)$ is:
where $\EE[\nu]{Y_i}$ denotes expected rewards under the considered new policy $\nu(\cdot)$. The optimal policy is defined as
where $p^*_{\nu}$ is the population equilibrium price under $\nu(\cdot)$.
When the targeting policy is restricted to have the same equilibrium effect as a baseline policy, then solving for that optimal policy takes the form of a linear optimization problem.
The above reveals that we can solve for the optimal price-stable policy without access to price elasticities. Thus, we can obtain a plug-in estimate for the optimal rule using CADE estimates derived from an RCT without price perturbations as in Design (ref); recall that the CADE itself can be identified without price perturbations (Theorem (ref)). Furthermore, following dantzig1951fundamental, we can verify that its solution takes on a simple parametric form: The optimal price-stable policy is a thresholding rule that compares the CADE for the outcomes against a shadow cost of net demand effects $ c \cdot \tau_{\text{CADE}}^{*,z}(x)$, where $c \in \RR^J$ can be interpreted as a shadow-price vector.
We can estimate the optimal equilibrium-neutral rule by finding a rule that meets an empirical version of the conditions in Theorem (ref), where $\tau^{*}_{\text{CADE}}(x)$ and $ \tau_{\text{CADE}}^{*,z}(x)$ are replaced with appropriate estimators. With consistent estimators of the conditional average direct effects, such as the estimator proposed in Theorem (ref), then the estimated equilibrium-neutral rule is consistent for the population equilibrium-neutral rule under some smoothness assumptions on the distribution of conditional average treatment effects. A description of the estimation procedure and a formal consistency result is provided in Appendix (ref).
The targeting rule in Theorem (ref) takes on a particularly simple form when we only have $J = 1$ good, and the intervention has a crowding-out type structure where direct effects are positive but spillovers are negative, $\tau^{*}_{\text{CADE}}(x) > 0$ and $\tau_{\text{CADE}}^{*,z}(x) < 0$. In this case, the optimal price-stable targeting rule will be of the form
with potential random tie-breaking at the cutoff. In other words, we want to give treatment to units whose direct effects are large relative to corresponding spillovers; we then pick $c$ to satisfy the price-stability constraint. This algorithmic structure is familiar from the literature on cost-sensitive treatment targeting sun2021treatment.
In this section, we estimate a simple model of the direct and indirect impacts of a conditional cash transfer on children's health outcomes in the Philippines using data from filmer2023. We use data simulated from the model to illustrate the performance of the average direct and indirect treatment effect estimators and the estimated targeting rule proposed in this paper. filmer2023 use a village-level, cluster-randomized experiment to analyze the effect of the Pantawid conditional cash transfer program on young children's health outcomes. The program provides a monthly sum for eligible families, where the amount depends on the number of children in the household. Families are eligible if they meet a proxy means test and have either children under 14 years of age or a pregnant woman in the household. The authors show that the health of eligible children in treated villages is better than in control villages. However, children whose families are ineligible for the transfer are worse off in treated villages compared to control villages.
The authors argue that this phenomenon is due to spillover effects, which largely arise from equilibrium effects in the market for perishable protein sources. Specifically, the authors provide evidence that eggs, which are a commonly consumed perishable protein source in the Philippines, have higher prices in treated compared to control villages where saturation of eligible households is high. This price effect is explained by the increase in demand for protein from eligible families and inelastic supply in remote villages. The higher prices in treated villages lead ineligible children to consume less protein, which explains a significant proportion of the negative effect of the cash transfer program on ineligible children. It thus appears that Assumption (ref) is reasonable, in that the spillovers described in detail by the authors are restricted to those that occur through market prices.
Under the assumption that markets in remote villages are isolated from one another, the village-level randomized experiment allowed filmer2023 to recover treatment effects that include equilibrium effects. The total effect $\tau_{\text{TOT}}$ could be estimated using data from the cluster-randomized experiment by comparing the average outcomes of all children in treated compared to control villages. In this paper, in contrast, our focus is on randomized experiments where only a single market is observed, but randomization is possible at the household level. While our econometric setting is different from that of filmer2023, we can use their dataset to conduct a calibrated evaluation of our approach. We first use moments from the cluster-randomized experiment in filmer2023 to estimate a model of supply, demand, and children's outcomes in a single remote village market. Then, we use this model to simulate a single-village, unit-randomized experiment, and assess the ability of our estimators of the average direct and indirect treatment effects to recover equilibrium effects from this simulated experiment.
Let $i \in \{1, \, \ldots, n \}$ index the people in the village and $h \in \{1, \, \ldots n_h \}$ index the households in the village. For each $h \in \{1, \, \ldots, n_h \}$, $A^h$ is a set of integers containing the indexes of the people that are part of household $h$ and $C^h$ is a set of integers containing the indexes of the children aged 0-5 that are part of household $h$. $h(\cdot)$ is a map that provides the household index given a person's index. The binary treatment $W_h \in \{0, 1\}$ is assigned at the household level. $E_h \in \{0, 1\}$ indexes the eligibility for a household. We assume that individual demand in eggs per week is determined by the following linear equation, where $p$ is the price of eggs: \[ D_i(W_h, p) = \theta_{d01} \cdot E_{h(i)} + \theta_{d00} \cdot ( 1- E_{h(i)}) + \theta_{dw} \cdot W_{h(i)} \cdot E_{h(i)} + \theta_{dp} \cdot p + \varepsilon_{d, h(i)} + \nu_{d, i}(W_i), \] where the household and individual-level error terms are normally distributed. Demand for eggs is increased for eligible adults and children in households that receive the Pantawid conditional cash transfer. Demand also responds to the market price of eggs.
We assume that the aggregate supply per person in the village is responds linearly to prices, $S(p) = \theta_{s0} + \theta_{sp} \cdot p$. This formulation assumes that the village supply is not directly affected by the treatment; in filmer2023, the authors argue that egg supply in the Philippines is dominated by larger commercial producers, which makes this a reasonable assumption.
Our target outcome is the height-for-age Z-score for children between 0 and 5 years of age. For our evaluation, we assume that these outcomes are generated as \[ Y_i (W_{h(i)}, p) = \theta_{y01}E_{h(i)} + \theta_{y00} (1 - E_{h(i)}) + \theta_{yd} D_i(W_{h(i)}, p) + \theta_{yw} W_{h(i)} + \varepsilon_{y, h(i)} + \nu_{yi}(W_i), \] where the household and individual-level error terms are normally distributed. Consumption of a perishable protein source increases child-level health outcomes. The conditional cash transfer also has impact on health metrics through other factors, which is captured by $\theta_{yw}$.
For the purpose of our analysis, we aggregate individual level outcomes at the household level:
The factor $\frac{n_h}{n}$ ensures that average treatment effects on household-level demand can be interpreted as the treatment effect on eggs consumed per-person in the village. The factor $\frac{n_h}{n_c}$ ensures that average treatment effects on household-level outcomes can be interpreted as the treatment effect per child aged 0 to 5 in the village. The realized egg price in the market matches aggregate supply and demand: \[ \frac{1}{n_h} \sum \limits_{h=1}^{n_h} D_h(W_{h(i)}, P(\bm W)) = S(P(\bm W)) \] Given a treatment vector $\bm W$, the realized household-level outcomes are $Y_h(\bm W) = Y_h(W_h, P(\bm W))$ and household level net demand is $Z_h(\bm W) = D_h(\bm W, P(\bm W)) - S(P(\bm W))$.
The 10 parameters in this model are estimated by matching 10 moments from a subset of the data in filmer2023, restricted to households in villages above the 70th percentile of the remoteness index defined in the paper, where equilibrium effects were most meaningful. The 10 moments are:
The total treatment effect here is the difference in average outcomes (or demand) in treated versus control villages. The estimation details and the estimated parameters of the model are reported in Appendix (ref).
Figure (ref) shows that aggregate demand in the village is decreasing in price, but is shifted upward by the cash transfer when eligible families are treated. Supply is increasing in price but is inelastic enough to explain the shift in prices observed in the data in treated villages. Figure (ref) shows the distribution of $\tau_{\text{ADE}}$ and $\tau_{\text{TOT}}$, simulated using repeated samples of a village of 1000 households from the model. In this linear model, $\tau_{\text{TOT}} = \tau_{\text{MPE}}$. An RCT estimates the direct effect, which is more than 50% higher than the total effect. Figure (ref) explains why the direct effect and the total effect are so different in this setting. The treatment raises demand for eggs among treated families, which impacts the market-clearing price for the treated market compared to the control market. Holding food prices fixed, the effect of the treatment is a large positive increase in health outcomes. However, when the effect on food prices is taken into account, which impacts both eligible and ineligible children, the total treatment effect is much lower.
In Table (ref), we show that both the direct and the indirect effect are estimable in finite samples using our proposed unit level augmented experiment. The size of the price-shifters in the simulated experiment is 0.15 Philippine pesos per egg, which is less than 2.5% of the market price. We report the results of a Monte Carlo simulation with 1,000 repetitions and a sample size of 2,000 households to evaluate the bias, variance, and coverage properties of the estimators and confidence intervals for $\hat \tau_{\text{ADE}}$ and $\hat \tau_{\text{AIE}}$. The table illustrates that the bias of the average direct effect and indirect effect estimator based on the augmented randomize experiment are low in finite samples. Furthermore, the coverage when $\hat \sigma_D$ and $\hat \sigma_I$ are used to construct confidence intervals in finite samples is slightly above the asymptotic confidence level of 95% for both population estimands. As expected from Theorem (ref), intervals that target $\tau^*_{\text{ADE}}$ are conservative for $\tau_{\text{ADE}}$. For the AIE, in simulations the coverage for the sample estimand is also more conservative than for the population estimand. In this simple linear model, $\nabla_p y(1, p) = \nabla_p y(0, p)$, so $\Delta_i(W_i, p) = 0$ in (ref), and confidence intervals for differences in means estimation that ignore price fluctuations are asymptotically valid. The augmented randomized experiment, however, is required to estimated the AIE in this model and to build asymptotically valid confidence intervals for the ADE in more general settings with additional heterogeneity and non-linearity.
In the setting of filmer2023, there may be heterogeneity in both outcome and demand effects that is correlated with observed pre-treatment covariates. If so, a targeting rule can improve outcomes compared to a randomized rule that induces the same equilibrium. Since the replication package did not contain data on pre-treatment household characteristics that can be matched to post-treatment data, we cannot estimate heterogeneous treatment effects directly. Instead, we augment the model in Section (ref) with an additional 10 generated household-level covariates that are observed by the planner, and a set of four heterogeneous household types that are correlated with the generated covariates but unobserved. These household types are generated purely for this simulation exercise and are not calibrated to real data. Household Type A has the same data-generating process in the previous section. Household Type B is especially focused on their children, and purchases enough food that outcomes improve even more than for Type A. For Household Type C, which makes up a small percentage of the households, increased cash is not beneficial for child health outcomes, as it makes consuming some welfare-decreasing product possible for adults, and the budget share of food of the household decreases. For Household Type D, there is a small positive effect on children's health outcomes, but a large effect on demand for the perishable protein source; in these houses, adults consume most of the additional food that is purchased with the cash transfer. The details of the augmented model are in Appendix (ref). Then, we estimate the optimal equilibrium-stable targeting rule on data simulated from this model.
In Figure (ref), we illustrate the optimal rule estimated using a sample of 5,000 households. Ignoring equilibrium effects, a good targeting rule would allocate treatment to those individuals who are estimated to have a positive CADE. Here, however, this rule would result in a larger impact on perishable food demand compared to the RCT since, on average, those who increase their children's health using the cash transfer also increase their consumption of perishable food. In contrast, the optimal rule that respects the equilibrium constraint allocates treatment to those with $\hat \tau_{\text{CADE}}(X_i) > c\hat \tau^z_{\text{CADE}} (X_i)$. We can see that for the most part, it is those with a positive CADE that are treated. However, households of Type D consume a lot of additional perishable food without raising a child's health outcomes very much. Dropping some of these households from the equilibrium-stable targeting rule allows the planner to target more households from Type B while keeping the equilibrium food price stable.
We then evaluate the optimal equilibrium stable rule by estimating the rule using the procedure in Section (ref) on a training sample of 5,000 households, and then evaluating the expected value of that rule numerically on a separated simulated dataset of 5,000 households. The results are reported in Table (ref). Note that our health outcome, which is average Z-Scores of children under 5 years in the simulated village, is negative since children living in remote and isolated villages in the Philippines are smaller than average. The closer Z-scores are to zero, the more a targeting rule has improved health outcomes for young children in the simulated village.
Compared to Target-Random, Target-Direct avoids treating some of the few households for which the cash transfer is not beneficial, which improves outcomes by 8.9%. However, for equilibrium stability, $\hat \alpha < 1$, and the rule does not distinguish between households based on their demand for the perishable protein source. Target-Optimal has improved health outcomes since it treats more households with a large positive direct effect on outcomes and avoids treating households for which the treatment has a small impact on outcomes but a large impact on demand. It improves health outcomes by 17% compared to Target-Random.
Analyzing the performance of randomized control trials in settings with equilibrium effects is needed given the rapid growth of experimentation both in practice and in research studies. The Neyman-Rubin framework that relies on SUTVA rules out interaction effects that can have an important impact on decision-relevant treatment effects. A parametric structural model may capture a variety of complex equilibrium effects, but is not robust to misspecification, which can be problematic when individuals behave in complex and heterogeneous ways. A model of treatment effects under general patterns of interference is intractable without clustering or other assumptions.
This paper shows that it is fruitful to marry ex-ante knowledge about the structure of an economic environment with a non-parametric stochastic model of treatment effects under interference. This leads to a characterization of asymptotic properties of treatment effects and estimators of those treatment effects based on new and existing experimental designs that are robust to a wide range of modeling choices. Results on estimation, inference, and optimal targeting in complex environments with some economic structure imposed can then be easily compared with the large body of work that studies causal inference under SUTVA.
Our results on the direct effect and on targeting rely on the assumption that all spillovers are mediated by the prices of a finite number of traded goods; our results for the indirect effect and for inference further rely on us being able to exogenously apply small shifts to the prices that different market participants are exposed to. As discussed in the previous section, such assumptions may be relevant to experiments run in communities with self-contained markets. aouad2024digitized describe an experimental setting where researchers open a subsidy store that they then use to randomize subsidy eligibility at the unit level; and experiments of this type could plausibly also integrate small, random price perturbations. Another class of applications our methods are a natural fit for are online platforms for ride sharing, freelance labor, short-term rentals, etc. Price-mediated spillovers are of central importance when experimenting in such systems; for example, in ride sharing, both drivers and riders respond to the average price of a mile and/or minute of transportation, and both supply- and demand-side interventions may alter the market-clearing price for transportation. Furthermore, technology companies running such platforms have already documented willingness to run experiments comparable to the price perturbations we describe, e.g., via random discounts or bonuses castillo2023,holtz2024reducing. There remains, however, a large class of settings where our methods cannot be applied, e.g., when spillovers are not mediated via observed equilibrium quantities cai2015social,manski1993identification, or in matching problems where equilibrium dynamics play a key role li2023experimenting.
There are a variety of avenues for future work possible. One limitation of our approach is that we analyze the large sample limit of the market place where the number of suppliers and the number of buyers grow large; an analysis of experiments in settings where firms have significant market power would likely require different techniques. A related potential direction for extending our results would be to allow for the market to depend on prices of a continuum of goods, e.g., localized prices that vary continuously in space. In general, extending our results to a broader class of equilibrium mechanisms would be of considerable interest.
{0.2pt plus 0.3ex}
\singlespacing