Identifying Socially Disruptive Policies

Identifying Socially Disruptive Policies

Introduction

Many policies are socially disruptive in that they alter a substantial fraction of agents' social or economic connections. Since networks determine a wide range of economic activities, disrupting them can lead to unintended consequences. For example, carrell2013natural study a change in classroom composition that was supposed to improve academic performance but instead segregated students which exacerbated inequality. barnhardt2017moving analyze an antipoverty program that was intended to provide economic opportunity but instead isolated participants which led to financial insecurity. Both policies were well intentioned but, because they were socially disruptive, ultimately hurt the agents that they were designed to help.\looseness=-1

In light of these and other examples, a growing literature recommends measuring and accounting for social disruption when evaluating the welfare impact of a policy banerjee2021changes,jackson2021inequality. But identifying social disruption from data is not always straightforward in practice. Economists typically characterize the disruptive impact of a policy by running dyadic regressions, which amounts to comparing the average number of connections between agents with and without the policy. While easy to implement, these regressions generally understate the total amount of social disruption. The reason for this is that economic policies usually have heterogeneous effects: they create some connections and destroy others. If the amount of created connections is roughly the same as the amount of destroyed connections, then the average difference will be small, even when the total number of connections affected by the policy is not.\footnote{This disruption is policy relevant. It represents actual relationships that are upended, requiring time and resources to replace. The literature shows that new connections may be of lower quality, associated with less trust, communication, peer influence, etc. As a result, this disruption impacts welfare, even when the average number of connections with and without the policy is similar.} \looseness=-1

In this paper, we go beyond comparing averages and consider the problem of separately identifying the amount of connections created and the amount of connections destroyed by a policy. We focus on a research design that is popular in the literature. Agents are first randomly (or as good as randomly) assigned to one of two groups. The policy is implemented in one of the groups but not the other. The agents in each group then interact and form network connections. Versions of this design are considered by carrell2013natural,feigenberg2013economic,graham2014complementarity,cai2015social,bajari2021multiple,banerjee2021changes,comola2021treatment,comola2023interplay, hess2021development,johari2022experimental. \looseness=-1

Our first contribution is to propose a new framework to characterize the impact of a policy on the structure of a network in a randomized experiment. We use the classical implication of random assignment, that the agents in the group subjected to the policy form connections that are representative of what the agents in the group not subjected to the policy would have realized had they been subjected to the policy. Formalizing this condition in the context of network data is not standard in econometrics, however, and our framework builds on ideas from the graph theory and operations research literatures lovasz2012large,cela2013quadratic. \looseness=-1

Our second contribution is to derive new identification results. We find that the amount of connections created or destroyed by a policy is partially identified. Sharp bounds on the identified set are given by a quadratic assignment problem (QAP), but these bounds are analytically and computationally intractable. Instead, we propose conservative outer bounds based on intersecting several relaxations of the QAP. These bounds are formed by simple rearrangements of the eigenvalues of the conditional distribution of network connections between pairs of agents identified from the experiment, which are relatively straightforward to analyze and compute. Though not the focus of our paper, we also show how to consistently estimate the bounds and construct valid confidence intervals in Online Appendix Section D.3, focusing on a class of network formation models called dyadic regression models that are popular in the network economics literature, including the motivating examples referenced above graham2020network. R code for implementation can be found at \url{https://github.com/yong-cai/MatrixHTE}. \looseness=-1

We demonstrate our bounds with an empirical illustration using data from banerjee2021changes. In the illustration, villages participate in a microfinance program and the network connections are informal risk sharing links between households. The authors compare the average number of connections between villages that do and do not participate with various dyadic regression models. They find that participation is associated with a roughly one percentage point decrease in connections between households. We find disruptive effects that are sixteen to twenty-three times larger using our bounds. We conclude that the microfinance program is substantially more disruptive than what is indicated by the authors' regression estimates.\footnote{The reason why we find a much larger effect size than banerjee2021changes is that their regressions only approximate the total amount of connections impacted by the microfinance program if the program's effect is monotonic: e.g. it only creates connections or it only destroys connections. See Section 6.4.3 below. Our results indicate that such a monotonicity assumption does not hold in this setting. }\looseness=-1

Our paper relates to two relatively new literatures on endogenous network formation and partial identification with network data de2016econometrics,bramoulle2020peer,graham2020network,molinari2020microeconometrics. Most of this work focuses on recovering the structural parameters of a social interaction or network formation model. Two exceptions we know of are chandrasekhar2011econometrics,thirkettle2019identification. While these authors focus on identifying centrality measures from sampled networks, our interest is in social disruption from an experiment. \looseness=-1

Our paper also relates to an older literature on Fr\'echet-Hoeffding-Makarov bounds hoeffding1940masstabinvariante,frechet1951tableaux,makarov1982estimates and quantile treatment effects doksum1974empirical,lehmann1975nonparametrics,whitt1976bivariate. See, for instance, manski1997mixing,manski2003partial,heckman1997making,bitler2006mean,firpo2007efficient,fan2010sharp,tamer2010partial,abadie2018econometric,masten2018identification,masten2020inference,firpo2019partial,frandsen2021partial for work in econometric program evaluation. However, the structure of our identification problem is fundamentally different, introducing challenges not present in this literature. Intuitively, what distinguishes our framework is that while agents are individually assigned to policies, connections are measured between pairs of agents. It turns out that this distinction substantially alters the identification problem. Standard results are not generally valid and standard tools when naively applied often fail to identify any social disruption. We provide intuition as to how our problem is different and why our methodology is appropriate in Section 2. Our formal framework is in Section 3 and identification results are in Sections 4 and 5. The empirical illustration is in Section 6. Proofs are in the appendix. Additional details and results are in the online appendix. \looseness=-1

An illustration of the main identification problem

In this section, we provide a simplified illustration of the main identification problem, deferring the formal framework and identification results to Sections 3-5. We focus on identifying the magnitude of connections destroyed by a change in policy. The identification problem is similar in spirit to that of bounding the joint distribution of two random variables using their marginals originally considered by hoeffding1940masstabinvariante,frechet1951tableaux. However, our problem has a fundamentally different structure. We give an example where naively applying the bounds from this literature fails to identify any social disruption, but our methodology does. \looseness=-1

A simplified setup

To learn about the disruptive impact of a new policy, we consider an experiment where we randomly assign $N$ agents to a treatment group and $N$ agents to a control group. We implement the new policy in the treatment group and maintain the status quo policy in the control group. \looseness=-1

The mechanics of the identification problem do not depend on what the two policies actually do. It only matters that they somehow determine the agents' network connections. So, for example, a policy could be that every agent in the group participates in an antipoverty program, no agent participates, a random subset of agents participate, a nonrandom subset of agents are encouraged but not required to participate, etc. Once the new and status quo policies are chosen and implemented, the $N$ agents in the treatment group interact and form one network. The $N$ agents in the control group interact and form another network. We take the size of the two groups to be the same to simplify the illustration, but this is straightforward to relax. \looseness=-1

We use potential outcome notation to describe the identification problem. Let policy $1$ refer to the new policy, policy $0$ refer to the status quo, group $1$ refer to the treatment group, and group $0$ refer to the control group. Then $Y_{ij,t}(s)$ is the potential connection between agents $i$ and $j$ in group $t$ under policy $s$. That is, $Y_{ij,t}(s)$ describes what the connection between agents $i$ and $j$ in group $t$ would be if policy $s$ were implemented in that group. Since, in the experiment, policy $s$ is implemented in group $s$, $Y_{ij,t}(s)$ is observed if $s = t$ and is unobserved if $s \neq t$. To simplify our illustration, we assume that the networks are unweighted and undirected so that the matrix $Y_{s}(t) = \{Y_{ij,s}(t)\}_{i,j = 1}^{N}$ is symmetric with $\{0,1\}$-valued entries. Weighted networks are accommodated in Section 3 by thresholding. Directed networks are accommodated in Online Appendix Section D.1 by symmetrization. \looseness=-1

In this simplified setup, we assume that the potential outcomes are not random. In practice, researchers often specify stochastic models of network formation where a random connection between pairs of agents reflects the fact that network formation is the result of human decision making which is naturally indeterminate. We ignore the issue of random networks to simplify the presentation of the main identification problem in this section, but incorporate it into our formal framework in Section 3.\looseness=-1

To illustrate the main identification problem, we take as the parameter of interest the number of network connections between the $N$ agents assigned to the control group that would be destroyed by implementing the new policy in that group. That is,

$Y_{ij,0}(0)$ is observed but $Y_{ij,0}(1)$ is unobserved and without additional assumptions could take any value in $\{0,1\}$. It follows that the identified set for ((ref)) is

This set is typically too large to be informative in practice, and so we refine it below by using the assumption that the agents are randomly assigned to the treatment and control groups. \looseness=-1

The main identification assumption

Our first main contribution is to propose a condition that formalizes how the random assignment of agents to groups refines the identified set. Our main identification assumption is that $Y_{0}(1)$ and $Y_{1}(1)$ are weakly isomorphic. We defer a formal definition of this assumption to Section 4.1, but, intuitively, it says that the configuration of network connections between agents assigned to the treatment group describe how the agents assigned to the control group would be connected if that group were assigned the new policy instead of the status quo. This assumption is a network analog of the conventional assumption that the entries of $Y_{0}(1)$ and $Y_{1}(1)$ have the same empirical distribution, which is a strong but ubiquitous implication of random assignment, see Chapter 7.3 of manski2009identification for a textbook discussion. \looseness=-1

As in a conventional experiment, randomization only generally implies that $Y_{0}(1)$ and $Y_{1}(1)$ are weakly isomorphic in expectation. However, to illustrate the main identification problem of our paper, we will, for this simplified illustration, make the strong and unrealistic assumption that it holds exactly in the realized experiment. \looseness=-1

Formally, the main identification assumption is that there exists an $N\times N$ permutation matrix $\Pi$,\footnote{A permutation matrix is a square matrix with $\{0,1\}$-valued entries. Every row and column sums to $1$.} unknown to the researcher, such that \looseness=-1

In words, condition ((ref)) says that there exists a one-to-one match between agents in the two groups such that the counterfactual outcomes of the agents in group $0$ are given by their matches in group $1$. It could be rationalized by a specific experimental design conducted on $N$ pairs or “clones” of agents. One member of each pair is randomly assigned to each group and the counterfactual connection between agents in the control group is given by their clones in the treatment group. If the researcher knows which pairs of agents are clones, then they can compute ((ref)) by simply substituting $Y_{ij,0}(1)$ with $Y_{c_{i}c_{j},1}(1)$ where $c_{i}$ is the identity of $i$'s clone. But the researcher has forgotten this information so that, in principle, any matching between the agents of the treatment and control groups (as represented by some permutation matrix) could be the correct one. \looseness=-1

If the researcher conducts such a clone experiment, then ((ref)) holds exactly in a finite sample. However, outside this specific setting, randomization does not generally imply this condition. Instead, it is an approximation to what randomization does in large samples. The idea that randomization can be characterized by an approximate matching is not original to our paper: an analogous condition plays a key role in the identification arguments of the Fr\'echet-Hoeffding-Makarov bounds and quantile treatment effects literature. See whitt1976bivariate,heckman1997making for detailed discussions. What is new in our setting is that the quadratic structure of ((ref)) makes the problem of identifying social disruption with network data fundamentally different. We discuss this complication in Section 2.2.2 below. \looseness=-1

The identified set under the main identification assumption

Though condition ((ref)) is intended to be an approximation, to illustrate its identifying content in this section we suppose that it holds exactly. Plugging ((ref)) into ((ref)) implies that the number of links destroyed by the policy is \looseness=-1

The substitution solves the initial problem that $Y_{0}(1)$ is not known because both $Y_{1}(1)$ and $Y_{0}(0)$ on the right-hand side are observed. However, the right-hand side now depends on the unknown $\Pi$. Since, under ((ref)), any permutation matrix suggests a number of destroyed links that is consistent with the observed network connections, the identified set is \looseness=-1

The identified set is typically uncomputable

It is straightforward to compute the identified set ((ref)) in our toy example because $N$ is small. However, computing the identified set is not possible in many cases of interest. This is because the problem of finding the largest or smallest element of ((ref)) is equivalent to solving a quadratic assignment problem which is strongly NP hard in theory and practically uncomputable for instances with more than a few dozen agents. See Section 1.5 of cela2013quadratic. \looseness=-1

Our second main contribution is to instead propose tractable outer bounds that are both informative about social disruption and computationally feasible even for large networks. Intuitively, it is hard to compute sharp bounds on ((ref)) because searching over permutation matrices is difficult. Our bounds instead search over orthogonal matrices.\footnote{An orthogonal matrix is a square matrix where the inner product of any row or column with itself is $1$ and the inner product of any two distinct rows or two distinct columns is $0$.} To illustrate this idea, we replace ((ref)) with

There are two reasons for this substitution. First, because all permutation matrices are orthogonal, ((ref)) contains ((ref)) and so any bounds on ((ref)) will also be valid for ((ref)). Second, solving for the smallest and largest element of ((ref)) is relatively straightforward: the minimum is $\sum_{r=1}^{N}\lambda_{r}(1)\lambda_{N-r}(0)$ and the maximum is $\sum_{r=1}^{N}\lambda_{r}(1)\lambda_{r}(0)$ where $\lambda_{r}(s)$ is the $r$th largest eigenvalue of $(1-Y_{s}(s))^{s}Y_{s}(s)^{1-s}$. See Lemma 2 in Appendix Section A.3 for a proof.\footnote{The idea of bounding a QAP by searching over orthogonal matrices was originally proposed by finke1987quadratic, however applying this logic to our general setting is not straightforward and requires arguments not typical of the QAP literature. See Section 4.1 for a discussion. } \looseness=-1

Our bounds can be more informative than conventional methods

Conventional methods such as computing the difference in the number of connections or the Fr\'echet-Hoeffding bounds are not generally effective at identifying social disruption.\footnote{Typically the literature runs dyadic regressions. They regress the magnitude of the connection between a pair of agents on an indicator for whether the agents are subjected to the new policy and additional covariates. The coefficient on the policy indicator is then used to measure social disruption. Without covariates, this is equivalent to computing the difference in the number of connections. Including covariates does not generally make these regressions informative about social disruption. See our empirical illustration in Section 6.} The difference in the number of connections, $\frac{1}{2}\sum_{i,j = 1}^{N}Y_{ij,0}(0) - \frac{1}{2}\sum_{i,j = 1}^{N}Y_{ij,1}(1)$, is also the number of connections destroyed by the new policy minus the number of connections created. It is only a good approximation of the number of destroyed connections if the number of created connections is close to zero, which is rare in practice. \looseness=-1

The Fr\'echet-Hoeffding lower bound on ((ref)) is $\max(\frac{1}{2}\sum_{i,j=1}^{N}Y_{ij,0}(0)- \frac{1}{2}\sum_{i,j=1}^{N}Y_{ij,1}(1),0)$. The upper bound is $\min(\frac{1}{2}\sum_{i,j=1}^{N}(1-Y_{ij,1}(1)),\frac{1}{2}\sum_{i,j=1}^{N}Y_{ij,0}(0))$. These bounds are valid in that the number of destroyed connections is necessarily between them, but they are not generally informative. The lower bound is essentially the difference in the number of connections and is zero if the policy creates at least as many connections as it destroys. The upper bound is large if there are many pairs of agents that are connected in the control group and many pairs of agents that are not connected in the treatment group. Both are common in practice. \looseness=-1

The problem with these methods is that they do not use all of the information available in condition ((ref)). They only use the relatively weak implication that $Y_{0}(1)$ and $Y_{1}(1)$ have the same number of connections. Our bounds often perform better because there can be important identifying information in the former restriction that is not in the latter. \looseness=-1

The performance of our methodology is not limited to the toy example. Section 6 provides empirical evidence showing that our bounds can be much more effective at identifying social disruption than conventional methods in a real world setting.\looseness=-1

Main framework

In this section, we describe our framework. We first define the parameters of interest and then specify a model for the data. We focus on undirected networks, i.e. networks that can be represented by a symmetric real-valued adjacency matrix. The framework and results immediately extended to the directed case by a symmetrization argument described in Online Appendix Section D.1. This use of symmetrization to extend results from the symmetric case to the asymmetric case is standard and comes from the literature on U-statistics. See, for instance, Section 5.1.1 of serfling2009approximation for a textbook presentation. \looseness=-1

Potential outcomes and the parameters of interest

Potential outcomes

A community of $N$ agents may be subjected to one of two policies indexed by $s \in \{0,1\}$. In our framework, we use the word “policy” to refer to any community-level intervention that influences the network connections between agents. One policy could, for instance, assign an individualized treatment to every agent in the community. Another policy could assign the treatment to a subpopulation of agents. Still another policy could be the introduction of a public good that is available to all members of the community. All of these examples fall within our framework. \looseness=-1

Once the community is subjected to one of the two policies, the agents interact and form connections. For two arbitrary agents $i$ and $j$ in the community, the variable $Y_{ij}(s)$ describes the magnitude of the connection between agents $i$ and $j$ when the community is assigned policy $s$. We refer to these quantities as potential outcomes or potential connections. \looseness=-1

Parameters of interest

We focus on the joint distribution of potential outcomes (DPO): \looseness=-1

where $y_{1},y_{0} \in \mathbb{R}$ and the measure $\mathbbm{P}$ refers to the joint distribution of potential outcomes, to be specified in Section 3.2 below. In words, $F(y_{1},y_{0}) $ is the mass of agent pairs with potential outcomes less than $y_{1}$ under policy $1$ and less than $y_{0}$ under policy $0$. Many measures of social disruption, including the fraction of network connections created or destroyed by a change in policy, can be written as simple functions of the DPO. For example, the fraction of binary $(\{0,1\}$-valued) network connections destroyed by a change in policy is $F(0,1) - F(0,0)$. The fraction created is $F(1,0) - F(0,0)$. \looseness=-1

A closely related parameter to the DPO is the distribution of treatment effects (DTE): \looseness=-1

where $y \in \mathbb{R}$ is arbitrary. In words, $Y_{ij}(1) - Y_{ij}(0)$ is the change in potential outcomes for agents $i$ and $j$ caused by switching from policy $0$ to policy $1$. $\Delta(y)$ is the mass of individual treatment effects that are less than $y$. While we focus our identification results on the DPO, we show how our proposed bounds for the DPO translate to bounds on the DTE in Proposition 3 of Section 5.2 below. \looseness=-1

Model of the data observed from the experiment

The researcher observes network data for two communities, each subjected to a different policy. It is assumed that the observed data are distributed as though a randomized experiment were conducted. Randomized experiments are common in the network economics literature, see, for examples, the literature cited in the third paragraph of the introduction. The assumption is also commonly employed in settings where randomization did not actually occur, but the researcher has reason to believe that the communities that make up the data are balanced in the sense that they contain a similar composition of agent types. \looseness=-1

In this subsection, we first specify a model of an experiment where agents are randomly assigned to communities. We then specify a stochastic model of link formation. Finally, we describe our main identifying assumption. This assumption is used to characterize the identified set for the DPO in Section 4. \looseness=-1

Experiment

We model a completely randomized balanced experiment conducted with $2N$ agents. In the experiment, the researcher defines two communities indexed by $\{0,1\}$. $N$ agents are then chosen uniformly at random and assigned to community $1$. The remaining $N$ agents are assigned to community $0$. We focus on the setting where the two communities are the same size because it greatly simplifies the exposition. It is straightforward to extend the framework to allow for differently-sized communities. For $s \in \{0,1\}$, policy $s$ is implemented in community $s$. We use $D_i \in \{0,1\}$ to indicate the policy/community assignment of agent $i$ and collect the community assignments for the $2N$ agents into the vector $D := \{D_i\}_{i=1}^{N}$. \looseness=-1

After the community assignments are made, the agents interact and form network connections, as described in Section 3.2.2 below. For two arbitrary agents $i$ and $j$ assigned to the same community $s \in \{0,1\}$ the researcher observes the potential outcome $Y_{ij}(s)$. That is, if $D_i = D_j$ then the observed magnitude of the connection between $i$ and $j$ is given by \looseness=-1

If $D_i \neq D_j$ then no connection is observed. We collect the $N(N-1)$ observed network connections into the vector $Y := \{Y_{ij}\}_{ij : D_i = D_j}$. A key assumption of our framework is that the potential outcomes and community assignments are unrelated, i.e. the entries of $Y(1)$ and $Y(0)$ are mutually independent of the entries of $D$. We describe this condition in Section 3.3 below. \looseness=-1

At the conclusion of the experiment, the researcher observes the vectors $\{Y,D\}$ as data. \looseness=-1

Network Formation Model

For a fixed $s \in \{0,1\}$, we assume that the connections between the $N$ agents in community $s$ are given by the model \looseness=-1

where $w_{i,s} \in [0,1]$ is a random agent effect, $\eta_{ij,s} \in [0,1]$ is a random idiosyncratic error, and $g_{N,s} : [0,1]^{3}\to \mathbb{R}$ is a measurable link function that may depend on the community size $N$. The entries of $\{w_{i,s}\}_{i=1}^{N}$ are independent across agents, the entries of $\{\eta_{ij,s}\}_{i,j = 1}^{N}$ are independent across pairs of agents, and the entries of $\{w_{i,s}\}_{i=1}^{N}$ and $\{\eta_{ij,s}\}_{i,j = 1}^{N}$ are mutually independent. The agent effects, idiosyncratic errors, and link function may all depend on the policy $s$. For our identification results, we assume that these three determinants of link formation are all unobserved. That is, the researcher only observes data on the network connections $Y$. \looseness=-1

A key feature of this network formation model is that for any two pairs of agents $\{i,j\}$ and $\{k,l\}$ in the community, the connections $Y_{ij}(s)$ and $Y_{kl}(s)$ are independent if $\{i,j\}$ and $\{k,l\}$ do not share an index. Though strong, this dependence structure describes a number of specifications that are popular in the economics literature. We provide three broad examples. \looseness=-1

Relating the network connections across policies

In the model ((ref)), the agent effects $w_{i,s}$, idiosyncratic errors $\eta_{ij,s}$, and link function $g_{s}$ (where the dependence on $N$ has been suppressed- see Remark 7 above) may all vary with the policy $s$. We do not place any additional restrictions on the agent effects or the link functions. That is, the two link functions $g_1$ and $g_2$ can be any measurable functions and the vector $\{w_{i,1},w_{i,0}\}$ is allowed to have an arbitrary joint distribution subject to the normalizing of the marginal distributions of $w_{i,1}$ and $w_{i,0}$ made in Remark 6. We represent the distribution of $\{w_{i,1},w_{i,0}\}$ in a particular way. Specifically, by Lemma 2.7 of Whitt (1976), there exists a policy-invariant $w_i$ and unknown measurable functions $\varphi_{1}, \varphi_{0}: [0,1] \to [0,1]$ such that $w_{i,s} = \varphi_{s}\left(w_i\right)$ for $s \in \{0,1\}$ where the entries of $\{w_i\}_{i=1}^{N}$ can be taken to be independent with standard uniform marginal distributions. We emphasize that this representation is without loss of generality. Since the marginal distributions of $w_{i,1}$, $w_{i,0}$ and $w_i$ are all standard uniform, the functions $\varphi_1$ and $\varphi_0$ are measure-preserving. This property plays a key role in our analysis, and so we display it as \looseness=-1

Intuitively, the measure-preserving transformation is an infinite-dimensional analog of the permutation that we used to characterize the identified set in the simplified setting of Section 2. It follows from this representation that the magnitude of the connection between agents $i$ and $j$ under policy $s$ is represented by the model \looseness=-1

\looseness=-1

Finally, we restrict the idiosyncratic errors to be unrelated across the two policies. That is, we assume that the entries of $\{\eta_{ij,1}\}_{i,j = 1}^{N}$ and $\{\eta_{ij,0}\}_{i,j = 1}^{N}$ are mutually independent. We interpret this restriction as saying that a change in policy does not alter the idiosyncratic errors in any systematic way. \looseness=-1

Main identifying assumption

We close our specification of the model by highlighting our main identifying assumption. It is that the community assignments are unrelated to the factors that determine the network connections under each policy. That is, \looseness=-1

Since $Y_{ij}(s)$ is determined by $\{w_{i,s},w_{j,s},\eta_{ij,s}\}$, Assumption 1 implies that the entries of $\{D_i\}_{i=1}^{N}$ and $\{Y_{ij}(1), Y_{ij}(0)\}_{i,j=1}^{N}$ are mutually independent. \looseness=-1

Researchers often justify Assumption 1 in practice by conducting a randomized experiment. The assumption is also common in non-experimental settings when the researcher believes that the communities in the data are balanced so that the agent interactions in one community under the policy implemented in that community describe how the agents in the other community would interact under that same policy. \looseness=-1

It follows from Assumption 1 that the distribution of the observed data $\{Y,D\}$ is parametrized by the functions $\{g_s,\varphi_s\}_{s \in \{0,1\}}$. Specifically, for any constants $\{y_{ij}, d_i\}_{i,j=1}^{2N} \in \mathbbm{R}^{2N\times 2N}\times \{0,1\}^{2N}$ the distribution function $\mathbbm{P}\left(\{Y_{ij} \leq y_{ij}, D_i = d_i\}_{i,j=1}^{2N}\right)$ can be written as \looseness=-1

In words, the first line describes the conditional distribution of the network connections given the community assignments, $\mathbbm{P}\left(\{Y_{ij} \leq y_{ij}\}_{i,j=1}^{2N}|\left\{D_i = d_i\right\}_{i=1}^{2N}\right)$. The second line describes the marginal distribution of the community assignments $\mathbbm{P}\left(\{D_i = d_i\}_{i=1}^{2N}\right)$. They are both determined by $N$, the constants $\{y_{ij}, d_i\}_{i,j=1}^{2N}$, and parameters $\{g_s,\varphi_s\}_{s \in \{0,1\}}$.\looseness=-1

Identification

In this section, we characterize the identified set for the DPO given the distribution of the data $\{Y,D\}$ under the model of Section 3 as parametrized by $\{g_s,\varphi_s\}_{s \in \{0,1\}}$. Our characterization uses a (slight variation on a) construction from the graph theory literature called a graph function. See Chapter 7.1 of lovasz2012large for a textbook introduction. We first define this construction and then use it to characterize the identified set. \looseness=-1

Graph function

For any $s \in \{0,1\}$ and $y_s \in \mathbb{R}$, we define the graph function $h_{s}: [0,1]^2\times \mathbbm{R} \to [0,1]$ to be \looseness=-1

In words, $h_{s}(a,b;y_s)$ is the conditional probability that the magnitude of a connection between agent $i$ with $w_i = a$ and agent $j$ with $w_j = b$ is less than $y_s$ under policy $s$. The function depends on a fixed choice of $y_s$, which we suppress to simplify our notation, simply writing $h_s(a,b,;y_s) = h_s(a,b)$. It also implicitly depends on the community size $N$ through $g_s$, which we also suppress. \looseness=-1

We do not distinguish between graph functions that are equivalent up to a measure-preserving transformation. That is,

The reason why we only consider graph functions defined up to a measure-preserving transformation is because it is the limit of what is identified from the data. That is, \looseness=-1

The proof of Lemma 1 can be found in Appendix Section A.2. When we write that $h_{s}$ is “identified up to a measure-preserving transformation,” we make two claims. The first claim is that the distribution of the data can distinguish between two graph functions that are not equivalent up to a measure-preserving transformation. That is, formally, suppose that $\{g_s,\varphi_s\}_{s \in \{0,1\}}$ and $\{g'_s,\varphi'_s\}_{s \in \{0,1\}}$ are two sets of model parameters that generate the data $\{Y,D\}$ and $\{Y',D'\}$. Let $h_{s} = \int \mathbbm{1}\{g_{s}(\varphi_s(a),\varphi_s(b),w) \leq y_s\}dw$ and $h'_{s} = \int \mathbbm{1}\{g'_{s}(\varphi'_s(a),\varphi'_s(b),w) \leq y_s\}dw$ be the graph functions associated with these two models. Then the first claim is that $h_{1} \not\sim h'_{1}$ or $h_{0} \not\sim h'_{0}$ implies that $\{Y,D\}$ and $\{Y',D'\}$ do not have the same distribution (i.e. they have different distribution functions ((ref))). \looseness=-1

The second claim in the statement of Lemma 1 is that the distribution of the data cannot distinguish between two graph functions that are equivalent up to a measure-preserving transformation. That is, in the above setting, if $h_{1} \sim h'_{1}$ and $h_0 \sim h'_0$ then $\{Y,D\}$ and $\{Y',D'\}$ have the same distribution function ((ref)).\looseness=-1

Identified set

To derive the identified set for the DPO, we represent it as the inner product of $h_{1}$ and $h_{0}$.

where the first equality is due to the definition of the DPO in Section 3.1.2, the second equality follows from the law of iterated expectations, and the third equality is due to the definition of the graph function in Section 4.1. The identified set for the DPO then follows immediately from the fact that $h_1$ and $h_0$ are identified up to a measure-preserving transformation (Lemma 1 of Section 4.1). That is, \looseness=-1

In words, ((ref)) is the set of all values that the DPO can potentially take given the distribution of $\{Y,D\}$ as determined by the model described in Section 3. \looseness=-1

Bounds on the identified set

We first describe sharp but infeasible bounds on the DPO. We then propose tractable outer bounds on the DPO and DTE. Our proposed bounds are based on rearrangements of the eigenvalues of the graph functions $h_1$ and $h_0$. Eigenvalues of functions are defined a bit differently than their matrix counterparts, see Appendix Section A.1 for details. Proof of claims are in Appendix Sections A.3 and A.4. \looseness=-1

Sharp bounds are intractable

Proposition 1 implies that (pointwise) sharp bounds on the identified set for the DPO are \looseness=-1

These bounds are sharp because $h_1$ and $h_0$ are only identified up to a measure preserving transformation by Lemma 1, and so, as far as the distribution of the data is concerned, any choice of $\psi_1, \psi_0 \in \mathcal{M}$ could be the one that defines the true DPO. They suggest a natural, though infeasible, two-step estimation strategy. In the first step, the researcher constructs an estimator $\hat{h}_{s}$ for the graph function $h_{s}$. In the second step, the researcher computes the empirical analog of ((ref)), \looseness=-1

The first step of this strategy is feasible. That is, even though both the graph function $\{h_1,h_0\}$ and the agent effects $\{w_i\}_{i=1}^{N}$ are unobserved, it is possible to consistently estimate the function $\hat{h}_s$ (up to a measure-preserving transformation) under relatively mild conditions. See, for instance, olhede2014network,chatterjee2015matrix,zhang2015estimating. \looseness=-1

It is the second step of the strategy that is infeasible. In particular, the optimization problem described in this step is both analytically and computationally intractable, even for relatively small community sizes, because the function $\psi_{s}$ appears twice in the optimization problems on the right and left-hand sides of ((ref)). As a result, is generalizes the quadratic assignment problem described in Section 2. Solutions to this problem are only known for stylized examples of $Y_{1}$ and $Y_{0}$ that are not relevant for the kinds of network data typically observed by economists. See Online Appendix Section C.3 or cela2013quadratic Section 1.5 for examples. \looseness=-1

Proposed outer bounds

Our proposed bounds are based on the intersection of several relaxations of ((ref)). Our bounds are not sharp, but they are tractable and use enough information from Lemma 1 to outperform conventional methods in many settings. Our main idea is to rearrange the eigenvalues of the graph functions associated with each policy. Specifically, let $\lambda_{1s}(y_{s}) \geq \lambda_{2s}(y_{s}) \geq ... \geq \lambda_{Rs}(y_{s})$ be the $R$ largest in magnitude eigenvalues of $h_{s}$ ordered to be decreasing and $\rho_{R}(r) = R - r + 1$. For $s, s' \in \{0,1\}$, let $\sum_{r}\lambda_{rs}\lambda_{rs'} := \lim_{R \to \infty}\sum_{r=1}^{R}\lambda_{rs}(y_{s})\lambda_{rs'}(y_{s'})$, $\sum_{r}\lambda_{rs}\lambda_{\rho(r)s'} := \lim_{R \to \infty}\sum_{r=1}^{R}\lambda_{rs}(y_{s})\lambda_{\rho_{R}(r)s'}(y_{t'})$ and $\sum_{r}\lambda_{rs}^{2} := \sum_{r}\lambda_{rs}\lambda_{rs}$.\footnote{These limits exist because the series $\{\lambda_{rs}\lambda_{rs'}\}_{r\in\mathbb{N}}$ and $\{\lambda_{rs}\lambda_{\rho_{R}(r)s'}\}_{r\in\mathbb{N}}$ are absolutely summable by the Cauchy-Schwarz inequality and the fact that $\sum_r \lambda_{rs}^2 = \int\int h_s(u,v)^2dudv \leq 1$ for $s,s' \in \{0,1\}$.} We note that the eigenvalues of $h_s$ are identified because $h_s$ is identified up to a measure preserving transformation by Lemma 1 of Section 4.1, and any two graph functions that are equivalent up to a measure preserving transformation necessarily have the same eigenvalues. \looseness=-1

Our proposed bounds on the DPO are \looseness=-1

where the eigenvalue $\lambda_{rs}$ is implicitly a function of $y_{s}$ through the definition of $h_{s}$.

The proof of Proposition 2 can be found in Appendix Section A.4. The result is similar in spirit to the conventional Fr\'echet-Hoeffding bounds, but builds on a relaxation of ((ref)), and so the arguments behind the proofs are fundamentally different. Intuitively, a common way to derive the Fr\'echet-Hoeffding bounds is to rearrange the quantiles of the distribution of outcomes associated with each policy. See, for instance, the second proof of Theorem 2.1 in whitt1976bivariate. Our bounds instead work by rearranging the eigenvalues of the graph functions building on a proposal by finke1987quadratic for the finite dimensional QAP described in Section 2.2.2. That their logic extends to graph functions and so applies to the nonparametric model described in Section 3.1 is not obvious, requires arguments from functional analysis that are not typical of the QAP literature, and is, to our knowledge, original to our paper. Intuitively, the finke1987quadratic bounds work in the finite dimensional case because the rows and columns of the matrix of eigenvectors are orthogonal. An analogous property does not hold for eigenfunctions and so we instead bound the DPO on a sequence of histogram-like approximations. See our Lemmas 1 and 3 in Appendix Section A.3. Unlike the bounds in ((ref)), those in ((ref)) are tractable because they only depend on the eigenvalues of $h_{s}$. The bounds are also straightforward to estimate, see Sections 5.3 and Online Appendix Section D.3 below. \looseness=-1

The bounds on the DPO can be used to bound the DTE. Our proposed bounds are

where the eigenvalue $\lambda_{rs}$ is implicitly a function of $y_{s}$ through the definition of $h_{s}$. The proof of Proposition 3 can be found in Appendix Section A.5. The result is similar in spirit to the conventional Makarov bounds, but uses our Proposition 2 instead of Fr\'echet-Hoeffding.

Estimating the bounds

We propose estimating the bounds in Propositions 2 and 3 using a feasible version of the plug-in strategy described in Section 5.1. In the first step, the researcher constructs an estimator for $\hat{h}_s$ for the graph function $h_s$. In our Online Appendix Section D.3 we estimate the graph function in the dyadic regression setting (the first example literature we provided in Section 3.2.2) under a parametric restriction, as presented in Section 4 of graham2020network. But parametric restrictions are not necessary here, see for instance the USVT estimator of chatterjee2015matrix. We emphasize that since $h_s$ is only identified up to a measure preserving transformation, the goal is not to approximate the function $h_s$ exactly, but only to approximate an element of the identified set. \looseness=-1

In the second step, the researcher computes the eigenvalues of the function $\hat{h}_s$ and uses them as a substitute for those of $h_s$ in ((ref)) and ((ref)). Intuitively, since eigenvalues are continuous (see Lemma B3 in Online Appendix Section B) and invariant to measure preserving transformations, the estimation error for these bounds should be small in large samples if the estimator $\hat{h}_s$ is consistent in the above sense. We formalize this logic in the dyadic regression setting in Online Appendix Section D.4. We also give a procedure to construct confidence intervals for the DPO and provide sufficient conditions for asymptotic validity. \looseness=-1

We focus on the dyadic regression models because some version of this model is used in nearly all of the motivating examples we reference in our introduction. We believe that it is straightforward to apply our arguments to the exchangeable graph and strategic link formation settings (the other two example literatures described in Section 3.2.2), following the estimation and inference arguments made in these literatures, but we leave this to future work. \looseness=-1

Empirical illustration

In this section, we apply our framework to the setting of banerjee2021changes. Section 6.1 reviews background information, Section 6.2 applies the framework of Section 3, and Section 6.3 applies the estimation and inference results of Online Appendix Section D.3. The results are in Section 6.4. \looseness=-1

Background

banerjee2021changes study the effect of a microfinance program on informal risk sharing in Karnataka, India. The authors find, among other things, that participating villages have one percentage point (1pp) less informal risk sharing links between households. We find disruptive effects that are sixteen to twenty-three times larger using our methodology. \looseness=-1

The Karnataka study is centered around the planned introduction of microfinance in 75 villages by Bharatha Swamukti Samsthe (BSS). BSS selected 43 of these villages in 2006 and implemented the program between 2007 and 2010. They originally planned to implement the program in all of the villages, but ultimately did not because of an external crisis. banerjee2021changes argue that the villages are comparable after controlling for village size. The authors collected data on informal risk sharing connections between households at two time periods: before and after BSS implemented the program in the selected villages. \looseness=-1

We reanalyze the banerjee2021changes data using our framework.\footnote{The data can be found at \url{https://zenodo.org/record/7706650\#.ZD9Tti-B2gQ}.} We focus on two villages: village 57, which participated in the microfinance program, and village 44, which did not. We chose these villages because they are the most similar in terms of pre-treatment covariates across all pairs of villages.\footnote{Specifically, we normalized the pre-treatment village-level covariates to have a standard deviation of 1. We picked the treatment-control pair whose pre-treatment covariates have the smallest Euclidean distance.} \looseness=-1

Application of our framework

We extend our Section 3 framework to the panel setting of banerjee2021changes. Let $Y_{ij,t}(s)$ indicate whether there would be a risk sharing connection between households $i$ and $j$ when a village participates $(s = 1)$ or does not participate $(s = 0)$ in the microfinance program before $(t = 0)$ or after $(t = 1)$ the program is implemented in the participating villages. Following banerjee2021changes, we take as the outcome of interest the change in network connections for a pair of households over time $\Delta Y_{ij}(s) := Y_{ij,1}(s) - Y_{ij,0}(s)$. These network connections are ternary, taking values in $\{-1,0,1\}$. \looseness=-1

Our proposed measures of disruption are the expected fraction of risk sharing links created by implementing the microfinance program in a village,

and the expected fraction of risk sharing links destroyed by implementing the microfinance program in the village,

where $\mathbbm{P}$ refers to the distribution of network connections as given by the model ((ref)) in Section 3.2. Following banerjee2021changes, we specify a dyadic regression model for our estimation and inference results in Section 6.3.\looseness=-1

To bound $C$ and $D$, we first bound a discrete analog of the DPO, the joint density function\looseness=-1

where $y_1, y_0 \in \{-1,0,1\}$ is arbitrary and $h_{s}(u,v) = \mathbbm{P}\left(\Delta Y_{ij}(s) = y_s | w_i = u, w_j = v \right)$. We then use the bounds on $f(y_1,y_0)$ to bound $C$ and $D$ via the formulas

We use a variation on the bounds in Proposition 2 of Section 4.1.2 that corrects for row and column heterogeneity, see Online Appendix Section D.2. Specifically, we define $\alpha_{s}(u) = \mathbbm{P}\left(\Delta Y_{ij}(s) = y_s | w_i = u\right) - \frac{1}{2}\mathbbm{P}\left(\Delta Y_{ij}(s) = y_s\right)$ and $\epsilon_s(u,v) = \mathbbm{P}\left(\Delta Y_{ij}(s) = y_s| w_i = u, w_j = v\right) - \alpha_{s}(u) - \alpha_{s}(v)$. Our lower bound on $f(y_1,y_0)$ is $$2\int\alpha_{1}^{+}(u)\alpha_{0}^{+}(1-u)du + 2\bar{\alpha}_{1}\bar{\alpha}_{0} + \max\left(\sum_{r}\left(\lambda_{r1}^{2} + \lambda_{r0}^{2}\right) - 1,\sum_{r}\lambda_{r1}\lambda_{\rho(r)0},0\right)$$ and our upper bound on $f(y_1,y_0)$ is $$2\int\alpha_{1}^{+}(u)\alpha_{0}^{+}(u)du + 2\bar{\alpha}_{1}\bar{\alpha}_{0} + \min\left(\sum_{r}\lambda_{r1}^{2},\sum_{r}\lambda_{r0}^{2},\sum_{r}\lambda_{r1}\lambda_{r0}\right)$$ where $\alpha_{s}^{+}$ is the quantile function of $\alpha_{s}$, $\bar{\alpha}_s = \int\alpha_s(u)du$, $\lambda_{rs}$ is the $r$th eigenvalue of the function $\epsilon_s$, and the sums are as defined in Section 4.1.2. As we discuss in Online Appendix Section D.2, $\alpha^{+}_s(\cdot)$ and $\bar{\alpha}_s$ are identified from the distribution of $Y$ and $D$, while $\epsilon_s(\cdot,\cdot)$ is identified up to a measure-preserving transformation following the logic of Lemma 1 in Section 4.1. \looseness=-1

Estimation and inference

Our estimation and inference results use a parametric dyadic regression model along the lines specified by banerjee2021changes in their Section 3.2. Specifically, we use \looseness=-1

where $\Lambda$ is the identity link function, $X_{i,s} = x_s(w_i)$ is a vector of $K$ observed household-specific characteristics represented as a function of $w_{i}$ following the logic of Example 1 in our Section 3.2.2, $\tau_s$ is an unknown intercept, $\beta_s$ is an unknown $K$-dimensional vector of coefficients, and $x_{ks}$ and $\beta_{ks}$ refer to the $k$th entry of $x_{s}$ and $\beta_{s}$.\footnote{While we follow banerjee2021changes and take $\Lambda$ to be the identity link function, our theoretical results in Online Appendix Section D.3 allow for an arbitrary twice continuously differentiable function such as the logistic function or normal cdf. We use the same household covariates as banerjee2021changes, but exclude network statistics (functionals of $Y$ such as degree or eigenvector centrality) because of endogeneity concerns.}

Empirical findings

Estimated amount of social disruption using standard tools

We first recreate the results of banerjee2021changes . Using difference-in-differences, banerjee2021changes find that the villages selected for the program experience a greater decline in social connections, with an ATE of approximately -1pp. They also estimate average treatment effects conditional on whether two households were linked in the pre-treatment period and whether households have a high (H) or low (L) propensity to borrow money from the microfinance program. For two households that were linked before treatment, those in the treated village were 6pp more likely to have their link destroyed. For two households that were not linked, those in the treated village were 2pp less likely to have a link form between them. The authors also find that the microfinance program has larger effects on connections between L households than H households. \looseness=-1

We reproduce these results using villages 57 and 44 in Table 1. Specifically, Table 1 contains sample analogs of the average treatment effect (ATE), the average treatment effect conditional on being connected in the pre-treatment period (CATE(1)), and the average treatment effect conditional on not being connected in the pre-treatment period (CATE(0))\looseness=-1

\looseness=-1 for the full sample of households, as well as subsamples of pairs of households that are both type $H$ ($HH$), both type $L$ ($LL$), or mixed ($HL$).

The average treatment effects in Table 1, particularly the ATEs, are on the order of $1$pp or smaller, consistent with the findings of banerjee2021changes. The table does reveal nominally large negative effects for CATE(1), particularly for the HL subsample. However, the fraction of household pairs that are connected in the pre-treatment period is relatively small, so that total disruptive effect revealed by these estimators is still on the order of $1$pp. We demonstrate this in columns 3 and 5 of Table 1, where we scale CATE(1) by the (empirical analog of) the mass of links in the treatment group in the pre-treatment period, $P(1) := \mathbbm{P}\left(Y_{ij,0}(1) = 1\right)$, and similarly scale CATE(0) by $P(0) := \mathbbm{P}\left(Y_{ij,0}(1) = 0\right)$. We find substantially larger effects using our bounds below. \looseness=-1

Estimated amount of social disruption using our bounds

Our main results are presented in Table 2. We report both our point estimates for the upper and lower bounds on $f(y_1,y_0)$, as well as confidence intervals, for $y_1,y_0 \in \{-1,0,1\}$. The first three rows report results using all of the households in villages 57 and 44. Recall that for link destruction, the mass of destroyed connections is given by $D = f(-1,0) + f(-1,1) + f(0,1)$. We find that, with high probability, \looseness=-1 $$f(-1,0) \in [0.0296, 0.0428] \quad , \quad f(0,1) \in [0.0517, 0.0725] $$ where $f(0,1)$ is the fraction of connections that would have been created if not for the microfinance program, $f(-1,0)$ is the fraction of connections that were destroyed because of the program, and $f(-1,1)$ is approximately $0$. The bounds together imply that the mass of destroyed connections $D \in [0.0830,0.1193]$ with high probability. Similarly for link creation, with $C = f(0,-1)+f(1,-1)+f(1,0)$, we find that, with high probability,\looseness=-1 $$f(0,-1) \in [0.0202, 0.0385] \quad ,\quad f(1,0) \in [0.0532, 0.0687]$$ where $f(1,-1)$ is also approximately $0$. These bounds together imply that the mass of created connections $C \in [0.0769,0.1103]$. The results for the HH, LL and HL subsamples are similar.

Our bounds indicate that the microfinance program disrupted an approximately $D+C \in [0.1599, 0.2296]$ fraction of the connections between households. Using the -1pp ATE as a baseline, we conclude that the microfinance program is sixteen to twenty-three times more disruptive than what is indicated by the average treatment effects of Section 6.4.1. \looseness=-1

Our results rule out monotonic treatment effects

The reason why our bounds in Section 6.4.2 reveal a much larger amount of disruption than the average treatment effects of Section 6.4.1 is because the microfinance program both creates and destroys many connections between households. If, in contrast, the program only created or only destroyed connections, then we would expect them to be roughly equivalent. To see this, suppose that $\Delta Y_{ij,1} \geq \Delta Y_{ij,0}$ with probability one. In this case, $f(-1,0) = f(-1,1) = f(0,1) = 0$ so that $D = 0$ and \looseness=-1

where the last line is because $f(1,-1) \approx 0$ (see Table 2). Similarly, $C = 0$ and $ATE \approx D$ whenever $\Delta Y_{ij,1} \leq \Delta Y_{ij,0}$ with probability one. \looseness=-1

Since our confidence intervals for $C$ and $D$ both exclude $0$, we reject both monotonicity assumptions $\Delta Y_{ij}(1) \geq \Delta Y_{ij}(0)$ and $\Delta Y_{ij}(1) \leq \Delta Y_{ij}(1)$. As a result, the average treatment effects, even conditional on whether the households are linked in the pre-treatment period or have a high or low propensity to borrow money, necessarily understate the program's disruptive effect. \looseness=-1

Conclusion

This paper is about identifying social disruption: the amount of network connections created or destroyed by a policy. It focuses on a research design that is popular in the literature. We first formalize the informational content of the random assignment of agents to communities. We then show that the sharp identified set is given by an intractable quadratic assignment problem and propose outer bounds constructed by rearranging the eigenvalues of two graph function parameters that are identified from the experiment. Our empirical illustration demonstrates that our methodology is effective at identifying social disruption in practice. Alternative methods used in the literature can substantially understate the disruptive impact of the policy. \looseness=-1

How should researchers use measurements of social disruption to evaluate and design policy? One way to do this would be to specify a welfare function that values both conventional economic benefits like the health or wealth of agents as well as the amount of social disruption. Researchers could then choose a policy that maximizes the economic benefits subject to the constraint that it does not alter too many social connections (i.e. that the estimate for the upper bound on the amount of social disruption is sufficiently small). The idea here is that policies that destroy only a few connections between agents are unlikely to have the kinds of negative unintended consequences reported in the empirical literature jackson2021inequality.\looseness=-1

Another way to use measurements of social disruption for policy is to actually assign a specific value to the number of created and destroyed connections, and directly weigh them against the other economic benefits or harms of the policy. This requires the researcher to take stance on the economic cost of social disruption. To our knowledge, there is currently little work explicitly on pricing disruption, and so we highlight this as an important area for future research.\looseness=-1