Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
71,974 characters · 20 sections · 74 citation commands
A Binary IV Model for Persuasion: Profiling Persuasion Types among Compliers
Empirical studies on persuasion are widespread in economics. Researchers have examined the effect of information treatments on individual behaviour in various contexts. For example, they have studied the impact of voter mobilisation on voting, partisan media on voting, advertising on consumption, donor outreach on charity donations, and financial disclosures on investment decisions (dellavigna2010persuasion).
To study the problem of identifying the effects of persuasion, jun2018identifying set up an econometric model of persuasion, which is a binary Imbens-Angrist instrumental variable (“IA IV" hereafter) model with monotone treatment response. Consider a study evaluating the effect of voter mobilisation, specifically a "Get Out the Vote" (“GOTV" hereafter) programme, on voting behaviour. In this case, the instrument is the randomly assigned information treatment, that is, mobilisation messages sent to voters. The treatment is whether voters received these messages, and the outcome is a binary behaviour measure, such as whether they voted.
This paper shows three sets of results that are economically relevant to empirical studies of persuasion. First, we show that under an econometric model of persuasion, the joint distribution of potential outcomes conditional on compliers is identified. The percentage of mobilised voters, conditional on compliers, is identifiable because the local persuasion rate is identified (jun2018identifying). Furthermore, under the monotone treatment response assumption, the event in which an individual is an always-voter is equivalent to the event that the individual would vote without receiving the mobilisation. A similar argument applies to never-voters.
We show that in an econometric model of persuasion, although analysts do not directly observe the three persuasion types—always-voters, never-voters, and mobilised voters among compliers—any statistical characteristic defined by the moments of the joint distribution of treatment and covariates is identifiable for these latent types. To show this, we first extend Abadie’s $\kappa$ results. Specifically, we show that in an IA IV model, any statistical characteristic defined by the moments of the joint distribution of potential outcomes, treatment, and covariates is identifiable for compliers. Building on this, we show that in an IA IV model, any statistical characteristic defined by the moments of the joint distribution of treatment and covariates can be identified, conditional on compliers and marginal potential outcomes. Finally, under the monotone treatment response assumption, we strengthen the interpretation of the conditioning set from types defined by marginal potential outcomes to types defined by joint potential outcomes.
These two new identification results are economically relevant. Persuasion involves shifting individuals from one type of action to another. To better understand the effectiveness of information treatments, researchers need insight about the joint distribution of potential outcomes (heckman1997making, dellavigna2010persuasion, jun2018identifying).
Additionally, profiling the statistical characteristic of persuasion types among compliers can help test the economic mechanisms through which persuasion works and is policy-relevant. Profiling these latent types can help researchers assess the mechanisms by which the treatment affects outcomes. For example, since the voter mobilisation treatment only slightly reduces voting costs, we would expect the voting propensity of mobilised voters to be marginally lower than that of always-voters. Given that voting behaviour is highly persistent, always-voters are likely to participate in multiple elections. Both hypotheses can be tested using the methods developed in this paper. These profiling results are also policy-relevant. Researchers can identify the probability of a mobilised voter being a Democrat, which allows them to estimate the number of mobilised voters who are Democrats. There are two key implications here. First, if mobilisation occurs in a swing state, the experiment could significantly influence election outcomes if mobilised voters are predominantly Democrats. Second, researchers can use these statistics to calculate the expected cost of mobilising a Democrat, enabling a better assessment of the cost-effectiveness of the intervention.
We also present three additional results that complement the analyses in jun2018identifying. First, we show that the local persuasion rate and the approximated persuasion rate, as summarised in dellavigna2010persuasion, are equal under one-sided non-compliance, provided that certain restrictions on the dependence between potential treatments and potential outcomes are satisfied. Next, we propose a sharp test for the identification assumptions by testing whether there are non-negative solutions to a system of linear equations implied by the model. Finally, we provide a simple sensitivity analysis for the monotone treatment response assumption.
Finally, we apply the methods to a GOTV experiment done by green2003getting. The results indicate that approximately $10\%$ of compliers are mobilised. Among compliers, always-voters had the highest turnout in the last presidential election, while never-voters had the lowest. These findings suggest that voting behaviour is habit-forming and persistent (gerber2003voting). Additionally, the voting propensity of those mobilised is close to that of always-voters conditional on compliers, which is consistent with the interpretation that the GOTV programme mobilises high-propensity voters, as the intervention only slightly lowers the voting cost. In Bridgeport, the results show a high likelihood that the mobilised complier voters were Democrats, although the estimate is noisy. Based on this likelihood, we estimate that approximately 28 Democrats were mobilised due to the mobilisation, with an average cost of \$1,066 per voter.
This paper contributes to three strands of literature. First, it is closely related to abadie2003semiparametric. We extend Abadie’s $\kappa$ results by identifying statistical characteristic measured by treatment and covariates of latent types, defined by either marginal or joint potential outcomes among compliers, under an econometric model of persuasion. This paper complements the analyses in jun2018identifying by: (1) providing conditions under which the "approximated" persuasion rate and local persuasion rate are equal; (2) proposing a sharp test for the identification assumptions; and (3) providing a simple sensitivity analysis for the monotone treatment response assumption. Finally, this paper contributes to the literature on identifying the joint distribution of potential outcomes in an IV model. Previous work assumes rank invariance (chernozhukov2004impact, vuong2017counterfactual), while our approach restricts both the support of the outcome variable and the direction of the causal effect.
Finally, comey2023supercompliers independently develop methods for profiling outcome types under Assumption (ref), but there are four key differences between their work and ours. First, we focus on identifying the joint distribution of potential outcomes among compliers, relevant to persuasion studies, which is not explicitly stated in their paper. Second, we show that statistical characteristic defined by the moments of the joint distribution of treatment and covariates are identifiable, while they do not consider profiling with treatment. Third, we provide a sharp test for Assumption (ref), based on the model’s implication of an under-determined system of linear equations. The testable implication applies to an IV model under various monotonicity restrictions with a discrete outcome, a discrete instrument, and a binary treatment. Lastly, we compare different estimands relevant to persuasion studies, which they do not address.
The remainder of the paper is organised as follows. Section (ref) sets up a binary IV model of persuasion. Section (ref) presents methodologies for identifying the joint distribution of potential outcomes among compliers and profiling persuasion types conditional on compliers. Additional discussions are provided in Section (ref). In Section (ref), we reanalyse the data from green2003getting and conclude in the final section.
In an empirical study of persuasion, researchers often collect data on a binary information treatment $T_{i}$, and a binary behavioural outcome $Y_{i}$. For example, in a GOTV experiment, the outcome of interest is whether or not voters vote, and the information treatment is a mobilization that contains information on the timing and the location of the upcoming election. Since information consumption involves self-selection, researchers often employ an instrument $Z_{i}$ which creates exogenous variations for an individual's information consumption decision. In many experiments, the instrument $Z_{i}$ is also binary. In a GOTV experiment, the instrument is the randomly assigned access to the GOTV treatment. Besides the aforementioned variables, researchers also collect pre-treatment covariates $X_{i} \in \mathbb{R}^{k}$. Define $Y_{i}(1)$ and $Y_{i}(0)$ as the potential outcomes that an individual would attain with and without being exposed to the treatment, and $T_{i}(1)$ and $T_{i}(0)$ as the potential treatments that an individual would attain with and without being exposed to the instrument. For a particular individual, the variable $Y_{i}(t, z)$ represents the potential outcome that this individual would obtain if $T_{i} = t$ and $Z_{i} = z$, where $t, z \in \{ 0, 1\}$.
Formally speaking, researchers make the following assumptions in a binary IV model of persuasion (jun2018identifying).
Assumptions 1 to 4 are the assumptions in an IA IV model. To simplify the notation, we will suppress the conditional notation throughout the paper, but all assumptions and analyses should be understood as conditional on the covariates. Note that it is not new to assume the direction of the treatment effect in causal inference literature (manski1997monotone, manski2000monotone). This assumption assumes that there are no demobilised voters.
Assumption (ref) can be applied in contexts beyond GOTV. For instance, this model can also be used to study the persuasion effects of political messages on voting behaviour (dellavigna2007fox), to persuade donors to contribute (landry2006toward), and to assess the impact of job training programme on reducing crime (blattman2016can), among others.
By Assumption (ref), we can classify individuals into 9 groups. Since the outcome is binary, the monotone treatment response assumption implies that we can classify individuals as always-voters, never-voters, and mobilised voters. By the IV monotonicity assumption, we can classify individuals as always-takers, never-takers, and compliers. The classification is presented in Table (ref).
In this section, we present our methodology for identifying three new estimands in Section (ref). We first show that under Assumption (ref), the joint distribution of potential outcomes among compliers is identifiable. We then show how to profile persuasion types, defined by either marginal or joint potential outcomes, using pre-treatment covariates. Finally, we briefly discuss estimation and inference issues.
Remarkably, the joint distribution of potential outcomes is identified under Assumption (ref). In other words, under the assumptions for a binary IV model of persuasion, we can know the percentage of always-voters, never-voters, and mobilised voters among compliers. This result strengthens the classic result that identifies the quantities of the marginal distribution of the potential outcome of compliers (imbens1994identification, imbens1997estimating). We state the results formally in Proposition (ref) below.
Here is the intuition behind Proposition (ref). Under the monotone treatment response in Assumption (ref), those who will vote without receiving the GOTV treatment (i.e., those with $Y_{i}(0) = 1$) will also vote if they receive the GOTV treatment (i.e., their $Y_{i}(1)$ is also $1$). Therefore, identifying the proportion of always-voters among compliers boils down to identifying the proportion of voters who will vote if they do not receive the GOTV treatment among compliers, which is identifiable under the IA IV assumption (imbens1997estimating). Similarly, the proportion of never-voter voters among compliers is identifiable by observing that, under the monotone treatment response assumption in Assumption (ref), those who will not vote if they receive the GOTV treatment (i.e., those with $Y_i(1) = 0$) will also not vote if they do not receive the GOTV treatment (i.e., their $Y_i(0)$ is also $0$).
An application of Bayes' theorem shows that the proportion of persuadable individuals among compliers is the product of the local persuasion rate and the proportion of voters who will not vote without receiving the GOTV treatment among compliers. Under Assumption (ref), the local persuasion rate is identifiable (jun2018identifying). Under the IA IV assumption, the latter quantity is also identifiable (imbens1997estimating). Therefore, the proportion of persuadable individuals among compliers is identifiable under Assumption (ref).
We also discuss the extension of the identification results in Proposition (ref) to non-binary outcomes and instruments in Appendix B. The results are negative for the former and positive for the latter.
This section presents the results that profile the persuasion types among compliers. First, we present a result that identifies any statistical characteristic of compliers, defined by the moments of the joint distribution of $(Y_{i}(t), T_{i}, X_{i})$, where $t \in \{0, 1 \}$. Then, we present a series of results identifying the statistical characteristics of compliers and persuasion types defined by the marginal potential outcomes. These statistical characteristics are measured by the moments of the joint distribution of $(T_{i}, X_{i})$. Finally, we provide results identifying the statistical characteristics of compliers and three persuasion types that are defined in Table (ref).
abadie2003semiparametric shows that any statistical characteristic that can be defined in terms of the moments of the joint distribution of $(Y_{i}(t), X_{i})$, where $t \in {0, 1}$, is identified for compliers under the IA IV assumption. We first strengthen the results in abadie2003semiparametric by showing that any statistical characteristic that can be defined in terms of the moments of the joint distribution of $(Y_{i}(t), T_{i}, X_{i})$, where $t \in \{0, 1\}$, is identified for compliers under the IA IV assumption.
The intuition behind the results is the following. Among compliers, their treatment-taking status equals the treatment assignment. Therefore, under the IV independence assumption, identifying the moments defined by $(Y_{i}(t), T_{i}, X_{i})$ reduces to identifying the weighted averages of the moments for $(Y_{i}(t), 0, X_{i})$ and $(Y_{i}(t), 1, X_{i})$ among compliers, with the weights determined by the treatment assignment probability. Finally, abadie2003semiparametric shows that the moments for $(Y_{i}(t), 0, X_{i})$ and $(Y_{i}(t), 1, X_{i})$ are identifiable under the IA IV assumption. We formally state the results in Theorem (ref).
We now present two examples that are special cases of Theorem 3.1. In the first example, we consider a function $g(\cdot)$ that is a trivial function of $T_{i}$: $g(Y_{i}(t), X_{i})$. Then, for $\mathbb{E}[g(Y_{i}(t), X_{i}) \mid T_{i}(1) > T_{i}(0)]$:
which matches the Theorem 3.1 part (b) and part (c) in abadie2003semiparametric. In the second example, we consider a function $g(\cdot)$ that is a trivial function of $(Y_{i}(t), X_{i})$, then, for $\mathbb{E}[T_{i} \mid T_{i}(1) > T_{i}(0)]$:
where the second equality uses the IV monotonicity in Assumption (ref).
The results in Theorem (ref) imply that we can identify any statistical characteristic defined in terms of the moments of the joint distribution of $(T_{i}, X_{i})$ for the subpopulations where $[Y_{i}(t) = y, T_{i}(1) > T_{i}(0)]$, with $t$ and $y \in \{ 0, 1 \}$. The intuition behind this result is the following. An immediate implication of Theorem (ref) is that the moments of the joint distribution of $(T_{i}, X_{i})$, conditional on compliers and a function of $Y_{i}(t)$, are also identifiable. We formally state the results in Proposition (ref).
We provide some examples of $g(T_{i}, X_{i})$ below. For instance, if we choose $g(T_{i}, X_{i}) = \left(X_{i}^{j} \right)^{p}$, where $X_{i}^{j}$ is the $j$-th component of $X_{i}$ and $p \in \mathbb{R}^{+}$, we can identify any moments of the covariate $X_{i}^{j}$ if the moments exist. In a GOTV experiment, $X_{i}^{j}$ could be a binary partisanship variable, indicating whether or not $i$ is a Democrat. By choosing $p = 1$, we can identify the probability that voters belonging to the type $[Y_{i}(t) = y, T_{i}(1) > T_{i}(0)]$ are Democrats. Another example is $g(T_{i}, X_{i}) = \mathbbm{1}\{ X^{j}_{i} \leq x \}$ where $x \in \mathbb{R}$. With this choice, we can identify the cumulative distribution function of $X_{i}^{j}$ among voters belonging to the type $[Y_{i}(t) = y, T_{i}(1) > T_{i}(0)]$. For instance, if $X_{i}$ represents personal income, we can identify the cumulative density function of income among voters of the type $[Y_{i}(t) = y, T_{i}(1) > T_{i}(0)]$.
Theorem 3.1 in abadie2003semiparametric shows that any statistical characteristic that can be defined in terms of moments of the joint distribution of $(Y_{i}, T_{i}, X_{i})$ is identified for compliers:
where $\kappa \equiv 1 - \frac{T_{i}(1 - Z_{i})}{\mathbb{P}[Z_{i} = 0]} - \frac{(1 - T_{i})Z_{i}}{\mathbb{P}[Z_{i} = 1]}$. Proposition (ref) strengthens Abadie's $\kappa$ by further conditioning on marginal potential outcomes. Thus, a natural question is whether or not we can point identify $\mathbb{E}[g(Y_{i}, T_{i}, X_{i}) \mid Y_{i}(t) = y, T_{i}(1) > T_{i}(0)]$ under the IA IV assumption. The answer is no. To see the intuition, we use $\mathbb{E}[g(Y_{i}, T_{i}, X_{i}) \mid Y_{i}(0) = 0, T_{i}(1) > T_{i}(0)]$ to illustrate:
where the first equality uses the fact that $T_{i} = Z_{i}$ for compliers, the fourth equality uses the IV independence assumption. Due to the presence of $\mathbb{E}[g(Y_{i}(1), 1, X_{i}) \mid Y_{i}(0) = 0, T_{i}(1) > T_{i}(0)] \mathbb{P}[Z_{i} = 1]$, which is about the joint distribution of potential outcomes, $\mathbb{E}[g(Y_{i}, T_{i}, X_{i}) \mid Y_{i}(0) = 0, T_{i}(1) > T_{i}(0)]$ is not point identified with the IA IV assumptions. The same intuition carries over to the remaining three cases in Proposition (ref).
Proposition (ref) can be applied to continuous $Y_{i}$ by defining a new indicator variable, $\Tilde{Y}_{i} = \mathbbm{1}\{ Y_{i} \in B \}$, where $B$ is a measurable set, and a new potential outcome, $\Tilde{Y}_{i}(t) = \mathbbm{1}\{ Y_{i}(t) \in B\}$. The result in Proposition (ref) holds for $\Tilde{Y}_{i}$ under the IA IV assumptions in Assumption (ref). An example of $B$ is: $B = \mathbbm{1}\{ Y_{i}(t) \leq \Tilde{y} \}$. That is, researchers can identify characteristics measured by $X_{i}$ of compliers and those with the potential outcome less than $\Tilde{y}$.
Under Assumption (ref), we can identify the statistical characteristics defined by the moments of the joint distribution of $(T_{i}, X_{i})$ for always-voters, never-voters, and mobilisable voters among compliers. The intuition follows the same reasoning as in Lemma (ref). We can strengthen the interpretation of the results in Proposition (ref) from conditioning on marginal potential outcomes to conditioning on joint potential outcomes under the monotone treatment response assumption in Assumption (ref). The results extend Theorem 3.1 in abadie2003semiparametric by further conditioning on persuasion types defined by the pair of potential outcomes. The results are formally stated in Theorem (ref).
Theorem (ref) is a powerful identification result. Although we cannot directly observe always-voters, never-voters, or mobilised voters among compliers, we can still profile these three unobservable subpopulations using treatment and covariates. We provide three remarks on the results. First, the conditional distribution functions of a covariate given persuasion types and compliers are identified because we can define $g(T_{i}, X_{i})$ as $g(T_{i}, X_{i}) = \mathbbm{1}\{ X^{j}_{i} \leq x \}$ with $X^{j}_{i}$ is the $j$-th component of $X_{i}$ and $x \in \mathbb{R}$. Furthermore, for measurable $g$, the expectations of $g(T_{i}, X_{i})$ conditional on the three unobservable subpopulations in Theorem (ref) are also identified, provided the expectation is well-defined. In other words, any statistical characteristics measured by the covariates $X_{i}$ of always-voters, never-voters, and mobilised voters among compliers are identified. Finally, by Bayes' rule, the conditional probability of belonging to a specific persuasion type, conditional on compliers and covariates, is also identified.
The estimands identified in Theorem (ref) provide important insights into the intervention's impact and mechanism. For instance, in a GOTV experiment, the theorem identifies the probability that a mobilised complier is a Democrat. Although GOTV experiments are typically non-partisan, they can result in partisan outcomes, such as disproportionately mobilising Democrats. This can affect closely contested elections and helps quantify how many Democrats were mobilised, aiding analysts in evaluating the cost-effectiveness of the intervention.
Theorem (ref) also helps assess the mechanisms by which the mobilisation affects voting. In a GOTV experiment, these results can test the hypothesis that voting is habit-forming (gerber2003voting). Prior voting records can serve as a measure of voting propensity, and if the hypothesis holds, always-voters among compliers should show the highest propensity, while never-voters should show the lowest.
In addition to Theorem (ref), there are other ways to profile voters using pre-treatment covariates. Consider this key quantity: conditional on compliers who will not vote without mobilisation, what traits define those who will vote when exposed to the treatment? Such a parameter is conditional on the voting outcome when voters are not mobilised, which is of interest to analysts focused on social justice (heckman1997making). For example, if a covariate measures whether a voter is African American, it allows us to determine, among compliers who would not vote without mobilisation, the percentage of additional African American voters who would vote after being mobilised. This parameter is particularly valuable for analysts seeking to better understand the effectiveness of mobilisation efforts, especially when the goal is to increase minority voter turnout. The identifiability of these estimands follows from the fact that the monotone treatment response assumption implies the identifiability of the joint distribution of the potential outcomes among compliers. These results are formally stated in Proposition (ref).
This section provides estimation and inference results for the estimands we proposed. Note that the estimands we proposed in prior sections usually take the form of a Wald estimand:
where $f$ and $h$ are measurable functions. For example, for the case of always-voters in Theorem (ref), $f(X_{i}, Y_{i}, T_{i}) = \sum_{z \in \{0, 1\}} g(z, X_{i}) \mathbbm{1}\{ Y_{i} = 1, T_{i} = 0 \}$, $h(Y_{i}, T_{i}) = Y_{i} (1 - T_{i})$. It is easy to see that the numerator in Equation (ref) is the coefficient of $Z_{i}$ from regressing $f(X_{i}, Y_{i}, T_{i})$ on $Z_{i}$ and a constant, while the denominator in Equation (ref) is the coefficient of $Z_{i}$ from regressing $h(Y_{i}, T_{i})$ on $Z_{i}$ and a constant. Therefore, the standard estimation and inference theory for Wald estimand applies immediately to the current case with an independently and identically distributed sample of $(Y_{i}, T_{i}, Z_{i}, X_{i})$. We can either employ the conventional asymptotic results for hypothesis testing or use the Anderson-Rubin test which is robust to weak identification. We provide a more detailed discussion on inference issues in Appendix F. Note that both inferential methods can be easily implemented in standard statistical software, say, ivreg2 and weakiv in Stata.
In this section, we discuss three points on identification results from previous sections. Firstly, we compare $\theta_{\text{local}}$ with classic estimands. Additionally, we propose a test for Assumption (ref) and a simple method to assess the sensitivity of the results to the monotone treatment response assumption.
As summarised in dellavigna2010persuasion, one popular estimand in the empirics of persuasion is the “approximated” persuasion rate $\tilde{\theta}_{\text{DK}}$:
Empirical researchers often use $\tilde{\theta}_{\text{DK}}$ to approximate the persuasion rate, $\mathbb{P}[Y_{i}(1) = 1 \mid Y_{i}(0) = 0]$. However, $\theta_{\text{DK}}$ is not a well-defined conditional probability, hence, it does not measure a persuasion rate for any subpopulation (jun2018identifying).
Instead, jun2018identifying propose the local persuasion rate, that measures the persuasion rate among compliers:
The local persuasion rate measures the percentage of compliers who take the action of interest if exposed to the treatment among those who will not take the action of interest without being exposed to the information treatment.
The results below show that under one-sided non-compliance, $\tilde{\theta}_{\text{DK}}$ equals $\theta_{\text{local}}$ under specific conditions on the distribution of potential outcomes and potential treatments. Suppose there is one-sided non-compliance in the control group. In this case, the two estimands are equal if and only if the proportion of untreated potential outcome being $0$ among untreated potential treatment being $0$ equals the proportion of never-voter among the always-takers. Suppose there is one-sided non-compliance in the treatment group. In this case, the two estimands are equivalent if and only if the untreated potential outcome is independent of the treated potential treatment.
Proposition (ref) complements the results in jun2018identifying. jun2018identifying show that $\tilde{\theta}_{\text{DK}} = \theta_{\text{local}}$ if certain conditions of the treatment effect homogeneity holds. Instead, after adding a one-sided non-compliance condition, Proposition (ref) shows that $\tilde{\theta}_{\text{DK}} = \theta_{\text{local}}$ if certain restrictions on the dependence between potential outcome and potential treatment hold.
The most closely related target parameter to the local persuasion rate is the complier causal attribution rate, which measures the proportion of observed outcome prevented by the hypothetical absence of the treatment among compliers (yamamoto2012understanding):
One main difference between $p_{C}$ and $\theta_{\text{local}}$ is that the conditioning set for $p_{C}$ is $[Y_{i}(1) = 1, T_{i} = 1, T_{i} > T_{i}(0)]$ but the conditioning set for $\theta_{\text{local}}$ is $[Y_{i}(0) = 0, T_{i} > T_{i}(0)]$. Therefore, a natural way to extend the local persuasion rate is to define the local persuasion rate on the untreated:
We can point identify $\theta_{\text{local untreated}}$ given Assumption (ref). The intuition of the identification of $\theta_{\text{local untreated}}$ is that, conditional on compliers, $T_{i} = Z_{i}$, thus, $\theta_{\text{local untreated}} = \theta_{\text{local}}$. We formally state the result in Proposition (ref).
The main identification results in Theorem (ref) rely on two assumptions: the IA IV assumptions and the monotone treatment response assumption. These assumptions impose restrictions on individuals' choice behaviours by ruling out the defiers and the demobilised voters. Therefore, we propose a sharp test for Assumption (ref).
The idea of the test closely relates to balke1997bounds. Assumptions 1, 2, 4, and 5 in Assumption (ref) imply that the observed quantity, $\mathbb{P}[Y_{i} = y, T_{i} = t, X_{i} \in A \mid Z_{i} = z]$, with $y, t, z \in \{0, 1\}$ and $A$ measurable, is a linear combination of the probability of the unobserved persuasion and compliance types:
Furthermore, the defiers and the demobilised voters are ruled out by the monotonicity assumptions in Assumption (ref) (that is, $\mathbb{P}[Y_{i}(0) = 1, Y_{i}(1) = 0] = \mathbb{P}[T_{i}(0) = 1, T_{i}(1) = 0] = 0$). Collecting these linear equations form a system of linear equations:
where $A_{\text{obs}}$ is a matrix that reflects the restrictions implied by Assumptions 1, 2, 4, and 5 in Assumption (ref), $\mathbf{p}$ is a non-negative vector that collects the probability of the unobserved persuasion and compliance types, and $\mathbf{b}$ is a collection of observed quantities. Note that matrix $A_{\text{obs}}$ can flexibly reflect the model restrictions. Different restrictions lead to different matrices $A_{\text{obs}}$. For example, if the model restrictions are the IA IV model without the IV relevance condition, the linear system of equations restricts the probability of defiers to zero (that is, $\mathbb{P}[T_{i}(0) = 1, T_{i}(1) = 0] = 0$).
The testable empirical implications, summarized in the system of linear equations above, provide a sharp characterization of Assumptions 1, 2, 4, and 5 in Assumption (ref). In other words, whenever the system of linear equations holds, there always exists another potential outcome and potential treatment model compatible with the data, in which Assumptions 1, 2, 4, and 5 of Assumption (ref) also hold (kitagawa2015test, mourifie2017testing, kedagni2020generalized).
An implication of Proposition (ref) is that to test the validity of Assumption (ref), for observed data $\{ Y_{i}, T_{i}, Z_{i}, X_{i} \}_{i = 1}^{n}$ that is an independently and identically distributed sample drawn from $P \in \mathbf{P}$, it suffices to test the null hypothesis:
where $\mathbf{P}_{0} \equiv \{ P \in \mathbf{P}: \exists \mathbf{p} \geq \mathbf{0} \text{ s.t. } A_{\text{obs}} \mathbf{p} = \mathbf{b} \}$, which is the set of distributions that is consistent with Assumptions 1, 2, 4, and 5 in Assumption (ref). Thus, if $H_{0}$ is rejected, we have strong evidence against the validity of the assumptions. However, if $H_{0}$ is not rejected, we cannot confirm the validity of the assumptions. In this precise sense, Assumptions 1, 2, 4, and 5 in Assumption (ref) are refutable but nonverifiable (kitagawa2015test).
In terms of the implementation of testing (ref), with discrete $X_{i}$, we can set $A$ to be the support of $X_{i}$, and proceed the test using the recent advancement on testing whether there exists a nonnegative solution to a possibly under-determined system of linear equations with known coefficients (bai2022testing, fang2023inference). bai2022testing propose to use subsampling method to test $H_{0}$, which can control size uniformly over $\mathbf{P}$ by the results in romano2012uniform. The test statistic in bai2022testing is given by:
where $\hat{\mathbf{b}}$ is an estimator of $\mathbf{b}$. For more discussions on the details of computing the test statistic, see Appendix (ref). Then, consider the following quantity:
where $N_{n} = \binom{n}{b}$, $j$ indexes the $j$th subsample of size $b$, $\hat{\mathbf{b}}_{j}$ is $\hat{\mathbf{b}}$ evaluated at $j$th subset of the data. The subsampling-based test is:
Besides testing the identification assumptions jointly in the previous subsection, we now develop a sensitivity analysis approach to help researchers assess to what extent the point identification results are sensitive to the monotone treatment response assumption. Note that we apply the sensitivity analysis to the identification results in Lemma (ref).
The sensitivity analysis builds on the idea in balke1997bounds. Note that the marginal distribution of the potential outcomes among compliers can be represented as the following system of linear equations:
Therefore, we can vary the size of $\mathbb{P}[Y_{i}(0) = 1, Y_{i}(1) = 0 \mid T_{i}(1) > T_{i}(0)]$ to see how the point identification results for the joint distribution of potential outcomes change. Here, with known $\mathbb{P}[Y_{i}(0) = 1, Y_{i}(1) = 0 \mid T_{i}(1) > T_{i}(0)]$, we can point identify $\mathbb{P}[Y_{i}(0) = 0, Y_{i}(1) = 0 \mid T_{i}(1) > T_{i}(0)]$, $\mathbb{P}[Y_{i}(0) = 0, Y_{i}(1) = 1 \mid T_{i}(1) > T_{i}(0)]$, and $\mathbb{P}[Y_{i}(0) = 1, Y_{i}(1) = 1 \mid T_{i}(1) > T_{i}(0)]$ from the system of equations above.
This section demonstrates the application of the methods using green2003getting as an example. First, we provide information on the empirical setup. Then, we illustrate our main identification results with data from green2003getting. Finally, we conduct the test for the identification assumptions and sensitivity analysis.
In 2001, green2003getting conducted randomised voter mobilisation experiments during local elections in the following six cities: Bridgeport, Columbus, Detroit, Minneapolis, Raleigh, and St. Paul. Detroit, Minneapolis, and St. Paul held mayoral elections, Bridgeport had a school board election, Columbus conducted a city council election, and Raleigh hosted a mayoral/city council election. The instrument $Z_{i}$ is a randomly assigned face-to-face contact from a coalition of nonpartisan student and community organizations, encouraging voters to vote. The face-to-face contact included a brief reminder of the upcoming election in the area. The treatment $T_{i}$ is whether or not voters received the face-to-face contact. The outcome variable $Y_{i}$ is voter turnout in the local election in 2001. There are two pre-treatment covariates that we are interested in. For the full sample, we are interested in whether or not voters voted in the 2000 presidential election. We also restrict the analysis to Bridgeport. For Bridgeport, we are interested in whether or not voters are Democrats. A summary statistics table is provided in Table (ref).
We first present the results for the marginal and joint distribution of potential outcomes of compliers in Table (ref). Our results reveal two interesting patterns. First, conditional on compliers, most of them are never-voters in both samples. Specifically, $61.9\%$ of voters are never-voters conditional on compliers in the full sample, and $75\%$ of voters are never-voters conditional on compliers in Bridgeport. The first stage results in Table (ref) imply that $29.3\%$ and $27.7\%$ of voters were compliers in the full sample and the Bridgeport sample, respectively. Hence, the estimated numbers of never-voters conditional on compliers in each sample, were 3434 and 375, respectively.
Second, only $7.9\%$ of voters are mobilised conditional on compliers in the full sample, and $13.9\%$ of voters are mobilised conditional on compliers in Bridgeport. Hence, the estimated number of mobilised voters conditioning on compliers in the full sample and the Bridgeport sample, were 438 and 70, respectively.
Moreover, the local persuasion rates in the full sample and the Bridgeport sample are $11.3\%$ and $15.7\%$, respectively. In other words, among the voters who are compliers and will not vote if they do not receive the GOTV intervention, $11.3\%$ and $15.7\%$ of them will vote in the full sample and the Bridgeport sample, respectively. The local persuasion rate is mechanically larger than the percentage of mobilised voters among compliers. This holds mechanically because the local persuasion rate reweights the percentage of mobilised voters by the proportion of the voters who will not vote if they do not receive the GOTV intervention.
We now apply Proposition (ref) and Theorem (ref) to this experiment. The results are presented in Table (ref). For the full sample, the probability of voting in the 2000 presidential election conditional on those who do not vote without the treatment and compliers is $60.3\%$. A more interesting finding is that the subpopulation of always-voters compliers has the highest probability (that is, $95.4\%$) of voting in the 2000 presidential election. The results show that if always-voters and compliers vote in the low-profile local elections regardless of the GOTV intervention, they were very likely vote in the high-profile 2000 presidential elections. This empirical pattern is consistent with the robust findings on the persistence of voting behaviour (gerber2003voting). One potential explanation of the persistence of the voting behaviour is that voting behaviour is habit-forming (gerber2003voting). As expected, the subpopulation of never-voters and compliers has the lowest probability of voting in the 2000 presidential election.
Another interesting finding is that the voting propensity in the 2000 presidential election of the mobilised compliers is very close to the always-voters and compliers. It is consistent with the findings that GOTV experiments mobilise the high-propensity voters. One potential explanation is that the GOTV programme only mobilises the voters who are on the margin of not voting, as the intervention only slightly lowers the voting cost. Hence, the mobilised voters should have a voting propensity that is close to the always-voters.
Moreover, we also apply Abadie's $\kappa$ to identify the statistical characteristic of the compliers based on whether a voter voted in the 2000 U.S. presidential election in Table (ref). The estimated likelihood that a complier voted in the 2000 U.S. presidential election is $67.4\%$. This propensity for voting is quite close to the average voting propensity in the full sample. In other words, compliers were statistically similar to the average voter in the full sample in terms of voting propensity.
In the Bridgeport sample, the most notable finding is that, among mobilised compliers, the estimated probability of being a Democrat is 81.3%. However, the confidence interval is quite wide. Additionally, we estimate that 3.1% of mobilised voters are also compliers and Democrats. Mobilising more Democrats in the Bridgeport school board election has practical implications for two main reasons. First, Democrats tend to be more pro-union, and second, turnout rates in these elections are typically low. For example, the turnout rate in the control group was 9.9% (green2003getting). The mobilised voters might vote for pro-union candidates and help elect candidates more likely to increase teachers' salaries, benefits, and improve their working conditions (anzia2011election).
Beyond these benefits, the methods developed in this paper also allows researchers to assess the cost of mobilising Democrats in a GOTV experiment. Since the mobilisation message was randomly assigned with a probability of 50%, the estimated likelihood of a Democratic, complier, and mobilised voter being mobilised is 1.6%, or around 28 people. Assuming the experiment's costs include (1) \$3,000 for administrative expenses (including but not limited to randomisation, training canvassers, etc.), and (2) \$30 per voter for outreach, the total cost of the experiment amounts to \$29,350. Therefore, the estimated average cost of mobilising a Democrat in this study is \$1,066.
Moreover, we also utilize Abadie's $\kappa$ to identify the statistical characteristic of the compliers based on whether a voter was a Democrat in Table (ref). The estimated likelihood of a complier being a Democrat is $57.3\%$. This propensity for being a Democrat is quite close to the average propensity for being a Democrat in the Bridgeport sample. In other words, compliers were statistically similar to the average voter in the Bridgeport sample in terms of the propensity for being a Democrat.
We implement the test for the Assumption (ref) by using Proposition (ref). We use the subsampling method in bai2022testing for this test. Note that the subsampling test in bai2022testing requires us to pick a size for the subsample with $b_{n} \xrightarrow[]{} \infty$ and $\frac{b_{n}}{n} \xrightarrow[]{} 0$. We set $b_{n}$ to $n^{\frac{2}{3}}$ here. Given that the observed test statistics are less than the $95$th quantiles of the resampled distribution, results in Figure (ref) show that we cannot reject the validity of the identification assumptions at the $5\%$ level for both the full sample and the Bridgeport sample.
Furthermore, we provide the sensitivity analysis results on the joint distribution of potential outcomes in Table (ref) by varying the degree to which the monotone treatment response assumption is violated among compilers. The results show that when the violation becomes larger, the proportion of mobilised voters among compliers increases.
This paper studies a binary IA IV model for persuasion. We show that in an econometric model of persuasion, it is possible to identify the joint distribution of potential outcomes among compliers. We develop a weighting method that helps researchers identify the statistical characteristics of persuasion types: always-voters and compliers, never-voters and compliers, and mobilised compliers. We apply the proposed methodology to profile persuasion types in the GOTV experiments by green2003getting. The results show that among compliers, roughly $10\%$ voters are mobilised. The results are also consistent with the findings that voters' voting behaviours are highly persistent and the mobilised voters are high propensity voters. In future research, researchers can apply the partial identification approach proposed by mogstad2018using to assess the welfare impact of the information treatment by partially identifying the persuasion rate, either with or without assuming a monotone treatment response.
I thank my advisors, Scott Gehlbach, Robert Gulotty, and Alexander Torgovitsky, who were gracious with their advice, support, and feedback. I additionally thank Alberto Abadie, Eric Auerbach, Stephane Bonhomme, Federico Bugni, Joshua Byun, Matias Cattaneo, Gustavo Diaz, Yingying Dong, Wayne Yuan Gao, Justin Grimmer, Peter Hull, Kosuke Imai, Sung Jae Jun, Michal Kolesar, Nadav Kunievsky, Xinran Li, Jonathan Mummolo, Molly Offer-Westort, Zhuan Pei, Maggie Penn, Kirill Ponomarev, James Robinson, Jonathan Roth, Francesco Ruggieri, Azeem Shaikh, Tymon Sloczynski, Joshua Ka Chun Shea, Liyang Sun, Max Tabord-Meehan, Christopher Walters, Linbo Wang, Yiqing Xu, Teppei Yamamoto, Boyang Zhang, as well as participants in conferences and seminars at the American Causal Inference Conference 2022, the Midwest Econometrics Group Conference 2022, Harvard, and Princeton, for their helpful comments on this paper. Finally, I thank the co-editor and the two anonymous referees for their constructive comments, which have substantially improved the paper.