The exact contents of citations.db main_text.text for this paper — one flattened LaTeX string, title through conclusion, appendix excluded, unmodified except for removing email addresses. This is what our citation measures are computed over.
58,792 characters
Interpreting TSLS Estimators in Information Provision Experiments
\title{Interpreting TSLS Estimators in Information Provision Experiments\thanks{We are grateful to Isaiah Andrews, Anna Mikusheva, and Frank Schilbach for their guidance and support; to Josh Angrist, David Autor, Dylan Balla-Elliott, Simon Jäger, Haruki Kono, and participants at the MIT Behavioral, Econometrics, and Third-Year Lunches for helpful comments and conversations; to Simon Jäger, Chris Roth, Nina Roussille, and Benjamin Schoefer for providing us with an excellent replication package; to Olivier Coibion, Yuriy Gorodnichenko, and Saten Kumar for making public an excellent replication package; to Chantal Pezold for providing us with preliminary experimental data. We acknowledge financial support from the George and Obie Shultz Fund, and the National Science Foundation Graduate Research Fellowship under Grant No. 1745302.}}
\author{Vod Vilfort\thanks{
Department of Economics, MIT, Cambridge, MA 02142, [email removed]} \and Whitney Zhang\thanks{
Department of Economics, MIT, Cambridge, MA 02142, [email removed]}}
\date{June 21, 2024}
\maketitle
\begin{abstract}
To estimate the causal effects of beliefs on actions, researchers often run information provision experiments. We consider the causal interpretation of two-stage least squares (TSLS) estimators in these experiments. We characterize common TSLS estimators as weighted averages of causal effects, and interpret these weights under general belief updating conditions that nest parametric models from the literature. Our framework accommodates TSLS estimators for both passive and active control designs. Notably, we find that some passive control estimators allow for negative weights, which compromises their causal interpretation. We give practical guidance on such issues, and illustrate our results in two empirical applications.
\end{abstract}
\pagebreak
\section{Introduction}
There has been a surge in research that uses the random provision of information to estimate the causal effects of beliefs on actions.\footnote{In macroeconomics, researchers have studied the effects of beliefs about inflation, GDP growth, and other macroeconomic indicators on firm and household decision making \citep{coibion2020inflation, coibion2021effect, coibion2022monetary, coibion2023forward, kumar2023effect}; in labor economics, researchers have studied the effects of beliefs about others' wages on one's own efforts and job search decisions \citep{cullen2022much, jager2022worker}, and how beliefs about labor market tightness affect support for unions \citep{pezold_labor_2023}; at the intersection of labor and public finance, researchers have studied the effect of beliefs about future government benefits on human capital investments \citep{deshpande2023lack}; yet others have studied the effects of beliefs about discrimination on policy preferences \citep{haaland2023racial, settele_how_2022}.} These information provision experiments provide a basis for testing the assumptions of economic models, differentiating across theoretical mechanisms, and informing economic policy \citep{haaland2023designing}. Given the importance of these goals, it is essential that the causal parameters of interest be carefully defined and accurately estimated.
However, in practice, the causal parameters of interest are often informally defined, or are developed in stylized models that impose strong conditions on agents' beliefs, learning, and actions. In such models, it is unclear which of the implied restrictions drive the conclusions of an information provision experiment. Relatedly, it is ambiguous which of the many estimation strategies from the literature recover interpretable causal parameters. This paper addresses both of these concerns.
An information provision experiment generally proceeds as follows. First, the experiment elicits \textit{features} (e.g., expectations) of agents' prior beliefs over a set of action-relevant states. Next, the experiment randomly assigns agents to different groups---either to a control group receiving no information, or to one of potentially multiple treatment groups receiving \textit{signals}, where a signal is a piece of relevant information. Then, the experiment elicits features of agents' posterior beliefs, and records the actions taken under those posterior beliefs. Finally, the experiment uses group assignment to instrument for beliefs in two-stage least squares (TSLS) regressions of actions on posterior features: TSLS specifications in \textit{passive control} experiments compare the control group to the treatment group(s), whereas specifications in \textit{active control} experiments compare across treatment groups.
In Section \ref{main:sec:setup}, we introduce an instrumental variables (IV) framework in which beliefs affect actions through features. We consider the \textit{partial effects} of features on actions, which we allow to vary across agents and feature values. In Section \ref{main:sec:TSLS}, we characterize TSLS estimators from the literature as weighted average partial effects (APEs) across these two margins. Notably, without further structure on belief updating to ensure IV monotonicity, the weights on agents' partial effects can be \textit{negative}, which compromises the causal interpretation of TSLS estimators \citep{imbens1994identification, blandhol2022tsls}.
In Section \ref{main:sec:interpretation}, we propose conditions that ensure IV monotonicity. First, we propose \textit{signal monotonicity}, which formalizes the idea that agents should update their beliefs ``towards'' the signals. We motive signal monotonicity in a model of posterior formation where agents perceive the signals as realizations from probability distributions that satisfy the monotone likelihood ratio (MLR) property. This allows for a broad class of signal distributions and belief updating rules. In particular, signal monotonicity nests existing models that assume Gaussian distributions and Bayesian updating \citep{armantier2016price, cavallo2017inflation, armona2019home, cullen2022much, fuster2022expectations, balla2022determinants}. We additionally propose \textit{control-group stability} for passive control comparisons, and \textit{treatment-group neutrality} for active control comparisons; both formalize common experimental design considerations.
Under these conditions, TSLS estimators recover APEs that weight agents according to how much their beliefs respond to information provision. In particular, the weights for active control estimators emphasize agents with larger differences in their counterfactual posterior features across treatment groups (i.e., low versus high signal). Analogously, the weights for passive control estimators emphasize agents with larger differences in their counterfactual posterior features across treatment and control (i.e., signal versus no signal). However, some passive control estimators (i) up-weight agents with larger \textit{perception gaps} (i.e., the difference between prior features and signals); or (ii) are contaminated by linear combinations of the TSLS first-stage coefficients. The latter allows for negative weights, which we advise against. The former ensures non-negative weights, but leads to a different APE than the baseline case above. We interpret these differences and give practical recommendations in Section \ref{main:sec:recs}.
In Section \ref{main:sec:applications}, we take our results on passive control estimation to \cite{kumar2023effect} and \cite{jager2022worker}. In the \cite{kumar2023effect} application, we find that for three outcomes, the uncontaminated estimators produce coefficients that are one-third to two-thirds of the magnitude of those produced by the contaminated estimators; for one outcome, while the contaminated coefficients are statistically significant from the null, the uncontaminated coefficients are not. Together, we take this as evidence that negative weights meaningfully impact the estimates from the contaminated specifications. In the \cite{jager2022worker} application, for one outcome, the non-up-weighted estimator produces a coefficient that is 30-40\% larger in magnitude than those of the other specifications. Altogether, these empirical results highlight that the choice of weights can substantially impact the magnitude and significance of one's estimates.
\paragraph{Related Literature.}
Our formulation of the causal effects of beliefs as the partial effects of features aligns with existing empirical practice, and nests models considered in the literature where agents optimize their actions as a function of their beliefs \citep{cullen2022much, balla2023identifying, jager2022worker}. In allowing for heterogeneous partial effects, we are consistent with characterizations of IV estimators from other settings with continuous endogenous variables \citep{angrist2000interpretation, rambachan2021common, andrews2023causal}. To our knowledge, heterogeneity in the feature value margin has not received explicit attention in the information provision literature---we show in Section \ref{main:sec:recs} that accounting for this margin affects the interpretation of the policy counterfactuals captured by passive and active control experiments.
Our proposed conditions correspond to notions from the IV literature. For active control estimators, signal monotonicity and treatment-group neutrality correspond to the canonical IV monotonicity condition \citep{imbens1994identification, angrist2000interpretation}. In particular, agents' posterior features are larger when provided a high signal as opposed to a low signal. For passive control estimators, signal monotonicity and control-group stability correspond to ``weak'' IV monotonicity, which allows the direction of IV monotonicity to vary by covariates \citep{sloczynski2020should, blandhol2022tsls}. In particular, agents' posterior features are larger when provided a signal above their prior feature, and smaller when provided a signal below their prior feature. Finally, the contamination that we find in some passive control estimators is an instance of TSLS specifications failing to be ``monotonicity correct'' in their first-stage \citep{blandhol2022tsls}.
\cite{haaland2023designing} survey applications of information provision experiments, and give guidance on experimental design, belief elicitation techniques, and other technical challenges. In contrast, we develop theory for the identification and estimation of causal effects. The closest to our paper in this regard is the independent and concurrent work of \cite{balla2023identifying}, who likewise studies the interpretation of TSLS estimators in information provision experiments. \cite{balla2023identifying} considers the partial effects of expectations, and targets an APE that places equal weights across agents. To identify this APE, \cite{balla2023identifying} restricts heterogeneity in the feature value margin, and appeals to the structure of (i) active control comparisons; and (ii) linear updating of expectations. In contrast, since we primarily seek to characterize existing specifications, we achieve identification under weaker conditions on agents' actions, in both passive and active control experiments, and for more general learning environments. Finally, while \cite{balla2023identifying} also interprets TSLS specifications from the literature, we discuss and characterize a more comprehensive set of specifications in a more general framework.
\section{Setup}\label{main:sec:setup}
Let $\Delta(\Omega)$ denote the set of probability distributions over states $\omega \in \Omega$. Agents' beliefs $B$ are contained in a subset of distributions $\mathcal{B} \subseteq \Delta(\Omega)$. Examples include (i) sets of distributions for which relevant moments exist; and (ii) families of parametric distributions considered in the literature. We use $\phi: \mathcal{B} \to \mathbb{R}$ to index features of interest. For instance, when $\Omega \subseteq \mathbb{R}$, examples include the mean $\mu(B) := \int \omega dB(\omega)$ and the variance $ \sigma^{2}(B) := \int (\omega - \mu(B))^{2} dB(\omega)$.
\subsection{Experiment}
An information provision experiment generally proceeds as follows. First, the experiment elicits features $\phi(B_{i0})$ of agents' prior beliefs $B_{i0}$. Next, the experiment randomly assigns agents to groups $g \in \mathcal{G}$. In passive control experiments, the groups are $\mathcal{G} = \{C, T\}$: Agents assigned to treatment receive signals $S_{i}^{T}$, whereas agents assigned to control receive no information $S_{i}^{C} := \varnothing$. In active control experiments, the groups are $\mathcal{G} = \{L, H\}$: Agents assigned to high treatment receive signals $S_{i}^{H}$, whereas agents assigned to low treatment receive signals $S_{i}^{L} < S_{i}^{H}$.
Given group assignment $G_{i} \in \mathcal{G}$, agents form posterior beliefs $B_{i1} \equiv B_{i1}^{G_{i}}$. These posterior beliefs influence agents' actions/outcomes $Y_{i} \in \mathbb{R}$, which the experiment records; we assume there exist functions $Y_{i}^{g}(B)$ such that $Y_{i} \equiv Y_{i}^{G_{i}}(B_{i1})$. Finally, the experiment elicits posterior features $\phi(B_{i1})$. The goal is to estimate the causal effects of beliefs on actions.
\begin{example}[name=Passive Control, label=main:lab:example:jager] \cite{jager2022worker} run a passive control experiment on a sample of workers from the German Socio-Economic Panel to investigate if workers have accurate beliefs about the wage distribution.
In our notation, workers $i$ have beliefs $B$ about the wages $\omega$ at the best job they would find (i.e., their outside option) if forced to leave their current job. These beliefs influence workers' labor market behavior $Y_{i}$, such as their intended probability of looking for a new job or asking for a raise. The treatment group receives the average wage $S_{i}^{T}$ of workers with similar characteristics. \cite{jager2022worker} elicit workers' prior and posterior expectations $\mu(B_{i0}), \mu(B_{i1})$ of the wages at their outside options.
\end{example}
\begin{example}[name=Active Control, label=main:lab:example:roth]
Households' personal economic outlook and economic decisions depend on their macroeconomic expectations. To study these relationships, \cite{roth2020expectations} run an active control experiment on a sample of U.S. households.
In our notation, households $i$ have beliefs $B$ about the probability $\omega$ of a recession. These beliefs influence households' actions/outcomes $Y_{i}$, such as their anticipated earnings growth and net stock purchases. For $g = L$, the experiment provides the predicted probability of a recession from a pessimistic forecaster. For $g = H$, the experiment provides the analogous information from an optimistic forecaster. Thus, $S_{i}^{L} < S_{i}^{H}$, with $S_{i}^{g}$ constant across $i$. \cite{roth2020expectations} elicit households' prior and posterior expectations $\mu(B_{i0}), \mu(B_{i1})$ of the probability of a recession.
\end{example}
\begin{remark}
We focus on experiments that provide quantitative information $S_{i}^{g} \in \mathbb{R} \cup \varnothing$, since that is the leading case in the literature. That said, there are settings where the information has qualitative components---Appendix \ref{main:sec:general.identification} covers these settings.
\end{remark}
\begin{remark}
The results that follow apply to \textit{pairs} of comparison groups $\{g, \Tilde{g}\}$. Therefore, in experiments with more than two treatment groups, one can condition on pairs of comparison groups, apply our results, and then aggregate as necessary---see Appendix \ref{main:sec:general.estimation} for details. In considering pairs, we also avoid complications that arise with the causal interpretation of TSLS with multiple instruments \citep{mogstad2021causal}.
\end{remark}
\subsection{Instrumental Variables}\label{main:sec:IV}
The random assignment of agents to groups provides a basis for estimating causal effects. Formally, we assume that group assignment $G_{i}$ is a valid instrument. In what follows, $X_{i}$ is a vector of agent characteristics.
\begin{assumption}[Valid Instrument]\label{main:ass:IV}
$G_{i}$ satisfies
\begin{enumerate}[label=(\roman*)]
\item independence: $G_{i} \perp\!\!\!\!\!\!\perp (X_{i}, B_{i0}, \{S_{i}^{g}, B_{i1}^{g}, Y_{i}^{g}(B)\}_{g \in \mathcal{G}, B \in \mathcal{B}})$;
\item exclusion: $Y_{i}^{g}(B) = Y_{i}(B)$, $\forall g, B$.
\end{enumerate}
\end{assumption}
Independence means that the experiment cannot condition group assignment $G_{i}$ on agent characteristics $X_{i}$, prior features $\phi(B_{i0})$, anticipated actions, and so on. However, it allows the content of the signals $S_{i}^{g}$ to depend on such variables \citep{jager2022worker, deshpande2023lack}. In any case, random assignment is sufficient for Assumption \ref{main:ass:IV}(i).
Exclusion means that group assignment only impacts actions through posterior beliefs. Therefore, one practical concern is that information provision may also affect actions through emotional responses \citep{haaland2023designing}. However, Assumption \ref{main:ass:IV}(ii) does accommodate emotional responses that solely affect posterior \textit{formation}---examples include belief-based utility or motivated reasoning \citep{brunnermeier2005optimal, epley2016mechanics}.
\subsection{Feature Action Functions}
To formalize a tractable notion for the causal effects of beliefs on actions, we assume that actions depend on beliefs through a single feature of interest $\phi$. In what follows, we suppose that the set of possible feature values $\phi(\mathcal{B}) := \{\phi(B): B \in \mathcal{B}\}$ is convex: For any two values in $\phi(\mathcal{B})$, any third value between them can be rationalized by some $B \in \mathcal{B}$. This ensures that the causal effects defined in Section \ref{main:sec:TSLS} correspond to beliefs that the agents could actually hold.\footnote{Convexity holds whenever the set of possible beliefs $\mathcal{B}$ is sufficiently rich---see Appendix \ref{main:sec:primitive.conditions}.}
\begin{assumption}[Feature Action Functions]\label{main:ass:parametric}
$\phi(\mathcal{B})$ is convex, and there exists a continuously differentiable function $Y_{i}^{\phi}(v)$ defined over it such that $Y_{i}(B) = Y_{i}^{\phi}(\phi(B))$.
\end{assumption}
Assumption \ref{main:ass:parametric} aligns with existing practice from the literature, which often frames the causal effects of beliefs on actions in terms of features. If $\mathcal{B}$ is parametrized by $\phi$, then actions depend on beliefs through $\phi$; examples include one-parameter exponential families \citep{lehmann2006theory}. If we broaden $\mathcal{B}$ to be the set of distributions with finite second moments, then another approach is to restrict preferences. For example, if we assume that agents are risk neutral in the sense that their optimal actions only depend on beliefs via first moments, then $Y_{i}(B) = Y_{i}^{\mu}(\mu(B))$. We formalize these arguments in Appendix \ref{main:sec:primitive.conditions}.
In some cases, Assumption \ref{main:ass:parametric} allows beliefs to affect actions through additional features. In particular, if $Y_{i}(B) = Y_{i}^{\phi, \eta}(\phi(B), \eta(B))$ and $\eta(B_{i1}^{g}) = \eta(B_{i1}^{\Tilde{g}})$ for $\mathcal{G} = \{g,\Tilde{g}\}$, then we can take $Y_{i}^{\phi}(\phi(B)) := Y_{i}^{\phi, \eta}(\phi(B), \eta(B_{i1}^{g}))$. This accommodates models from the literature that predict $\sigma^{2}(B_{i1}^{H}) = \sigma^{2}(B_{i1}^{L})$ in active control experiments with $\phi = \mu$---see Appendix \ref{main:sec:gaussian.model}. An analogous argument applies for \textit{cross-learning}, which is when information about one state affects beliefs about another state \citep{haaland2023designing}. For example, information about inflation may influence expectations of both inflation \textit{and} economic growth \citep{coibion2023forward}. In such cases, it suffices to rule out \textit{differences} in cross-learning across $\{g,\Tilde{g}\}$.
One caveat is that, given a pair of comparison groups, we cannot allow multiple features to change at the same time. For example, we rule out comparisons where agents in one of the groups receive multiple signals with the intention of shifting multiple features \citep{cullen2022much, kumar2023effect, coibion2021effect}. Otherwise, there would be multiple endogenous variables; in such cases, TSLS estimators generally do not recover interpretable causal parameters \citep{bhuller20222sls}.
\section{TSLS}\label{main:sec:TSLS}
We formulate the causal effects of beliefs on actions as the partial effects $\partial_{v}Y_{i}^{\phi}(v)$ of feature $\phi$.\footnote{In Appendix \ref{main:sec:alt.formulations}, we discuss alternative formulations, including one that accomodates discrete actions.} TSLS recovers APEs across feature values $v$ and agents $i$. In Sections \ref{main:sec:passive.specifications} and \ref{main:sec:active.specifications}, we derive these APEs for passive and active control specifications from the literature. In Section \ref{main:sec:takeaways.specifications}, we summarize key takeaways and motivate the need for assumptions on belief updating behavior.
In what follows, $W_{i}$ is a vector that includes $1$ and potentially other variables that are independent of $G_{i}$. Let $\textnormal{sign}(v) := \mathds{1}\{v \geq 0\} - \mathds{1}\{v \leq 0\}$ and $\psi_{i}^{g\tilde{g}} := \textnormal{sign}(\phi(B_{i1}^{\tilde{g}}) - \phi(B_{i1}^{g}))$. Let $\lambda_{i}^{g\tilde{g}}(v)$ denote the density function of the uniform distribution on the values $v$ between $\phi(B_{i1}^{g})$ and $\phi(B_{i1}^{\tilde{g}})$. Propositions \ref{main:prop:passive.baseline} and \ref{main:prop:active.baseline} follow from TSLS algebra and the fundamental theorem of calculus, in the spirit of \citet[Theorem 1]{angrist2000interpretation}. See Appendix \ref{main:sec:proofs} for details.
\subsection{Passive Control TSLS}\label{main:sec:passive.specifications}
\begin{specification}\label{main:spec:passive}
Consider $I_{i}$, a sub-vector of $W_{i}$. Let $F_{i} = W_{i}'\pi_{0} + \mathds{1}\{G_{i} = T\}I_{i}'\pi$ be the first-stage population linear regression of $\phi(B_{i1})$ on $W_{i}$ and $I_{i}\mathds{1}\{G_{i} = T\}$. The passive control specification is
\begin{align*}
\begin{split}
\phi(B_{i1}) &= W_{i}'\pi_{0} + \mathds{1}\{G_{i} = T\}I_{i}'\pi + \zeta_{i}, \\
Y_{i} &= W_{i}'\gamma_{0} + \gamma^{CT} F_{i} + \upsilon_{i},
\end{split}
\end{align*}
where $\zeta_{i}$ and $\upsilon_{i}$ are population residuals. The coefficient of interest is $\gamma^{CT}$.
\end{specification}
\begin{proposition}\label{main:prop:passive.baseline}
Let $\mathcal{G} = \{C, T\}$ and suppose Assumptions \ref{main:ass:IV} and \ref{main:ass:parametric} are satisfied. If (i) $\mathbb{E}[W_{i}W_{i}']$ is full-rank and $\mathbb{P}(G_{i} = T) \in (0,1)$; and (ii) $\mathbb{E}[I_{i}\phi(B_{i1})|G_{i}=T]- \mathbb{E}[I_{i}\phi(B_{i1})|G_{i}=C] \neq 0$, then $\gamma^{CT}$ identifies
\begin{align}\label{main:eq:passive.baseline}
\begin{split}
\beta^{CT} := \mathbb{E}[w_{i}^{CT}\Bar{\beta}_{i}^{CT}], \quad \quad w_{i}^{CT} &= \frac{|\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})|\psi_{i}^{CT}I_{i}'\pi}{\mathbb{E}[|\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})|\psi_{i}^{CT}I_{i}'\pi]}, \\[10pt]
\Bar{\beta}_{i}^{CT} &:= \int \lambda_{i}^{CT}(v)\partial_{v}Y_{i}^{\phi}(v)dv.
\end{split}
\end{align}
\end{proposition}
Passive control specifications recover an APE, constructed as follows. First, for each agent $i$, the partial effect curve $ \partial_{v}Y_{i}^{\phi}(v)$ is aggregated across the range of values $v$ between $\phi(B_{i1}^{C})$ and $\phi(B_{i1}^{T})$. The density function $\lambda_{i}^{CT}(v)$ places uniform weight across this range of feature values, which generates $\Bar{\beta}_{i}^{CT}$. Then, $\Bar{\beta}_{i}^{CT}$ is averaged across agents with weights $w_{i}^{CT}$ that are proportional to (i) the signed difference $|\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})|\psi_{i}^{CT}$ in $i$'s counterfactual posterior features across treatment and control; and (ii) the agent-specific linear combination $I_{i}'\pi$ of first-stage coefficients:
\begin{align*}
w_{i}^{CT} \propto |\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})|\psi_{i}^{CT}I_{i}'\pi.
\end{align*}
The weights $\lambda_{i}^{CT}(v)$ for the feature values are non-negative and integrate to one. The weights $w_{i}^{CT}$ for the agents integrate to one, but can be negative due to $\psi_{i}^{CT}$ and $I_{i}$. The literature considers a number of ``interactions'' $I_{i}$---note that $\pi$ cancels out when $I_{i}$ is a scalar.
\begin{interaction}\label{main:int:IPIV}
If $I_{i} = I_{i}^{sign} := \textnormal{sign}(S_{i}^{T} - \phi(B_{i0}))$, then
\begin{align*}
w_{i}^{CT} \propto |\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})|\psi_{i}^{CT}\textnormal{sign}(S_{i}^{T} - \phi(B_{i0})).
\end{align*}
This interaction is in the spirit of \cite{cantoni2019protests}. In the literature, $S_{i}^{T} - \phi(B_{i0})$ is known as the \textit{perception gap}.
\end{interaction}
\begin{interaction}\label{main:int:cullen}
If $I_{i} = I_{i}^{gap} := S_{i}^{T} - \phi(B_{i0})$, then
\begin{align*}
w_{i}^{CT} \propto |\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})|\psi_{i}^{CT}(S_{i}^{T} - \phi(B_{i0})).
\end{align*}
This interaction is in the spirit of \cite{cullen2022much}, \cite{galashin2020macroeconomic}, and \cite{balla2022determinants}.
\end{interaction}
\begin{interaction}\label{main:int:jager}
If $I_{i} = I_{i}^{1, gap} := (1, S_{i}^{T} - \phi(B_{i0}))'$, then
\begin{align*}
w_{i}^{CT} \propto |\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})|\psi_{i}^{CT}(\pi_{1} + \pi_{2}(S_{i}^{T} - \phi(B_{i0}))).
\end{align*}
This interaction is in the spirit of \cite{jager2022worker}.
\end{interaction}
\begin{interaction}\label{main:int:kumar}
If $I_{i} = I_{i}^{1, prior} := (1, \phi(B_{i0}))'$, then
\begin{align*}
w_{i}^{CT} \propto |\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})|\psi_{i}^{CT}(\pi_{1}+ \pi_{2}\phi(B_{i0})).
\end{align*}
This interaction is in the spirit of \cite{kumar2023effect} and \citet{coibion2021effect, coibion2022monetary, coibion2023forward}. To the best of our knowledge, this interaction is only used when the signals $S_{i}^{T}$ are common across agents, which is allowed in our framework. This interaction is also analogous to an interaction from \cite{deshpande2023lack} with $I_{i} = (1, S_{i}^{T}, \phi(B_{i0}))'$. In this case, the signals must be heterogeneous, or else the first-stage coefficients cannot be identified.
\end{interaction}
\subsection{Active Control TSLS}\label{main:sec:active.specifications}
\begin{specification}\label{main:spec:active} $F_{i} = W_{i}'\pi_{0} + \pi \mathds{1}\{G_{i} = H\}$ is the first-stage population linear regression of $\phi(B_{i1})$ on $W_{i}$ and $\mathds{1}\{G_{i} = H\}$. The active control specification is
\begin{align*}
\begin{split}
\phi(B_{i1}) &= W_{i}'\pi_{0} + \pi \mathds{1}\{G_{i} = H\} + \zeta_{i}, \\
Y_{i} &= W_{i}'\gamma_{0} + \gamma^{LH} F_{i} + \upsilon_{i},
\end{split}
\end{align*}
where $\zeta_{i}$ and $\upsilon_{i}$ are population residuals. The coefficient of interest is $\gamma^{LH}$.
\end{specification}
\begin{proposition}\label{main:prop:active.baseline}
Let $\mathcal{G} = \{L, H\}$ and suppose Assumptions \ref{main:ass:IV} and \ref{main:ass:parametric} are satisfied. If (i) $\mathbb{E}[W_{i}W_{i}']$ is full-rank and $\mathbb{P}(G_{i} = H) \in (0,1)$; and (ii) $\mathbb{E}[\phi(B_{i1})|G_{i} = H] - \mathbb{E}[\phi(B_{i1})|G_{i} = L] \neq 0$, then $\gamma^{LH}$ identifies
\begin{align}\label{main:eq:active.baseline}
\begin{split}
\beta^{LH} := \mathbb{E}[w_{i}^{LH}\Bar{\beta}_{i}^{LH}], \quad \quad w_{i}^{LH} &:= \frac{|\phi(B_{i1}^{H}) - \phi(B_{i1}^{L})|\psi_{i}^{LH}}{\mathbb{E}[|\phi(B_{i1}^{H}) - \phi(B_{i1}^{L})|\psi_{i}^{LH}]}, \\[10pt]
\Bar{\beta}_{i}^{LH} &:= \int \lambda_{i}^{LH}(v)\partial_{v}Y_{i}^{\phi}(v)dv.
\end{split}
\end{align}
\end{proposition}
Active control specifications recover an APE that is constructed analogously to the passive control case. The weights $\lambda_{i}^{LH}(v)$ for the feature values are non-negative and integrate to one. The weights $w_{i}^{LH}$ for the agents integrate to one, but can be negative due to $\psi_{i}^{LH}$.
\subsection{Takeaways}\label{main:sec:takeaways.specifications}
Thus far, we assumed that (i) group assignment is a valid instrument for posterior beliefs; and (ii) actions depend on beliefs through a single feature of interest. Under these assumptions, we showed that TSLS specifications from the literature recover APEs $\beta^{CT}$ and $\beta^{LH}$ with weights $w_{i}^{CT}$ and $w_{i}^{LH}$ that can be negative. This negative weighting compromises the causal interpretation of $\beta^{CT}$ and $\beta^{LH}$. For instance, if some of the weights are negative, then it is possible for $\beta^{CT}$ and $\beta^{LH}$ to be negative even when all partial effects are positive \citep{blandhol2022tsls}. To address this concern, we now place structure on belief updating.
\section{Interpretation}\label{main:sec:interpretation}
The weights $w_{i}^{g\tilde{g}}$ are contaminated by $\psi_{i}^{g\tilde{g}} := \textnormal{sign}(\phi(B_{i1}^{\tilde{g}}) - \phi(B_{i1}^{g}))$. This source of contamination suggests that we should restrict the ``direction'' of agents' responses to information provision. Section \ref{main:sec:signal.monotonicity} introduces a \textit{signal monotonicity} condition that makes this idea precise. To achieve IV monotonicity for passive control, Section \ref{main:sec:stability} additionally introduces a \textit{control-group stability} condition---Section \ref{main:sec:passive.parameters} then characterizes $w_{i}^{CT}$. To achieve IV monotonicity for active control, Section \ref{main:sec:neutrality} additionally introduces a \textit{treatment-group neutrality} condition---Section \ref{main:sec:active.parameters} then characterizes $w_{i}^{LH}$.
\subsection{Signal Monotonicity}\label{main:sec:signal.monotonicity}
For agent $i$ in group $g$, let $B_{i1}^{g}(\cdot|s)$ be the belief updating rule, which maps signals $s \in \mathbb{R} \cup \varnothing$ to beliefs $B \in \mathcal{B}$. Note that $B_{i1}^{g}(\cdot|s)$ implicitly depends on priors $B_{i0}$, and $B_{i1}^{g} \equiv B_{i1}^{g}(\cdot|S_{i}^{g})$ in the realized experiment.
\begin{assumption}[Signal Monotonicity]\label{main:ass:signal.monotonicity}
If $s' > s$, then $\phi(B_{i1}^{g}(\cdot|s')) \geq \phi(B_{i1}^{g}(\cdot|s))$.
\end{assumption}
Assumption \ref{main:ass:signal.monotonicity} restricts the direction of agents' responses to information provision; intuitively, agents should update ``towards'' the signal. This assumption is reasonable in many experiments. For example, in the setting of \cite{kumar2023effect}, firms in one treatment group are given the average forecasts of mean GDP from a panel of experts. It is reasonable to assume that larger average forecasts lead firms to form higher expectations of mean GDP. In another treatment group, firms are given the difference in forecasts between the most and least optimistic experts. It is reasonable to assume that larger dispersion in forecasts leads firms to be more uncertain in their beliefs.
However, Assumption \ref{main:ass:signal.monotonicity} can sometimes be invalid. For example, consider \cite{coibion2022monetary}, who study households' expectations of inflation. In one treatment group, they provide information about unemployment. Some households may believe that higher unemployment implies lower inflation, whereas others may believe the opposite. Therefore, signal monotonicity is unreasonable for this treatment group.
As demonstrated, we can often intuit the validity of signal monotonicity. To complement this intuition, Appendix \ref{main:sec:posterior.formation} formalizes conditions under which Assumption \ref{main:ass:signal.monotonicity} is satisfied for the leading case of $\phi = \mu$. In summary, the main condition is that agents perceive the signals as realizations from distributions that satisfy a monotone likelihood ratio (MLR) property. In particular, agents assigned to treatment assume that the experiment tends to provide larger signals under larger realizations of the state.
The MLR property is satisfied for the set of Gaussian beliefs considered in the information provision literature---see Appendix \ref{main:sec:gaussian.model}. More generally, many exponential families satisfy the MLR property in their respective sufficient statistics \citep{casella2021statistical}. Given the MLR structure on agents' perceptions, we show in Appendix \ref{main:sec:posterior.formation} that Assumption \ref{main:ass:signal.monotonicity} is satisfied for $\phi = \mu$ under various belief updating rules, including (i) the Bayesian baseline considered in the information provision literature \citep{armantier2016price, cavallo2017inflation, armona2019home, cullen2022much, fuster2022expectations, balla2022determinants}; and (ii) systematic deviations from Bayesian updating considered in the behavioral economics literature \citep{gabaix2019behavioral, benjamin2019errors}.
\subsection{Stability}\label{main:sec:stability}
We first consider passive control comparisons. Let $S_{i}^{\phi} := \phi(B_{i0})$ denote agent $i$'s prior feature.
\begin{definition}[Stability]
A passive control comparison is \textit{control-group stable} if
\begin{align*}
\phi(B_{i1}^{T}(\cdot|S^{\phi}_{i})) = \phi(B_{i1}^{C}(\cdot|S^{C}_{i})).
\end{align*}
\end{definition}
If a passive control comparison is control-group stable, then the provision of $s = S_{i}^{\phi}$ in the treatment group is the same as not providing any information in the control group. Notice
\begin{align*}
\psi_{i}^{CT} = \textnormal{sign}([\phi(B_{i1}^{T}(\cdot|S^{T}_{i})) - \phi(B_{i1}^{T}(\cdot|S^{\phi}_{i}))] + [\phi(B_{i1}^{T}(\cdot|S^{\phi}_{i})) - \phi(B_{i1}^{C}(\cdot|S^{C}_{i}))]).
\end{align*}
Thus, under control-group stability, signal monotonicity implies
\begin{align*}
\textnormal{sign}(S_{i}^{T} - \phi(B_{i0}))\psi_{i}^{CT} = \textnormal{sign}(S_{i}^{T} - \phi(B_{i0}))\textnormal{sign}(\phi(B_{i1}^{T}(\cdot|S^{T}_{i})) - \phi(B_{i1}^{T}(\cdot|S^{\phi}_{i}))) \geq 0.
\end{align*}
In particular, with an appropriate TSLS specification, we can use the sign of the perception gap to correct for the contamination from $\psi_{i}^{CT}$. Thus, signal monotonicity and control-group stability correspond to ``weak'' IV monotonicity, which allows the direction of canonical IV monotonicity to vary across covariate values \citep{sloczynski2020should, blandhol2022tsls}.
The stability condition assumes that receiving information ``consistent'' with one's prior feature is equivalent to not receiving any information, in the sense that the posterior features are the same. For example, consider an agent that (i) has prior expectations of 3\% for inflation; and (ii) without additional information, maintains their prior beliefs. Stability assumes that this agent's posterior expectations remain at 3\% if given a signal that inflation will be 3\%.\footnote{Here it is fine if the agent's uncertainty decreases---we only need expectations to be stable.}
The stability condition restricts the signal to be directly comparable to the state of interest. For example, consider a design in which the elicited priors and posteriors are expectations $\mu$ over the extent of discrimination against Black individuals in the housing market, and the signal measures the extent of discrimination against Black individuals in the labor market. This treatment group plausibly satisfies signal monotonicity: It is reasonable that agents believe that higher levels discrimination against Black individuals in the labor market tend to imply higher levels of discrimination in the housing market. However, the value of $\mu(B_{i0})$ refers to a different state (labor market) than $S_{i}^{T}$ (housing market). In particular, being told that the level of discrimination in the housing market is $S_{i}^{\mu} := \mu(B_{i0})$ need not confirm one's priors over that discrimination.\footnote{An alternative design is to elicit prior expectations over the state corresponding to the signal, as is done in \cite{haaland2023racial}, who estimate an intent-to-treat effect. Our framework considers a common state space $\Omega$ for the prior and posterior for simplicity---to our knowledge, no existing paper that provides TSLS estimates uses the alternative design in \cite{haaland2023racial}.}
The stability condition limits the extent to which agents may update their beliefs beyond the signal provided, between the elicitation of priors and posteriors. For example, if agent $i$ searches for information with greater intensity when assigned to control than when assigned to treatment with a signal $s = S_{i}^{\phi}$ that ``confirms'' their priors, then $\phi(B_{i1}^{T}(\cdot|S^{\phi}_{i})) \neq \phi(B_{i1}^{C})$. However, in such cases it suffices to assume a weaker version of stability:
\begin{align*}
|\phi(B_{i1}^{T}(\cdot|S^{\phi}_{i})) - \phi(B_{i1}^{C}(\cdot|S_{i}^{C}))| \leq |\phi(B_{i1}^{T}(\cdot|S_{i}^{T})) - \phi(B_{i1}^{T}(\cdot|S^{\phi}_{i}))|.
\end{align*}
This allows agents to acquire outside information \citep{cavallo2017inflation, armona2019home}, provided that the difference in $\phi$ from treatment with $s = S_{i}^{T}$ versus $s = S_{i}^{\phi}$ is large enough.
\subsection{Passive Control Weights}\label{main:sec:passive.parameters}
Proposition \ref{main:prop:passive.MLRP}, which follows from the analysis in Section \ref{main:sec:stability}, characterizes $w_{i}^{CT}$.
\begin{proposition}\label{main:prop:passive.MLRP}
Let Assumption \ref{main:ass:signal.monotonicity} be satisfied. If the passive control comparison is control-group stable and $S_{i}^{T} \neq \phi(B_{i0})$, then
\begin{align*}
w_{i}^{CT} &\propto |\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})|, & I_{i}^{sign} &:= \textnormal{sign}(S_{i}^{T} - \phi(B_{i0})), \\
w_{i}^{CT} &\propto |\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})| \times |S_{i}^{T} - \phi(B_{i0})|, & I_{i}^{gap} &:= S_{i}^{T} - \phi(B_{i0}), \\
w_{i}^{CT} &\propto |\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})| \times (\pi_{1}\psi_{i}^{CT} + \pi_{2}|S_{i}^{T} - \phi(B_{i0})|), & I_{i}^{1 + gap} &:= (1, S_{i}^{T} - \phi(B_{i0}))', \\
w_{i}^{CT} &\propto |\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})| \times (\pi_{1}\psi_{i}^{CT} + \pi_{2}\psi_{i}^{CT}\phi(B_{i0})), & I_{i}^{1 + prior} &:= (1, \phi(B_{i0}))'.
\end{align*}
\end{proposition}
Thus, under signal monotonicity and control-group stability, only \textit{some} of the passive control specifications from the literature recover positive-weighted APEs. The weights from $I_{i}^{sign}$ are positive, and proportional to the absolute difference in $i$'s counterfactual posterior features across treatment and control.\footnote{If $S_{i}^{T} = \phi(B_{i0})$ for some $i$, then the weights for $I_{i}^{sign}$ are $w_{i}^{CT} \propto |\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})|\mathds{1}\{S_{i}^{T} \neq \phi(B_{i0})\}$.} Relative to $I_{i}^{sign}$, interaction $I_{i}^{gap}$ additionally up-weights agents with larger absolute perception gaps. Relative to $I_{i}^{sign}$ and $I_{i}^{gap}$, interactions $I_{i}^{1,gap}$ and $I_{i}^{1,prior}$ allow for negative weights. Intuitively, the signs of the scalars $I_{i}^{sign}$ and $I_{i}^{gap}$ correctly predict $\psi_{i}^{CT}$. On the other hand, the vectors $I_{i}^{1,gap}$ and $I_{i}^{1,prior}$ induce linear combinations $I_{i}'\pi$ that need not correctly predict $\psi_{i}^{CT}$.
\subsection{Neutrality}\label{main:sec:neutrality}
We now consider active control comparisons.
\begin{definition}[Neutrality]
An active control comparison is \textit{treatment-group neutral} if
\begin{align*}
\phi(B_{i1}^{H}(\cdot|S^{L}_{i})) = \phi(B_{i1}^{L}(\cdot|S^{L}_{i})).
\end{align*}
\end{definition}
If an active control comparison is treatment-group neutral, then the provision of $s = S_{i}^{L}$ in both treatment groups leads agent $i$ to form the same posterior features. Notice
\begin{align*}
\psi_{i}^{LH} = \textnormal{sign}([\phi(B_{i1}^{H}(\cdot|S^{H}_{i})) - \phi(B_{i1}^{H}(\cdot|S^{L}_{i}))] + [\phi(B_{i1}^{H}(\cdot|S^{L}_{i})) - \phi(B_{i1}^{L}(\cdot|S^{L}_{i}))]).
\end{align*}
Thus, under treatment-group neutrality, signal monotonicity implies
\begin{align*}
\textnormal{sign}(S^{H}_{i} - S^{L}_{i})\psi_{i}^{LH} = \textnormal{sign}(S^{H}_{i} - S^{L}_{i})\textnormal{sign}(\phi(B_{i1}^{H}(\cdot|S^{H}_{i})) - \phi(B_{i1}^{H}(\cdot|S^{L}_{i}))) \geq 0.
\end{align*}
In particular, we can use $\textnormal{sign}(S^{H}_{i} - S^{L}_{i})$ to correct for the contamination from $\psi_{i}^{LH}$. However, since $S_i^H > S_i^L$ for all $i$, then $\textnormal{sign}(S^{H}_{i} - S^{L}_{i}) = 1$ so that existing active control specifications ``automatically'' make the correction. Therefore, signal monotonicity and treatment-group neutrality correspond to the canonical IV monotonicity condition \citep{imbens1994identification, angrist2000interpretation}.
Intuitively, the neutrality condition assumes that agents perceive the provision of information to be ``equally credible'' across treatment groups.\footnote{It suffices to assume that the belief updating rules $B_{i1}^{g}(\cdot|s)$ are the same across $g \in \{L, H\}$.} This equality may not hold when there are asymmetric effects of information provision across treatment groups. For example, agents may perceive information provision to be less credible in groups that provide information that more strongly challenges their prior beliefs \citep{gentzkow2006, haaland2023designing}. Such issues can be mitigated with appropriate experimental design, including framing the information as coming from similar sources and using the same presentation strategies across treatment groups. Furthermore, as with control-group stability, we can also entertain a weaker version of neutrality:
\begin{align*}
|\phi(B_{i1}^{H}(\cdot|S^{L}_{i})) - \phi(B_{i1}^{L}(\cdot|S^{L}_{i}))| \leq |\phi(B_{i1}^{H}(\cdot|S^{H}_{i})) - \phi(B_{i1}^{H}(\cdot|S^{L}_{i}))|.
\end{align*}
This allows for some perceived asymmetry, provided that the difference in $\phi$ from treatment with $s = S_{i}^{H}$ versus $s = S_{i}^{L}$ is large enough.
\subsection{Active Control Weights}\label{main:sec:active.parameters}
Proposition \ref{main:prop:active.MLRP}, which follows from the analysis in Section \ref{main:sec:neutrality}, characterizes $w_{i}^{LH}$.
\begin{proposition}\label{main:prop:active.MLRP}
Let Assumption \ref{main:ass:signal.monotonicity} be satisfied. If the active control comparison is treatment-group neutral, then
\begin{align*}
w_{i}^{LH} \propto |\phi(B_{i1}^{H}) - \phi(B_{i1}^{L})|.
\end{align*}
\end{proposition}
In particular, under signal monotonicity and treatment-group neutrality, existing active control specifications recover positive-weighted APEs. The weights $w_{i}^{LH}$ are proportional to the absolute difference in agents' posterior features across treatment groups, which parallels $w_{i}^{CT}$ for the passive control specification with interaction $I_{i}^{sign}$.
\section{Practical Recommendations}\label{main:sec:recs}
\subsection{Passive versus Active Control}
Given our framework, when deciding between active and passive control designs, researchers should consider how the associated comparisons $\phi(B_{i1}^{H}) - \phi(B_{i1}^{L})$ and $\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})$ differ in (i) the causal parameters they recover; and (ii) the identification constraints they place on the design of information provision.
Within a given agent $i$, passive control aggregates the partial effects $\partial_{v}Y_{i}^{\phi}(v)$ over the range of values $v$ between $\phi(B_{i1}^C)$ and $\phi(B_{i1}^T)$, directly answering the question ``what is the effect of giving agent $i$ information, relative to not giving them information?'' In contrast, active control aggregates over values $v$ between $\phi(B_{i1}^L)$ and $\phi(B_{i1}^H)$, answering the question ``what is the effect of giving agent $i$ one piece of information relative to another?'' The latter counterfactual may have less policy relevance, as noted in \cite{haaland2023designing}.
Notably, even if $|\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})| = |\phi(B_{i1}^{H}) - \phi(B_{i1}^{L})|$, we can still have $\beta^{CT} \neq \beta^{LH}$. For a simple illustration of this difference, let us assume that $S_i^T = S_i^H$, such that $B_{i1}^T = B_{i1}^C = B_{i1}^*$. If we trace out a partial effects curve $\partial_{v}Y_{i}^{\phi}(v)$ as in Figure \ref{fig:example}, then $\Bar{\beta}^{LH}_{i}$ is proportional to the green region and $\Bar{\beta}^{CT}_{i}$ is proportional to the orange region. Due to the shape of the partial effects curve for this agent, $\Bar{\beta}^{CT}_{i} > \Bar{\beta}^{LH}_{i}$. Thus, even if $|\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})| = |\phi(B_{i1}^{H}) - \phi(B_{i1}^{L})|$ for all $i$, we can still have $\beta^{CT} \neq \beta^{LH}$.
\begin{figure}[h!]
\centering
\includegraphics[width=0.5\textwidth]{diagram-20240504.png}
\caption{Comparing Active and Passive Control Counterfactuals}
\label{fig:example}
\floatfoot{\textit{Note:} The blue curve illustrates the partial effects curve $\partial_{v}Y_{i}^{\phi}(v)$ for $i$. We let $B_{i1}^* = B_{i1}^T = B_{i1}^H$. Therefore, the green region is proportional to $\Bar{\beta}^{LH}_{i}$---the within-agent APE associated with the active control comparison for agent $i$. The orange region is proportional to $\Bar{\beta}^{CT}_{i}$---the within-agent APE associated with the passive control comparison for agent $i$.}
\end{figure}
Passive and active control also differ in how they aggregate partial effects across agents. The latter weights agents according to $|\phi(B_{i1}^H) - \phi(B_{i1}^L)|$, while the former with interaction $I_i^{sign}$ weights agents according to $|\phi(B_{i1}^T) - \phi(B_{i1}^C)|$.\footnote{We focus on $I_i^{sign}$ because it induces weights $w_{i}^{CT}$ that are most analogous to $w_{i}^{LH}$.} Because agents assigned to control are not given information, $|\phi(B_{i1}^T) - \phi(B_{i1}^C)|$ is more sensitive to agents' priors. When $|\phi(B_{i1}^T) - \phi(B_{i1}^C)|$ is more heterogeneous than $|\phi(B_{i1}^H) - \phi(B_{i1}^L)|$, passive control places less equal weights across agents; this is more likely in settings with highly heterogeneous priors---see Appendix \ref{main:sec:gaussian.model}. Therefore, although active control generates a potentially less policy-relevant counterfactual, it may place more equal weights across agents, which may be desirable in some settings \citep{balla2023identifying}. This analysis informs discussions from \cite{roth2020expectations}
and \cite{haaland2023designing}, which posit that active control identifies causal effects for a ``broader population'' of agents. This claim is not exactly true; rather, it is the relative weight on agents with differing priors that active control may better equalize.
Finally, active control requires treatment-group neutrality, whereas passive control requires control-group stability. Under neutrality, $S_i^L$ and $S_i^H$ must be perceived to be equally credible; this imposes design constraints on how distinct the two signals can be (since more extreme signals may lead to loss of credibility), and what sources may be used to generate the signals. Under stability, $S_i^T$ must convey similar numerical content to the prior feature (i.e., same units for the signals as the prior features), and agents should not differentially search for additional information between prior and posterior elicitation, dependent on group assignment.
\subsection{Choosing a Passive Control Specification}
Among the four passive control specifications, interactions $I_i^{sign}$ and $I_i^{gap}$ generate positive weights, whereas $I_i^{1,gap}$ and $I_i^{1,prior}$ do not. Therefore, the latter two specifications should not be used: Agents' partial effects may enter \textit{negatively} into the weighted average, attenuating or reversing the sign of the estimated APE.
Between $I_i^{sign}$ and $I_i^{gap}$, the former directly maps to canonical formulations of the local average treatment effect for continuous endogenous variables \citep{angrist2000interpretation}, which weights by the size of agents' ``first-stages.'' In this setting, these agent first-stages are given by $|\phi(B_{i1}^{T}) - \phi(B_{i1}^{C})|$. However, if researchers are especially interested in agents with large perception gaps, then $I_i^{gap}$ may generate a more useful APE. Moreover, $I_i^{sign}$ and $I_i^{gap}$ may provide different levels of estimation precision.
\section{Applications}\label{main:sec:applications}
We now compare the different passive control interactions using data from \cite{kumar2023effect} and \cite{jager2022worker}. We show that the chosen specification affects the magnitude and significance of estimates, and provide evidence that these effects are driven by contaminated weights in the former setting, and by up-weighting of agents with larger perception gaps in the latter setting.
\subsection{Kumar et al. (2023)}
\cite{kumar2023effect} study how firms' expectations and uncertainty of future GDP growth affect their economic decisions. We compare two arms in their experiment: the control arm, which receives no information, and their first treatment arm, which tells firms that the average prediction of GDP growth among a panel of professional forecasters is 4\%. \cite{kumar2023effect} elicit agents' expectations and uncertainty---we focus on expectations.\footnote{As noted in Section \ref{main:sec:setup}, econometric issues arise when there is more than one endogenous variable; hence, we avoid including both expectations and uncertainty.} The outcome variable $Y_{i}$ is defined as the difference between realized and planned changes in business choices, such as prices and employment.
Figure \ref{fig:kumar-coeff} plots TSLS coefficients for various outcomes: price, employment, and advertising budget. Other outcomes are given in Appendix \ref{main:sec:appendix.applications.1}. Since all firms in the treatment group receive the same professional forecast, interactions $I_i^{1,gap}$ and $I_i^{1,prior}$ provide the same estimates.
\begin{figure}[h!]
\centering
\includegraphics[width = \textwidth]{kumar_maintext_coefficients.pdf}
\caption{Coefficient Estimates---Data from \cite{kumar2023effect}}
\label{fig:kumar-coeff}
\floatfoot{\textit{Note:} The points are point estimates and the bars are 95\% confidence intervals. The outcomes $Y_{i}$ are defined as the difference between the planned change and the actualized change, i.e., the planned change in price versus the actualized changed in price. Price is the price of the firm's main product, employment is total employment at the firm, and advert. budget is the advertising budget of the firm. The signal $S_{i}^{T}$ is the average prediction of 4\% GDP growth among professional forecasters, the prior feature $\mu(B_{i0})$ is pre-treatment GDP growth expectations, and the posterior feature $\mu(B_{i1})$ is post-treatment GDP growth expectations. $I_i^{sign}$ regresses the outcome on the sign of the perception gap and posterior GDP growth expectations, instrumenting the posterior by the treatment indicator times the sign of the perception gap $S_i^T - \mu(B_{i0})$. $I_i^{gap}$ regresses the outcome on the perception gap $S_i^T - \mu(B_{i0})$ and the posterior, instrumenting the posterior by the treatment indicator times the perception gap. $I_i^{1, gap}$ regresses the outcome on the perception gap and the posterior, instrumenting the posterior by the treatment indicator and the treatment indicator times the perception gap. $I_i^{1, prior}$ regresses the outcome on the prior and the posterior, instrumenting the posterior by the treatment indicator and the treatment indicator times the prior. In all specifications, the coefficient of interest is the coefficient on the posterior expectation.}
\end{figure}
For price, the magnitude of the coefficient for interaction $I_i^{sign}$ is halved and the magnitude of the coefficient for interaction $I_i^{gap}$ is reduced by one-third, relative to the coefficients for interactions $I_i^{1,gap}$ and $I_i^{1,prior}$. A t-test of these coefficients finds that $I_i^{sign}$ is marginally significantly different (p-value = 0.078) from $I_i^{1,gap}$ and $I_i^{1,prior}$. Similarly, for employment, the magnitude of the coefficients for $I_i^{sign}$ and $I_i^{gap}$ are 60\% and 50\% of the other coefficients; $I_i^{gap}$ is marginally significantly different (p-value = 0.081) from $I_i^{1,gap}$ and $I_i^{1,prior}$. Finally, for advertising budget, the magnitudes of the coefficients for $I_i^{sign}$ and $I_i^{gap}$ are a third of the coefficients for interactions $I_i^{1,gap}$ and $I_i^{1,prior}$. In fact, under $I_i^{sign}$ and $I_i^{gap}$, the 95\% confidence interval on the advertising budget estimate include zero effects.
What explains the above differences? Recall that interactions $I_i^{sign}$ and $I_i^{gap}$, which estimate coefficients of similar magnitude, both generate positive weights. In contrast, interactions $I_i^{1,gap}$ and $I_i^{1,prior}$, which estimate coefficients that are twice as large, both allow for negative weights. Although we cannot formally reject the non-existence of negative weights for interactions $I_i^{1,gap}$ and $I_i^{1,prior}$, the above grouping pattern suggests that negative weights may be driving larger TSLS estimates for $I_i^{1,gap}$ and $I_i^{1,prior}$.
\subsection{Jäger et al. (2024)}
\cite{jager2022worker} study how information about workers' outside options---that is, workers' wages if they left their current job to find a new one---affects their labor market decisions. First, \cite{jager2022worker} elicit workers' prior expectations of their outside options. Next, they split their sample into a control group and a single treatment group. The control arm receives no information; those in the treatment group are told the mean wage of workers similar to themselves (based on gender, age, occupation, labor market region, and education level). Then, \cite{jager2022worker} elicit workers' posterior expectations of their outside options.
Figure \ref{fig:jager-coeff} displays estimated coefficients from all four interactions using intended negotiation probability as the outcome---estimates for other outcomes are in Appendix \ref{main:sec:appendix.applications.1}. The interaction $I_i^{sign}$ coefficient is 30-40\% larger (though not statistically significantly so) than the coefficients from interactions $I_i^{gap}$, $I_i^{1,gap}$, and $I_i^{1,prior}$, each of which are similar to each other. The fact that interaction $I_i^{gap}$, which ensures positive weights, is similar to interactions $I_i^{1,gap}$ and $I_i^{1,prior}$ suggests that the potential negative weighting in the latter is not consequential in this setting. Instead, the smaller coefficients for interactions $I_i^{gap}$, $I_i^{1,gap}$, and $I_i^{1,prior}$ may be due to the up-weighting of agents with larger perception gaps. We provide further evidence that up-weighting may be driving these results in Appendix \ref{main:sec:appendix.applications.2}.
\begin{figure}[h!]
\centering
\includegraphics[width = 0.7\textwidth]{jager_maintext_coefficients.pdf}
\caption{Coefficient Estimates---Data from \cite{jager2022worker}}
\label{fig:jager-coeff}
\floatfoot{\textit{Note:} The points are point estimates and the bars are 95\% confidence intervals. The outcome is the intended probability of negotiating for a raise. The signal $S_{i}^{T}$ is the mean wage of workers similar to worker $i$, and the prior $\mu(B_{i0})$ and posterior $\mu(B_{i1})$ are worker $i$'s expectations of how much their wages would change, as a percentage of their current wage, pre- and post- treatment, respectively. $I_i^{sign}$ regresses the outcome on the sign of the perception gap and the posterior, instrumenting the posterior by the treatment indicator times the sign of the perception gap. $I_i^{gap}$ regresses the outcome on the perception gap and the posterior, instrumenting the posterior by the treatment indicator times the perception gap. $I_i^{1, gap}$ regresses the outcome on the perception gap and the posterior, instrumenting the posterior by the treatment indicator and the treatment indicator times the perception gap. Following \cite{jager2022worker}, we normalize agents' perception gaps $S_{i}^{T} - \mu(B_{i0})$ to be a percentage of $S_{i}^{T}$ for $I_i^{gap}$ and $I_i^{1, gap}$. $I_i^{1, prior}$ regresses the outcome on the prior and the posterior, where the posterior is instrumented by the treatment indicator, the treatment indicator times the prior, and the treatment indicator times the signal. We estimate the version of interaction $I^{1,prior}_{i}$ that includes both the signal and the prior because signals are personalized to each agent---see Interaction \ref{main:int:kumar} for a discussion. In all specifications, the coefficient of interest is the coefficient on the posterior expectation.
}
\end{figure}
One explanation for why up-weighting may have attenuated the estimates is endogenous information acquisition, such as in \cite{balla2023identifying}. Agents who are more responsive to information are more likely to seek it out, and therefore have more accurate priors. At the same time, agents that are very accurate will not respond much to the information treatment; their actions are already near-optimal. Therefore, the agents that are most responsive are those that have medium-sized perception gaps, and the agents that are least responsive are those that have large or no perception gap. Interactions $I_i^{gap}$, $I_i^{1,gap}$, and $I_i^{1,prior}$ will up-weight agents with large perception gaps---and thus those who are least responsive---attenuating treatment effects towards zero.
\section{Conclusion}
This paper examines identification and estimation in information provision experiments. We developed a causal framework for these experiments, and provided conditions for recovering interpretable APEs in both passive and active control designs. Using this framework, we evaluated TSLS specifications from the literature. We found that the standard active control specification identifies an APE with non-negative weights. However, two of the four common passive control specifications allow for negative weights, which we advise against.
\newpage
\bibliography{lit.bib}
\newpage