The exact contents of citations.db main_text.text for this paper — one flattened LaTeX string, title through conclusion, appendix excluded, unmodified except for removing email addresses. This is what our citation measures are computed over.
72,569 characters
Sharp Sensitivity Analysis for Inverse Propensity Weighting via Quantile Balancing
\if11
{
\title{\Large \textbf{Sharp Sensitivity Analysis for Inverse Propensity Weighting via Quantile Balancing}
\footnote{This material is based upon work supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. DGE-2039656. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. We are grateful to the associate editor and several anonymous referees for careful feedback. We are also grateful for comments from Guillaume Basse, Bo Honor{\'e}, Nathan Kallus, Michal Koles\'{a}r, David Lee, Xinran Li, Ulrich M{\"u}ller, Karl Schulze, Dylan Small, Angela Zhou, Qingyuan Zhao, and seminar participants at Berkeley, Cambridge, and Princeton.}}
\author{Jacob Dorn\\ Department of Economics \\ Princeton University \and Kevin Guo \\ Department of Statistics \\ Stanford University}
\maketitle
} \fi
\if01
{
\bigskip
\bigskip
\bigskip
\begin{center}
{\LARGE\bf Sharp Sensitivity Analysis for Inverse Propensity Weighting via Quantile Balancing}
\end{center}
\medskip
} \fi
\bigskip
\begin{abstract}
Inverse propensity weighting (IPW) is a popular method for estimating treatment effects from observational data. However, its correctness relies on the untestable (and frequently implausible) assumption that all confounders have been measured. This paper introduces a robust sensitivity analysis for IPW that estimates the range of treatment effects compatible with a given amount of unobserved confounding. The estimated range converges to the narrowest possible interval (under the given assumptions) that must contain the true treatment effect. Our proposal is a refinement of the influential sensitivity analysis by Zhao, Small, and Bhattacharya (2019), which we show gives bounds that are too wide even asymptotically. This analysis is based on new partial identification results for Tan (2006)'s marginal sensitivity model.
\end{abstract}
\noindent
{\it Keywords:} unobserved confounding, partial identification, quantile regression
\section{Introduction}
Estimating treatment effects from observational data is difficult because ``treated" and ``control" samples typically differ on many characteristics besides treatment status. For example, consumers of nutritional supplements may be wealthier or more health-conscious than those not taking supplements. One popular tool for adjusting for such baseline imbalances is Inverse Propensity Weighting (IPW) \citep{ipw_review, ATTViaIPW}. This technique re-weights treated and untreated samples to be similar along all observed characteristics and then compares outcomes in the weighted samples. The crucial assumption underlying this approach is that the weighted samples do not systematically differ along important \emph{unobserved} characteristics. This ``unconfoundedness" assumption is untestable, and often implausible.
This paper studies how much can be learned when unconfoundedness does not hold, but one can bound the plausible degree of unobserved confounding. In particular, given a ``sensitivity assumption" controlling the degree of selection, we aim to answer two questions:\\
\begin{enumerate}[label=(\arabic*),topsep=0pt,itemsep=-1ex]
\item \emph{Sensitivity analysis}. Can we bound how much the IPW point estimate from our ``primary analysis" might change if unobserved confounding were properly accounted for? \label{sensitivity_analysis}
\item \emph{Partial identification}. Can we characterize the most informative bounds that could possibly be obtained from the sensitivity assumption with even an infinite amount of observational data? \label{partial_identification}\\
\end{enumerate}
The specific sensitivity assumption used in this paper is the ``marginal sensitivity model" of \cite{tan2006}, which is a variant of Rosenbaum's famous ``$\Gamma$ sensitivity model" \citep{RosenbaumDesign, Rosenbaum1987, rosenbaum2002} that is better suited for IPW analyses. This sensitivity assumption is quite popular in causal inference; see \cite{tan2006, confounding_robust_policy_improvement, kallus2020confoundingrobust, kallus_zhou2020, kallus2018interval, zsb2019, causal_rule_ensemble, rosenman2020combining, rosenman2021designing, soriano2021interpretable} for an incomplete list of references. As we will see, it lends itself to computationally-efficient sensitivity analyses which are simple enough to explain to any practitioner comfortable with IPW.
Recently, Zhao, Small, and Bhattacharya \cite{zsb2019} (hereafter ZSB)
introduced an interpretable IPW sensitivity analysis for the marginal sensitivity model that has been largely responsible for the recent resurgence of interest in this sensitivity assumption. However, they did not answer the partial identification question, leaving open the possibility that more informative bounds could be obtained from the same data and assumptions. Indeed, there are no existing partial identification results for the marginal sensitivity model that can be used to benchmark a sensitivity analysis.
The first main contribution of this paper is to provide a complete answer to the partial identification question \ref{partial_identification}. We derive closed-form expressions for the largest and smallest values of the ``usual" estimands (e.g. average treatment effect) compatible with the marginal sensitivity assumption. These expressions show that the ZSB bounds are essentially always conservative because they ignore an infinite collection of constraints implied by the distribution of observed characteristics. \cite{tan2006} also identified these constraints, but deemed it intractable to incorporate them all in a sensitivity analysis. In contrast, our partial identification results show that this collection can actually be reduced to a \emph{single} constraint which is easy to incorporate.
Our second main contribution is to introduce a new IPW sensitivity analysis, which we call the \emph{quantile balancing} method. The method is a simple refinement of the ZSB sensitivity analysis, and has several desireable features:\\
\begin{enumerate}[label=(\roman*),topsep=0pt,itemsep=-1ex]
\item The quantile balancing sensitivity interval is always a subset of the ZSB interval. Outside of knife-edge cases, it is a strict subset.
\item When the outcome's conditional quantiles can be estimated consistently, the bounds converge to the sharp partial identification region for the average treatment effect (the best possible bounds that can be obtained under the marginal sensitivity model). With some abuse of terminology, we say that quantile balancing is ``sharp."
\item Under standard assumptions for IPW inference, the bounds can be converted into confidence intervals using the same percentile bootstrap scheme proposed by ZSB.
\item When the estimated quantiles are inconsistent, the sensitivity interval is too wide rather than too narrow and the confidence intervals over-cover rather than under-cover. In other words, our intervals are guaranteed to be valid, regardless of the quality of the additional input we demand. \\
\end{enumerate}
We apply the quantile balancing method in several simulated examples and one real-data application, and find that it can substantially tighten the ZSB bounds when the covariates are good predictors of the outcome. We also extend our analysis to Augmented IPW (AIPW) estimators. That analysis shows that a slight refinement of the ZSB method is sharp under ``additive-noise" data generating processes, though the refinement makes little difference in practice. One shortcoming we will mention up-front is that our statistical guarantees assume the outcome is continuously-distributed in order to enable quantile regression. Since our partial identification results also apply to discrete outcomes, we conjecture that the quantile balancing procedure could be modified to give sharp bounds in that setting too.
\subsection{Setting and background}
We consider the Neyman-Rubin potential outcomes model with a binary treatment \citep{neyman, rubin1974}. We observe i.i.d. samples $(X_i, Y_i, Z_i)$ from a distribution $P$, where $X_i \in \mathcal{X} \subseteq \mathds{R}^d$ is a vector of covariates, $Z_i \in \{ 0, 1 \}$ is a binary treatment assignment indicator, and $Y_i \in \mathds{R}$ is a real-valued outcome.
We assume that each sample $(X_i, Y_i, Z_i)$ is obtained by coarsening a ``full data" sample $(X_i, Y_i(0), Y_i(1), Z_i, U_i)$. Here, $Y_i(0)$ and $Y_i(1)$ are potential outcomes and $U_i$ is a vector of unobserved confounders of unspecified dimension. The observed outcome is related to the potential outcomes through the consistency relation $Y_i = Z_i Y_i(1) + (1 - Z_i) Y_i(0)$.
The goal is to use the observed data to draw inferences about a causal estimand $\psi_0$. For the purposes of exposition, we initially focus on the counterfactual means $\psi_{\textup{T}} = \mathbb{E}[ Y(1)]$ and $\psi_{\textup{C}} = \mathbb{E}[ Y(0)]$, although the examples of most practical interest are the average treatment effect (ATE) and the average treatment effect on the treated (ATT).
\begin{align*}
\psi_{\textup{ATE}} &= \mathbb{E}[ Y(1) - Y(0)] \\
\psi_{\textup{ATT}} &= \mathbb{E}[ Y(1) - Y(0) | Z = 1].
\end{align*}
With minor modification, our identification results can also be applied to more complex estimands, including policy values \citep{athey2017efficient, confounding_robust_policy_improvement} and weighted average treatment effects. However, we do not present those extensions in this paper.
Under the unconfoundedness assumption $(Y(0), Y(1)) \, \rotatebox[origin=c]{90}{$\models$} \, Z \mid X$, all of the above quantities can be consistently estimated from the observed data using inverse propensity weighting. IPW estimators work by reweighting the observed sample by some function of the propensity score $e(x) := P(Z = 1 | X = x)$. For example, if the estimand of interest is $\psi_{\textup{T}}$, the (stabilized) IPW estimator is given by (\ref{ipw1}):
\begin{align}
\hat{\psi}_{\textup{T}} = \frac{\mathbb{E}_n[ YZ/\hat{e}(X)]}{\mathbb{E}_n[ Z/\hat{e}(X)]} \label{ipw1}
\end{align}
Here, $\hat{e}(\cdot)$ is an estimate of the propensity score $e(\cdot)$ and $\mathbb{E}_n[\cdot]$ is shorthand for $\tfrac{1}{n} \sum_{i = 1}^n [\cdot]_i$. An unstabilized version of $\hat{\psi}_{\textup{T}}$ which uses only the numerator of (\ref{ipw1}) is also common. Related estimators for the other estimands considered will be denoted by $\hat{\psi}_{\textup{C}}, \hat{\psi}_{\text{ATE}}$, and $\hat{\psi}_{\text{ATT}}$. See the articles by \cite{ipw_review} or \cite{ATTViaIPW} for their exact formulas.
We will assume some conditions which are required for identification and estimation under unconfoundedness: overlap ($0 < e(X) < 1$ almost surely) and one outcome moment ($\mathbb{E}_{P}[|Y|] < \infty$). However, we will not assume unconfoundedness.
\section{The marginal sensitivity model} \label{section:msm}
The marginal sensitivity model introduced by \cite{tan2006} is a relaxation of unconfoundedness which has been applied in many causal inference problems. This one-parameter sensitivity assumption allows for the existence of unobserved confounders $U$, but limits the degree of selection bias that can be attributed to these confounders.
\begin{manualassumption}{$\Lambda$}
\label{assumption:msm}
\textup{\textbf{(Marginal sensitivity model)}}\\
There exists a vector of unmeasured confounders $U$ that, if measured, would lead to unconfoundedness: $(Y(0), Y(1)) \, \rotatebox[origin=c]{90}{$\models$} \, Z \mid (X, U)$. However, within each stratum of the observed covariates, measuring $U$ can only change the odds of treatment by at most a factor of $\Lambda$, i.e. if we set $e_0(x, u) := P(Z = 1 | X = x, U = u)$, then (\ref{or_bound}) holds with probability one.
\begin{align}
\Lambda^{-1} \leq \frac{e_0(X, U)/[1 - e_0(X, U)]}{e(X)/[1 - e(X)]} \leq \Lambda \label{or_bound}
\end{align}
\end{manualassumption}
The statement of the marginal sensitivity model presented in \cite{tan2006} and \cite{zsb2019} uses the potential outcomes $(Y(0), Y(1))$ in place of the unobserved variable $U$. However, as pointed out by a referee, these assumptions are equivalent.
To avoid confusion between $e_0$ and $e$, we will follow \cite{kallus_zhou2020} and refer to $e_0$ as the ``true propensity score" and $e$ as the ``nominal propensity score."
Like Rosenbaum's famous ``$\Gamma$ sensitivity model", Assumption \ref{assumption:msm} controls the degree of unobserved confounding with a single parameter. When $\Lambda = 1$, measuring additional confounders cannot change the odds of treatment at all, i.e. treatment assignment is unconfounded. As $\Lambda$ increases, stronger forms of confounding are allowed. For advice on how to choose this parameter, see \cite{hsu_small2013}. For more on the relationship between this and Rosenbaum's model, see \cite{zsb2019} Section 7.1. The marginal sensitivity assumption is ``nonparametric" in the sense that no assumptions are needed about how $e_0$ depends on $u$. Even the dimension of the vector $U$ does not need to be specified.
To see how Assumption \ref{assumption:msm} can be used for sensitivity analysis, begin by considering how an oracle statistician who observed the confounders $U_i$ might estimate $\psi_{\textup{T}}$. One strategy would be to use the IPW estimator (\ref{oracle}), which is consistent under weak assumptions.
\begin{align}
\hat{\psi}_{\textup{T}}^* = \frac{\sum_{i = 1}^n Y_i Z_i / e_0(X_i, U_i)}{\sum_{i = 1}^n Z_i / e_0(X_i, U_i)} \label{oracle}.
\end{align}
In reality, $\{ U_i \}_{i \leq n}$ are not observed, but under Assumption \ref{assumption:msm}, it is possible to \emph{bound} the true propensity scores $e_0(X_i, U_i)$. In particular, the vector $(e_0(X_1, U_1), \cdots, e_0(X_n, U_n))$ must belong to the ZSB constraint set $\mathcal{E}_n(\Lambda)$ defined in (\ref{msm_set}).
\begin{align}
\mathcal{E}_n(\Lambda) = \left\{ \bar{e} \in \mathds{R}^n \, : \, \Lambda^{-1} \leq \frac{\bar{e}_i/(1 - \bar{e}_i)}{e(X_i)/[1 - e(X_i)]} \leq \Lambda \right\} \label{msm_set}
\end{align}
ZSB proposed bounding the oracle statistician's IPW estimator (\ref{oracle}) with the largest and smallest IPW estimates that can be obtained using putative propensities in $\mathcal{E}_n(\Lambda)$.
\begin{align}
[\hat{\psi}_{\textup{T,ZSB}}^-, \hat{\psi}_{\textup{T,ZSB}}^+] = \left[ \min_{\bar{e} \in \mathcal{E}_n(\Lambda)} \frac{\sum_{i = 1}^n Y_i Z_i / \bar{e}_i}{\sum_{i = 1}^n Z_i / \bar{e}_i}, \max_{\bar{e} \in \mathcal{E}_n(\Lambda)} \frac{\sum_{i = 1}^n Y_i Z_i / \bar{e}_i}{\sum_{i = 1}^n Z_i / \bar{e}_i} \right] \label{zsb_bounds}.
\end{align}
Since the interval (\ref{zsb_bounds}) contains the consistent estimator $\hat{\psi}_{\textup{T}}^*$, the distance between the true estimand $\psi_{\textup{T}}$ and the nearest point in the sensitivity interval tends to zero. ZSB show that this conclusion holds even if the nominal propensity score $e$ is replaced by a suitably consistent estimate $\hat{e}$ in the definition of $\mathcal{E}_n(\Lambda)$, which is important for practical applications as $e$ is typically not known in observational studies.
This simple idea is intuitive enough to explain to any practitioner who is comfortable with IPW and has been extended to estimands other than $\psi_{\textup{T}}$. ZSB also consider $\psi_{\text{ATE}}$ and $\psi_{\text{ATT}}$. Related work by \cite{confounding_robust_policy_improvement, kallus2020confoundingrobust, kallus2018interval, causal_rule_ensemble} takes the idea substantially further. \cite{tan2006} applied a similar idea to a different propensity-score-based estimator and \cite{AronowLeeInterpretable, MiratrixEtAl, tudballZhaoEtAl2019interval} used similar approaches in survey sampling problems.
\subsection{Sharpness and data-compatibility} \label{section:sharpness}
The aforementioned works do not address the asymptotic optimality of the interval $[\hat{\psi}_{\textup{T,ZSB}}^-, \hat{\psi}_{\textup{T,ZSB}}^+]$. Does it converge to a limiting set containing all values of $\psi_{\textup{T}}$ compatible with Assumption \ref{assumption:msm} and no others? Sensitivity analyses with this asymptotic optimality property are called ``sharp" in the partial identification literature.
Sharpness is important for interpreting the results of a sensitivity analysis. If the primary analysis finds a positive treatment effect but the bounds associated with a very small value of $\Lambda$ include zero, one might be tempted to conclude that the primary analysis is sensitive to unobserved confounding. However, unless the bounds are known to be sharp, this inference is not warranted even in large samples. Perhaps the bounds were just too conservative.
Despite its attractive features, the ZSB sensitivity analysis is not sharp. It can be arbitrarily conservative. To illustrate this, consider a simple joint distribution of observables:
\begin{align}
\begin{split}
X &\sim \mathcal{N}(0, \sigma^2)\\
Z \mid X &\sim \textup{Bernoulli}( \tfrac{1}{2})\\
Y \mid X, Z &\sim \mathcal{N}(X, 1). \label{example}
\end{split}
\end{align}
Suppose that a data analyst receives i.i.d. samples $(X_i, Y_i, Z_i)$ from this distribution and is willing to posit that Assumption \ref{assumption:msm} is satisfied with $\Lambda = 2$. Let $\phi(\cdot)$ and $z_{\tau}$ denote the density and $\tau$-th quantile of the standard normal distribution, respectively. The following result, which follows from Theorem \ref{theorem:psiATE_identified_set} in Section \ref{section:balancing_bounds_simpler}, writes the set of values of $\psi_{\textup{T}}$ compatible with Assumption \ref{assumption:msm} explicitly in terms of these quantities and shows that this ``partially identified" set is smaller than the limiting ZSB interval.
\begin{corollary}
\label{corollary:zsb_not_sharp}
\emph{\textbf{(ZSB is asymptotically conservative)}}\\
Let $(X_i, Y_i, Z_i)$ be i.i.d. samples from the joint distribution (\ref{example}).
\begin{enumerate}[label=(\roman*),topsep=0pt,itemsep=-1ex]
\item The set of values of $\psi_{\textup{T}}$ compatible with the bound $\Lambda = 2$ and the distribution (\ref{example}) is the interval $[ \pm \tfrac{3}{4} \phi( z_{2/3}) ] \approx [\pm 0.27]$.\label{item:ExampleIdentifiedSet}
\item However, with probability one, $[ \pm 0.27 \sqrt{\sigma^2 + 1} ] \subseteq [ \hat{\psi}_{\textup{T,ZSB}}^-, \hat{\psi}_{\textup{T,ZSB}}^+]$ for all large $n$.\label{item:ZSBLimitSet}
\end{enumerate}
\end{corollary}
The precise meaning of \ref{item:ExampleIdentifiedSet} is the following: for any $\psi_{\textup{T}} \in [\pm \tfrac{3}{4} \phi(z_{2/3})]$, it is possible to construct a distribution $Q$ for the full data $(X, Y(0), Y(1), Z, U)$ which marginalizes to (\ref{example}), satisfies Assumption \ref{assumption:msm} with $\Lambda = 2$, and has $\mathbb{E}_{Q}[ Y(1)] = \psi_{\textup{T}}$. On the other hand, for any $\psi_{\textup{T}}$ not in this interval, it is impossible to construct such a distribution.
Corollary \ref{corollary:zsb_not_sharp} implies that the ZSB interval typically includes many values of $\psi$ which cannot possibly be reconciled with the data. The explanation for this conservatism is that the odds-ratio bound (\ref{or_bound}) does not capture all of the restrictions on the true propensity score $e_0$. Additional information can be found in the marginal distribution of the \emph{observed} characteristics. For example, in the context of Corollary \ref{corollary:zsb_not_sharp}, consider the putative propensity score (\ref{putative}).
\begin{align}
\bar{e}(x, u) =
\left\{
\begin{array}{ll}
1/3 &\text{if } x < 0\\
2/3 &\text{if } x \geq 0
\end{array}
\right. \label{putative}
\end{align}
This certainly satisfies the odds-ratio bound (\ref{or_bound}) --- and is therefore a possible value of $\bar{e}$ in the ZSB optimization problem (\ref{zsb_bounds}) --- but it could not possibly be the true propensity score $e_0$. If it were, we would observe $P(Z = 1 | X \geq 0) = \tfrac{2}{3}$, while the observed data distribution $P$ demands that $P(Z = 1 | X \geq 0) = \tfrac{1}{2}$. Another way of saying this is that $\bar{e}$ does not \emph{marginalize} to the nominal propensity score:
\begin{align*}
1/2 &= P(Z = 1 | X = x)\\
&= \int P(Z = 1 | X = x, U = u) \, \textup{d} P(u | X = x)\\
&\neq \int \bar{e}(x, u) \, \textup{d} P(u | X = x)\\
&= \left\{
\begin{array}{ll}
1/3 &\text{if } x < 0\\
2/3 &\text{if } x \geq 0
\end{array}
\right. .
\end{align*}
In short, this choice of $\bar{e}$ is allowed in the domain of the ZSB optimization problem but is incompatible with the distribution of observed data.
This example suggests that it should be possible to improve upon the ZSB bounds by only optimizing over the subset of $\mathcal{E}_n(\Lambda)$ which is ``data compatible." However, this is easier said than done, because the observed data distribution actually imposes an infinite number of constraints on putative propensity scores $\bar{e}$. For example, the true $e_0$ ``balances" all integrable functions $h : \mathcal{X} \rightarrow \mathds{R}$:
\begin{align}
\begin{split}
\mathbb{E}[ h(X) Z / e_0(X, U)] & = \mathbb{E}[h(X) \mathbb{E}[Z | X, U] / e_0(X, U)] \\
& = \mathbb{E}[h(X) e_0(X, U) / e_0(X, U)] \\
& = \mathbb{E}[h(X)]. \label{population_balance_h}
\end{split}
\end{align}
Every such $h$ gives rise to a testable ``balancing constraint" (\ref{balance_h}) which can be used to rule out incompatible values of $\bar{e}$.
\begin{align}
\frac{\mathbb{E}_n[ h(X) Z / \bar{e}]}{\mathbb{E}_n[ Z / \bar{e}]} \approx \mathbb{E}[h(X)] \label{balance_h}
\end{align}
In other words, any sharp sensitivity analysis must contend with an infinite number of constraints, which is typically computationally intractable \citep{BeresteanuEtAl, DaveziesDHault}. Previous works have considered relaxing these constraints by balancing only a finite set of functions \citep{tan2006, tudballZhaoEtAl2019interval}, but the resulting bounds are generally not sharp.
While this paper proceeds under the ``superpopulation" model of causal inference, the idea that observable quantities can constrain unobserved variables can also be applied in the ``finite population" model. See \cite{tudballZhaoEtAl2019interval} for an application of this idea to partial identification in survey sampling problems.
\section{Partial identification results} \label{section:partial_identification}
In this section, we show that at the \emph{population} level, it is possible to characterize the sharp bounds for $\psi_0 \in \{ \psi_{\textup{T}}, \psi_{\textup{C}}, \psi_{\textup{ATT}}, \psi_{\textup{ATE}} \}$ without ignoring or relaxing any of the infinitely many balancing constraints on the true propensity score. We apply these partial identification results to finite-sample sensitivity analysis in Section \ref{section:sensitivity_analysis}.
To state these results formally, we need a few pieces of additional notation. Recall that Assumption \ref{assumption:msm} requires the true propensity score $e_0(X, U)$ to satisfy the following odds-ratio bound:
\begin{align*}
\Lambda^{-1} \leq \frac{e_0(X, U)/[1 - e_0(X, U)]}{e(X)/[1 - e(X)]} \leq \Lambda.
\end{align*}
Therefore, it is natural to define $\mathcal{E}_{\infty}(\Lambda)$ to be the set of all random variables $\bar{E}$ which satisfy the same condition:
\begin{align}
\mathcal{E}_{\infty}(\Lambda) := \left\{ \bar{E} \, : \, \Lambda^{-1} \leq \frac{\bar{E} / (1 - \bar{E})}{e(X)/(1 - e(X))} \leq \Lambda \text{ with probability one} \right\}.
\end{align}
This can be viewed as the ``population" version of the ZSB constraint set $\mathcal{E}_n(\Lambda)$.
Additionally, we define the conditional distribution function $F(y | x, z)$ and quantile function $Q_t(x, z)$ by:
\begin{align*}
F(y | x, z) &= P(Y \leq y \mid X = x, Z = z)\\
Q_t(x, z) &= \inf \{ q \in \mathds{R} \, : \, F(q | x, z) \geq t \}.
\end{align*}
Since these functions only refer to observed quantities, they are identified from the observed-data distribution.
\subsection{Partial identification via quantile balancing}\label{section:balancing_bounds_simpler}
Our first partial identification result shows that to compute optimal bounds for $\psi_{\textup{T}}$, the infinitely-many balancing constraints described in Section \ref{section:sharpness} can actually be reduced to a \emph{single} constraint. In particular, it suffices to minimize/maximize the function $\bar{E} \mapsto \mathbb{E}[YZ/\bar{E}]$ over the set of putative propensity scores $\bar{E} \in \mathcal{E}_{\infty}(\Lambda)$ that ``balance" a particular conditional quantile of $Y$.
\begin{theorem}
\label{theorem:psiT_identified_set}
\textup{\textbf{(Optimal bounds for $\psi_{\textup{T}}$)}}\\
For any $\Lambda \geq 1$, the set of values of $\psi_{\textup{T}}$ compatible with the observed data distribution and Assumption \ref{assumption:msm} is a closed interval $[\psi_{\textup{T}}^-, \psi_{\textup{T}}^+]$. Moreover, if we define $\tau = \tfrac{\Lambda}{\Lambda + 1}$, then the interval endpoints solve (\ref{psiT_minus}) and (\ref{psiT_plus}).
\begin{align}
\psi_{\textup{T}}^- &= \min_{\bar{E} \in \mathcal{E}_{\infty}(\Lambda)} \mathbb{E}[ YZ/\bar{E}] \quad \text{subject to} \quad \mathbb{E}[ Q_{1 - \tau}(X, 1) Z / \bar{E}] = \mathbb{E}[ Q_{1 - \tau}(X, 1)] \label{psiT_minus} \\
\psi_{\textup{T}}^+ &= \max_{\bar{E} \in \mathcal{E}_{\infty}(\Lambda)} \mathbb{E}[ Y Z / \bar{E}] \quad \text{subject to} \quad \mathbb{E}[ Q_{\tau}(X, 1) Z / \bar{E}] = \mathbb{E}[ Q_{\tau}(X, 1)] \label{psiT_plus}.
\end{align}
\end{theorem}
We will highlight a few important takeaways from this theorem. First, if one adds additional balancing constraints of the form $\mathbb{E}[ h(X) Z / \bar{E}] = \mathbb{E}[ h(X)]$ in (\ref{psiT_minus}) and (\ref{psiT_plus}), the value of these problems will not change. Thus, for the purposes of computing population-level bounds, the quantile balancing constraints in Theorem \ref{theorem:psiT_identified_set} capture all the information in the observed data. Second, the fact that only a single conditional quantile appears in each of the sharp bounds for $\psi_{\textup{T}}$ reflects a special advantage of the marginal sensitivity model. For alternative sensitivity assumptions, sharp bounds often involve distinct quantiles $Q_{\tau(x)}$ for each covariate level \citep{LeeSelection, MastenPoirerSharp}, complicating estimation by potentially requiring estimates of the entire conditional quantile process \citep{masten2020assessing, semenova2020better}. Third, this result shows that the ZSB sensitivity analysis for IPW can only be sharp when the conditional quantiles of $Y$ do not depend on $X$ at all, and can therefore be refined outside pathological cases. AIPW-based variants of the ZSB sensitivity analysis will generally refine the IPW bounds since some of the variability in the quantiles of $Y$ will be absorbed by the regression function. We discuss AIPW sensitivity analysis in Section \ref{section:balance_aipw}.
We can extend the theorem to other estimands. To bound $\psi_{\textup{C}}$, exchange the labels ``treated" and ``control" and apply Theorem \ref{theorem:psiT_identified_set}. Sharp bounds on $\psi_{\textup{C}}$ can be translated into sharp bounds on $\psi_{\textup{ATT}}$ using the relation $\psi_{\text{ATT}} = \tfrac{\mathbb{E}[ Y ] - \psi_{\textup{C}}}{P(Z = 1)}$.
\begin{corollary}
\label{corollary:psiC_att_identified_set}
\textup{\textbf{(Optimal bounds for $\psi_{\textup{C}}$ and $\psi_{\textup{ATT}}$)}}\\
In the setting of Theorem \ref{theorem:psiT_identified_set}, the partially identified set for $\psi_{\textup{C}}$ is the interval $[\psi_{\textup{C}}^-, \psi_{\textup{C}}^+]$, where the interval endpoints solve (\ref{psiC_minus}) and (\ref{psiC_plus}).
\begin{align}
\psi_{\textup{C}}^- &= \min_{\bar{E} \in \mathcal{E}_{\infty}(\Lambda)} \mathbb{E} [ Y \tfrac{1 - Z}{1 - \bar{E}}] \quad \text{subject to} \quad \mathbb{E}[Q_{1 - \tau}(X, 0) \tfrac{1 - Z}{1 - \bar{E}}] = \mathbb{E}[ Q_{1 - \tau}(X, 0)] \label{psiC_minus}\\
\psi_{\textup{C}}^+ &= \max_{\bar{E} \in \mathcal{E}_{\infty}(\Lambda)} \mathbb{E}[ Y \tfrac{1 - Z}{1 - \bar{E}}] \quad \text{subject to} \quad \mathbb{E}[ Q_{\tau}(X, 0) \tfrac{1 - Z}{1 - \bar{E}}] = \mathbb{E}[ Q_{\tau}(X, 0)] \label{psiC_plus}
\end{align}
The partially identified set for $\psi_{\textup{ATT}}$ is the interval $[ \psi_{\textup{ATT}}^-, \psi_{\textup{ATT}}^+]$, where $\psi_{\textup{ATT}}^{\mp} = \tfrac{\mathbb{E}[ Y ] - \psi_{\textup{C}}^{\pm}}{P(Z = 1)}$.
\end{corollary}
Sharp bounds for $\psi_{\textup{ATE}}$ can be obtained by subtracting sharp bounds for $\psi_{\textup{T}}$ and $\psi_{\textup{C}}$. Equivalently, these bounds can be obtained by solving optimization problems with two quantile balancing constraints. Although this result is superficially similar to Theorem \ref{theorem:psiT_identified_set} and Corollary \ref{corollary:psiC_att_identified_set}, its proof requires a novel construction, which we discuss in Section \ref{section:explaining_bounds_ATE}.
\begin{theorem}
\label{theorem:psiATE_identified_set}
\textup{\textbf{(Optimal bounds for $\psi_{\textup{ATE}}$)}}\\
For any $\Lambda \geq 1$, the set of values of $\psi_{\textup{ATE}}$ compatible with the observed data distribution and Assumption \ref{assumption:msm} is a closed interval $[\psi_{\textup{ATE}}^-, \psi_{\textup{ATE}}^+]$ where $\psi_{\textup{ATE}}^- = \psi_{\textup{T}}^- - \psi_{\textup{C}}^+$ and $\psi_{\textup{ATE}}^+ = \psi_{\textup{T}}^+ - \psi_{\textup{C}}^-$.
\end{theorem}
In certain special cases, the partially identified set for $\psi_{\textup{ATE}}$ can be computed more explicitly. These explicit bounds are useful for gaining intuition about the main factors that make a causal estimate more or less robust to unobserved confounding. Corollary \ref{corollary:additive_noise_identified_set}, which is a corollary of our later work, gives such bounds in the Gaussian outcome model (\ref{eq:additive_noise}).
\begin{align}
\begin{split}
X &\sim P_X\\
Z \mid X & \sim \textup{Bernoulli}(e(X))\\
Y \mid X, Z& \sim \mathcal{N}(\mu(X, Z), \sigma^2(X)). \label{eq:additive_noise}
\end{split}
\end{align}
\begin{corollary}
\label{corollary:additive_noise_identified_set}
\emph{\textbf{(Simpler bounds for Gaussian data)}}\\
Suppose the observed-data distribution has the factorization (\ref{eq:additive_noise}), with $0 < e(X) < 1$ almost surely and $\mathbb{E}[ | \mu(X, Z)|] < \infty$.
Let $\psi_{\textup{ATE}} = \mathbb{E}[\mu(X, 1) - \mu(X, 0)]$ be the nominal ATE. Then the partially identified set for the ATE under Assumption \ref{assumption:msm} is:
\begin{align}
[\psi_{\textup{ATE}}^-, \psi_{\textup{ATE}}^+] &= [\psi_{\textup{ATE}} \pm \tfrac{\Lambda^2 - 1}{\Lambda} \phi(\Phi^{-1} (\tfrac{\Lambda}{\Lambda + 1})) \mathbb{E}[ \sigma(X)] ] \label{additive_noise_formula}.
\end{align}
Here, $\phi$ and $\Phi$ are the standard normal density and distribution function, respectively.
\end{corollary}
For a fixed bound $\Lambda$ on the degree of unobserved confounding, the formula (\ref{additive_noise_formula}) shows that two key features map the observed data distribution to robustness. The first is the magnitude of the nominal ATE: all else equal, larger nominal effects are more robust. The second is the average noise level $\mathbb{E}[ \sigma(X)]$: the better the measured variables predict the outcome, the less unobserved confounding can affect our estimates. In the extreme case where $X$ and $Z$ perfectly predict $Y$, then the ATE remains point-identified no matter how large $\Lambda$ is, as long as overlap holds. These insights are not specific to the marginal sensitivity model. In alternative sensitivity models, they have also been observed by
\cite{rosenbaum2005, hsu_small_rosenbaum2013}, \cite{CinelliHazlett}, and others.
\cite{rosenbaum2005, hsu_small_rosenbaum2013, CinelliHazlett}, and others.
\subsection{Data-compatible propensity scores} \label{section:explaining_bounds_APO}
Although the qualitative implications of Corollary \ref{corollary:additive_noise_identified_set} are plausible, we nevertheless find the quantile balancing formulas of Section \ref{section:balancing_bounds_simpler} to be counterintuitive. After all, it is certainly not true that every random variable $\bar{E} \in \mathcal{E}_{\infty}(\Lambda)$ satisfying $\mathbb{E}[ Q_{\tau}(X, 1) Z / \bar{E}] = \mathbb{E}[ Q_{\tau}(X, 1)]$ could plausibly be the true propensity score $e_0(X, U)$. Indeed, the constraints of the quantile-balancing optimization problems do not even enforce that $\mathbb{E}[ Z / \bar{E}] = 1$. Our intuition for why the ZSB procedure is conservative suggests the quantile balancing formulas should be conservative as well.
To explain how these results are possible, we begin by characterizing which random variables $\bar{E}$ could plausibly be the true propensity score $e_0(X, U)$. The calculation (\ref{population_balance_h}) indicates that $\bar{E}$ should at least satisfy $\mathbb{E}[ h(X) Z / \bar{E}] = \mathbb{E}[ h(X)]$ for all integrable $h$, or equivalently, $\mathbb{E}[Z / \bar{E} | X] = 1$. Proposition \ref{proposition:data_compatibility_psiT} shows that for the purposes of bounding $\psi_{\textup{T}}$, this is actually the \emph{only} constraint on $\bar{E}$ implied by the distribution of observables. Similar results appear in \cite{BirminghamJRSSB, Robins_etal_2000, tan2006, graham2011, hristache_patilea2017, FranksEtAl, zsb2019}.
\begin{proposition}
\label{proposition:data_compatibility_psiT}
\emph{\textbf{(Characterizing data-compatible propensity scores)}}\\
For any random variable $\bar{E} \in \mathcal{E}_\infty(\Lambda)$ satisfying $\mathbb{E}[ Z / \bar{E} | X] = 1$, there is a distribution $Q$ for $(X, Y(0), Y(1), Z, U)$ with the following properties:
\begin{enumerate}[label=(\roman*),topsep=0pt,itemsep=-1ex]
\item The distribution of the observables $(X, Y, Z)$ is the same under $P$ and $Q$.
\item $Q$ satisfies Assumption \ref{assumption:msm}.
\item $\mathbb{E}_{Q}[Y(1)] = \mathbb{E}_{P}[ YZ / \bar{E}]$.
\end{enumerate}
\end{proposition}
In short, this result says that $\mathbb{E}[ YZ / \bar{E}]$ is a plausible value of $\psi_{\textup{T}}$ as long as $\mathbb{E}[ Z / \bar{E} | X] = 1$. It is not hard to show that the converse also holds: if $\psi$ is a plausible value of $\psi_{\textup{T}}$, then $\psi = \mathbb{E}[ YZ / \bar{E}]$ for some random variable $\bar{E}$ satisfying $\mathbb{E}[ Z / \bar{E} | X] = 1$. As a result, the optimal bounds for $\psi_{\textup{T}}$ can be obtained by solving the variational problems in Corollary \ref{corollary:variational_problems}.
\begin{corollary}
\label{corollary:variational_problems}
The partially identified set for $\psi_{\textup{T}}$ is an interval whose endpoints solve:
\begin{align}
\psi_{\textup{T}}^- &= \min_{\bar{E} \in \mathcal{E}_{\infty}(\Lambda)} \mathbb{E}[ YZ / \bar{E}] \quad \text{subject to} \quad \mathbb{E}[ Z / \bar{E} | X] = 1 \label{psi_minus_variational}\\
\psi_{\textup{T}}^+ &= \max_{\bar{E} \in \mathcal{E}_{\infty}(\Lambda)} \mathbb{E}[ YZ/\bar{E}] \quad \text{subject to} \quad \mathbb{E}[ Z / \bar{E} | X] = 1 \label{psi_plus_variational}
\end{align}
\end{corollary}
Even though the variational problems (\ref{psi_minus_variational}) and (\ref{psi_plus_variational}) can be infinite-dimensional optimization problems with infinitely-many constraints, they have several nice features that enable them to be solved explicitly. Some straightforward algebraic manipulation shows that the problem (\ref{psi_plus_variational}) can be written as:
\begin{align}
\begin{split}
\text{maximize} &\quad \mathbb{E}[ \mathbb{E}[ YZ / \bar{E} | X]]\\
\text{subject to} &\quad \mathbb{E}[ Z / \bar{E} | X] = 1\\
\text{and} &\quad 1 + \tfrac{1 - e(X)}{e(X)} \Lambda^{-1} \leq 1 /\bar{E} \leq 1 + \tfrac{1 - e(X)}{e(X)} \Lambda.
\end{split}
\end{align}
Not only is this problem \emph{linear} in the decision ``variable" $1/\bar{E}$, it also separates across levels of $X$. Therefore, it suffices to separately solve (\ref{x_specific_problem}) for each $x \in \mathcal{X}$.
\begin{align}
\begin{split}
\text{maximize} &\quad \mathbb{E}[ YZ / \bar{E} | X = x]\\
\text{subject to} &\quad \mathbb{E}[ Z / \bar{E} | X = x] = 1\\
\text{and} &\quad 1 + \tfrac{1 - e(x)}{e(x)} \Lambda^{-1} \leq 1 / \bar{E} \leq 1 + \tfrac{1 - e(x)}{e(x)} \Lambda \label{x_specific_problem}
\end{split}
\end{align}
The problem (\ref{x_specific_problem}) requires us to maximize one expectation subject to an equality constraint on another expectation. This resembles the problem solved by the Neyman-Pearson lemma, and in fact is a special case of the generalization due to \cite{dantzig_wald_1951}. The optimization problems posed in Theorem \ref{theorem:psiT_identified_set} also fall in this class. It turns out that both of these problems have a common solution, given in Proposition \ref{proposition:psiT_formulas}.
\begin{proposition}
\label{proposition:psiT_formulas}
\emph{\textbf{(Formulas for the worst-case propensity scores)}}\\
There exist $\bar{E}_-$, $\bar{E}_+ \in \mathcal{E}_{\infty}(\Lambda)$ satisfying $\mathbb{E}[ Z / \bar{E}_- | X] = \mathbb{E}[ Z / \bar{E}_+ | X] = 1$ and also (\ref{cutoff_minus}) and (\ref{cutoff_plus}).
\begin{align}
1 / \bar{E}_- &=
\left\{
\begin{array}{ll}
1 + \tfrac{1-e(X)}{e(X)} \Lambda^{+1} &\text{if } Y < Q_{1 - \tau}(X, 1)\\
1 + \tfrac{1-e(X)}{e(X)} \Lambda^{-1} &\text{if } Y > Q_{1 - \tau}(X, 1)
\end{array}
\right. \label{cutoff_minus}\\
1 / \bar{E}_+ &= \left\{
\begin{array}{ll}
1 + \tfrac{1-e(X)}{e(X)} \Lambda^{+1} &\text{if } Y > Q_{\tau}(X, 1)\\
1 + \tfrac{1-e(X)}{e(X)} \Lambda^{-1} &\text{if } Y < Q_{\tau}(X, 1)
\end{array}
\right. \label{cutoff_plus}
\end{align}
Further, $\bar{E}_-$ solves both (\ref{psiT_minus}) and (\ref{psi_minus_variational}), and $\bar{E}_+$ solves both (\ref{psiT_plus}) and (\ref{psi_plus_variational}).
\end{proposition}
The form of the propensity score $\bar{E}_+$ gives us insight into the confounding structure which maximizes $\psi_\textup{T}$: in the worst case, all observations with ``high" values of $Y$ are unlikely to be treated and thus receive large propensity weight, while all observations with ``low" values of $Y$ are likely to be treated and thus receive small propensity weight. The cutoff between high and low is chosen to satisfy the data-compatibility condition $\mathbb{E}[ Z / \bar{E}_+ | X ] = 1$.
This argument presented in this section extends immediately to $\psi_{\textup{C}}$ by swapping treatment and control labels, extends to $\psi_{\textup{ATT}}$ by the argument given in Section \ref{section:balancing_bounds_simpler}, and can extend to other sensitivity models of the form $e_{\min}(X) \leq e_0(X, U) \leq e_{\max}(X)$ by modifying the constraints of (\ref{x_specific_problem}).
\subsection{Data compatibility for the ATE}\label{section:explaining_bounds_ATE}
To extend the argument from Section \ref{section:explaining_bounds_APO} to the ATE requires additional care. Although $\psi_{\textup{ATE}}^+ = \psi_{\textup{T}}^+ - \psi_{\textup{C}}^-$ is certainly a \textit{valid} upper bound for the partially identified set for $\psi_{\textup{ATE}}$, it is not obviously a sharp one. Proposition \ref{proposition:data_compatibility_psiT} only implies that there exists a distribution $Q$ matching the observed-data distribution which has $\mathbb{E}_{Q}[Y(1)] = \psi_{\textup{T}}^+$ and another distribution $Q'$ which has $\mathbb{E}_{Q'}[ Y(0)] = \psi_{\textup{C}}^-$, but these distributions need not be the same. In other words, the two bounds may not be simultaneously achievable.
Theorem \ref{theorem:psiATE_identified_set} indicates that the worst-case bounds on the counterfactual means are simultaneously achievable in the marginal sensitivity model. This is a surprising result, given that simultaneous achievability is \emph{not} expected to hold in the closely-related Rosenbaum sensitivity model. In that model, \cite{yadlowsky2018bounds} derived sharp bounds on $\psi_{\textup{T}}$ and $\psi_{\textup{C}}$ but required an extra symmetry assumption on the distribution of potential outcomes to establish sharpness of the resulting ATE bounds.
The key to our bounds on $\psi_{\textup{ATE}}$ is the following claim, which strengthens Proposition \ref{proposition:data_compatibility_psiT}.
\begin{proposition}
\label{proposition:data_compatibility_ATE}
\emph{\textbf{(Simultaneous achievability)}}\\
For any random variable $\bar{E} \in \mathcal{E}_\infty(\Lambda)$ satisfying $\mathbb{E} [Z / \bar{E} | X] = \mathbb{E} [ (1-Z)/(1-\bar{E}) | X] = 1$, there is a distribution $Q$ for the full data $(X, Y(0), Y(1), Z, U)$ with the following properties:
\begin{enumerate}[label=(\roman*),topsep=0pt,itemsep=-1ex]
\item The distribution of the observables $(X, Y, Z)$ is the same under $P$ and $Q$.\label{item:data_compatibility_ATE:data_compatible}
\item $Q$ satisfies Assumption \ref{assumption:msm}.\label{item:data_compatibility_ATE:MSM}
\item $\mathbb{E}_{Q}[Y(1)] = \mathbb{E}_{P}[ YZ/\bar{E}]$ and $\mathbb{E}_{Q}[Y(0)] = \mathbb{E}_{P}[ Y(1-Z)/(1-\bar{E})]$.\label{item:data_compatibility_ATE:Means}
\end{enumerate}
\end{proposition}
Unlike Proposition \ref{proposition:data_compatibility_psiT}, this result does not follow from the existing data-compatibility characterizations of \cite{BirminghamJRSSB, Robins_etal_2000, tan2006, zsb2019} and instead requires an original construction. Given this result, one can derive Theorem \ref{theorem:psiATE_identified_set} as a consequence of Theorem \ref{theorem:psiT_identified_set} and Corollary \ref{corollary:psiC_att_identified_set}.
\section{Sensitivity analysis} \label{section:sensitivity_analysis}
In this section, we give our proposals for translating the population-level partial identification results of Section \ref{section:partial_identification} into practical sensitivity analyses. Our main proposal, which we call the \emph{quantile balancing} method, conducts a sensitivity analysis for IPW estimators by modifying the ZSB proposal to incorporate the sufficient constraints derived in Section \ref{section:balancing_bounds_simpler}. We also discuss extensions of our sensitivity analysis to the AIPW estimator of \cite{robins_rotnitzky_zhao1994} which are simpler to implement but only sharp under homoscedasticity.
Throughout this section, we take $\Lambda \geq 1$ to be fixed and set $\tau = \Lambda/(\Lambda + 1)$.
\subsection{Sensitivity analysis via quantile balancing} \label{section:introducing_qb}
We begin by describing our IPW sensitivity analysis for the average treated potential outcome. Theorem \ref{theorem:psiT_identified_set} implies that the largest value of $\psi_{\textup{T}}$ compatible with Assumption \ref{assumption:msm} solves the optimization problem (\ref{population_optimization}):
\begin{align}
\psi_{\textup{T}}^+ &= \max_{\bar{E} \in \mathcal{E}_{\infty}(\Lambda)} \frac{\mathbb{E}[ YZ / \bar{E}]}{\mathbb{E}[ Z / \bar{E}]} \quad \text{s.t.} \quad
\binom{\mathbb{E}[ Q_{\tau}(X, 1) Z / \bar{E}]}{\mathbb{E}[ Z / \bar{E}]} = \binom{\mathbb{E} [ Q_{\tau}(X, 1) Z / e(X)]}{\mathbb{E}[ Z / e(X)]}.
\label{population_optimization}
\end{align}
In the above display, we have included an additional constraint $\mathbb{E}[ Z / \bar{E}] = \mathbb{E}[ Z / e(X)]$ which motivates our finite-sample procedure without affecting the optimization problem value.
Our proposal is to estimate $\psi_{\textup{T}}^+$ by replacing all of the unknown quantities in (\ref{population_optimization}) with empirical counterparts. We estimate $\psi_{\textup{T}}^-$ by following the same principle. To translate these estimates into confidence intervals, we employ the same simple percentile bootstrap scheme as ZSB.
We will be concrete about what optimization problem we are proposing to solve. Let $\hat{Q}_{\tau}(x, z)$ be an estimate of the conditional quantile function of $Y$ obtained by some kind of quantile regression (e.g. \cite{generalized_random_forests, koenker_bassett_1978, quantile_random_forest, stone1977}). Let $\hat{e}$ be the data analyst's estimate of the nominal propensity score $e$ from their primary analysis. We define $\hat{\psi}_{\textup{T}}^+$ as the solution to the empirical maximization problem (\ref{qbalance_psiT}).
\begin{align}
\hat{\psi}_{\textup{T}}^+ &= \max_{\bar{e} \in \mathcal{E}_n(\Lambda)} \frac{\mathbb{E}_n[YZ/\bar{e}]}{\mathbb{E}_n[Z/\bar{e}]} \quad \text{s.t.} \quad \binom{\mathbb{E}_n[ \hat{Q}_{\tau}(X, 1) Z / \bar{e}]}{\mathbb{E}_n[Z / \bar{e}]} = \binom{\mathbb{E}_n[\hat{Q}_{\tau}(X, 1) Z / \hat{e}(X)]}{\mathbb{E}_n[Z / \hat{e}(X)]} \label{qbalance_psiT}
\end{align}
The lower bound $\hat{\psi}_{\textup{T}}^-$ is defined similarly, but with maximization replaced by minimization and $\hat{Q}_{\tau}(x, z)$ replaced by another quantile estimate $\hat{Q}_{1 - \tau}(x, z)$. We call $\hat{\psi}_{\textup{T}}^+$ and $\hat{\psi}_{\textup{T}}^-$ the \emph{quantile balancing bounds} for $\psi_{\textup{T}}$.
Two features of this proposal require some explanation. The first feature to explain is the inclusion of the constraint $\mathbb{E}_n[Z / \bar{e}] = \mathbb{E}_n[ Z / \hat{e}(X)]$ in (\ref{qbalance_psiT}). At the population level, Theorem \ref{theorem:psiT_identified_set} shows that only the constraint $\mathbb{E}[ Q_{\tau}(X, 1) Z / \bar{E}] = \mathbb{E}[ Q_{\tau}(X, 1) Z / e(X)]$ is relevant. However, in finite samples, this additional constraint improves robustness when $\hat{Q}_{\tau}$ is an inaccurate estimate of $Q_{\tau}$ and also simplifies the associated computation. The second feature to explain is why the right-hand side of the constraints in (\ref{qbalance_psiT}) have an ``IPW" form (i.e. $\mathbb{E}_n[ \hat{Q}_{\tau}(X, 1) Z / \hat{e}(X)]$) rather than a ``sample average" form (i.e. $\mathbb{E}_n[ \hat{Q}_{\tau}(X, 1)]$). If $\mathbb{E}_n[ \hat{Q}_{\tau}(X, 1) Z/ \hat{e}(X)] \neq \mathbb{E}_n[ \hat{Q}_{\tau}(X, 1)]$, then a sample average version of (\ref{qbalance_psiT}) may have no feasible propensities. With the IPW form, $\bar{e}_i = \hat{e}(X_i)$ is always feasible.
Now that we have explained our proposed sensitivity analysis, we will collect several immediate properties of the quantile balancing bounds:\\
\begin{enumerate}[label=(\roman*),topsep=0pt,itemsep=-1ex]
\item When $\Lambda = 1$ (i.e. no confounding is allowed), the quantile balancing bounds collapse to the usual IPW estimate of $\psi_{\textup{T}}$ under unconfoundedness. \label{prop:nest_ipw}
\item The quantile balancing bounds are sample bounded, i.e. $\min_i Y_i \leq \hat{\psi}_{\textup{T}}^- \leq \hat{\psi}_{\textup{T}}^+ \leq \max_i Y_i$.\label{prop:SampleBound}
\item The quantile balancing bounds are always a subset of the ZSB bounds and, outside of knife-edge cases, are a strict subset.\label{prop:ZSBSubset}
\item The optimization problem (\ref{qbalance_psiT}) is convex and can be solved efficiently. In fact, it reduces to a standard quantile regression problem. See Appendix A
for implementation details.\\
\end{enumerate}
The property \ref{prop:nest_ipw} leads us to call quantile balancing a ``sensitivity analysis for IPW." One can also apply quantile balancing to unstabilized IPW estimators at the cost of properties \ref{prop:SampleBound} and \ref{prop:ZSBSubset}. See Appendix B for computational details, including for Augmented IPW estimators.
The quantile balancing idea extends easily to other causal estimands. To compute bounds for $\psi_{\textup{C}}$, one only needs to exchange the definitions of ``treated" and ``control" and solve the same optimization problem. Subtracting the bounds for $\psi_{\textup{T}}$ and $\psi_{\textup{C}}$ gives bounds for $\psi_{\textup{ATE}}$, and bounds for $\psi_{\textup{ATT}}$ follow from a similar principle (see Appendix A
for the exact formula).
To form confidence intervals based on quantile balancing, we follow \cite{zsb2019} and propose using the percentile bootstrap. If $[\hat{\psi}_b^-, \hat{\psi}_b^+]$ are quantile balancing bounds estimated in the $b^{\text{th}}$ of $B$ bootstrap samples, we report the quantile balancing $1 - \alpha$ confidence interval as:
\begin{align}
\textup{CI}(\alpha) = [ Q_{\alpha/2} ( \{ \hat{\psi}_b^- \}_{b \in [B]}), Q_{1 - \alpha/2}( \hat{\psi}_b^+ \}_{b \in [B]})]. \label{ci}
\end{align}
As is standard for bootstrap-based IPW inference, we require re-estimating the nominal propensity score separately in each bootstrap replication. That requirement does not extend to the conditional quantiles. While the conditional quantiles can be re-estimated within bootstraps, our inference results will also apply if they are taken from the main dataset. This helps keep inference computationally tractable.
\subsection{Implications for AIPW sensitivity analysis} \label{section:balance_aipw}
The quantile balancing sensitivity analysis described above requires the data analyst to perform several quantile regressions. Our partial identification results imply that, in certain ``additive-noise" data generating processes, a data analyst whose primary analysis was conducted using the AIPW estimator can perform sharp sensitivity analysis without performing any quantile regressions.
To explain how, we begin by describing the modeling assumption. Suppose the observed outcome $Y$ has the following signal-plus-noise representation:
\begin{align}
Y = \mu(X, Z) + \epsilon \quad \text{with} \quad \mathbb{E}[ \epsilon ] = 0, \epsilon \, \rotatebox[origin=c]{90}{$\models$} \, (X, Z). \label{eq:signal_plus_noise}
\end{align}
Such models frequently arise in the regression applications \citep[see, e.g.][Chapter 3]{esl2001} and fit quite well in the real-data example we present in Section \ref{section:real_data} below.
The additive-noise assumption (\ref{eq:signal_plus_noise}) implies that the conditional quantiles of the residuals $\epsilon$ are constant. In particular, the assumption implies $Q_\tau(x, z) = \mu(x, z) + Q_\tau(\epsilon)$, where $Q_\tau(\epsilon)$ is the $\tau$-th quantile of the noise. Therefore, Theorem \ref{theorem:psiT_identified_set} and some algebra imply that the sharp upper bound for $\psi_{\textup{T}}$ has the following formula:
\begin{align}
\psi_{\textup{T}}^+ &= \max_{\bar{E} \in \mathcal{E}_{\infty}(\Lambda)} \left\{ \mathbb{E}[ \mu(X, 1) + \frac{\mathbb{E}[ (Y - \mu(X, 1)) Z / \bar{E}]}{\mathbb{E}[ Z / \bar{E}]} \right\} \quad \text{s.t.} \quad \mathbb{E}[Z/\bar{E}] = \mathbb{E}[Z/e(X)].
\end{align}
Similar formulas can be derived for $\psi_{\textup{T}}^-, \psi_{\textup{C}}^+, \psi_{\textup{C}}^-$. This formula is convenient after an AIPW primary analysis, which requires estimates of all the nuisance parameters in this equation.
A natural estimate of $\psi_{\textup{T}}^+$ is the finite-sample analogue of (\ref{eq:additive_noise}).
\begin{align}
\hat{\psi}_{\textup{T,AIPW}}^+ &= \max_{\bar{e} \in \mathcal{E}_n(\Lambda)} \left\{ \mathbb{E}_n[ \hat{\mu}(X, 1)] + \frac{\mathbb{E}_n [ (Y - \hat{\mu}(X, 1)) Z / \bar{e}]}{\mathbb{E}_n[Z/\bar{e}]} \right\} \quad \text{s.t.} \quad \mathbb{E}_n[Z/\bar{e}] = \mathbb{E}_n[Z/\hat{e}(X)] \label{qbalance_aipw}
\end{align}
The estimated bound $\hat{\psi}_{\textup{T,AIPW}}^+$ grows with $\Lambda$ and recovers the original (stabilized) AIPW estimator when $\Lambda = 1$. One can also not divide by $\mathbb{E}_n[Z/\bar{e}]$ in (\ref{qbalance_aipw}) to recover the unstabilized AIPW estimator at $\Lambda = 1$.
The estimator (\ref{qbalance_aipw}) slightly modifies the proposal in Section 6.2 of \cite{zsb2019} to include the balancing constraint $\mathbb{E}_n[ Z / \bar{e}] = \mathbb{E}_n[Z / \hat{e}(X)]$. In theory, this constraint is necessary to achieve sharpness in the additive-noise model (\ref{eq:signal_plus_noise}). However, the simulations presented in Section \ref{section:numerical_examples} find that when the additive-noise model holds, this constraint scarcely refines the stabilized point estimates while somewhat degrading the coverage of bootstrap confidence intervals.
\subsection{Theoretical properties}
We now state some theoretical properties of the quantile balancing bounds $[\hat{\psi}^-, \hat{\psi}^+]$ which apply when the outcome $Y$ has a continuous distribution. In short, the bounds are sharp when quantiles are estimated consistently and are valid even when quantiles are estimated inconsistently. Moreover, the percentile bootstrap yields valid confidence intervals if standard IPW inference conditions are satisfied and quantiles are estimated parametrically.
To obtain these results, we need a few conditions. The first condition collects some standard IPW consistency requirements which we expect the data analyst to have already assumed in their primary analysis.
\begin{condition}
\label{condition:ipw_conditions}
\textup{\textbf{(IPW assumptions)}}\\
The nominal propensity score $e$ satisfies $\varepsilon \leq e(X) \leq 1 - \varepsilon$ with probability one for some $\varepsilon > 0$. The estimated nominal propensity score $\hat{e}(\cdot) \equiv \hat{e}( \cdot, \{ X_i, Z_i \}_{i \leq n} )$ is uniformly consistent, and the variance of $Y$ is finite.
\end{condition}
The second condition requires that the outcome $Y$ has a bounded conditional density which is positive near the relevant conditional quantiles. This is a common identification condition for quantile regression \citep{generalized_random_forests, BelloniEtAl2019}. However, it means our theoretical guarantees do not apply when $Y$ is discrete.
\begin{condition}
\label{condition:density}
\textup{\textbf{(Density)}}\\
The conditional distribution of $Y \mid X, Z$ has a uniformly bounded density $f(y | x, z)$. For each $(x, z) \in \mathcal{X} \times \{ 0, 1 \}$, the map $y \mapsto f( y | x, z)$ is continuous and positive near $Q_{1 - \tau}(x, z)$ and $Q_{\tau}(x, z)$.
\end{condition}
Finally, we make some assumptions about how the quantiles are estimated. For the standard linear quantile regression method of \cite{koenker_bassett_1978}, one only needs to check that the regressors in the quantile regression have finite variance. We cover generic (possibly nonlinear) methods by requiring sample splitting to avoid overfitting. The specific form of sample splitting analyzed in our proofs is ``cross-fitting" \citep{schick1986, newey_robins_crossfitting, doubleML}, but leave-one-out or out-of-bag quantile estimates perform similarly in simulations. Based on our practical experience, we recommend using some kind of sample splitting even when the quantile model is linear.
\begin{condition}
\label{condition:quantile_estimates}
\textup{\textbf{(Quantile estimates)}}\\
For each $t \in \{ 1 - \tau, \tau \}$, one of the following holds for the estimated quantile function $\hat{Q}_t$:
\begin{enumerate}[label=(\roman*),topsep=0pt,itemsep=-1ex]
\item $\hat{Q}_t(x, z) = \hat{\beta}_t(z)^{\top} h(x)$ for some fixed ``features" $h_j(X)$ with finite variance. \label{linear}
\item $\hat{Q}_t(x, z)$ is estimated using cross-fitting and satisfies Condition N in the supplementary materials. \label{crossfit}
\end{enumerate}
\end{condition}
Condition \ref{condition:quantile_estimates} is essentially ``algorithmic," and neither \ref{linear} nor \ref{crossfit} impose any accuracy requirements on the estimated conditional quantiles. The appendix conditions in \ref{crossfit} are technical to state but very mild. Under Conditions \ref{condition:ipw_conditions} and \ref{condition:density}, they are satisfied by quantile estimates based on nearest-neighbors \citep{stone1977}, kernels \citep{kernel_quantile1990}, and random forests \citep{generalized_random_forests, quantile_random_forest}.
Under these conditions, we have the following result on the asymptotic sharpness of the quantile balancing bounds.
\begin{theorem}
\label{theorem:sharpness}
\textup{\textbf{(Sharpness and robustness)}}\\
For any $\psi_0 \in \{ \psi_{\textup{T}}, \psi_{\textup{C}}, \psi_{\textup{ATT}}, \psi_{\textup{ATE}} \}$, let $[\psi^-, \psi^+]$ be its partially identified interval under Assumption \ref{assumption:msm} and let $[ \hat{\psi}^-, \hat{\psi}^+]$ be the corresponding quantile balancing interval. Assume Conditions \ref{condition:ipw_conditions}, \ref{condition:density}, and \ref{condition:quantile_estimates}.
\begin{enumerate}[label=(\roman*),topsep=0pt,itemsep=-1ex]
\item If the quantile regression estimates are consistent, then $\hat{\psi}^- \xrightarrow{p} \psi^-$ and $\hat{\psi}^+ \xrightarrow{p} \psi^+$. \label{sharpness}
\item Even if the quantile models are misspecified, we still have $\hat{\psi}^- \leq \psi^- + a_n$ and $\psi^+ - b_n \leq \hat{\psi}^+$, where $a_n = o_P(1)$ and $b_n = o_P(1)$. \label{robustness}
\end{enumerate}
\end{theorem}
The same conclusions hold for the AIPW-based bounds introduced in Section \ref{section:balance_aipw} when the outcome regression is estimated by linear regression, i.e. sharpness under an additive-noise model and validity in general. However, while AIPW is doubly-robust under unconfoundedness, the validity of the corresponding AIPW quantile balancing bounds relies on correct specification of the nominal propensity score.
The result \ref{robustness} shows that even when quantiles are not estimated consistently, the quantile balancing bounds are still valid; we will offer some intuition on why this novel robustness property holds. At the population level, the worst-case propensity score $\bar{E}_+$ defined in Proposition \ref{proposition:psiT_formulas} ``balances" all integrable function of $X$, so intuitively, we should expect that it ``nearly" balances the estimated quantile function $\hat{Q}_{\tau}(\cdot, 1)$ in finite samples even if $\hat{Q}_{\tau}(\cdot, 1)$ is not particularly close to $Q_{\tau}(\cdot, 1)$. That suggests a vector of propensities very close to the true worst-case propensity vector will belong to the feasible set $\mathcal{E}_n(\Lambda)$. Since the quantile balancing upper bound $\hat{\psi}_{\textup{T}}^+$ is defined as a maximum over the feasible set, it will be at least as large as a quantity close to $\psi_{\textup{T}}^+$. This roughly explains why validity holds even under misspecification.
The validity of the confidence interval (\ref{ci}) follows under stronger parametric assumptions. We prove an inference result assuming the nominal propensity score is estimated by a correctly-specified parametric model and the conditional quantiles are estimated by a (potentially misspecified) parametric model.
\begin{theorem}
\label{theorem:inference}
\textup{\textbf{(Inference)}}\\
Let $[ \psi^-, \psi^+]$ be as in Theorem \ref{theorem:sharpness}, and let $\textup{CI}(\alpha)$ be as in (\ref{ci}). Suppose Conditions \ref{condition:ipw_conditions}, \ref{condition:density}, and \ref{condition:quantile_estimates}.\ref{linear} are satisfied, and also that the nominal propensity score is estimated by a regular parametric model (e.g. logistic regression). Then we have
\begin{align}\label{eq:InferentialGuarantee}
\liminf_{n \rightarrow \infty} \mathbb{P}( [\psi^-, \psi^+] \subseteq \textup{CI}(\alpha)) \geq 1 - \alpha
\end{align}
for any $\alpha \in (0, 1)$.
\end{theorem}
We have found that these confidence intervals can under-cover the identified set in finite samples when the quantiles are correctly specified. In our simulations, the use of cross-fit conditional quantile estimates largely resolves the issue with minimal effect on point estimates, so we advocate for the use of such estimators in practice. Although we do not have theoretical support for the confidence interval $\text{CI}(\alpha)$ when quantiles are estimated by a nonlinear model, we find that approach performs reasonably well in the simulations of Section \ref{section:numerical_examples} as long as cross-fit quantiles are used.
\section{Numerical examples}\label{section:numerical_examples}
In this section, we illustrate the finite-sample performance of our proposed sensitivity analyses on several simulated datasets and one real-data example.
\subsection{Simulated data}
We consider two data-generating processes (DGPs) in our simulated examples. The two DGPs differ in the conditional distribution of $Y$ given $(X, Z)$, but otherwise can be described as follows:
\begin{align}
\begin{split}
X &\sim \text{Uniform}([-1, 1]^5)\\
Z \mid X & \sim \text{Bernoulli} \left( \tfrac{1}{1 + \exp(-\sum_{j = 1}^5 X_{j}/\sqrt{5})} \right)\\
Y \mid X, Z &\sim \mathcal{N}( \mu(X), \sigma^2(X)).
\end{split}
\end{align}
In the first DGP, we use $\mu(x) = x_1 + \cdots + x_5$ and $\sigma(x) = 1$. In the second DGP, we use $\mu(x) = \tfrac{3}{2} \text{sign}(x_1) + \text{sign}(x_2)$ and $\sigma(x) = 2 + \text{sign}(x_3) + \text{sign}(x_4)$. The estimand of interest is the ATE and we fix $\Lambda = 2$, i.e. unobserved confounders can double or halve the odds of treatment.
We compare five methods for obtaining bounds on the partially identified set:
\begin{enumerate}[itemsep=-1ex]
\item \texttt{QB-Linear} applies the quantile balancing method of Section \ref{section:sensitivity_analysis} with quantiles estimated using linear quantile regression on $X_1, \ldots, X_5$.
\item \texttt{QB-Forest} applies quantile balancing with quantiles estimated using the random forest method from \cite{generalized_random_forests}.
\item \texttt{ZSB} applies the main IPW method from \cite{zsb2019}, described in Section \ref{section:sharpness}.
\item \texttt{ZSB-AIPW} applies the AIPW-based method from Section 6.2 of \cite{zsb2019}, described in Section \ref{section:balance_aipw}. This requires an estimate of the outcome model $\mu(X, Z) = \mathbb{E}[Y | X, Z]$. We use a situationally-appropriate outcome model, linear regression in DGP1 and random forest regression in DGP2.
\item \texttt{AIPW+1} applies the AIPW-based method introduced in Section \ref{section:balance_aipw}. We call this \texttt{AIPW+1} because it refines \texttt{ZSB-AIPW} to incorporate an additional ``one-balancing" constraint $\mathbb{E}_n[Z/\bar{e}] = \mathbb{E}_n[Z/\hat{e}(X)]$.
\end{enumerate}
All methods estimate the nominal propensity score by logistic regression. We use 5-fold cross-fitting in all of our quantile regressions. We do not re-estimate quantiles or random forest models within bootstraps.
Figure \ref{fig:simulation} shows the distribution of upper and lower bound point estimates from each of these five methods, estimated using 2,000 simulations with $n = 1,000$ observations each. Simulations at other sample sizes are presented in Appendix B.
Dashed lines indicate the true partially identified region. The results conform to the asymptotic predictions of Section \ref{section:sensitivity_analysis}: (i) when the quantile models are ``correctly specified," the quantile balancing point estimates are nearly unbiased; (ii) under misspecification, the range of \texttt{QB} point estimates is too wide rather than too narrow; (iii) the \texttt{ZSB} range of point estimates is too wide in both cases; and (iv) AIPW-based methods give nearly-sharp bounds in the additive-noise DGP1 but conservative bounds in the heteroscedastic DGP2. We also find that the \texttt{+1} constraint in \texttt{AIPW+1}, which is necessary for sharpness in theory, has minimal practical impact in either DGP.
\begin{figure}[!ht]
\centering
\includegraphics[width=15cm]{n1000.pdf}
\caption{\textit{Boxplots of the ATE upper and lower bound point estimates for both DGPs and all considered methods. The dashed line indicates the boundary of the true partially identified set. In DGP1, all methods but \texttt{ZSB} are correctly specified and give reasonably accurate bounds. In DGP2, the \texttt{Forest} method is well-suited to the piecewise-constant conditional quantiles and gives the most accurate bounds.}}
\label{fig:simulation}
\end{figure}
Figure \ref{fig:coverage1000} shows the coverage for 95\% bootstrap confidence intervals based on each of the five methods. In DGP1, both quantile balancing methods have nearly nominal coverage, but AIPW-based methods undercover and the \texttt{+1} constraint exacerbates the undercoverage. In DGP2 the \texttt{QB-Forest} method achieves nearly nominal coverage, while all other methods overcover. The \texttt{ZSB} method overcovers for both DGPs.
\begin{figure}[!ht]
\centering
\includegraphics[width=15cm]{coverage1000.pdf}
\caption{\textit{The coverage of nominal level 95\% bootstrap confidence intervals based on five different methods. Estimates are based on $2,000$ simulations each with $n = 1,000$ observations. The error bars are binomial confidence intervals for the true coverage probability.}}
\label{fig:coverage1000}
\end{figure}
\subsection{Real data} \label{section:real_data}
In this section, we apply our proposed sensitivity analysis to a subsample of data from the 1966-1981 National Longitudinal Survey (NLS) of Older and Young Men. We wish to estimate the impact of union membership on wages. Specifically, we consider the ATE of union membership on log wages. For illustrative reasons, we focus on the 1978 cross-section of Young Men and restrict our attention to craftsmen and laborers not enrolled in school. Our estimates are thus based on a sample of 668 respondents with measurements of wages, union membership, and eight covariates.
For our primary analysis, we use IPW to adjust for baseline imbalances in covariates between union and nonunion samples. Table \ref{table:balance} reports the covariate balance between union and nonunion samples before and after weighting by the (estimated) inverse propensity score. On several important characteristics, inverse propensity weighting dramatically improves balance across the two samples.
\begin{table}[!ht]
\centering
\small{\begin{tabular}{rrrrr}
\hline
&\multicolumn{2}{c}{Unweighted} &\multicolumn{2}{c}{Weighted}\\
Covariate &Union &Nonunion &Union &Nonunion\\
\hline
Age & 30.1 & 30.0 & 30.0 & 30.0 \\
Black & 24\% & 24\% & 23\% & 24\% \\
Metropolitan & \textcolor{red}{74\%} & \textcolor{red}{57\%} & 66\% & 65\% \\
Southern & \textcolor{red}{32\%} & \textcolor{red}{53\%} & 42\% & 42\% \\
Married & 78\% & 75\% & 76\% & 76\% \\
Manufacturing & \textcolor{red}{42\%} & \textcolor{red}{32\%} & 37\% & 38\% \\
Laborer & \textcolor{red}{23\%} & \textcolor{red}{15\%} & 18\% & 18\% \\
Education & 12.2 & 11.7 & 12.1 & 12.0 \\
\hline
\end{tabular}}
\caption{\small{\textit{Covariate means among the nonunion and union subsamples, along with the means in the weighted samples. In \textcolor{red}{red}, we highlight particularly large imbalances. In the weighted samples, propensity weights are estimated using logistic regression.}}}
\label{table:balance}
\end{table}
The IPW point estimate of the ATE is 0.23 with an associated 90\% confidence interval of $[0.18, 0.27]$. Thus, our primary analysis concludes that union membership has a positive effect on wages, at least on average among craftsmen and laborers. Both the point estimate and the confidence interval are in agreement with prior literature studying the same problem using cross-sectional data. See \cite{ jakubson1991, johnson1975} for overviews. An AIPW-based primary analysis gives the same point estimate and confidence interval, up to rounding.
\cite{freemn1984_unions}, \cite{mellow1981}, and many other economists have argued that cross-sectional estimates of the union premium overestimate the true causal effect because higher-skill workers are simultaneously more likely to be selected for union jobs and earn higher wages. Here, ``skill" refers to an unobserved confounder which is only partially captured by the measured covariates. Is it plausible that the positive effect we find in the IPW analysis could be entirely due to selection on skill? A sensitivity analysis may help address this question.
Figure \ref{fig:union_results} reports point estimate ranges and 90\% bootstrap confidence intervals from quantile balancing, the ZSB-IPW method, and the ZSB-AIPW method for several values of the sensitivity parameter $\Lambda$. For quantile balancing, we estimate conditional quantiles using linear quantile regression with five-fold cross fitting. For AIPW, we use linear regression for the outcome model.
\begin{figure}[!ht]
\centering
\includegraphics[width=15cm]{nls78.pdf}
\caption{\textit{Point estimate ranges and 90\% bootstrap confidence intervals for the ATE in the NLS dataset. For the quantile balancing method, conditional quantiles are estimated using the linear quantile regression method of \cite{koenker_bassett_1978}, with five-fold cross-fitting.}}
\label{fig:union_results}
\end{figure}
All three sensitivity analyses show that the positive effect found in the primary analysis is fairly robust to unobserved confounding, but quantile balancing and ZSB-AIPW refine the baseline ZSB-IPW interval. Even if the odds of union membership for ``skilled" workers were nearly double ($\Lambda = 1.9$) the odds for ``typical" workers with the same observed covariates, the quantile balancing and AIPW sensitivity analyses analysis would still find a statistically significant positive treatment effect. Meanwhile, when $\Lambda = 1.8$, the ZSB confidence intervals already include the null. In this application, quantile balancing only slightly refines the ZSB range. Moreover, quantile balancing and ZSB-AIPW yield very similar ranges and confidence intervals. This is to be expected from the discussion in Section \ref{section:balance_aipw}, as an ``additive noise" model appears quite plausible in this application.
To put these sensitivities in context, we follow \cite{kallus_zhou2020} and compute the degree to which the (estimated) odds of union membership could change if \textit{measured} confounders were omitted from the dataset. Caveats to this approach and more sophisticated empirical calibration strategies are discussed in
\cite{hsu_small2013, zhang_small2020, CinelliHazlett}.
No measured confounders except \texttt{Laborer} and \texttt{South} were able to nearly double or halve the odds of union membership for any respondent. We interpret these results as showing that the qualitative conclusions of the primary analysis are fairly robust to unobserved confounding by skill.
Incidentally, longitudinal estimates of union wage effects --- which control for individual-specific effects like ``skill" --- come to similar conclusions as the one suggested by our sensitivity analysis. Although treatment effect estimates from longitudinal studies are generally smaller than those from cross-sectional studies, they still find evidence in favor of the ``union premium" \citep{CHAMBERLAIN1982, jakubson1991, freemn1984_unions}.
\section{Conclusion}\label{section:conclusion}
We have shown that quantile balancing --- a simple modification of the popular ZSB sensitivity analysis --- is feasible, robust, and sharp. This new sensitivity analysis for IPW is based on novel partial identification results for \cite{tan2006}'s marginal sensitivity model.
We will point to several interesting directions for future work. While our partial identification results focus on counterfactual means and a few treatment effects, it should be possible to extend our partial identification results to more complex estimands of the type considered in \cite{kallus2018interval, kallus2020confoundingrobust, confounding_robust_policy_improvement, kallus_zhou2020, causal_rule_ensemble}. Perhaps a similarly compact sensitivity analysis could even apply to dynamic treatment regimes. Future work could also investigate data-compatibility in the finite population model. In addition, while our IPW identification arguments generalize to any sensitivity assumption that only restricts the propensity score in a pointwise fashion (i.e. $e_{\min}(x) \leq e_0(x, u) \leq e_{\max}(x)$), the practicality of our sensitivity analysis and its theoretical properties rely on the marginal sensitivity model quite heavily. It would be interesting to see if a practical and sharp sensitivity analysis could be developed for other sensitivity assumptions in this class.
\bibliographystyle{chicago}
\bibliography{bibliography.bib}