Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
56,119 characters · 11 sections · 49 citation commands
Sensitivity Analysis in Unconditional Quantile Effects
\doublespacing \normalem
} \\ Department of Economics, UC Santa Cruz} \thispagestyle{empty}
Keywords: unconditional quantile effects, partial identification, sensitivity analysis.
\pagenumbering{arabic}
In this paper we propose a sensitivity analysis on the effect of counterfactual policies that change the proportion of treated individuals. Consider a situation where a policy maker is interested in treating non-treated individuals. The key identification challenge is that we do not observe the counterfactual outcome of individuals who switch groups, that is, the newly treated individuals. In some cases, however, it is still possible to recover the distribution of the unobserved counterfactual outcome. For example, suppose that treatment status is randomly assigned, and a policy maker increases the proportion of treated individuals by randomly selecting non-treated individuals.\footnote{We assume full compliance in both randomizations.} Although we do not observe the counterfactual outcome of the newly treated individuals, we know it is drawn from the same distribution as the already treated individuals. Hence, we identified the counterfactual distribution of newly treated individuals.
When treatment status is not randomly assigned in the first place, the identification strategy previously described breaks down. The reason is that due to the selection bias in the original treatment status, a random selection of individuals from the control group will be drawn from a different distribution. Thus, in the presence of selection bias, identification of the counterfactual distribution requires that the policy maker has enough information to devise a policy such that the (unobservable) distribution of the newly treated “matches” the distribution of the already treated individuals. This is usually infeasible. Even if the policy maker has this information, such as when treatment status is randomly assigned, they might not be interested in a policy that merely selects the newly treated individuals at random.
The previous discussion highlights that identification of counterfactual distributions results in either very stringent information requirements, or in policies that might not be interesting. In both cases, the distribution of the newly treated individuals is restricted. From the point of view of the policy maker, this can rule out many interesting policies. To see this, consider the following example. A policy maker might like to know if an increase in the unionization rate reduces inequality. If unionized workers are relatively high-skilled, and a policy expands unionization with low-skilled workers, then the distribution of wages conditional on being in the union, is likely to change.
In order to analyze a richer set of counterfactual policies, we drop the restrictions on the distribution of the newly treated individuals and provide partial identification results for two effects. The first one is a global effect that compares the quantiles of the observed outcome, to those of the counterfactual outcome, where the proportion of treated individuals has been increased by $\delta$. The second one is a marginal effect where we let $\delta$ go to zero, and analyze its limiting effect on the unconditional quantiles of the outcome.
Another important contribution of this paper is to propose a framework for a sensitivity analysis on certain conclusions of interest. To do this, we quantify the departure from point identification by the vertical distance between the distributions of the newly treated individuals and the already treated individuals. We introduce a curve called the quantile breakdown frontier, which first indicates whether a sensitivity analysis is possible, and second, it quantifies the maximum departure from point identification such that a given set of conclusions holds across different quantiles. Using this curve, we bound the global effects curve using this maximum departure derived from the quantile breakdown frontier. In this way, we obtain an identified region for the global effect curve consistent with the desired conclusions. Estimation of both the quantile breakdown frontier and the bounds on the global effect are based on empirical distribution functions and empirical quantiles, and are $\sqrt n$-consistent.
The departure from point identification is due to the selection bias induced by the counterfactual policy. We call this the policy selection bias. The usual selection bias states that treated and non-treated individuals are different in a sense, and that is what explains the selection in the first place. Instead, the policy selection bias is the difference between the distributions of the newly treated individuals and the already treated individuals. Returning to the unionization example, the policy selection bias arises because the union wages of newly unionized workers may not be drawn from distribution of the already unionized workers. We do not know the distribution of union wages of newly unionized workers, hence we can only partially identify the global and marginal effects.
The policy selection bias can be non-negligible even if the original selection into treatment is randomly assigned. The reason is that, for the policy selection bias, what matters is who the newly treated individuals are. Conversely, if there is selection bias initially, but the distribution of the newly treated “matches” the distribution of the already treated individuals, then there will be no policy selection bias. Thus, the policy selection bias depends on the particular counterfactual policy being analyzed, not whether there is selection bias in the original selection mechanism.
We apply these methods to the study of unions and inequality, which has long been of interest to labor economics. A recent comprehensive review of this extensive literature is provided by Farber2020. Using the data in Firpo2009, our empirical application considers the effect of expanding unionization on the quantiles of the distribution of (log) wages. Our approach allows us to tackle the question from a different perspective. Using the tools developed in this paper, we can quantify the amount of policy selection bias that is consistent with a policy that increases the unionization rate by unionizing low earnings workers. By looking at the global effect in the $30$\textsuperscript{th} quantile of the distribution of wages we investigate the amount of policy selection bias consistent with unions reducing overall inequality. To this end, we examine the following conclusion: whether the $30$\textsuperscript{th} quantile increases by more than 10%. We find that this is consistent with moderate values of policy selection bias. The bounds on the global effect for other quantiles reveals that this policy can have a bigger effect on quantiles below the $30$\textsuperscript{th} quantile, without hurting those at the middle and top of the distribution of income.
Related Literature There is an extensive literature devoted to the analysis of counterfactual distributions. A good reference is Firpo2011. In this paper, we focus on counterfactual distributions that arise as a result of a counterfactual policy that changes the proportion of treated individuals. The Policy Relevant Treatment Effect (PRTE) of heckman_prte, Heckman2005, and the Marginal PRTE (MPRTE) of Carneiro2010,Carneiro2011 are examples of the aforementioned global and marginal effects. The difference is that they analyze the unconditional mean of the outcome. Identification relies on the a separable threshold model for the selection equation, and the availability of a continuous instrumental variable. In this setting, the proportion of treated individuals is changed by manipulating the instrumental variable. Our analysis does not make any assumptions on the selection equation. We do not require an instrumental variable either.
The marginal effect on the unconditional quantiles of an outcome was first studied by Firpo2009. The identification arguments of Firpo2009 are based on a distributional invariance assumption: the distribution of the outcome for the original treatment group (under the original policy regime) is the same as that for the new treatment group (under the new policy regime), and this also holds for the control groups under the two policy regimes.\footnote{See the proof to Corollary 3 of the working paper version Firpo2007.} For the case of an endogenous binary covariate, where distributional invariance might not hold, yixiao2020 achieve identification by generalizing the Marginal Treatment Effect framework. Kasy2016 also analyzes counterfactual policies which assign a binary treatment, but focuses on a welfare ranking.
Rothe2012 provides a general treatment for functionals of the unconditional distribution of the outcome. What we call a global effect, Rothe2012 refers to as a Fixed Partial Policy Effect, and what we call a marginal effect, Rothe2012 refers to as a Marginal Partial Distributional Policy. However, Rothe2012 imposes different identifying assumptions, namely a form of conditional exogeneity, which also yield a partial identified set. We do not impose such assumptions in order to broaden the types of policies we can analyze.
It is important to highlight that we do not estimate a quantile treatment effect. The quantile treatment effect is the difference between the $\tau$-quantile under treatment and the $\tau$-quantile under control, and depends on the distribution of the covariates. In a recent contribution, Lieli2020 investigate the changes in this effect when the distribution of the covariates is manipulated. Aside from treatment status, we do not manipulate the distribution of covariates.
Our sensitivity analysis is based on the breakdown analysis of Kline2013 and Masten2020. Kline2013 perform a sensitivity analysis in a different context: departures from a missing (data) at random assumption. In a manner similar to us, this departure is measured as the Kolmogorov-Smirnov distance between the distribution of observed outcomes and the (unobserved) distribution of missing outcomes. Our quantile breakdown frontier builds on the breakdown frontier introduced by Masten2020. However, instead of relaxing two parameters, we relax just one, and plot it against different quantiles. Another recent application of the breakdown analysis is noack in the context of LATE.
Notation All the CDFs are denoted by $F$ with a subscript indicating the random variable. So, the CDF of $Y$ is $F_Y(y)$. Conditional CDFs are denoted similarly. For example, the CDF of $Y$ conditional on $D=1$ and $X=x$ is denoted by $F_{Y|D=1,X=x}(y)$. The $\tau$-quantile of $Y$ is denoted by $F^{-1}_Y(\tau)$. Weak convergence is denoted by $\rightsquigarrow$.
We will work with the potential outcomes framework. For some unknown functions $h_0$ and $h_1$
where $X$ are observed covariates and $U_0$ and $U_1$ consist of unobservables. We do not impose any restriction on the dimension of the unobservables. The observed outcome is thus
for a general nonseparable function $h$, where $D$ is a binary random variable taking values $0$ and $1$, and $U:=(U_0,U_1)'$. The variable $D$ can be interpreted as the treatment status, and $p:=\Pr(D=1)$ is the proportion of treated individuals.
A counterfactual policy is an alternative assignment of individuals to treatment. It is given by a binary random variable $D_\delta$, such that $\Pr(D_\delta=1)=p+\delta$ for a fixed $\delta \in(-p,1-p)$. It is called counterfactual because it may assign $D_\delta=1$ to an individual whose $D=0$. As $\delta$ varies over $(-p,1-p)$, we obtain a collection of (counterfactual) policies which is denoted by $\mathcal D$. When a particular counterfactual policy $D_\delta$ belongs to $\mathcal D$ we write $D_\delta\in\mathcal D$. The counterfactual outcome we would observe for a given $D_\delta\in\mathcal D$ is
where we implicitly assumes that the potential outcomes are not affected by the manipulation of $D$.
Strictly speaking, the counterfactual outcome $Y_{D_\delta}$ is not well defined until we define $\mathcal D$, the collection of counterfactual policies. We will restrict ourselves to policies that shift a portion of individuals in the control group to the treatment group. We refer to such individuals as newly treated. This means that for every individual, $D_\delta-D\geq 0$. This is shown in Figure (ref).
The monotonicity assumption $D_\delta-D\geq 0$ is mainly for expositional simplicity. We can do without this assumption, but we need to make some minor changes to our approach. However, there is also a practical purpose. In a context where $D$ is union status, and $D=1$ denotes unionized individuals, Assumption (ref) requires that we increase the unionization rate by unionizing previously nonunionized workers. It would probably be hard to simultaneously unionize and deunionize different workers.
Another way to look at the monotonicity assumption is by inspecting the joint distribution of $D$ and $D_\delta$ it induces:
In other words, Assumption (ref) rules out the presence of newly untreated individuals. Also, in the limit, when $\delta=0$, we return to the original distribution of individuals. We will evaluate the effect of a counterfactual policy with two parameters: the global and the marginal effects. Let $F_Y^{-1}(\tau)$ and $F_{Y_{D_\delta}}^{-1}(\tau)$ denote the $\tau$-quantiles of $Y$ and $Y_{D_\delta}$ respectively.
The global effect $G_{\tau, D_\delta}$ is the comparison of quantiles of the counterfactual distribution vs. the observed distribution for a fixed policy $D_\delta$. Naturally, for a collection $\mathcal D$, we have a corresponding collection on global effects. The marginal effect $M_{\tau,\mathcal D}$ can be interpreted as an ordinary derivative: for small $\delta$, it provides an approximation to the direction of the change in a given $\tau$-quantile. The main text will focus on the global effect, while the marginal effect is treated in detail in Appendix (ref).
The next task is to define who are the newly treated individuals, that is, how does $D_\delta$ determine who receives treatment among the individuals whose $D=0$? In this paper we will focus on two types of policies: a policy that simply chooses individuals whose $D=0$ at random and assigns them to $D_\delta=1$, and a policy that chooses individuals based on a user-specified criterion. We will refer to these two types of policies as randomized policy and non-randomized policy respectively.
For a collection of policies $\mathcal D$ that satisfies Assumptions (ref), the counterfactual distribution $F_{Y_{D_\delta}}(y)$ can be decomposed as
for each $D_\delta$. Here, $F_{Y(1)|D=0,D_\delta=1}(y)$ corresponds to the distribution of the newly treated. This distribution cannot be identified from the data because it requires observing $Y(1)$ for a subpopulation for which we only observe their $Y(0)$. Consequently, $F_{Y_{D_\delta}}(y)$ is not identified either. The goal is to bound the quantiles of $F_{Y_{D_\delta}}(y)$. To that end, we make the following regularity assumptions.
The next assumption is our main working assumption to obtain the bounds on the quantiles of $F_{Y_{D_\delta}}(y)$.
We refer to the left hand side of (ref) as the policy selection bias. The idea is that in the absence of policy selection bias, we can take $c=0$ and $F_{Y(1)|D=0,D_\delta=1}(y)$ can be recovered by matching already unionized individuals with newly unionized individuals and integrating against the characteristics of the newly unionized individuals, using $F_{X|D=0,D_\delta=1}(x)$. This is akin to a joint unconfoundedness assumption: $D\perp Y(1)\| X$ and $D_\delta\perp Y(1)\| X$.\footnote{If $D, D_\delta\perp Y(1)\| X$, then $F_{Y|D=1, X=x}(y) = F_{Y(1)|D=0,D_\delta = 1, X=x}(y)$, so that integrating against $F_{X|D=0,D_\delta=1}(x)$ yields $F_{Y(1)|D=0,D_\delta=1}(y) $.} By allowing $c$ to be non-zero, we are relaxing this particular type of conditional independence assumption (Masten2018). Here, Assumption (ref).(ref) becomes relevant. Kline2013 also use the Kolmogorov-Smirnov distance to bound the quantiles of the outcome to allow for the possibility that data might not be missing at random.
The quantile breakdown frontier is a curve that allows to perform to a sensitivity analysis with respect to $c$, the policy selection bias. Suppose we are interested in a target conclusion $G_{\tau,D_\delta}\geq g_{\tau,L} $ for some $g_{\tau,L}$. If the conclusion does not hold when $c=0$, then there is no point in performing a sensitivity analysis. On the other hand, if the conclusion does hold under $c=0$, we would like to know the maximum amount of $c$ such that the conclusion continues to hold. This is the sensitivity analysis. The quantile breakdown frontier for the global effect tackles both of these issues. The frontier is the map
for $\tau\in(\delta,1-\delta)$.\footnote{We can also look at conclusions of the type $G_{\tau,D_\delta}\leq g_{\tau,U}$. In this case, the quantile breakdown frontier is the map
} An explanation of the derivation is in section (ref) of the appendix. When $c_{\tau,L}<0$, the desired target conclusion does not hold. If $c_{\tau,L}>0,$ then for $c\leq c_{\tau,L}$ then target conclusion holds under point identification. If $c> c_{\tau,L}>0$, then the target conclusion might not hold. In this sense, when $c_{\tau,L}>0,$ the frontier provides an amount of policy selection bias which is compatible with the conclusion.
The next lemma contains some properties of the quantile breakdown frontier.
Part $(i)$ of the lemma puts a restriction on the types of conclusion by requiring continuity of the family of target conclusions. This is not essential, but illustrates the fact that the smoothness of the frontier depends, among other things, on the conclusions. For notational simplicity, later on we assume $g_{\tau,L}\equiv g$. Part $(iv)$ addresses a potential “conservadurism” in the breakdown analysis. This stems from the fact there is a possibility that the lower bound for the global effect does not depend on $c$. In the notation of Theorem (ref), this means that we cannot rule out $\max\left\{\tilde F_A^{-1}(\tau-\delta) ,F_A^{-1}(\tau-\delta c_{\tau,L})\right\} = \tilde F_A^{-1}(\tau-\delta).$ This is related to part $(ii)$. Indeed, if the opposite is true, $\tilde F_A^{-1}(\tau-\delta)< F_A^{-1}(\tau-\delta c_{\tau,L})$, then we can modify the language of part $(ii)$ to say that for $c> c_{\tau,L}$ the conclusion will not hold.\footnote{In the empirical application we test for $\tilde F_A^{-1}(\tau-\delta)-F_Y^{-1}(\tau) < g_{\tau,L}$.}
It is possible to examine the behavior of $c_{\tau,L}$ with respect to $g_{\tau,L}$. Differentiating (ref), we get, for a fixed $\tau$:
provided $\delta>0$. This means that if the conclusion is less stringent, i.e., $g_{\tau,L}$ is reduced, then the frontier moves upwards, indicating that more policy selection bias can be allowed before the conclusion breaks down. In this sense, $c_{\tau,L}$ provides a minimum amount of policy selection bias for all $g$ such that $g\leq g_{\tau,L}$.
Now consider the derivative of (ref) with respect to $\tau$:
It is not possible to sign the slope of the frontier a priori. It can depend largely on $\tau$, that is, whether we are the center or the tails of the distribution of $Y$, and whether the outcome is bounded or not, among other reasons.
The derivative of (ref) with respect to $\delta$ is more complicated due to the fact that $F_A$ depends on $\delta$. A general expression is the following:
where, following the definition of $F_A ( y)$ given in (ref), we get (heuristically)
Suppose that a policy maker choose a particular quantile $\tau^*$ and a target conclusion $g_{\tau^*,L}$. Then, provided that $c_{\tau^*,L}\in [0,1]$, we can obtain bounds on the global effect other quantiles $\tau\neq \tau^*$. The interpretation is that, as long as the policy selection bias satisfies $c\leq c_{\tau^*,L}$, the global effect will be bounded across quantiles by the bounds of Theorem (ref) evaluated at $c_{\tau^*,L}$. In a sense, this allows us to extend the sensitivity analysis to other quantiles. That is for $\tau\in(\delta,1-\delta)$, we have
We work in the space $\ell^{\infty}(\delta,1-\delta)$ of bounded real-valued functions defined on $(\delta,1-\delta)$. As usual, we endow this space with the supremum norm: $\|x\|_\infty:=\sup_{t\in(\delta,1-\delta)}|x(t)|$.\footnote{The reason we restrict the space to be $\ell^{\infty}(\delta,1-\delta)$ and not $\ell^{\infty}(0,1)$ is due to the fact that for a given $\delta$, we cannot reach quantiles below $\delta$ or above $1-\delta$. See Remark (ref) above.} In order to simplify notation, and ensure the continuity of the quantile breakdown frontier, we are going to focus on the case where the threshold is constant across $\tau$.
This assumption can be relaxed at the expense of more complicated notation. However, we still require smoothness in the map $\tau\mapsto g_\tau.$ For the case of the quantile breakdown for the sign of the marginal effect, we will set $g=0$.
To estimate $ c_{\tau,L}$ given in (ref) we need to specify two components: $(i)$ $\delta$, the increase in the treated proportion, and $(ii)$ $D_{i,\delta}$, the counterfactual treatment assignment, as in examples (ref) and (ref). Once this has been done, we use sample analogs. The estimator of the quantile breakdown frontier for the global effect is
where $ \hat F_{Y}^{-1}(\tau)$ is the empirical $\tau$-quantile of $Y$: $\hat F_Y^{-1}(\tau) :=\inf \left\{ y:\hat F_Y(y)\geq \tau \right\}$, and $\hat F_A(y)$ is the empirical counterpart of $F_A(y)$ given in Theorem (ref). Detailed expressions are provided in Appendix (ref). This is similar to a quantile-quantile transformation (see Exercise 4 in Chapter 3.9 in vandervaart1996). We base our proof of the asymptotic distribution of $\sqrt n(\hat c_{\tau,L}-c_{\tau,L})$ on the proof of Lemma A.1 in Beare2019.\footnote{Beare2019 also offer some interesting historical context for the result.} We view the map $\tau\mapsto\hat c_{\tau,L}$ as a random element of $\ell^{\infty}(\delta,1-\delta)$. In that case, we denote it simply by $\hat c_{L}.$
The main assumption is the following.
The following assumption is needed to establish the Hadamard differentiable of different functions used in the construction of $\theta$.
The first item in Assumption (ref) concerns the support $\mathcal Y$ and the smoothness of $F_Y$. It is used to guarantee the Hadamard differentiability of the quantile process $\tau\mapsto F_Y^{-1}(\tau)$ for $\tau\in(\delta,1-\delta)$. The second item ensures that $F_A(y)$ has a uniformly continuous and bounded derivative which we denote by $f_A(y)$. It is needed to establish the Hadamard differentiability of the composition map $(F_A,F_Y^{-1})\mapsto F_A\circ (F_Y^{-1}+g)$.\footnote{Section 3.9 in vandervaart1996 studies the Hadamard differentiability of composition maps.}
We propose a two-step inference procedure to accommodate the results of Lemma (ref). First, we test the following null hypothesis: $ H_{1,0}: \tilde F_A^{-1}(\tau-\delta)-F_Y^{-1}(\tau)< g,$ against the alternative $ H_{1,a}: \tilde F_A^{-1}(\tau-\delta)-F_Y^{-1}(\tau)\geq g.$ If the null $H_{1,0}$ is not rejected, we proceed to test: $H_{2,0}: c_{\tau,L} = \bar c$, against the alternative $ H_{2,a}: c_{\tau,L} \neq \bar c,$ for some user-specified $\bar c$.
Let $\alpha_1$ denote the size of the first test, and let $\alpha_2$ denote the conditional size of the second test. The family-wise error rate (FWER), defined as the probability of making at least one false rejection among all true null hypotheses, is $ \alpha_1 + (1-\alpha_1)\alpha_2$. For example, to keep the FWER at $5\%$, then setting $\alpha_1=0.025$ yields $\alpha_2\le(0.05-\alpha_1)/(1-\alpha_1)\approx 0.02564$.
The first null hypothesis requires estimation of $d_{\tau}:=\tilde F_A^{-1}(\tau-\delta)-F_Y^{-1}(\tau)$. Estimation follows by resorting to the sample analogs: $\hat d_{\tau}=\hat{\tilde F}_A^{-1}(\tau-\delta)-\hat F_Y^{-1}(\tau)$. Details can be found in Appendix (ref).
To estimate the bounds derived the from the QBF for the global effect, we use the sample counterpart of (ref). Besides the target conclusion $g$ for the global effect, we need to supply, a quantile level $\tau^*$ of interest. For $\tau\in(\delta,1-\delta)$, the estimator of the derived bounds is
where
Here, $\hat c_{\tau^*,L}$ is the estimator given in (ref) evaluated $\tau=\tau^*$. The asymptotic distribution of the bounds cannot be Gaussian due to being the composition of maps which are not Hadamard fully differentiable--only directional. Hence, by Corollary 3.1 in Fang2019, the standard bootstrap will fail. This means that if we attempt to construct confidence intervals in the usual way by resampling, we will not obtain correct asymptotic coverage. Instead, we use the numerical delta method of Hong2018. A detailed analysis of the asymptotic distribution of the bounds is in Appendix (ref).
There is an extensive literature that studies unions and inequality. A recent contribution by Farber2020 contains a review of the literature. In our empirical application, in particular, we look at how unions affect the distribution of wages for all workers. Unions can have a variety of effects on the distribution of wages. As argued by Freeman1980, unions can raise the wages of unionized workers relative to non-unionized workers, possibly through more bargaining power. So, if higher paid workers unionize, the dispersion of wages can increase, but if lower paid workers unionize, the dispersion of wages can decrease. Furthermore, within a given industry, the union can reduce the dispersion of wages by standardizing the wages. This will impact the distribution of wages more or less depending on the size of the industry and the wages it pays.
A key difficulty in identifying the causal effect of unions on wages is that selection into unions is non-random. Hence, any measurement of the union premium--the difference in wages between similar union and nonunion workers--will be biased for the causal effect. Indeed, this has been a long standing concern of labor economists.\footnote{Indeed, the opening words of Card1996 are:
} With respect to selection into unions, Card1996 argues that unionized workers with low observed skills, tend to have high unobserved skills. The reverse happens with high skilled unionized workers: they tend to have low unobservable skills. Due to this selection bias, it might be impossible for a policy maker to devise a policy where the newly unionized workers are selected in a way such that they are drawn from the distribution of the already unionized workers.
Using the techniques developed in this paper, we are going to consider the effect of both globally and marginally expanding union coverage. We will explicitly allow for non-random selection into unions. This allows for the newly unionized and already unionized workers to be drawn from different distributions. We do not use any imputation method to impute the union premium of the newly unionized workers.
Following Freeman1980, Card2001 and Card2004 we consider a two sector economy. Each worker has a well-defined pair of potential (log) wages: $Y_i(1)$ for the unionized sector and $Y_i(0)$ for the nonunionized sector. Under Assumption (ref), and for any policy $D_\delta$, we have the following classification of individuals:
The relevant unobserved distribution is then $F_{Y(1)|D=0, D_\delta=1}$: the union wages of the newly unionized workers. Following (ref), we look at departures of $F_{Y(1)|D=0, D_\delta=1}$ from
which is observed. This difference is what we refer to as the policy selection bias.
Using the data in Firpo2009 we estimate the quantile breakdown frontier for marginal and global effects of different type of policies on the distribution of real log hourly wages. We use the 1983-1985 Outgoing Rotation Group (ORG) Supplement of the Current Population Survey. Our sample consists of 266,956 observations on U.S. males, of which $73.8\%$ are non-unionized, and $26.2\%$ are unionized. See Lemieux2006 for more details about the data. The covariates are: years of education, age, marital status, dummy for race (nonwhite), years of experience, and union status indicator. Table (ref) contains sample means by union status, and the difference. All of the differences are significantly different from $0$ at the $5\%$ level, possibly reflecting selection into unions.
The unionization rate in the dataset is $0.26$. Figure (ref) shows the typical hump-shaped pattern of the unionization rates by quantiles of the distribution of wages. For lower quantiles, unionization rates are quite low. They peak in the past the middle of the distribution and then drop at the higher quantiles.
Consider a policy that increase in the unionization rate by $10\%$. It consists of unionizing workers whose wages are below the $.10/(1-p)$-quantile $\approx 0.14$-quantile of the wages of the nonunionized sector.\footnote{This guarantees that the unionization rate increases by roughly 10%. Indeed, the mean of $D_\delta$ is now $0.36$.} In the notation of this paper, we have $D=1$ if a worker is unionized, $D_\delta=1$ if a worker is unionized under the policy, $Y$ is (log) wage, and $\delta=0.1$. That is, $D_\delta$ is given by
Figure (ref) shows a family of quantile breakdown frontiers for $g=0, 0.025, 0.05, 0.075, 0.1$, for a grid of $\tau\in (0.15,0.85)$. The top one corresponds to $g=0$. As $g$ increases, the frontiers shift down. This is in agreement with (ref), which states that, for a fixed $\tau$, the derivative of $c_{\tau,L}$ with respect to $g$ is negative.
Figure (ref) takes a closer look at the quantile breakdown frontier for $g=0.05$. Confidence intervals are obtained via 1,000 bootstrap replications. Since the outcome variable is log wages, this means approximately a $5\%$ increase in wages. First, we can see that for lower quantiles, the conclusion will hold as long as $c$ is below the frontier. On the other hand, for upper quantiles, the conclusion does not hold under point identification. Thus there is no point in doing a sensitivity analysis for upper quantiles.
Suppose the policy maker is interested in $\tau^*=.3$. This is indicated by the vertical dashed red line. The $.3$-quantile of the unconditional distribution of log wages is $1.465$. An increase of $0.05$ would bring this the $.3$-quantile to $1.515$ which is close to the $.33$-quantile which is $1.521$. In this case, $\hat c_{\tau^*,L}=\hat c_{.3,L}=0.223$, and $[0.176,0.270]$ is the approximately $97.5\%$ confidence interval. Importantly, $\hat d_{\tau^*} = \hat d_{.3} = 0.0001$ which is less that $g=0.05$, and, as shown in Figure (ref), it is (uniformly) statistically less that $g=0.05$. This means, by parts $(iii)$ and $(iv)$ of Lemma (ref), that the conclusion only holds for $c\leq 0.223$, and hence the quantile breakdown frontier is sharp at $\tau=.3$. Using equation (ref), we can find the bounds for the global effect for all quantiles which are consistent with this amount of policy selection bias. This is shown in Figure (ref). Note that at $\tau=0.3$, the lower bound is $.05$ by construction. At upper quantiles, the lower bound is slightly negative.
In order to interpret the magnitude of the QBF, we compare it to the observable difference
This is different from the KS-distance in Assumption (ref), which is stated in terms of $F_{Y(1)|D=0,D_\delta=1}(y)$, an unobserved distribution. In (ref), we are comparing the difference between the newly-treated when they are not treated, versus the treated matched with the newly treated covariates. The empirical counterpart of (ref) is $\hat c_{obs}=0.243$, and $(0.231 ,0.256)$ is the $95\%$ confidence interval computed via the usual bootstrap. This is similar to the value of the QBF at $\tau=.3$ which is $\hat c_{.3,L}=0.223$. If the treatment does not have a very large effect on the distribution of the outcome, then one could argue that the conclusion of interest might hold. However, if the treatment might have a very strong effect on the distribution, then it might be less likely that the conclusion holds.
This paper provides a way to perform a sensitivity analysis on the unconditional quantiles effects of policies that manipulate the proportion of treated individuals. To do so, we introduce the quantile breakdown frontier as a tool to examine, across quantiles, whether a sensitivity analysis is possible, and how much policy selection bias is compatible with a given conclusion. Next, we use the information from the curve at a particular quantile to provide bounds on the effect of a policy across quantiles. Our empirical application looks at the effect of increasing the proportion of unionized workers by unionizing lower earners. We find that an increase of $10\%$ for the $30$\textsuperscript{th} quantile is consistent under moderate values of selection bias. Moreover, this implies that the effect on lower quantiles would be even bigger, and that lower quantiles would not be hurt.