Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
88,419 characters · 14 sections · 75 citation commands
Breaking Ties: Regression Discontinuity DesignMeets Market Design
\onehalfspacing
\thispagestyle{empty}
Large school districts increasingly use sophisticated centralized assignment mechanisms to match students and schools. In addition to producing fair and transparent admissions decisions, centralized assignment offers a unique resource for research on schools: the data these systems generate can be used to construct unbiased estimates of school value-added. This research dividend arises from the tie-breaking embedded in centralized assignment. Many school assignment schemes rely on the deferred acceptance (DA) algorithm, which takes as input information on applicant preferences and school priorities. In settings where seats are scarce, DA rations seats at oversubscribed schools using tie-breaking variables, producing quasi-experimental assignment of students to schools.
Many districts break ties with a uniformly distributed random variable, often described as a lottery number. mdrd1:17 show that DA with lottery tie-breaking assigns students to schools as if in a stratified randomized trial. That is, conditional on preferences and priorities, the assignments generated by such systems are randomly assigned and therefore independent of potential outcomes. In practice, however, preferences and priorities, which we call applicant type, are too finely distributed for full non-parametric conditioning to be useful. We must therefore pool applicants of different types, while avoiding any omitted variables bias that might arise from the fact that type predicts outcomes.
The key to type pooling is the DA propensity score, defined as the probability of school assignment conditional on applicant type. In a mechanism with lottery tie-breaking, conditioning on the scalar DA propensity score is sufficient to make school assignment independent of potential outcomes. Moreover, the distribution of the scalar propensity score turns out to be much coarser than the distribution of types.\footnote{The propensity score theorem says that for research designs in which treatment status, $D_i$, is independent of potential outcomes conditional on covariates, $X_i$, treatment status is also independent of potential outcomes conditional on the propensity score, that is, conditional on $E[D_i|X_i]$. In work building on mdrd1:17, the DA propensity score has been used to study schools bergman2018risks, management training abebe2019learning, and entrepreneurship training argentina.}
This paper generalizes the propensity score to DA-based assignment mechanisms in which tie-breaking variables are not limited to randomly assigned lottery numbers. Selective exam schools, for instance, admit students with high test scores, and students with higher scores tend to have better achievement and graduation outcomes regardless of where they enroll. We refer to such scenarios as involving general tie-breaking.\footnote{Non-lottery tie-breaking embedded in centralized assignment schemes has been used in econometric research on schools in Chile hastings/neilson/zimmerman:13, zimmerman2016making, Ghana ajayi:13, Italy fort2016cognitive, Kenya lucas/mbiti:14, Norway kirkeboen/leuven/mogstad:15, Romania pop-eleches/urquiola:13, Trinidad and Tobago jackson:10, jackson:12, beuermann2015privately, and the U.S. abdulkadiroglu/angrist/pathak:14, dobbie/fryer:11, barrow2016role. These studies treat individual schools and tie-breakers in isolation, without exploiting centralized assignment. Related methodological work exploring regression discontinuity designs with multiple assignment variables and multiple cutoffs includes papay2011extending, zajonc2012regression, wong2013analyzing, cattaneo2016interpreting.} Matching markets with general tie-breaking raise challenges beyond those addressed in the mdrd1:17 study of DA with lottery tie-breaking.
The most important complication raised by general tie-breaking arises from the fact that seat assignment is no longer independent of potential outcomes conditional on applicant type. This problem is intimately entwined with the identification challenge raised by regression discontinuity (RD) designs, which typically compare candidates for treatment on either side of a qualifying test score cutoff. In particular, non-lottery tie-breakers play the role of an RD running variable and are likewise a source of omitted variables bias. The setting of interest here, however, is far more complex than the typical RD design: DA may involve many treatments, tie-breakers, and cutoffs.
A further barrier to causal inference comes from the fact that the propensity score in this general tie-breaking setting depends on the unknown distribution of non-lottery tie-breakers conditional on type. Consequently, the propensity score under general tie-breaking may be no coarser than the underlying high-dimensional type distribution. When the score distribution is no coarser than the type distribution, score conditioning is pointless.
These problems are solved here by introducing a local DA propensity score that quantifies the probability of school assignment induced by a combination of non-lottery and lottery tie-breakers. This score is “local” in the sense that it is constructed using the fact that continuously distributed non-lottery tie-breakers are locally uniformly distributed. Combining this property with the (globally) known distribution of lottery tie-breakers yields a formula for the assignment probabilities induced by any DA match. Conditional on the local DA propensity score, school assignments are shown to be asymptotically randomly assigned. Moreover, like the DA propensity score for lottery tie-breaking, the local DA propensity score has a distribution far coarser than the underlying type distribution.
Our analytical approach builds on hahn2001identification and other pioneering econometric contributions to the development of non-parametric RD designs. We also build on the more recent local random assignment interpretation of nonparametric RD.\footnote{See, among others, frolich2007regression, cattaneo2015randomization, cattaneo2017comparing, frandsen2017party, sekhon2017interpreting; frolich/huber2019; and arai2019testing.} The resulting theoretical framework allows us to quantify the probability of school assignment as a function of a few features of student type and tie-breakers, such as proximity to the admissions cutoffs determined by DA and the identity of key cutoffs for each applicant. By integrating nonparametric RD with rosenbaum/rubin:83's propensity score theorem and large-market matching theory, our theoretical results provide a framework suitable for causal inference in a wide variety of applications.
The research value of the local DA propensity score is demonstrated through an analysis of New York City (NYC) high school report cards. Specifically, we ask whether schools distinguished by “Grade A” on the district's school report card indeed signify high quality schools that boost their students' achievement and improve other outcomes. Alternatively, the good performance of most Grade A students may reflect omitted variables bias. The distinction between causal effects and omitted variables bias is especially interesting in light of an ongoing debate over access to New York's academically selective schools, also called screened schools, which are especially likely to be graded A (see, e.g., brody:19 and veiga:18). We identify the causal effects of Grade A school attendance by exploiting the NYC high school match. NYC employs a DA mechanism integrating non-lottery screened school tie-breaking with a common lottery tie-breaker at lottery schools. In fact, NYC screened schools design their own tie-breakers based on middle school transcripts, interviews, and other factors.
The effects of Grade A school attendance are estimated here using instrumental variables constructed from the school assignment offers generated by the NYC high school match. Specifically, our two-stage least squares (2SLS) estimators use assignment offers as instrumental variables for Grade A school attendance, while controlling for the local DA propensity score. The resulting estimates suggest that Grade A attendance boosts SAT math scores modestly and may increase high school graduation rates a little. But these effects are much smaller than those the corresponding ordinary least squares (OLS) estimates of Grade A value-added would suggest.
We also compare 2SLS estimates of Grade A effects computed separately for NYC's screened and lottery schools, a comparison that shows the two sorts of schools to have similar effects. This finding therefore implies that OLS estimates showing a large Grade A screened school advantage are especially misleading. The distinction between screened and lottery schools has been central to the ongoing debate over NYC school access and quality. Our comparison suggests that the public concern with this sort of treatment effect heterogeneity may be misplaced. Treatment effect heterogeneity may be limited, supporting our assumption of constant treatment effects conditional on observables.\footnote{The analysis here allows for treatment effect heterogeneity as a function of observable student and school characteristics. Our working paper shows how DA in markets with general tie-breaking identifies average causal affects for applicants with tie-breaker values away from screened-school cutoffs (abdulkadiroglu/angrist/narita/pathak:19). We leave other questions related to unobserved heterogeneity for future work.}
The next section shows how DA can be used to identify causal effects of school attendance. Section (ref) illustrates key ideas in a setting with a single non-lottery tie-breaker. Section (ref) derives a formula for the local DA propensity score in a market with general tie-breaking. This section also derives a consistent estimator of the local propensity score. Section (ref) uses these theoretical results to estimate causal effects of attending Grade A schools.\footnote{Our theoretical analysis covers any mechanism that can be computed by student-proposing DA. This DA class includes student-proposing DA, serial dictatorship, the immediate acceptance (Boston) mechanism ergin/sonmez:05, China's parallel mechanisms chen/kesten:17, England's first-preference-first mechanisms pathak/sonmez:13, and the Taiwan mechanism dur/pathak/song/sonmez:18. In large markets satisfying regularity conditions that imply a unique stable matching, the relevant DA class also includes school-proposing as well as applicant-proposing DA (these conditions are spelled out in azevedo/leshno:14). The DA class omits the Top Trading Cycles (TTC) mechanism defined for school choice by abdulkadiroglu/sonmez:03.}
The NYC school report cards published from 2007-13 graded high schools on the basis of student achievement, graduation rates, and other criteria. These grades were part of an accountability system meant to help parents choose high quality schools. In practice, however, report card grades computed without extensive control for student characteristics reflect students' ability and family background as well as school quality. Systematic differences in student body composition are a powerful source of bias in school report cards. It's therefore worth asking whether a student who is randomly assigned to a Grade A high school indeed learns more and is more likely to graduate as a result.
We answer this question using instrumental variables derived from NYC's DA-based assignment of high school seats. The NYC high school match generates a single school assignment for each applicant as a function of applicants' preferences over schools, school-specific priorities, and a set of tie-breaking variables that distinguish between applicants who share preferences and priorities.\footnote{Seat assignment at some of NYC's selective enrollment “exam schools” is determined by a separate match. NYC charter schools use school-specific lotteries. Applicants are free to seek exam school and charter school seats as well as an assignment in the traditional sector.} Because they're a function of student characteristics like preferences and test scores, NYC assignments are not randomly assigned. We show, however, that conditional on the local DA propensity score, DA-generated assignments of a seat at school $s$ provide credible instruments for enrollment at school $s$. This result motivates a two-stage least squares (2SLS) specification where the endogenous treatment is enrollment at any Grade A school while the instrument is DA-generated assignment of a seat at any Grade A school.
Our identification strategy builds on the large-market “continuum” model of DA detailed in mdrd1:17. The large-market model is extended here to allow for multiple and non-lottery tie-breakers. To that end, let $s=0,1,...,S$ index schools, where $s=0$ represents an outside option. Applicants are assumed to be identified by an index, $i$, drawn from the unit interval $[0, 1]$. The large market model is “large” by virtue of this assumption.
Applicant $i$'s preferences over schools constitute a strict partial ordering, $\succ_{i}$, where $a\succ_{i}b$ means that $i$ prefers school $a$ to school $b$. Each applicant is also granted a priority at every school. For example, schools may prioritize applicants who live nearby or with currently enrolled siblings. Let $\rho_{is}\in\{1,...,K,\infty\}$ denote applicant $i$'s priority at school $s$, where $\rho_{is}<\rho_{js}$ means school $s$ prioritizes $i$ over $j$. We use $\rho_{is}=\infty$ to indicate that $i$ is ineligible for school $s.$ The vector $\rho_{i}=(\rho_{i1},...,\rho_{iS})$ records applicant $i$'s priorities at each school. Applicant type is then defined as $\theta_{i}=(\succ_{i},\rho_{i})$, that is, the combination of an applicant's preferences and priorities at all schools. Let $\Theta_s$ denote the set of types, $\theta$, that ranks $s$.
In addition to applicant type, DA matches applicants to seats as a function of a set of tie-breaking variables. We leave DA mechanics for Section (ref); at this point, it's enough to establish notation for DA inputs. Most importantly, our analysis of markets with general tie-breaking requires notation to keep track of tie-breakers. Let $v\in\{1,...,V\}$ index tie-breakers and let $S_v$ be the set of schools using tie-breaker $v$. We assume that each school uses a single tie-breaker. Scalar random variable $R_{iv}$ denotes applicant $i$'s tie-breaker $v$. Some of these are uniformly distributed lottery numbers. The set of non-lottery $R_{iv}$ used at schools ranked by applicant $i$ are collected in the vector $\mathcal{R}_i$. Without loss of generality, we assume that ties are broken in favor of applicants with the smaller tie-breaker value. DA uses $\theta_i$, $\mathcal{R}_i$, and the set of lottery tie-breakers for all $i$ to assign applicants to schools.
We are interested in using the assignment variation resulting from DA to estimate the causal effect of $C_{i}$, a variable indicating student $i$'s attendance at (or years of enrollment in) any Grade A school. Outcome variables, denoted $Y_i$, include SAT scores and high school graduation status. In a DA match like the one in NYC, $C_i$ is not randomly assigned, but rather reflects student preferences, school priorities, tie-breaking variables, as well as decisions whether or not to enroll at school $s$ when offered a seat there through the match. The potential for omitted variables bias induced by the process determining $C_i$ can be eliminated by an instrumental variables strategy that exploits our understanding of the structure of matching markets.
The instruments used for this purpose are a function of individual school assignments, indicated by $D_i(s)$ for the assignment of student $i$ to a seat at school $s$. Because DA generates a single assignment for each student, a dummy for any Grade A assignment, denoted $D_{Ai}$, is the sum of dummies indicating all assignments to individual Grade A schools. $D_{Ai}$ provides a natural instrument for $C_i$. In particular, we show below that 2SLS consistently estimates the effect of $C_i$ on $Y_i$ in the context of a linear constant-effects causal model that can be written as:
where $\beta$ is the causal effect of interest and the associated first stage equation is
The terms $f_1(\theta_i, \mathcal{R}_i, \delta)$ and $f_2(\theta_i, \mathcal{R}_i, \delta)$ are functions of type and non-lottery tie-breakers, as well as a bandwidth, $\delta\in \mathbb{R}$, that's integral to the local DA propensity score.
Our goal is to specify $f_1(\theta_i, \mathcal{R}_i, \delta)$ and $f_2(\theta_i, \mathcal{R}_i, \delta)$ so that 2SLS estimates of $\beta$ are consistent. Because (ref) is seen as a model for potential outcomes rather than a regression equation, consistency requires that $D_{Ai}$ and $\eta_i$ be uncorrelated. The relevant identification assumption can be written:
where $\approx$ means asymptotic equality as $\delta\rightarrow0$, in a manner detailed below. Briefly, our main theoretical result establishes limiting local conditional mean independence of school assignments from applicant characteristics and potential outcomes, yielding (ref). This result specifies $f_1(\theta_i, \mathcal{R}_i, \delta)$ and $f_2(\theta_i, \mathcal{R}_i, \delta)$ to be easily-computed functions of the local propensity score and elements of $\mathcal{R}_i$.
mdrd1:17 derive the relevant DA propensity score for a scenario with lottery tie-breaking only. Lottery tie-breaking obviates the need for a bandwidth and control for components of $\mathcal{R}_i$. Many applications of DA use non-lottery tie-breaking, however. The task here is to derive the propensity score for elaborate matches like that in NYC, which combines lottery tie-breaking with many school-specific non-lottery tie-breakers. The resulting estimation strategy integrates propensity score methods with the nonparametric approach to RD (introduced by hahn2001identification), and the local random assignment model of RD (discussed by frolich2007regression, cattaneo2015randomization, cattaneo2017comparing, frandsen2017party, among others). Our theoretical results can also be seen as generalizing nonparametric RD to allow for many schools (treatments), many tie-breakers (running variables), and many cutoffs.
An analysis of a market with a single non-lottery tie-breaker and no priorities illuminates key elements of our approach. DA in this case is known as serial dictatorship. Like the general local DA score, the local DA score for serial dictatorship depends only on a handful of statistics, including admissions cutoffs for schools ranked, and whether applicant $i$'s tie-breaker is close to cutoffs for schools using non-lottery tie-breakers. Conditional on this local propensity score, school offers are asymptotically randomly assigned.
Serial dictatorship can be described as follows:
Seating is constrained by a capacity vector, $q=(q_0,q_1,q_2,...,q_S)$, where $q_{s}\in[0, 1]$ is defined as the proportion of the unit interval that can be seated at school $s$. We assume $q_0=1$. Serial dictatorship is used in Boston and New York City to allocate seats at selective public exam schools.
Because serial dictatorship relies on a single tie-breaker, notation for the set of non-lottery tie-breakers, $\mathcal{R}_i$, can be replaced by a scalar, $R_{i}$. As in mdrd1:17, tie-breakers for individuals are modelled as stochastic, meaning they are drawn from a distribution for each applicant. Although $R_{i}$ is not necessarily uniform, we assume that it's distributed with positive density over $[0, 1]$, with continuously differentiable cumulative distribution function, $F^i_R$. These common support and smoothness assumptions notwithstanding, tie-breakers may be correlated with type, so that $R_{i}$ and $R_{j}$ for applicants $i$ and $j$ are not necessarily identically distributed, though they're assumed to be independent of one another. The probability that type $\theta$ applicants have a tie-breaker below any value $r$ is $F_R(r| \theta) \equiv E[F^i_R(r)| \theta_i=\theta]$, where $F^i_R(r)$ is $F^i_R$ evaluated at $r$.
The serial dictatorship allocation is characterized by a set of tie-breaker cutoffs, denoted $\tau_s$ for school $s$. For any school $s$ that's filled to capacity, $\tau_s$ is given by the tie-breaker of the last (highest tie-breaker value) student assigned to $s$. Otherwise, $\tau_s=1$, a non-binding cutoff reflecting excess capacity. We say an applicant qualifies at $s$ when they have a tie-breaker value that clears $\tau_s$. Under serial dictatorship, students are assigned to $s$ if and only if they:
In large markets, cutoffs are constant, so stochastic variation in seat assignments arises solely from the distribution of tie-breakers.
Which cutoffs matter? Under serial dictatorship, the assignment probability faced by an applicant of type $\theta$ at school $s$ is determined by the cutoff at $s$ and by cutoffs at schools preferred to $s$. By virtue of single tie-breaking, it's enough to know only one of the latter. In particular, an applicant who fails to clear the highest cutoff among those at schools preferred to $s$ surely fails to do better than $s$. This leads us to define most informative disqualification (MID), a scalar parameter for each applicant type and school. MID tells us how the tie-breaker distribution among type $\theta$ applicants to $s$ is truncated by disqualification at the schools type $\theta$ applicants prefer to $s$.
Because MID for type $\theta$ at school $s$ is defined with reference to the set of schools $\theta$ prefers to $s$, we define:
the set of schools type $\theta$ prefers to $s$. For each type and school, $MID_{\theta s}$ is a function of tie-breaker cutoffs at schools in $B_{\theta s}$, specifically:
$MID_{\theta s}$ is zero when school $s$ is ranked first since all who rank $s$ first compete for a seat there. The second line reflects the fact that an applicant who ranks $s$ second is seated there only when disqualified at the school they've ranked first, while applicants who rank $s$ third are seated there when disqualified at their first and second choices, and so on. Moreover, anyone who fails to clear cutoff $\tau_{b}$ is surely disqualified at schools with less forgiving cutoffs. For example, applicants who fail to qualify at a school with a cutoff of 0.6 are disqualified at a school with cutoff 0.4.
Note that an applicant of type $\theta$ cannot be seated at $s$ when $MID_{\theta s} > \tau_s$. This is the scenario sketched in the top panel of Figure (ref), which illustrates the forces determining SD assignment rates. On the other hand, assignment rates when $MID_{\theta s} \leq \tau_s$ are given by the probability that: $$MID_{\theta s} < R_i \leq \tau_s,$$ an event described in the middle panel of Figure (ref). These facts are collected in the following proposition, which is implied by a more general result for DA proved in the online appendix.
Proposition (ref) says that the serial dictatorship assignment probability, positive only when the tie-breaker cutoff at $s$ exceeds $MID_{\theta s}$, is given by the size of the group with $R_i$ between $MID_{\theta s}$ and $\tau_s$. This is $$F_R(\tau_{s}| \theta)-F_R(MID_{\theta s}| \theta).$$ With a uniformly distributed lottery number, the serial dictatorship propensity score simplifies to $\tau_s - MID_{\theta s}$, a scenario noted in Figure (ref). In this case, the assignment probability for each applicant is determined by $\tau_s$ and $MID_{\theta s}$ alone. Given these two cutoffs, seats at $s$ are randomly assigned.
With non-lottery tie-breaking, the serial dictatorship propensity score depends on the conditional distribution function, $F_R(\cdot| \theta)$ evaluated at $\tau_s$ and $MID_{\theta s}$, rather than the cutoffs themselves. This dependence leaves us with two econometric challenges. First, $F_R(\cdot| \theta)$ is unknown. This precludes computation of the propensity score by repeatedly sampling from $F_R(\cdot| \theta)$. Second, $F_R(\cdot| \theta)$, is likely to depend on $\theta$, so the score in Proposition (ref) need not have coarser support than does $\theta$. This is in spite of the fact many applicants with different values of $\theta$ share the same $MID_{\theta s}$. Finally, although controlling for $p_{s}(\theta)$ eliminates confounding from type, assignments are a function of tie-breakers as well as type. Confounding from non-lottery tie-breakers remains even after conditioning on $p_{s}(\theta)$.
These challenges are met here by focusing on assignment probabilities for applicants with tie-breaker realizations close to key cutoffs. Specifically, for each $\tau_s$, define an interval, $(\tau_s - \delta, \tau_s + \delta]$, where parameter $\delta$ is a bandwidth analogous to that used for nonparametric RD estimation. A local propensity score treats the qualification status of applicants inside this interval as randomly assigned. This assumption is justified by the fact that, given continuous differentiability of tie-breaker distributions, non-lottery tie-breakers have a limiting uniform distribution as the bandwidth shrinks to zero.
\setcounter{figure}{0}
The following Proposition uses this fact to characterize the local serial dictatorship propensity score:
This follows from a more general result for DA presented in the next section.
Proposition (ref) describes a key conditional independence result: the limiting local probability of seat assignment in serial dictatorship takes on only three values and is unrelated to applicant characteristics. Note that the cases enumerated in the proposition (when $\tau_s > MID_{\theta s}$) partition the tie-breaker line as sketched in Figure (ref). Applicants with tie-breaker values above the cutoff at $s$ are disqualified at $s$ and so cannot be seated there, while applicants with tie-breaker values below $MID_{\theta s}$ are qualified at a school they prefer to $s$ and so will be seated elsewhere. Applicants with tie-breakers strictly between $MID_{\theta s}$ and $\tau_s$ are surely assigned to $s$. Finally, type $\theta$ applicants with tie-breakers near either $MID_{\theta s}$ or the cutoff at $s$ are seated with probability approximately equal to one-half. Nearness in this case means inside the interval defined by bandwidth $\delta$.
The driving force behind Proposition (ref) is the assumption that the tie-breaker distribution is continuously differentiable. In a shrinking window, the tie-breaker density therefore approaches that of a uniform distribution, so the limiting qualification rate is one-half (See mdrd2:17 or bugni2018testing for formal proof of this claim). The assumption of a continuously differentiable tie-breaker distribution is analogous to the continuous running variable assumption invoked in lee2008randomized and to a local smoothness assumption in dong2018alternative. Continuity of tie-breaker distributions implies a weaker smoothness condition asserting continuity at cutoffs of the conditional expectation functions of potential outcomes given running variables. We favor the stronger continuity assumption because the implied local random assignment provides a scaffold for construction of assignment probabilities in more complicated matching scenarios.\footnote{The connection between continuity of running variable distributions and conditional expectation functions is noted by dong2018alternative and arai2019testing. Antecedents for the local random assignment idea include an unpublished appendix to frolich2007regression and an unpublished draft of frandsen2017party, which shows something similar for an asymmetric bandwidth. See also cattaneo2015randomization and frolich/huber2019.}
Many school districts assign seats using a version of student-proposing DA, which can be described like this:
Different schools may use different tie-breakers. For example, the NYC high school match includes a diverse set of screened schools apr:05,abdulkadiroglu/pathak/roth:09. These schools order applicants using school-specific tie-breakers that are derived from interviews, auditions, or GPA in earlier grades, as well as test scores. The NYC match also includes many unscreened schools, referred to here as lottery schools, that use a uniformly distributed lottery number as tie-breaker. Lottery numbers are distributed independently of type and potential outcomes, but non-lottery tie-breakers like entrance exam scores almost certainly depend on these variables.
We adopt the convention that tie-breaker indices are ordered such that lottery tie-breakers come first. That is, $v\in\{1,...,U\}$, where $U\leq V$, indexes $U$ lottery tie-breakers. Each lottery tie-breaker, $R_{iv}$ for $v=\{1,...U\}$, is uniformly distributed over $[0,1]$. Non-lottery tie-breakers are indexed by $v \in \{U+1,...,V\}$. The assumptions employed with general tie-breaking are summarized as follows:
Let $v(s)$ be a function that returns the index of the tie-breaker used at school $s$. By definition, $s\in S_{v(s)}$. To combine applicants' priority status and tie-breaking variables into a single number for each school, we define applicant position at school $s$ as: \[ \pi_{is} = \rho_{is}+R_{iv(s)}. \] Since the difference between any two priorities is at least 1 and tie-breaking variables are between 0 and 1, applicant order by position at $s$ is lexicographic, first by priority then by tie-breaker. As noted in the discussion of serial dictatorship, we distinguish between tie-breakers and priorities because the latter are fixed, while the former are random variables.
We also generalize cutoffs to incorporate priorities; these DA cutoffs are denoted $\xi_s$. For any school $s$ that ends up filled to capacity, $\xi_s$ is given by $\max_i\{\pi_{is}| D_i(s)=1\}$. Otherwise, we set $\xi_s=K+1$ to indicate that $s$ has slack (recall that $K$ is the lowest possible priority).
DA assigns a seat at school $s$ to any applicant $i$ ranking $s$ who has
This is a consequence of the fact that the student-proposing DA is stable.\footnote{In particular, if an applicant is seated at $s$ but prefers $b$, she must be qualified at $s$ and not have been assigned to $b$. Moreover, since DA-generated assignments at $b$ are made in order of position, applicants not assigned to $b$ must be disqualified there.} In large markets, $\xi_s$ is fixed as tie breakers are drawn and re-drawn. DA-induced school assignment rates are therefore determined by the distribution of stochastic tie-breakers evaluated at fixed school cutoffs. Condition ((ref)) nests our characterization of seat assignment under serial dictatorship since we can set $\rho_{is}=0$ for all applicants and use a single tie-breaker to determine position. Statement ((ref)) then says that $R_i\leq\tau_s$ and $R_{i}>MID_{\theta s}$ for applicants with $\theta_i=\theta$.
The DA propensity score is the probability of the event described by (ref). This probability is determined in part by marginal priority at school $s$, denoted $\rho_s$ and defined as $\text{int}(\xi_s)$, the integer part of the DA cutoff. Conditional on rejection by all preferred schools, applicants to $s$ are assigned $s$ with certainty if $\rho_{is}<\rho_{s}$, that is, if they clear marginal priority. Applicants with $\rho_{is}>\rho_{s}$ have no chance of finding a seat at $s$. Applicants for whom $\rho_{is} = \rho_{s}$ are marginal: these applicants are seated at $s$ when their tie-breaker values fall below tie-breaker cutoff $\tau_{s}$. This quantity can therefore be written as the decimal part of the DA cutoff: $$\tau_s=\xi_{s}-\rho_{s}.$$ Applicants with marginal priority have $\rho_{is} = \rho_{s}$, so $\pi_{is} \leq \xi_{s} \Leftrightarrow R_{iv(s)} \leq \tau_s.$
In addition to marginal priority, the local DA propensity score is conditioned on applicant position relative to screened school cutoffs. To describe this conditioning, define a set of variables, $t_{is}(\delta)$, as follows:
where the mnemonic value labels $n,a,c$ stand for never seated, always seated, and conditionally seated. It's convenient to collect these variables in a vector, \[ T_{i}(\delta)=[t_{i 1}(\delta), ..., t_{i s}(\delta), ..., t_{i S}(\delta)]. \]
Elements of $T_i(\delta)$ for unscreened schools are a function only of the partition of types determined by marginal priority. For screened schools, however, $T_i(\delta)$ also encodes the relationship between tie-breakers and cutoffs. Never-seated applicants to $s$ cannot be seated there, either because they fail to clear marginal priority at $s$ or because they're too far above the cutoff when $s$ is screened. Always-seated applicants to $s$ are assigned $s$ for sure when they can't do better, either because they clear marginal priority at $s$ or because they're well below the cutoff at $s$ when $s$ is screened. Finally, conditionally-seated applicants to $s$ are randomized marginal priority applicants. Randomization is by lottery number when $s$ is a lottery school or by non-lottery tie-breaker within the bandwidth when $s$ is screened.
With this machinery in hand, the local DA propensity score is defined as follows: \[ \psi_s(\theta,T)=\lim_{\delta\rightarrow 0} E[D_i(s)|\theta_{i}=\theta,T_i(\delta)=T], \] for $T=[t_{1}, ..., t_{s}, ..., t_{S}]$ where $t_{s}\in\{n,a,c\}$ for each $s$. This describes assignment probabilities as a function of type and cutoff proximity at each school. As in Proposition (ref), formal characterization of $\psi_s(\theta,T)$ requires cutoffs be distinct:
The formula characterizing $\psi_s(\theta,T)$ builds on an extension of the $MID$ idea to a general tie-breaking regime. First, the set of schools $\theta$ prefers to $s$, $B_{\theta s}$, is partitioned by tie-breakers by defining $B^{v}_{\theta s}\equiv\{b \in S_v \hspace*{.1cm} | \hspace*{.1cm} b \succ_{\theta}s\}$ for each $v$. We then have:
$MID^{v}_{\theta s}$ quantifies the extent to which qualification at schools using tie-breaker $v(s)$ and that type $\theta$ applicants prefer to $s$ truncates the tie-breaker distribution among those contending for seats at s.
Next, define: \[ m_{s}(\theta,T) = |\{v>U: MID^{v}_{\theta s}=\tau_b \text{ and } t_b=c \text{ for some } b\in B^{v}_{\theta s} \}|. \] This quantity counts the number of RD-style experiments created by the screened schools that type $\theta$ prefers to $s$.
The last preliminary to a formulation of local DA assignment scores uses $MID^{v}_{\theta s}$ and $m_{s}(\theta,T)$ to compute disqualification rates at all schools preferred to $s$. We break this into two pieces: variation generated by screened schools and variation generated by lottery schools. As the bandwidth shrinks, the limiting disqualification probability at screened schools in $B_{\theta s}$ converges to
The disqualification probability at lottery schools in $B_{\theta s}$ is
without regard to bandwidth.
To recap: the local DA score for type $\theta$ applicants is determined in part by the screened schools $\theta$ prefers to $s$. Relevant screened schools are those determining $MID^{v}_{\theta s}$, and at which applicants are close to tie-breaker cutoffs. The variable $m_{s}(\theta,T)$ counts the number of tie-breakers involved in such close encounters. Applicants drawing screened school tie-breakers close to $\tau_b$ for some $b\in B^{v}_{\theta s}$ face qualification rates of $0.5$ for each tie-breaker $v$. Since screened school disqualification is locally independent over tie-breakers, the term $\sigma_s(\theta,T)$ computes the probability of not being assigned a screened school preferred to $s$. Likewise, since the qualification rate at preferred lottery schools is $MID^{v}_{\theta s}$, the term $\lambda_s(\theta)$ computes the probability of not being assigned a lottery school preferred to $s$.
The following theorem combines these in a formula for the local DA propensity score:
Theorem (ref) starts with a scenario where applicants to $s$ are either disqualified there or assigned to a preferred school for sure.\footnote{See the appendix for proof of the Theorem, along with other theoretical results, including derivation of a non-limit form of the DA propensity score.} In this case, we need not worry about whether $s$ is a screened or lottery school. In other scenarios where applicants are surely qualified at $s$, the probability of assignment to $s$ is determined entirely by disqualification rates at preferred screened schools and by truncation of lottery tie-breaker distributions at preferred lottery schools. These sources of assignment risk combine to produce the first line of (ref). The conditional assignment probability at any lottery $s$, described on the second line of (ref), is determined by the disqualification rate at preferred schools and the qualification rate at $s$, where the latter is given by $\tau_{s}-MID^{v(s)}_{\theta s}$ (to see this, note that $\lambda_s(\theta)$ includes the term $1-MID^{v(s)}$ in the product over lottery tie-breakers). Similarly, the conditional assignment probability at any screened $s$, on the third line of (ref), is determined by the disqualification rate at preferred schools and the qualification rate at $s$, where the latter is given by $0.5$.
The Theorem covers the non-lottery tie-breaking serial dictatorship scenario in the previous section. With a single non-lottery tie-breaker, $\lambda_s(\theta)=1$. When $t_s=n$ or $t_b=a$ for some $b \in B_{\theta s}$, the local propensity score at $s$ is zero. Otherwise, suppose $t_b=n$ for all $b \in B_{\theta s}$, so that $m_s(\theta,T)=0$. If $t_s=a$, then the local propensity score is $1$. If $t_s=c$, then the local propensity score is $0.5$. Suppose, instead, that $MID_{\theta s} = \tau_b$ for some $b \in B_{\theta s}$, so that $m_s(\theta,T)=1$. In this case, $t_s\neq c$ because cutoffs are distinct. If $t_s=a$, then the local propensity score is $0.5$. Online Appendix (ref) uses an example to illustrate the Theorem in other scenarios.
Theorem (ref) characterizes the theoretical probability of school assignment in a large market with a continuum of applicants. In reality, of course, the number of applicants is finite and propensity scores must be estimated. We show here that, in an asymptotic sequence that increases market size with a shrinking bandwidth, a sample analog of the local DA score described by Theorem (ref) converges uniformly to the corresponding local score for a finite market. Our empirical application establishes the relevance of this asymptotic result by showing that applicant characteristics are balanced by assignment status conditional on estimates of the local DA propensity score.
The asymptotic sequence for the estimated local DA score works as follows: randomly sample $N$ applicants from a continuum economy. The applicant sample (of size $N$) includes information on each applicant's type and the vector of large-market school capacities, $q_s$, which give the proportion of $N$ seats that can be seated at $s$. We observe realized tie-breaker values for each applicant, but not the underlying distribution of non-lottery tie-breakers. The set of finitely many schools is unchanged along this sequence.
Fix the number of seats at school $s$ in a sampled finite market to be the integer part of $Nq_s$ and run DA with these applicants and schools. We consider the limiting behavior of an estimator computed using the estimated $\hat{MID}^v_{\theta_i s}$, $\hat \tau_s$, and marginal priorities generated by this single realization. Also, given a bandwidth $\delta_N>0$, we compute $t_{is}(\delta_N)$ for each $i$ and $s$, collecting these in vector $T_i(\delta_N)$. These statistics then determine: $$\hat{m}_{s}(\theta_i, T_i(\delta_N))=|\{v>U: \hat{MID}^{v}_{\theta_i s}= \hat \tau_b \text{ and } t_{ib}(\delta_N)=c \text{ for some } b\in B^{v}_{\theta_i s} \}|.$$
Our local DA score estimator, denoted $\hat \psi_{s}(\theta_i,T_i(\delta_N))$, is constructed by plugging these ingredients into the formula in Theorem (ref). That is, if (a) $\hat t_{is}(\delta_N)=n$; or (b) $\hat t_{ib}(\delta_N)=a \ \text{ for some } b \in B_{\theta_i s}$, then $\hat \psi_{s}(\theta_i,T_i(\delta_N))=0.$ Otherwise,
where $$\hat \sigma_s(\theta_i,T_i(\delta_N)) = 0.5^{\hat{m}_{s}(\theta_i, T_i(\delta_N))} $$ and $$\hat \lambda_s(\theta_i)=\prod_{v=1}^{U}(1-\hat{MID}^{v}_{\theta_i s}).$$
As a theoretical benchmark for the large-sample performance of $\hat \psi_{s}$, consider the true local DA score for a finite market of size $N$. This is
where $E_N$ is the expectation induced by the joint tie-breaker distribution for applicants in the finite market. This quantity is defined by fixing the distribution of types and the vector of proportional school capacities, as well as market size. $\psi_{Ns}(\theta, T)$ is then the limit of the average of $D_i(s)$ across infinitely many tie-breaker draws in ever-narrowing bandwidths for this finite market. Because tie-breaker distributions are assumed to have continuous density in the neighborhood of any cutoff, the finite-market local propensity score is well-defined for any positive $\delta$.
We're interested in the gap between the estimator $\hat \psi_{s}(\theta,T(\delta_N))$ and the true local score $\psi_{Ns}(\theta, T)$ as $N$ grows and $\delta_N$ shrinks. We show below that $\hat \psi_{s}(\theta,T(\delta_N))$ converges uniformly to $\psi_{Ns}(\theta, T)$ in our asymptotic sequence.
This result uses a regularity condition:
Uniform convergence of $\hat \psi_{s}(\theta,T(\delta_N))$ is formalized below:
This result (proved in the online appendix) justifies conditioning on an estimated local propensity score to eliminate omitted variables bias in school attendance effect estimates.
Theorems (ref) and (ref) provide a foundation for causal inference. In combination with an exclusion restriction discussed below, these results imply that a dummy variable indicating Grade A assignments is asymptotically independent of potential outcomes (represented by the residuals in a equation (ref)), conditional on an estimate of the Grade A local propensity score. Let $S_A$ denote the set of Grade A schools. Because DA generates a single offer, the local propensity score for Grade A assignment can be computed as: $$\hat \psi_{A}(\theta_i,T_i(\delta_N))=\sum_{s\in S_A} \hat \psi_{s}(\theta_i,T_i(\delta_N)).$$ In other words, the local score for Grade A assignment is the sum of the scores for all Grade A schools in the match.
These considerations lead to a 2SLS estimator with second and first stage equations that can be written in stylized form as:
where $d_i(x)=1\{\hat \psi_{A}(\theta_i,T_i(\delta_N))=x\}$ and the set of parameters denoted $\alpha_2(x)$ and $\alpha_1(x)$ provide saturated control for the local propensity score. As detailed in the next section, functions $g_2(\mathcal{R}_i; \delta_N)$ and $g_1(\mathcal{R}_i; \delta_N)$ implement local linear control for screened school tie-breakers for applicants to these schools with $\hat{t}_{is}(\delta_N)=c$. Linking this with the empirical strategy sketched at the outset, equation (ref) is a version of of equation (ref) that sets $$f_2(\theta_i, \mathcal{R}_i, \delta) = \sum_{x} \alpha_{2}(x)d_{i}(x) + g_2(\mathcal{R}_i; \delta_N).$$ Likewise, equation (ref) is a version of equation (ref) with $f_1(\theta_i, \mathcal{R}_i, \delta)$ defined similarly.
Our implementation of score-controlled instrumental variables is inspired by the calonico_et_al:2019_RD analysis of RD designs with covariates. Using a mix of simulation evidence and theoretical reasoning, calonico_et_al:2019_RD argues that additive control for covariates in a local linear regression model requires fewer assumptions and is likely to have better finite-sample behavior than more elaborate procedures. The covariates of interest to us are a full set of dummies for values in the support of the Grade A local propensity score. We'd like to control for these while also benefiting from the good performance of local linear regression estimators of conditional mean functions near cutoffs.\footnote{calonico_et_al:2019_RD discuss both sharp and fuzzy RD designs. The conclusions for sharp design carry over to the fuzzy case in which cutoff clearance is used as an instrument. Equations (ref) and (ref) are said to be stylized because they omit a number of implementation details supplied in the following section.}
Note that saturated regression-conditioning on the local propensity score eliminates applicants with score values of zero or one. This is apparent from an analogy with a fixed-effects panel model. In panel data with multiple annual observations on individuals, estimation with individual fixed effects is equivalent to estimation after subtracting person means from regressors. Here, the “fixed effects” are coefficients on dummies for each possible score value. When the score value is 0 or 1 for applicants of a given type, assignment status is constant and observations on applicants of this type drop out. We therefore say an applicant has Grade A risk when $\hat \psi_{A}(\theta_i,T_i(\delta_N)) \in (0, 1)$. The sample with risk contributes to parameter estimation in models with saturated score control.
Propensity score conditioning facilitates control for applicant type in the sample with risk. In practice, local propensity score conditioning yields considerable dimension reduction compared to full-type conditioning, as we would hope. The 2014 NYC high school match, for example, involved 52,124 applicants of 47,074 distinct types. Of these, 42,461 types listed a Grade A school on their application to the high school match. By contrast, the local propensity score for Grade A school assignment takes on only 2,054 values.
Since the 2003-04 school year, the NYC Department of Education (DOE) has used DA to assign rising ninth graders to high schools. Many high schools in the match host multiple programs, each with their own admissions protocols. Applicants are matched to programs rather than schools. Each applicant for a ninth grade seat can rank up to twelve programs. All traditional public high schools participate in the match, but charter schools and NYC's specialized exam high schools have separate admissions procedures.\footnote{Some special needs students are also matched separately. The centralized NYC high school match is detailed in apr:05,abdulkadiroglu/pathak/roth:09. abdulkadiroglu/angrist/pathak:14 describe NYC exam school admissions.}
The NYC match is structured like the general DA match described in Section (ref): lottery programs use a common uniformly distributed lottery number, while screened programs use a variety of non-lottery tie-breaking variables. Screened tie-breakers are mostly distinct, with one for each school or program, though some screened programs share a tie-breaker. In any case, our theoretical framework accommodates all of NYC's many tie-breaking protocols.\footnote{Screened tie-breakers are reported as an integer variable encoding the underlying tie-breaker order such as a test score or portfolio summary score. We scale these so as to lie in $(0, 1]$ by computing $[R_{iv}-\min_j{R_{jv}}+1]/[\max_jR_{jv}-\min_jR_{jv}+1]$ for each tie-breaker $v$. This transformation produces a positive cutoff at $s$ when only one applicant is seated at $s$ and a cutoff of 1 when all applicants who rank $s$ are seated there.}
Our analysis uses Theorems (ref) and (ref) to compute propensity scores for programs rather than schools since programs are the unit of assignment. For our purposes, a lottery school is a school hosting any lottery program. Other schools are defined as screened.\footnote{Some NYC high schools sort applicants on a coarse screening tie-breaker that allows ties, breaking these ties using the common lottery number. Schools of this type are treated as lottery schools, with priority groups defined by values of the screened tie-breaker. Seats for NYC's ed-opt programs are allocated to two groups, one of which screens applicants using a single non-lottery tie-breaker and the other using the common lottery number. The online appendix explains how ed-opt programs are handled by our analysis.}
In 2007, the NYC DOE launched a school accountability system that graded schools from A to F. This mirrors similar accountability systems in Florida and other states. NYC's school grades were determined by achievement levels and, especially, achievement growth, as well as by survey- and attendance-based features of the school environment. Growth looked at credit accumulation, Regents test completion and pass rates; performance measures were derived mostly from four- and six-year graduation rates. Some schools were ungraded. Figure (ref) reproduces a sample letter-graded school progress report.\footnote{walcott:12 details the NYC grading methodology used in this period. Note that the computation of the grade of a school for a particular year uses only information from past years, so that there is no feedback between school grades and the school's current outcomes.}
The 2007 grading system was controversial. Proponents applauded the integration of multiple measures of school quality while opponents objected to the high-stakes consequences of low school grades, such as school closure or consolidation. rockoff/turner:10 provide a partial validation of the system by showing that low grades seem to have sparked school improvement. In 2014, the DOE replaced the 2007 scheme with school quality measures that place less weight on test scores and more on curriculum characteristics and subjective assessments of teaching quality. The relative merits of the old and new systems continue to be debated.
The results reported here use application data from the 2011-12, 2012-13, and 2013-14 school years (students in these application cohorts enrolled in the following school years). Our sample includes first-time applicants seeking 9th grade seats, who submitted preferences over programs in the main round of the NYC high school match. We obtained data on school capacities and priorities, lottery numbers, and screened school tie-breakers, information that allows us to replicate the match. Details related to match replication appear in the online appendix.\footnote{Our analysis assigns report card grades to a cohort's schools based on the report cards published in the previous year. For the 2011/12 application cohort, for instance, we used the grades published in 2010/11. On the other hand, applicant SAT scores from tests taken before 9th grade are dropped.}
Students at Grade A schools have higher average SAT scores and higher graduation rates than do students at other schools. Differences in graduation rates across schools feature in popular accounts of socioeconomic differences in school access (see, e.g., harris/fessenden:17 and disare:17). Grade A students are also more likely than students attending other schools to be deemed “college- and career-prepared” or “college-ready.”\footnote{These composite variables are determined as a function of Regents and AP scores, course grades, vocational or arts certification, and college admission tests.} These and other school characteristics are documented in Table (ref), which reports statistics separately by school grade and admissions regime. Achievement gaps between screened and lottery Grade A schools are especially large, likely reflecting selection bias induced by test-based screening.
Screened Grade A schools have a majority white and Asian student body, the only group of schools described in the table to do so (the table reports shares black and Hispanic). These schools are also over-represented in Manhattan, a borough that includes most of New York's wealthiest neighborhoods (though average family income is higher on Staten Island). Teacher experience is similar across school types, while screened Grade A schools have somewhat more teachers with advanced degrees.
The first two columns of Table (ref) describe the roughly 180,000 ninth graders enrolled in the 2012-13, 2013-14, and 2014-15 school years. Students enrolled in a Grade A school, including those enrolled in the Grade A schools assigned outside the match, are less likely to be black or Hispanic and have higher baseline scores than the general population of 9th graders. The 153,000 eighth graders who applied for ninth grade seats are described in column 3 of the table. Roughly 130,000 listed a Grade A school for which seats are assigned in the match on their application form and a little over a third of these were assigned to a Grade A school.\footnote{The difference between total 9th grade enrollment and the number of match participants is accounted for by special education students outside the main match, direct-to-charter enrollment, and a few schools that straddle 9th grade.} Applicants in the match have baseline scores (from tests taken in 6th grade) above the overall district mean (baseline scores are standardized to the population of test-takers). As can be seen by comparing columns 3 and 4 in Table (ref), however, the average characteristics of Grade A applicants are mostly similar to those of the entire applicant population.
The statistics in column 5 of Table (ref) show that applicants enrolled in a Grade A school (among schools participating in the match) are somewhat less likely to be black and have higher baseline scores than the total applicant pool. These gaps likely reflect systematic differences in offer rates by race at screened Grade A schools. Column 5 of Table (ref) also shows that most of those attending a Grade A school were assigned there, and that most Grade A students ranked a Grade A school first. Grade A students are about twice as likely to go to a lottery school as to a screened school. Interestingly, enthusiasm for Grade A schools is far from universal: just under half of all applicants in the match ranked a Grade A school first.
Because NYC has a single lottery tie-breaker, the disqualification probability at lottery schools in $B_{\theta s}$ described by equation (ref) simplifies to $$\lambda_s(\theta)=(1-MID^{1}_{\theta s}),$$ where $MID^{1}_{\theta s}$ is most informative disqualification at schools using the common lottery tie-breaker, $R_{1i}$. The local DA score described by equation (ref) therefore also simplifies, in this case to:
Estimates of the local DA score based on (ref) reveal that roughly 35,000 applicants have Grade A risk, that is, an estimated local DA score value strictly between 0 and 1. As can be seen in column 6 of Table (ref), applicants with Grade A risk have mean baseline scores and demographic characteristics much like those of the sample enrolled at a Grade A school. The ratio of screened to lottery enrollment among those with Grade A risk is also similar to the corresponding ratio in the sample of enrolled students (compare 32.9/15.3 in the former group to 66.3/25.0 in the latter). Online Appendix Figure (ref) plots the distribution of Grade A assignment probabilities for applicants with risk. The modal probability is $0.5$, reflecting the fact that roughly 25% of those with Grade A risk rank a single Grade A school and that this school is screened.
The balancing property of local propensity score conditioning is evaluated using score-controlled differences in covariate means for applicants who do and don't receive Grade A assignments. Score-controlled differences by Grade A assignment status are estimated in a model that includes a dummy indicating assignments at ungraded schools as well as a dummy for Grade A assignments, controlling for the propensity scores for both. We account for ungraded school attendance to ensure that estimated Grade A effects compare schools with high and low grades, omitting the ungraded.\footnote{Ungraded schools were mostly new when grades were assigned or had data insufficient to determine a grade.} Specifically, let $D_{Ai}$ denote Grade A assignments as before, and let $D_{0i}$ indicate assignments at ungraded schools. Assignment risk for each type of school is controlled using sets of dummies denoted $d_{Ai}(x)$ and $d_{0i}(x)$, respectively, for score values indexed by $x$.
The covariates of interest here, denoted by $W_i$, are those that are unchanged by school assignment and should therefore be mean-independent of $D_{Ai}$ in the absence of selection bias. The balance test results reported in Table (ref) are estimates of parameter $\gamma_A$ in regressions of $W_i$ on $D_{Ai}$ of the form:
Local piecewise linear control for screened tie-breakers is parameterized as:
where $S\backslash S_0$ is the set of screened programs, $a_{is}$ indicates whether applicant $i$ applied to screened program $s$, and $k_{is} = 1[\hat{t}_{is}(\delta_N)=c]$. The sample used to estimate (ref) is limited to applicants with Grade A risk.
Parameters in (ref) and (ref) vary by application cohort (three cohorts are stacked in the estimation sample). Bandwidths are estimated two ways, as suggested by imbens/kalyanaraman:12 (IK) using a uniform kernel, and using methods and software described in rdrobust2017 (CCFT). These bandwidths are computed separately for each program (the notation ignores this), for the set of applicants in the relevant marginal priority group.\footnote{The IK bandwidths used here are identical to those yielded by the IK implementation referenced in armstrong2018optimal and distributed via the \href{https://github.com/kolesarm/RDHonest}{RDhonest} package. Bandwidths are computed separately for each outcome variable; we use the smallest of these for each program. The bandwidth for screened programs is set to zero when there are fewer than five in-bandwidth observations on one or the other side of the relevant cutoff. The control function $g(\mathcal{R}_i; \delta_N)$ is unweighted and can therefore be said to use a uniform kernel. We also explored bandwidths designed to produce balance as in locrand2016. These results proved to be sensitive to implementation details such as the p-value used to establish balance.}
As can be seen in column 2 of Table (ref), which reports raw differences in means by Grade A assignment status, applicants assigned to a Grade A school are much more likely to have ranked a Grade A school first, and ranked more Grade A schools highly than did other applicants. These applicants are also more likely to rank a Screened Grade A school first and among their top three. Minority and free-lunch-eligible applicants are less likely to be assigned to a Grade A school, while those assigned to a Grade A school have much higher baselines scores, with gaps of $0.3-0.4$ in favor of those assigned. These raw differences notwithstanding, our theoretical results suggest that estimates of $\gamma_A$ in equation (ref) should be close to zero.
This is borne out by the estimates reported in column 4 of the table, which shows small, mostly insignificant differences in covariates by assignment status when estimated using using imbens/kalyanaraman:12 bandwidths. The estimated covariate gaps in column 6, computed using rdrobust2017 bandwidths, are similar. These estimates establish the empirical relevance of both the large-market model of DA and the local DA propensity score derived from it.\footnote{Our balance assessment relies on linear models to estimate mean differences rather than comparisons of distributions. The focus on means is justified because the IV reduced form relationships we aspire to validate are themselves regressions. Recall that in a regression context, reduced form causal effects are unbiased provided omitted variables are mean-independent of the instrument, $D_{Ai}$. Since treatment variable $D_{Ai}$ is a dummy, the regression of omitted control variables on it is given by the difference in conditional control variable means computed with $D_{Ai}$ switched on and off.}
Causal effects of Grade A attendance are estimated by 2SLS using assignment dummies as instruments for years of exposure to schools of a particular type, as suggested by equations (ref) and (ref). As in the setup used to establish covariate balance, however, the 2SLS estimating equations include two endogenous variables, $C_{Ai}$ for Grade A exposure and $C_{0i}$ measuring exposure to an ungraded school. Exposure is measured in years for SAT outcomes; otherwise, $C_{Ai}$ and $C_{0i}$ are enrollment dummies. As in equation (ref), local propensity score controls consist of saturated models for Grade A and ungraded propensity scores, with local linear control for screened tie-breakers as described by equation (ref). These equations also control for baseline math and English scores, free lunch, special education, and English language learner dummies, and gender and race dummies (estimates without these controls are similar, though less precise).\footnote{Replacing $W_i$ on the left hand side of (ref) with outcome variable $Y_i$, equations (ref) and (ref) describe the reduced form for our 2SLS estimator. In an application with lottery tie-breaking, mdrd1:17 compare score-controlled 2SLS estimates with semiparametric instrumental variables estimates based on abadie:03. The former are considerably more precise than the latter.}
OLS estimates of Grade A effects, reported as a benchmark in the second column of Table (ref), indicate that Grade A attendance is associated with higher SAT scores and graduation rates, as well as increased college and career readiness. The OLS estimates in Table (ref) are from models that omit local propensity score controls, computed in a sample that includes all participants in the high school match without regard to assignment probability. OLS estimates of the SAT gains associated with Grade A enrollment are around 6-7 points. Estimated graduation gains are similarly modest at 2.4 points, but effects on college and career readiness are substantial, running 7-10 points on a base rate around 40.
The first stage effects of Grade A assignments on Grade A enrollment, shown in columns 4 and 6 of Panel A in Table (ref), show that Grade A offers boost Grade A enrollment by about 1.8 years between the time of application and SAT test-taking. Grade A assignments boost the likelihood of any Grade A enrollment by about 67 percentage points. This can be compared with Grade A enrollment rates of 16-19 percent among those not assigned a Grade A seat in the match.\footnote{The gap between assignment and enrollment arises from several sources. Applicants remaining in the public system may attend charter or non-match exam schools. Applicants may also reject a match-based assignment, turning instead to an ad hoc administrative assignment process later in the year.}
In contrast with the OLS estimates in column 2, the 2SLS estimates shown in columns 4 and 6 of Table (ref) suggest that most of the SAT gains associated with Grade A attendance reflect selection bias. Computed with either bandwidth, 2SLS estimates of SAT math gains are around 2 points, though still (marginally) significant. 2SLS estimates of SAT reading effects are even smaller and not significantly different from zero, though estimated with similar precision. At the same time, the 2SLS estimate for graduation status shows a statistically significant gain of 3-4 percentage points, exceeding the corresponding OLS estimate. The estimated standard error of $0.009$ associated with the graduation estimate in column 4 seems especially noteworthy, as this suggests that our research design has the power to uncover even modest improvements in high school completion rates.\footnote{Estimates reported in Online Appendix Table (ref) show little difference in follow-up rates between applicants who are and aren't offered a Grade A seat. The 2SLS estimates in Table (ref) are therefore unlikely to be compromised by differential attrition.}
The strongest Grade A effects appear in estimates of effects on college and career preparedness and college readiness. This may in part reflect the fact that Grade A schools are especially likely to offer advanced courses, the availability of which contributes to the college- and career-related composite outcome variables (the online appendix details the construction of these variables). 2SLS estimates of effects on these outcomes are mostly close to the corresponding OLS estimates (three out of four are smaller). Here too, switching bandwidth matters little for magnitudes. Throughout Table (ref), however, 2SLS estimates computed with an IK bandwidth are more precise than those computed using CCFT.
In New York, education policy discussions often focus on access to academically selective screened schools such as Townsend Harris in Queens, a school consistently ranked among the top American high schools by U.S. News and World Report. Public interest in screened schools motivates an analysis that distinguishes screened from lottery Grade A effects. The possibility of different effects within the Grade A sector also raises concerns related to the exclusion restriction underpinning a causal interpretation of 2SLS estimates. In the context of our causal model of Grade A effects, the exclusion restriction fails when the offer of a Grade A seat moves applicants between schools of different quality within the Grade A sector. We therefore explore multi-sector models that distinguish causal effects of attendance at different sorts of Grade A schools, focusing on differences by admissions regime since this is widely believed to matter for school quality.
The multi-sector estimates reported in Table (ref) are from models that include separate endogenous variables for screened and lottery Grade A schools, along with a third endogenous variable for the ungraded sector. Instruments in this just-identified set-up are two dummies indicating each sort of Grade A offer, as well as a dummy indicating the offer of a seat at an ungraded school. 2SLS models include separate saturated local propensity score controls for screened Grade A offer risk, unscreened Grade A offer risk, and ungraded offer risk. These multi-sector estimates are computed in a sample limited to applicants at risk of assignment to either a screened or lottery Grade A school. In view of the relative precision of estimates using IK bandwidth, multi-sector estimates using CCFT bandwidths are omitted.
OLS estimates again provide an interesting benchmark. As can be seen in the first two columns of Table (ref), screened Grade A students appear to reap a large SAT advantage even after controlling for baseline achievement and other covariates. In particular, OLS estimates of Grade A effects for schools in the screened sector are on the order of 14-18 points. At the same time, Grade A lottery schools appear to generate achievement gains of only about 2 points. Yet the corresponding 2SLS estimates, reported in columns 3 and 4 of the table, suggest the achievement gains yielded by enrollment in both sorts of Grade A schools are equally modest. The 2SLS estimates here run less than 2 points for math scores, with smaller (not significant) negative estimates for reading. The sole statistically significant SAT effect is that for the lottery Grade A school impact on math scores.
The remaining 2SLS estimates in the table likewise show similar screened-school and lottery-school effects. With one marginal exception, p-values in the table reveal estimates for the two sectors to be statistically indistinguishable. As in Table (ref), the 2SLS estimates in Table (ref) suggest that screened and lottery Grade A schools boost graduation rates by about 3 points. Effects on college and career preparedness are larger for lottery schools than for screened, but this impact ordering is reversed for effects on college readiness. On the whole, Table (ref) leads us to conclude that OLS estimates showing a large screened Grade A advantage are driven by selection bias.
Centralized student assignment opens new opportunities for the measurement of school quality. The research potential of matching markets is enhanced here by marrying the conditional random assignment generated by lottery tie-breaking with RD-style variation at screened schools. The key to this intermingled empirical framework is a local propensity score that controls for differential assignment rates in DA matches with general tie-breakers. This new tool allows us to exploit all sources of quasi-experimental variation arising from any mechanism in the DA class.
Our analysis of NYC school report cards suggests Grade A schools boost SAT math scores and high school graduation rates by a few points. OLS estimates, by contrast, show considerably larger effects of Grade A attendance on test scores. Grade A screened schools enroll some of the city's highest achievers, but large OLS estimates of achievement gains from attendance at these schools appear to be an artifact of selection bias. Concerns about access to such schools (expressed, for example, in harris/fessenden:17) may therefore be overblown. On the other hand, Grade A attendance increases measures of college and career preparedness. These results may reflect the greater availability of advanced courses in Grade A schools, a feature that should be replicable at other schools.
In principle, Grade A assignments may act to move applicants between schools within the Grade A sector as well as to boost overall Grade A enrollment. Offer-induced movement between screened and lottery Grade A schools may violate the exclusion restriction that underpins our 2SLS results if schools within the Grade A sector vary in quality. It's therefore worth asking whether screened and lottery schools should indeed be treated as having the same effect. Perhaps surprisingly, our analysis supports the idea that screened and lottery Grade A schools can be pooled and treated as having a common average causal effect.
Our provisional agenda for further research prioritizes an investigation of econometric implementation strategies for DA-founded research designs. This work is likely to build on the asymptotic framework in bugni2018testing and the study of RD designs with multiple tie-breakers in papay2011extending, zajonc2012regression, WSC:2013_RD and Cattaneo_et_al:2019:RD_multi. It may be possible to extend the reasoning behind doubly robust nonparametric estimators, such as discussed by rothe2019properties and rothe2020flexible, to our setting.
Statistical inference in Section (ref) relies on conventional large sample reasoning of the sort widely applied in empirical RD applications. It seems natural to consider permutation or randomization inference along the lines suggested by cattaneo2015randomization, cattaneo2017comparing, and canay2017approximate, along with optimal inference and estimation strategies such as those introduced by armstrong2018optimal and imbens2017optimized. Also on the agenda, narita2017non suggests a path toward generalization of the large-market model of DA assignment risk. Finally, we look forward to a more detailed investigation of the consequences of heterogeneous treatment effects for identification strategies of the sort considered here.
\onehalfspacing