The exact contents of citations.db main_text.text for this paper — one flattened LaTeX string, title through conclusion, appendix excluded, unmodified except for removing email addresses. This is what our citation measures are computed over.
36,213 characters
Heterogeneous Policy Effects in Comparative Case Studies with Treated-Unit Microdata
\pagenumbering{gobble}
\maketitle
\thispagestyle{empty}
\begin{center}
\begin{minipage}{0.80\textwidth}
\begin{abstract}
Policy reforms are sometimes accompanied by detailed individual-level data in the implementing jurisdiction, while only aggregate outcomes are available for potential comparison jurisdictions. This article develops an identification framework for heterogeneous policy effects when individual-level data are unavailable for the comparison units. The framework combines treatment-effect contrasts from difference-in-differences comparisons within the treated jurisdiction with a compatible population-average effect identified from aggregate panel data. Identification requires relative parallel trends within the treated jurisdiction together with the assumptions needed to identify the population-average effect from the aggregate panel. The within-jurisdiction component can be estimated from repeated cross-sections with a single pretreatment period.
\end{abstract}
\vspace{0.65em}
\small\noindent
\textbf{Keywords:}
causal inference;
treatment effect heterogeneity;
comparative case studies;
difference-in-differences;
synthetic control
\end{minipage}
\end{center}
\clearpage
\pagenumbering{arabic}
\singlespacing
\section{Introduction}
Major policy reforms can prompt dedicated data collection in the implementing jurisdiction. Following the 2006 Massachusetts health reform, for example, the Massachusetts Health Reform Survey collected detailed information from nonelderly adults on insurance coverage, access to care, and affordability \citep{long2008road}. Evaluation of the Stockholm congestion-charging trial likewise drew on detailed travel surveys and administrative data \citep{eliasson2009stockholm}, while evaluation of minimum unit pricing in Scotland included repeated surveys alongside administrative and other population data \citep{katikireddi2019mup}. Such data can also contain income, age, and other characteristics relevant to treatment-effect heterogeneity. When potential comparison jurisdictions are observed only in aggregate, these characteristics may be unavailable or measured much more coarsely outside the jurisdiction implementing the policy.
Systematic variation in policy effects can matter for targeting and subsequent policy design \citep{manski2004statistical,athey2021policy}. Methods that estimate effect heterogeneity by observed characteristics generally require comparable individual-level information on those characteristics in treated and comparison populations. Recent difference-in-differences methods, for example, use repeated cross-sections from both populations to estimate heterogeneous effects \citep{deb2024flexible,wooldridge2026nonlinear}. Related methods use richer individual-level outcome information from treated and comparison units: changes-in-changes uses individual-level outcome distributions in treated and comparison populations \citep{athey2006identification}, while extensions of synthetic controls use individual observations \citep[e.g.,][]{robbins2017framework,abadie2021penalized} or outcome distributions \citep{gunsilius2023distributional} from the comparison units.
In this paper, I develop an identification framework for heterogeneous treatment effects when individual-level data are available only in the treated jurisdiction. Within that jurisdiction, differences in pre--post outcome changes across values of an observed characteristic can identify differences in treatment effects under a relative parallel-trends assumption, as in subgroup and factorial difference-in-differences designs \citep{shahn2023subgroup,shahn2024group,xu2026factorial}. These comparisons determine how treatment effects vary around their population average, but not the average itself. An aggregate comparative-case design can provide the missing anchor by identifying the population-average effect from aggregate panel data using synthetic-control, difference-in-differences, or related causal panel estimators that accommodate a single treated unit \citep[e.g.,][]{abadie2010synthetic,xu2017generalized,arkhangelsky2021synthetic,ben2021augmented,athey2021matrix}. Combining the two components then identifies treatment effects at each value of the characteristic without requiring individual-level data from the comparison jurisdictions. In an empirical application to the Flint water crisis, \citet{trejo2024flint} use a closely related construction with a synthetic-control counterfactual when subgroup characteristics are unavailable for the comparison units. The framework developed here formalizes the identifying conditions for this data structure and characterizes how the result changes when observed covariates predict untreated trends or treatment effects.
Section~\ref{sec:setup} develops the identification argument, including the roles of covariate adjustment, standardization, and pretreatment information. Section~\ref{sec:estimation} discusses estimation and inference. Section~\ref{sec:numerical} examines finite-sample behavior, and Section~\ref{sec:massachusetts} applies the framework to the Massachusetts health reform and compares the results with estimates using individual-level data from the comparison states. Section~\ref{sec:discussion} concludes.
\section{Setup and identification}
\label{sec:setup}
Consider a policy adopted in one jurisdiction. Aggregate outcomes are observed for that jurisdiction and a set of untreated comparison jurisdictions, while individual-level repeated cross-sections are observed within the treated jurisdiction before and after treatment. I use \emph{aggregate comparative-case design} to refer to a design based on the aggregate panel that identifies the population-average policy effect using the treated jurisdiction and the comparison jurisdictions. The repeated cross-sections within the treated jurisdiction are used to recover how effects vary across population groups.
\subsection{A two-group illustration}
\label{subsec:simple-illustration}
To illustrate the basic argument, suppose the policy effect is of interest for two population groups, indexed by $g\in\{0,1\}$, both of which are exposed to the policy after its introduction. Let $\Delta_g$ denote the observed pre--post change and $\tau_g$ the treatment effect for group $g$. If the untreated outcome would have changed by the same amount $\delta$ in both groups, then
\[
\Delta_0=\delta+\tau_0,
\qquad
\Delta_1=\delta+\tau_1,
\qquad\text{so that}\qquad
\Delta_1-\Delta_0=\tau_1-\tau_0.
\]
The difference-in-differences comparison therefore identifies the difference between the two treatment effects, but not either effect separately.
Let $p$ denote the population share of group 1. If a comparative-case analysis identifies the population-average effect $\bar\tau=(1-p)\tau_0+p\tau_1$, and $\bar\Delta=(1-p)\Delta_0+p\Delta_1$ denotes the corresponding population-average observed change, the average and the contrast determine both group effects:
\[
\tau_g=\bar\tau+\Delta_g-\bar\Delta,
\qquad g\in\{0,1\}.
\]
\subsection{Identification from within-jurisdiction outcome changes}
\label{subsec:relative-identification}
I now formalize this argument for a general observed characteristic in repeated cross-sections. Let $t\in\{0,1\}$ index the two repeated cross-sections and let $a\in\{0,1\}$ denote policy exposure, with $a=0$ corresponding to no policy and $a=1$ to exposure to the policy. The observed outcomes satisfy $Y_0=Y_0(0)$ before treatment and $Y_1=Y_1(1)$ after treatment.\footnote{This notation incorporates consistency of the observed outcome with the realized policy state and no anticipation, so that subsequent policy adoption does not affect the pretreatment potential outcome.}
Let $Z$ denote a predetermined characteristic, or vector of characteristics, along which treatment-effect heterogeneity is of interest. Because different individuals may be observed in the two periods, let $\mathbb{E}_t$ denote expectation in the population represented by the period-$t$ cross-section and write $\mu_t^a(z)=\mathbb{E}_t[Y_t(a)\mid Z=z]$. Assumption~\ref{ass:relative-parallel-trends} concerns changes in these period-specific population means rather than longitudinal changes for the same individuals.\footnote{The same argument can be applied when individual-level panel data are available, but I focus on repeated cross-sections because the framework does not require the same individuals to be followed over time.}
The post-treatment effect and the observed pre--post change at $Z=z$ are
\[
\tau(z)=\mu_1^1(z)-\mu_1^0(z),
\qquad
\Delta(z)=\mu_1^1(z)-\mu_0^0(z).
\]
The basic identification argument assumes a common untreated change across values of $Z$.
\begin{assumption}[Relative parallel trends]
\label{ass:relative-parallel-trends}
For every $z$ in the common support of $Z$ in the period-$0$ and period-$1$ populations,
\begin{equation}
\mu_1^0(z)-\mu_0^0(z)=\delta
\label{eq:relative-parallel-trends}
\end{equation}
for an unrestricted scalar $\delta$.
\end{assumption}
Assumption~\ref{ass:relative-parallel-trends} imposes parallel untreated changes across values of $Z$ within the treated jurisdiction. In a conventional two-group difference-in-differences comparison, one group remains untreated. Here all subgroups receive the policy, so differences in their observed changes identify differences in treatment effects rather than an effect relative to an untreated subgroup.\footnote{For discrete $Z$, the corresponding subgroup parallel-trends condition is studied by \citet{shahn2023subgroup} and \citet{shahn2024group}; related identification arguments for treatment-effect heterogeneity are also developed by \citet{xu2026factorial}.} Untreated outcome levels may differ across groups, while $\delta$ collects period changes that would affect all groups equally in the absence of the policy. Under the assumption, $\Delta(z)=\delta+\tau(z)$.
Let $F_Z^\star$ denote the distribution of $Z$ in the target population, with support contained in the common support of $Z$ in the period-$0$ and period-$1$ populations, and define
\begin{equation}
\bar\tau
=
\int\tau(z)\,dF_Z^\star(z).
\label{eq:average-effect}
\end{equation}
The corresponding population-average observed change is $\bar\Delta=\int\Delta(z)\,dF_Z^\star(z)$. Substituting $\Delta(u)=\delta+\tau(u)$ gives
\begin{align}
\Delta(z)-\bar\Delta
&=
\{\delta+\tau(z)\}
-
\int\{\delta+\tau(u)\}\,dF_Z^\star(u)
\nonumber\\
&=
\delta+\tau(z)-\delta-\bar\tau
\nonumber\\
&=
\tau(z)-\bar\tau.
\label{eq:identified-heterogeneity}
\end{align}
Thus the within-jurisdiction comparison identifies the treatment-effect contrast $h(z)=\tau(z)-\bar\tau$, but leaves $\bar\tau$ undetermined.
Recovering effect levels additionally requires identification of the population-average effect for the same causal target.
\begin{assumption}[Aggregate identification and compatibility]
\label{ass:estimand-compatibility}
The aggregate comparative-case design identifies $\bar\tau$ in \cref{eq:average-effect} for the same outcome, policy contrast, post-treatment period, and target population as the repeated-cross-sectional analysis.
\end{assumption}
In practice, the aggregate and within-jurisdiction analyses must use the same outcome scale and represent the same target population; for example, a population-average effect for the full population cannot anchor subgroup effects defined only among nonelderly adults. Compatibility also concerns the post-treatment period represented by each analysis. If the aggregate comparative-case estimator averages effects over several post-treatment periods while the repeated cross-sections represent a particular period, the two generally target different effects unless treatment effects are approximately constant over time.
\begin{proposition}[Identification of heterogeneous effects]
\label{prop:anchored-identification}
Suppose that \cref{ass:relative-parallel-trends,ass:estimand-compatibility} hold. Then $\tau(z)$ is identified throughout the common support of $Z$ in the period-$0$ and period-$1$ populations, with
\begin{equation}
\tau(z)
=
\bar\tau+\Delta(z)-\bar\Delta.
\label{eq:anchored-effect}
\end{equation}
\end{proposition}
\noindent\emph{Proof.} In Appendix~\ref{app:proofs}.
\subsection{Covariate adjustment and standardization}
\label{subsec:covariate-adjustment}
Relative parallel trends across $Z$ may fail when the groups being compared differ in characteristics that predict untreated outcome changes. For example, if age predicts secular changes in an outcome and age composition differs across income groups, untreated outcome changes may differ across those groups.
A covariate $X$ unaffected by the policy can matter for two distinct reasons. First, if $X$ predicts untreated outcome changes and its distribution differs across $Z$, failing to condition on $X$ can violate relative parallel trends. Second, treatment effects may also vary with $X$, in which case the effect associated with $Z=z$ depends on the distribution of $X$ over which it is averaged. These roles need not coincide. If $X$ affects untreated changes but treatment effects depend only on $Z$, conditioning on $X$ addresses the identifying assumption without changing the subgroup-effect target. When treatment effects also vary with $X$, standardization to a common $X$ distribution instead defines a treatment-effect comparison that holds covariate composition fixed.
Write $\mu_t^a(z,x)=\mathbb{E}_t[Y_t(a)\mid Z=z,X=x]$ and $\Delta(z,x)=\mu_1^1(z,x)-\mu_0^0(z,x)$. The conditional version of relative parallel trends is
\begin{equation}
\mu_1^0(z,x)-\mu_0^0(z,x)=m(x),
\label{eq:conditional-relative-parallel-trends}
\end{equation}
where $m$ is unrestricted. Counterfactual changes may therefore vary with $X$, but not additionally with $Z$ after conditioning on $X$. A sufficient overlap condition is that, for every $z$ in the common support of $Z$, the support of $X$ in the target population is contained in the support of $X\mid Z=z$ in both period-specific populations, with positive probability or density as appropriate.
If treatment effects depend on $Z$ but not further on $X$, then $\Delta(z,x)=m(x)+\tau(z)$. Under the overlap condition above, centering the $Z$-specific component and combining it with $\bar\tau$ identifies $\tau(z)$. Because the treatment effect does not vary further with $X$, this is also the treatment effect averaged over the $X$ distribution represented within subgroup $z$.
When treatment effects also vary with $X$, write $\tau(z,x)=\mu_1^1(z,x)-\mu_1^0(z,x)$. Let $F_{X,Z}^\star$ denote the target joint distribution of $(X,Z)$ and $F_X^\star$ its marginal distribution of $X$. The effect for $Z=z$ standardized to this common distribution is
\[
\tau^S(z)
=
\int\tau(z,x)\,dF_X^\star(x),
\]
with corresponding standardized observed change
\[
\Delta^S(z)
=
\int\Delta(z,x)\,dF_X^\star(x).
\]
The actual target-population averages remain
\[
\bar\Delta
=
\int\Delta(z,x)\,dF_{X,Z}^\star(x,z),
\qquad
\bar\tau
=
\int\tau(z,x)\,dF_{X,Z}^\star(x,z).
\]
Under \cref{eq:conditional-relative-parallel-trends},
\[
\Delta^S(z)
=
\int m(x)\,dF_X^\star(x)+\tau^S(z),
\qquad
\bar\Delta
=
\int m(x)\,dF_X^\star(x)+\bar\tau,
\]
where the second equality follows because $F_X^\star$ is the $X$-marginal of $F_{X,Z}^\star$. The same marginal average of $m(X)$ therefore cancels when the two quantities are differenced, so no separate aggregate estimate for a standardized population is needed. The same anchoring argument as in Proposition~\ref{prop:anchored-identification} then gives the following result.
\begin{proposition}[Identification of standardized heterogeneous effects]
\label{prop:standardized-effects}
Suppose that \cref{eq:conditional-relative-parallel-trends} holds. Assume also that, for every $z$ in the common support of $Z$ in the period-$0$ and period-$1$ populations, the support of $F_X^\star$ is contained in the support of $X\mid Z=z$ in both periods. If the comparative-case design identifies $\bar\tau$ for the target population, then
\begin{equation}
\tau^S(z)
=
\bar\tau+\Delta^S(z)-\bar\Delta.
\label{eq:standardized-anchored-effect}
\end{equation}
\end{proposition}
\noindent\emph{Proof.} In Appendix~\ref{app:proofs}.
Unlike the treatment-effect contrast $h(z)$ in \cref{eq:identified-heterogeneity}, $\tau^S(z)-\bar\tau$ need not average to zero over the marginal distribution of $Z$. Each $\tau^S(z)$ averages over the same marginal distribution $F_X^\star$, whereas $\bar\tau$ averages over the observed joint distribution $F_{X,Z}^\star$.
The standardized effect also differs from the treatment effect averaged over the covariate distribution actually observed within subgroup $z$,
\begin{equation}
\tau^J(z)
=
\int\tau(z,x)\,dF_{X\mid Z=z}^\star(x).
\label{eq:subgroup-average-effect}
\end{equation}
For example, if $Z$ denotes income and $X$ age, $\tau^S(z)$ compares income groups at a common age distribution, whereas $\tau^J(z)$ averages over the age distribution represented within each income group.
The two-period data do not generally identify $\tau^J(z)$ when $X$ affects both untreated changes and treatment effects. If $\Delta^J(z)=\int\Delta(z,x)\,dF_{X\mid Z=z}^\star(x)$ denotes the observed change averaged over subgroup $z$'s own covariate distribution, then
\[
\Delta^J(z)-\tau^J(z)
=
\int m(x)\,dF_{X\mid Z=z}^\star(x).
\]
The aggregate comparative-case design determines only the population average $\int m(x)\,dF_X^\star(x)=\bar\Delta-\bar\tau$, not the corresponding average within each subgroup. Recovering $\tau^J(z)$ therefore requires additional information about how untreated changes vary with $X$.
\subsection{Using pretreatment trends}
\label{subsec:pretreatment-trends}
Additional pretreatment waves provide information about relative changes before treatment. Under an assumption linking those changes to the treatment transition, they can also support adjustment for differential untreated trends and recovery of subgroup-average effects.
For an untreated transition ending in period $s\leq0$, let $r_s(z)$ denote the change at $Z=z$ minus the population-average change over the same transition. Pretreatment estimates of $r_s(z)$ therefore show how relative trends evolved before treatment.
Let $r_1(z)$ denote the corresponding counterfactual differential trend over the treatment transition. Under an assumption that determines $r_1(z)$ from its pretreatment history,
\begin{equation}
\tau(z)-\bar\tau
=
\Delta(z)-\bar\Delta-r_1(z).
\label{eq:trend-adjusted-heterogeneity}
\end{equation}
For equally spaced periods, the restriction $r_1(z)=r_0(z)$ carries the most recent differential pretrend into the treatment transition, as in sequential difference-in-differences arguments \citep{egami2023multiple}. Alternatively, pretreatment differences can be used to restrict subsequent departures from parallel trends rather than determine them exactly \citep{rambachan2023credible}.
Pretreatment information can also support recovery of the subgroup-average effect $\tau^J(z)$ when $X$ affects both untreated changes and treatment effects. The additional requirement is an assumption that identifies how the counterfactual untreated change over the treatment transition varies with $X$ relative to its population average; the overall average untreated change is still supplied by $\bar\Delta-\bar\tau$. The resulting expression and plug-in estimator are given in Online Appendix Section~\ref{sec:pretrend}.
\section{Estimation and inference}
\label{sec:estimation}
\subsection{Estimation}
\label{subsec:estimation}
Estimation combines an estimate of the within-jurisdiction treatment-effect contrast with an estimate of the population-average effect $\bar\tau$ from the aggregate comparative-case estimator. Under Assumption~\ref{ass:relative-parallel-trends}, combining these components gives the plug-in estimator
\[
\widehat\tau(z)
=
\widehat{\bar\tau}
+
\widehat\Delta(z)
-
\int\widehat\Delta(u)\,d\widehat F_Z^\star(u).
\]
For categorical $Z$, $\widehat\Delta(z)$ can be estimated from group-specific pre--post changes. For continuous $Z$, it can instead be estimated from a model for the conditional mean change.
Under \cref{eq:conditional-relative-parallel-trends}, when treatment effects depend on $Z$ but not further on $X$, the additive components in $\Delta(z,x)=m(x)+\tau(z)$ can be estimated from repeated cross-sections. For categorical $Z$, a repeated-cross-sectional regression with group-by-post interactions and post-period changes allowed to vary with $X$ provides a convenient implementation. If $\widehat a(z)$ denotes the fitted $Z$-specific component, the corresponding effect estimate is $\widehat{\bar\tau}+\widehat a(z)-\int\widehat a(u)\,d\widehat F_Z^\star(u)$.
For standardized effects, the conditional changes are averaged over the common marginal covariate distribution and the joint target distribution, respectively. If $\widehat\Delta(z,x)$ denotes an estimate of the conditional change, the standardized estimator is
\begin{equation}
\widehat\tau^S(z)
=
\widehat{\bar\tau}
+
\int\widehat\Delta(z,x)\,d\widehat F_X^\star(x)
-
\int\widehat\Delta(u,x)\,d\widehat F_{X,Z}^\star(x,u).
\label{eq:estimated-standardized-effect}
\end{equation}
The trend-adjusted and multi-period estimators use the same plug-in logic. The subgroup-average pretreatment extension is given in Online Appendix Section~\ref{sec:pretrend}.
\subsection{Inference}
\label{subsec:inference}
For any of the estimands above, write the estimator generically as $\widehat\tau(z)=\widehat{\bar\tau}+\widehat c(z)$, where $\widehat c(z)$ is the within-jurisdiction contribution. When the two components are estimated from independent samples, a natural variance estimator is
\[
\widehat{\operatorname{Var}}\{\widehat\tau(z)\}
=
\widehat{\operatorname{Var}}(\widehat{\bar\tau})
+
\widehat{\operatorname{Var}}\{\widehat c(z)\}.
\]
Pointwise confidence intervals can then be obtained using a normal approximation. For standardized effects, $\widehat c(z)$ is the difference between two averages obtained from the same fitted conditional-change function, so its variance should include their covariance as well as uncertainty in the estimated target distribution. A bootstrap can account for these sources of uncertainty by refitting the conditional-change model, re-estimating the target distribution, and recomputing both averages within each resample. The same principle applies when the within-jurisdiction component includes an estimated pretreatment-trend adjustment.
When bootstrap distributions are available for both components, uncertainty can instead be propagated through component bootstrap errors. Let $e_A^{*(b)}$ denote a bootstrap error for $\widehat{\bar\tau}$ and $e_C^{*(b)}(z)$ the corresponding error for $\widehat c(z)$. When the two components are estimated from independent samples, independent bootstrap errors can be combined as
\begin{equation}
\widehat\tau^{*(b)}(z)
=
\widehat\tau(z)
+
e_A^{*(b_A)}
+
e_C^{*(b_C)}(z).
\label{eq:independent-bootstrap}
\end{equation}
The bootstrap distribution of $\widehat c(z)$ should incorporate uncertainty from the estimated target distribution and any pretreatment-trend adjustment. If the same data contribute to both components, their covariance must also be accounted for. When both estimates can be recomputed from a common resample, a joint bootstrap provides one way to preserve this dependence.
When the aggregate comparative-case estimator supplies a confidence interval but no bootstrap distribution, component intervals can still be combined conservatively. If $[L_A,U_A]$ is an interval for $\bar\tau$ with coverage at least $1-\alpha_A$ and $[L_C(z),U_C(z)]$ is an interval for $c(z)$ with coverage at least $1-\alpha_C$, their endpoint sum $[L_A+L_C(z),\,U_A+U_C(z)]$ has pointwise coverage at least $1-\alpha_A-\alpha_C$ by Bonferroni's inequality, without an independence assumption.
\section{Numerical illustrations}
\label{sec:numerical}
I use two Monte Carlo experiments to examine two main components of the framework. The first asks whether combining a separately estimated population-average effect with within-jurisdiction treatment-effect contrasts introduces appreciable finite-sample bias or coverage distortion, and how precision responds to information in the aggregate panel and repeated cross-sections. Aggregate outcomes for 40 jurisdictions follow a rank-two interactive fixed-effects model, with one jurisdiction treated in the final period. Independent repeated cross-sections within the treated jurisdiction contain three population groups with shares $(0.35,0.30,0.35)$ and treatment effects $(0.35,0.25,0.15)$, giving a population-average effect of 0.25. I estimate the population-average effect using the generalized synthetic control method \citep{xu2017generalized}, with the factor rank fixed at its true value. The designs combine $T_0\in\{10,30\}$ pretreatment periods with $n\in\{500,2500\}$ observations per survey wave; complete data-generating processes and subgroup-specific results are given in Online Appendix Section~\ref{sec:simulations}.
\begin{table}[t]
\centering
\caption{Finite-sample performance of the combined estimator}
\label{tab:simulation}
\begin{threeparttable}
\small
\begin{tabular}{S[table-format=2.0] S[table-format=4.0] S[table-format=1.3] S[table-format=1.3] c}
\toprule
{$T_0$} & {$n$ per wave} & {Weighted mean $|$bias$|$} & {Weighted mean RMSE} & {95\% coverage range} \\
\midrule
10 & 500 & 0.005 & 0.275 & 0.940--0.964 \\
10 & 2500 & 0.001 & 0.271 & 0.956--0.964 \\
30 & 500 & 0.004 & 0.237 & 0.954--0.968 \\
30 & 2500 & 0.009 & 0.223 & 0.946--0.958 \\
\bottomrule
\end{tabular}
\begin{tablenotes}[flushleft]
\footnotesize
\item \textit{Notes:} Bias and RMSE refer to the three subgroup-specific combined treatment-effect estimators. Weighted mean $|$bias$|$ is the target-population-share-weighted mean of the absolute subgroup biases, and weighted mean root mean squared error (RMSE) is the corresponding weighted mean of the subgroup RMSEs. Bias and RMSE are based on 1,000 Monte Carlo replications. Coverage reports the range across the three subgroup-specific nominal 95\% Wald intervals and is based on 500 replications.
\end{tablenotes}
\end{threeparttable}
\end{table}
The combined estimator shows little finite-sample bias, and coverage remains close to the nominal level across all four designs (Table~\ref{tab:simulation}). The aggregate component accounts for the larger share of variance in this design, so increasing the pretreatment history from 10 to 30 periods reduces root mean squared error more than increasing the repeated-cross-sectional sample from 500 to 2,500 observations. The largest absolute subgroup bias is 0.011, and coverage of nominal 95\% intervals ranges from 0.940\ to 0.968.
The second experiment investigates whether standardization recovers effects defined at a common $X$ distribution when $X$ affects both untreated trends and treatment effects. The distribution of a continuous covariate $X$ differs across the three groups, untreated changes include the term $0.30X$, and treatment effects include $0.15X$. The standardized effects are 0.35, 0.25, and 0.15. Without standardization, differences in $X$ composition instead imply population values of 0.08, 0.25, and 0.42 for the three group-specific effects constructed from the observed changes and population-average anchor. Figure~\ref{fig:standardized-simulation} compares these quantities with Monte Carlo mean estimates from 1,000 replications, using $T_0=20$ and 2,000 observations per survey wave. As expected, the unadjusted estimates center on the composition-driven values, whereas the standardized estimates recover the target standardized effects with a maximum absolute bias of 0.011.
\begin{figure}[t]
\centering
\includegraphics[width=0.72\textwidth]{figure1.pdf}
\caption{Covariate standardization. Symbols show the target standardized effects and Monte Carlo mean estimates of the unadjusted and standardized estimators across 1,000 replications. The design uses $T_0=20$ and $n=2{,}000$ observations per survey wave.}
\label{fig:standardized-simulation}
\end{figure}
\section{Massachusetts health reform}
\label{sec:massachusetts}
Massachusetts's health reform sought near-universal insurance coverage through expanded subsidized coverage and an individual mandate \citep{long2008road}. I illustrate the framework using this reform. The Massachusetts Health Reform Survey (MHRS) provides detailed repeated cross-sections within the state around the main implementation of the reform, including education, age, sex, and current insurance coverage. I construct a state-level comparison panel from insurance coverage rates in the Behavioral Risk Factor Surveillance System (BRFSS). The combined estimator requires only aggregate coverage rates from the comparison states. Because the underlying BRFSS microdata are also available, the application also permits direct education-specific comparative-case analyses that use individual-level information from those states. Additional data and estimation details are given in Online Appendix Section~\ref{sec:mass-data}.
Massachusetts enacted its health-insurance reform in 2006, but the major components took effect during 2007 \citep{long2008road}. Consistent with previous evaluations, I use the 2006 MHRS as the baseline wave: although some provisions were already in place, the survey was fielded before the main components of the reform took effect. For the aggregate analysis, I date the policy transition from 2007Q1 and evaluate the effect in 2007Q4, matching the timing of the 2007 MHRS. For both components, the target population is Massachusetts adults aged 18--64 and the outcome is current health insurance coverage. I use the quarterly BRFSS panel to estimate the population-average effect and the 2007 MHRS to define the target covariate distribution and estimate the within-jurisdiction treatment-effect contrasts.
For the aggregate comparative-case analysis, I use the generalized synthetic control method \citep{xu2017generalized}, which models untreated outcomes with unit and period effects and a low-rank interactive fixed-effects component. I select the number of latent factors by rolling pretreatment cross-validation, as implemented in the \texttt{fect} package for R \citep{liu2024practical}. The selected specification contains no interactive factors, so the fitted counterfactual reduces to an additive state-and-period outcome model, closely related to the imputation-based difference-in-differences specification in \citet{borusyak2024revisiting}. For inference on the population-average effect, I use the parametric bootstrap procedure in \citet{xu2017generalized}.
The fitted untreated trajectory closely tracks the observed pretreatment series (Figure~\ref{fig:massachusetts-application}, panel~(a)). In 2007Q4, observed coverage exceeds the estimated counterfactual by 4.64\ percentage points (95\% confidence interval [0.75, 8.44]).
I examine heterogeneity across three education groups: high school education or less, some college or an associate degree, and a bachelor's degree or higher. Age and sex differ across education groups and may affect both untreated changes in coverage and responses to the reform. I therefore allow the conditional pre--post change to vary jointly with education, age, and sex and standardize each education group to the same age-sex distribution from the 2007 MHRS. The within-Massachusetts identification assumption is that, absent the 2007 policy transition, coverage would have changed equally across education groups of the same age and sex. For inference on the within-jurisdiction component, I use a survey bootstrap that respects the MHRS sampling design (see Online Appendix Section~\ref{sec:mass-data} for details). I combine independent bootstrap draws from the two components because the BRFSS panel and MHRS are based on separate survey samples.
Panel~(b) of Figure~\ref{fig:massachusetts-application} reports the resulting standardized treatment-effect contrasts. In ascending order of education, the standardized treatment-effect contrasts are 3.20, -0.77, and -2.45\ percentage points. The open circles show the corresponding unadjusted relative changes. Because each education group is standardized to the same age-sex distribution, these contrasts need not average to zero across the observed education shares.
Panel~(c) of Figure~\ref{fig:massachusetts-application} combines the population-average effect with the within-Massachusetts treatment-effect contrasts. The resulting standardized effects on insurance coverage are 7.85, 3.87, and 2.20\ percentage points in ascending order of education. For the lowest education group, the fitted standardized change is 10.15\ percentage points, while the fitted change averaged over the target joint distribution of education, age, and sex is 6.95\ points. Their difference, 3.20\ points, is the standardized treatment-effect contrast; combining it with the population-average estimate gives a standardized effect of 7.85\ points. The estimated difference between the lowest and highest education groups is 5.65\ percentage points (95\% confidence interval [2.78, 8.47]), and an omnibus test of equal standardized effects gives $\chi^2(2)=15.58$ ($p<0.001$).
Panel~(c) also reports direct education-specific comparative-case estimates from BRFSS state panels standardized to the same age-sex target distribution. These analyses use individual-level BRFSS data from Massachusetts and the comparison states, information that is not required by the combined estimator. The resulting point estimates are similar: 6.56, 2.51, and 2.73\ percentage points from the education-specific BRFSS panels, compared with 7.85, 3.87, and 2.20\ points from the combined estimator. Both approaches therefore produce the same broad pattern, with the largest estimated coverage increase in the lowest education group and smaller effects in the other two groups.
\begin{figure}[t]
\centering
\includegraphics[width=0.98\textwidth]{figure2.pdf}
\caption{Massachusetts health reform. Panel (a) shows observed quarterly insurance coverage among Massachusetts adults aged 18--64 (solid line) and the estimated untreated trajectory (dashed line); the vertical dashed line marks the 2007Q1 treatment onset. Panel (b) shows education-specific changes relative to the target-population change before (open circles) and after (filled circles) standardization to the 2007 Massachusetts Health Reform Survey age-sex distribution. Panel (c) shows the combined estimates (filled circles) together with estimates from separate education-specific BRFSS state panels (open squares). Vertical bars are pointwise 95\% confidence intervals, and the dashed horizontal line denotes the estimated population-average effect.}
\label{fig:massachusetts-application}
\end{figure}
\section{Discussion}
\label{sec:discussion}
This article develops an identification framework for heterogeneous policy effects in comparative case studies. Treatment-effect contrasts can be identified from repeated cross-sections within the treated jurisdiction under relative parallel trends, while a compatible population-average effect from an aggregate comparative-case design determines their level. The two components therefore identify heterogeneous effects even when individual-level data are unavailable for the comparison jurisdictions. The within-jurisdiction component requires only a single pretreatment cross-section.
I also show how the identification argument extends when untreated changes vary across population groups. Relative parallel trends can instead be imposed conditional on observed covariates, while additional pretreatment information can be used under explicit assumptions linking earlier differential trends to the treatment transition. These results clarify when heterogeneity can still be identified when relative parallel trends do not hold marginally, without requiring individual-level data from the comparison jurisdictions.
\clearpage