Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
49,466 characters · 12 sections · 40 citation commands
Feature Selection for Personalized Policy Analysis
\ifarxiv
\ Keywords: causal forests, feature selection, policy analysis.
\else \fi
Randomized control trials play an important role for treatment, policy, or a program effect analysis in economics, statistics, medicine, and other fields (banerjee2009experimental, bertrand2017field, chernozhukov2018generic). To design and implement effective interventions, policymakers and researchers are often interested in partitions of the population that are particularly susceptible to a new program or a policy. Identifying such subgroups can be challenging, especially when there are a large number of observable characteristics that can influence the outcome. In such cases, conventional estimation methods, such as ordinary least squares, may produce inefficient estimates. This is because these methods often struggle to identify relevant variables in a sparse feature space johnstone2009statistical, belloni2013least.
We propose Forest-PLS, a data-driven approach for selecting the target components of the population for personalized policy analysis. In our approach, the target components represent linear combinations of the explanatory variables. These components are characterized by large weights on the variables that are most strongly associated with the outcome, or carry a significant amount of information for explaining policy effect heterogeneity. As a second step, we identify and estimate personalized policy effects across the chosen components. By focusing on these key components, rather than considering the full set of characteristics, policymakers can design targeted interventions tailored to the most diversified segments of the population.
The procedure is a combination of two distinct methods, the partial least squares geladi1986partial, vinzi2010handbook and causal forest athey2016recursive, wager2018estimation algorithms. The partial least squares method is used to detect the policy-relevant components in the first step. These components represent a reduced explanatory variable space. The reduced space reflects the highest variation in explanatory features and the most relevant information for predicting the outcome. These components are continuous rather than discrete clusters of the data space, allowing us to analyze policy effects across a full spectrum of the population segments. In the second step, we use the causal forest algorithm to identify different quantiles of policy effects within each component value. This allows us to capture heterogeneity in the policy effects at a finer granularity.
The primary contribution of this paper is to advance our understanding of the distribution of policy effects from a theoretical and empirical perspective. The theoretical component of the article demonstrates that our approach is consistent and leads to asymptotically normally distributed policy effects. Our framework and findings extend beyond a single coefficient of interest wager2018estimation to multiple (plausibly) correlated policy effects. The empirical contribution of the paper is to identify and analyze two types of heterogeneity of the policy effects: within-group heterogeneity and between-group heterogeneity. The proposed method allows us to estimate quantiles of policy effects within and across the values of target components. Analyzing individual explanatory variables separately meinshausen2006quantile can be challenging in high-dimensional data. Our approach allows us to focus on aggregate aspects of these characteristics without losing the economic interpretation of the resulting subgroups.
Our framework is closely related to the papers dedicated to the estimation of personalized treatment effects athey2015machine, athey2016recursive, wager2018estimation, chernozhukov2018generic, chernozhukov2018sorted, kunzel2019metalearners, hahn2020bayesian, nie2021quasi, xiong2021federated, and feature selection for the inference on treatment effects belloni2012sparse, belloni2014inference, chernozhukov2015valid, chernozhukov2015post, urminsky2016using, banerjee2021selecting. Previous work for personalized treatment effect analysis considers a single source of heterogeneity, such as quantile treatment effects, or treatment effects across the original set of covariates (features). We unify the feature selection methods with personalised policy analysis. This allows us to investigate a full density of policy effects within and across a pooled variable space (target components).
Other related methods are proposed by hahn2002capturing, chun2010sparse, mehmood2012review, mehmood2020comparison, polson2021deep, nareklishvili2022deep, dixon2022deep that use the partial least squares algorithm for dimension reduction. These methods suggest that partial least squares as a precursor to a more general framework of deep learning and instrumental variables can increase efficiency. By comparison, our study shows that the method can pool statistically and economically significant variables for policy effect heterogeneity. Additionally, nekipelov2018moment, li2020asymptotic, nareklishvili2022adaptive investigate large sample properties for random forests under multiple outcomes, coefficients or network effects. We show that the theoretical properties hold even after the feature selection procedure by partial least squares. chernozhukov2018generic and jacob2019group propose group average One notable advantage of the Forest-PLS method is its ability to estimate heterogeneous group-average policy effects (GATE) in situations where the groups involved are both unordered and continuous. This characteristic distinguishes Forest-PLS from the approaches presented by chernozhukov2018generic and jacob2019group, which primarily focus on estimating GATE for ordered. It is important to note that the Bayesian approach is an alternative to policy effect heterogeneity. ansari2000hierarchical, taddy2016nonparametric, santos2018tree, hahn2020bayesian, woody2020estimating, starling2021targeted, krantsevich2022stochastic, he2023stochastic formulate Bayesian Additive Regression Trees (BART) for heterogeneous treatment or policy effect analysis. The advantage of the approach lies in the regularization effect through predetermined priors of the tree parameters. Our work can be extended to accommodate Bayesian priors.
We design various simulated experiments to unveil the inherent predictive advantages of the approach when compared to the traditional benchmark algorithms. The results reveal two notable advantages of Forest-PLS over the causal forest algorithm. First, Forest-PLS exhibits remarkable resilience in recovering the true density of policy effects, even when confronted with a limited number of observations. This characteristic renders the algorithm highly robust and consistent across varying sample sizes. Second, Forest-PLS effectively mitigates the influence of redundant variables and noise present in the experimental setup, enabling accurate estimation of the variance of policy effects.
This article examines the impact of financial incentives on unemployment duration based on data from the Pennsylvania "Reemployment Bonus" Demonstration, a randomized control trial conducted in 1988-1989. The analysis reveals significant variation in the policy effects both within and across different subgroups of the population. Specifically, the results show that the effects of the policy are more significantly dispersed for young, non-white male claimants who joined the experiment early on, compared to middle-age and older female participants with a high number of dependents. The difference between the 97.5th and 2.5th percentiles of policy effects is 92.8% for the first vigintile of the target component, and decreases to 22.1% for the final vigintile of the component. These findings highlight the need for targeted, personalized measures for specific subgroups, such as young non-white male participants.
Consider the outcome $y_i \in \mathbb{R}$ (e.g., unemployment duration) for a subject $i = 1, \dots, N$. Each subject is characterized with an observable vector of features $\mathbf{X}_{i} \in \mathbb{R}^p$ (e.g., age, gender, occupation). We assume that $p = N^\psi$, with $\psi < 1$. A policy is denoted by $P_i \in \{0, 1\}$, and we let $y_i(1)$ and $y_i(0)$ denote potential outcomes with and without the policy, respectively. We assume, the unconfoundedness holds:
A policymaker wishes to identify the dimensions of the feature space that contain the most relevant information about the policy effects. To this end, we consider a mapping $f: \mathbf{X}_{i} \mapsto \mathbf{C}_{i} \in \mathbb{R}^q$ that maps the original features to a set of target components $\mathbf{C}_i$. In other words, $\mathbf{C}_i$ is a collection of $q$-dimensional linear combinations of the features $x_{ij}$ for $j = 1, \dots, p$ (with $q \leq p$). This transformation allows us to focus on a smaller, more interpretable set of features while preserving the information about the policy effects.
The coefficient of interest is the effect of $P_i$ on the outcome:
Personalized policy effects are not directly observable, as an individual is only exposed to one policy state (either with or without the policy). Therefore, we typically consider the expectations of the potential outcomes:
Due to Assumption (ref), the policy effect of interest is given as
where $\mathbb{E}\big(y_i|P_i = d,\mathbf{C}_i\big)$ for $d \in \{1, 0\}$ denotes the observed expected outcomes with and without the policy, respectively.
To estimate the average policy effect in (ref), we need to determine the optimal target components $\mathbf{C}_i$ and use a method that estimates group-level policy effects, conditional on the chosen components.
We seek to identify the linear combinations of features, also known as target components/scores/factors, $\mathbf{C} = [\mathbf{c}_1, \mathbf{c}_2, \dots, \mathbf{c}_q$] (i.e., $\mathbf{c}_1 = \mathbf{X}\mathbf{w}_1$) that explain the highest variation in covariates $\mathbf{X}$, as well as the outcome $\mathbf{y}$ tobias1995introduction, abdi2003partial. $\mathbf{X}$ and $\mathbf{y}$ can be decomposed as:
where $\mathbf{X} = [\mathbf{x}_1, \mathbf{x}_2, \dots, \mathbf{x}_p]$ is the matrix of covariates, $\mathbf{V}^T$ is the matrix of loadings (weights), and $\mathbf{E}$ is the matrix of errors for the covariates. $\mathbf{y}$ denotes the outcome as before, $\mathbf{b}$ is the vector of coefficients (the influence of components on the outcome), and $\mathbf{e}$ is the vector of errors for the response.
We use the iterative procedure to obtain the target components (aka partial least squares). Consider, the weight $\tilde{\mathbf{w}}_1 = \big( cov(\mathbf{x}_1, \mathbf{y}), cov(\mathbf{x}_2, \mathbf{y}), \dots, cov(\mathbf{x}_p, \mathbf{y}) \big) = \big(\tilde{w}_{11}, \tilde{w}_{21}, \dots, \tilde{w}_{p1} \big)$. We normalize it to get a unit vector:
where $||\tilde{\mathbf{w}}_1||$ denotes the Euclidean norm. We use these weights to compute the first principal component:
The last equality in (ref) follows by the fact that the weights are unit vectors. A linear regression of a $j-$th covariate on the first component yields a loading. The vector of loadings associated with the first component is given by regressing the covariates on it:
Similarly, the first coefficient $b_1$ is obtained by regressing the outcome on the first component:
The next step is to obtain the approximation of the covariate matrix and the outcome, and predict residuals:
where $ \hat{\mathbf{X}} = \mathbf{c}_1\mathbf{v}_1^T$ and $\hat{\mathbf{y}} = b_1\mathbf{c}_1$. We obtain the subsequent components by repeating the described procedure for the first, second, and higher order residuals of the covariate matrix $\mathbf{X}_1, \mathbf{X}_2 \dots$, and the outcome $\mathbf{y}_1, \mathbf{y}_2 \dots$, respectively.
A desirable property of the procedure is that the coefficients have a closed-form solution. The estimator of these coefficients is given as (helland1990partial, stone1990continuum):
where $\hat{R} = (s_{xy}, S_{xx}s_{xy}, \dots, S_{xx}^{q-1}s_{xy})$ is the $p \times q$ matrix of the Krylov sequence with a $p \times p$ matrix $S_{xx}$ and a $p \times 1$ vector $s_{xy}$ defined as follows:
where $I$ is an identity matrix and $1$ is a matrix of ones. Intuitively, the algorithm searches for factors that capture the highest variability in $\mathbf{X}$, and at the same time maximizes the covariance between $\mathbf{X}$ and $\mathbf{y}$. If the number of components equals the dimension of the covariates, $q = p$, the method is equivalent to the ordinary least squares (helland1990partial).
phatak1997geometry show that the partial least squares estimator can geometrically be interpreted as the tangent rotation and projection of the OLS estimator on the ellipsoid. Consequently, that property allows us to extract the dimensions of the feature space that are relevant for predicting the outcome as well as the policy effect heterogeneity. We determine the optimal number of target components based on the cross-validation results. Particularly, we choose the minimum number of components beyond which the prediction performance stabilizes. It is noteworthy to emphasize that the target components identified through partial least squares are inherently derived from the data. This ensures that the composition of the identified groups may differ depending on the specific dataset under investigation. However, an article by cao2018partial provides compelling evidence demonstrating the robustness of the partial least squares method even in the presence of potential misspecifications in the treatment assignment model.
To identify smaller subgroups within the population, we use a causal forest algorithm. A tree in causal forests recursively partitions the feature space, in this setting, the space of identified target components $\mathbf{C}$, and makes axis-aligned splits to estimate the conditional mean of the outcome $\mu(c) = \mathbb{E}(y_{i}|P_i, \mathbf{C}_i = c)$ at a point $c$ for $P_i \in \{0, 1\}$.
An axis-aligned split is a pair $m = (j, c)$, where $j = 1, \dots, q$ is a specific component (the splitting coordinate) and $c \in \mathbb{R}$ is the corresponding value (the splitting index). The recursive partitioning procedure begins by considering the set $\mathcal{P}^{(0)} = \mathbf{C}\in R^{q}$ (the parent node of the tree). For this set, we select the splitting coordinate $j: 1 \leq j \leq q$ and the splitting index $c$ that divide $\mathcal{P}^{(0)}$ into two non-overlapping rectangles (child nodes):
After the first split, the process is repeated for $\mathcal{P}^{(1, 1)}$ and $ \mathcal{P}^{(1, 2)}$separately until the desired level of partitioning is achieved.
The sequence of $k$ splits defines a partition of the component space $\mathbf{C}$, which we denote by $\Pi$. This partition (or equivalently, a tree) consists of non-overlapping rectangular regions $\ell_n$ called the leaves or terminal nodes of the tree. These leaves represent the final subgroups or subpopulations identified by the algorithm. The union of all these partitions is the entire component space:
athey2016recursive propose a method for estimating heterogeneous policy effects under the assumption of unconfoundedness. To implement this method, we split the data into two different samples: a training sample $S^{tr}$ used to build and find the splitting variables and values, and an estimation sample $S^{est}$ used to estimate policy effects across different subgroups of the population. The unbiased sample analogue of $\mathbb{E}(\theta(\mathbf{C}_i))$ is denoted as follows:
where $|\Pi|$ is the total number of the terminal nodes. $1(c \in \ell_n, P_i = d)$ is a binary variable and equals one when, for a given $d \in \{0, 1\}$, a generic test data point $c$ belongs to a terminal leaf $\ell_n$, and zero otherwise. Additionally, let $\Sigma$ be the variance of $\widetilde{\theta}(X_i, S^{est}, \Pi)$. \footnote{While our analysis is based on a single policy variable and a single outcome, the proposed framework can handle multiple policy variables and outcomes with correlated coefficients. }
To estimate policy effects from the available data, we aim to maximize the variance of the policy effect estimator:
The proof of (ref) is provided in Appendix (ref). Intuitively, the objective function in (ref) encourages the causal forest algorithm to search for subsets of target components with the highest variation in policy effects. To further increase the robustness of the estimates, we build multiple trees on bootstrapped data and average the resulting coefficients. This approach, known as the causal forest algorithm, has been described in detail by athey2016recursive and wager2018estimation.
To show the asymptotic normality of the estimated policy effects, we need to make certain assumptions about the underlying data-generating process.
Under Assumption (ref), the relation between the response and the independent characteristics follows a predetermined functional form. Additionally, the subject characteristics are assumed to have an elliptical distribution, meaning they are shaped like an ellipse in a multi-dimensional coordinate system. While this assumption is not always satisfied in practice, it has been shown that the results obtained under this assumption do not significantly differ from those obtained when the features have other types of distributions (see brillinger2012generalized).
The proof of Lemma (ref) is provided in Appendix (ref). Lemma (ref) shows that the identified target components are consistent. The causal forest method described in this article relies on the same assumptions as those introduced by wager2018estimation. One of them is the "honesty" of the tree.
There are various ways to satisfy Assumption (ref). In this article, we use a two-sample approach, where we split the data into a training sample $S^{tr}$ and an estimation sample $S^{est}$. The splitting coordinates and indices ($m$) of the trees are determined based on the observations in $S^{tr}$, while the predicted outcomes are based on the observations in $S^{est}$. This separation of the data into two different samples ensures that the splitting parameters and the outcomes are independent of each other.
In order to guarantee the consistency of the causal forest method, it is necessary for the leaves of the trees to become small in all dimensions of the component space as the sample size $N$ increases. To ensure this, we adopt Assumption (ref), which is based on the assumptions of meinshausen2006quantile and wager2018estimation. This assumption states that for all splitting steps, each component has a probability of at least $\pi/d$ of being selected as the splitting coordinate, for some $0 < \pi \leq 1$.
Assumption (ref) ensures that each half-space produced by a split in the tree construction process contains a sufficient number of observations. As shown by wager2015adaptive, this assumption also implies that the half-spaces are large in Euclidean volume. Assumption (ref) places an upper bound on the number of observations that can be contained in a terminal node of the tree. Specifically, when a tree is fully grown to depth $k$, we have that each terminal node contains between $[k, 2k-1]$ observations. One important consequence of this assumption is that it places an upper bound on the variance of the tree estimator at any test point $c$.
Lipschitz continuity and bounded variances are widely used assumptions in the field of statistics and machine learning (wager2018estimation, biau2012analysis). In the context of this paper, the results do not depend explicitly on the distributional assumptions of $\mathbf{C}_i$, however, they affect the constants that we carry throughout this paper (constants borrowed from Lemma 2 and Theorem 3 in Section 3.2 in wager2018estimation).
Assumption (ref) ensures that, as the number of observations $N$ increases, there will be a sufficient number of subjects with and without a policy at any given test point $c$. Under the given assumptions, wager2018estimation show that the random forest estimator is consistent and asymptotically normally distributed. They generalize the properties to a single parameter of interest. Additionally, nareklishvili2022adaptive in Theorem 6.3 shows that the causal forest estimator is asymptotically normally distributed even for multiple (possibly correlated) parameters. Appendix (ref) presents supplementary definitions and elaborates on the algorithm. To quantify the uncertainty of the policy effects, we employ the jackknife variance estimator, as outlined in Subsection (ref) of Appendix (ref) (wager2018estimation). The results of wager2018estimation and nareklishvili2022adaptive directly apply to our setting when conditioned on the target components.
We explore different simulation designs aimed at evaluating the estimation performance of Forest-PLS. The accompanying Figure (ref) visually represents two distinct scenarios. In Panel (a), we present a randomized controlled experiment. In this scenario, the outcome of interest is influenced by a specific policy, and the assignment of the policy is under the complete control of the experimenter. This design allows us to assess the direct impact of the policy on the outcome while minimizing potential confounding factors. Panel (b) highlights an alternative situation where unobservable factors, which are not accessible to a policy-maker, may correlate with both the policy and the outcome. This design aims to mimic real-world scenarios where policy decisions are made under uncertainty, and there exist latent factors that introduce bias in the analysis.
In the framework of randomized controlled trials (RCT), the policymaker has access to the outcome, denoted as $Y_i$, the policy $P_i$, and four distinct variables, represented as $X_{i1}$, $X_{i2}$, $X_{i3}$, and $X_{i4}$. Formally, the outcome ($Y_i$) is determined by a combination of the policy intervention and the individual characteristics associated with each participant $i = 1, \dots, N$:
In our paper, the policy variable ($P_i$) is generated from a binomial distribution, denoted as $\mathcal{B}$. This variable represents the policy intervention assigned to each participant and captures the binary nature of the treatment. On the other hand, the individual characteristics ($X_{i1}$, $X_{i2}$, $X_{i3}$, and $X_{i4}$) are drawn from a normal distribution, denoted as $\mathcal{N}$. These characteristics could correspond to important demographic factors such as age, gender, education level, and race, and are continuous in nature. The outcome also consists of a normally distributed noise ($\varepsilon_i$). To investigate the effects of the policy intervention on the outcome, we simulate the randomized controlled trial (RCT) fifty times with different sample sizes denoted by $N$ = (70, 100, 500, 1000, and 5000), and average the results. The variance-covariance matrix of the participant characteristics ($\Sigma$) is assumed to be the identity matrix $\mathbf{1}_{4 \times 4}$, indicating that the individual characteristics are independent and have equal variances.
The randomized controlled trial design in (ref) presents two significant challenges for the Forest-PLS algorithm. First, the policy effects exhibit heterogeneity and vary across individuals based on their values of $X_{i3}$ and $X_{i4}$, i.e., $\mathbb{E}(Y_i|P_i = 1) - \mathbb{E}(Y_i|P_i = 0) = X_{i3} + 0.1\times X_{i4} + 0.2\times X_{i3}\times X_{i4}$. Second, the outcome is heavily influenced by independent variables $X_{i1}$ and $X_{i2}$, which are not particularly relevant for explaining the heterogeneity of policy effects. In this setting, the causal forest needs to correctly identify the sources of heterogeneity when provided with the target components.
Figure (ref) depicts the simulated density of policy effects alongside their estimated counterparts based on the causal forest and Forest-PLS algorithms. The illustration reveals two notable advantages of Forest-PLS over the causal forest algorithm. First, Forest-PLS exhibits remarkable resilience in recovering the true density of policy effects, even when confronted with a limited number of observations. This characteristic renders the algorithm highly robust and consistent across varying sample sizes. Second, Forest-PLS effectively mitigates the influence of redundant variables and noise present in the experimental setup, enabling accurate estimation of the variance of policy effects.
The success of Forest-PLS can be attributed to the inherent diversity and relevance of the target components. As demonstrated in Table (ref) in Appendix (ref), the first component predominantly captures the features $X_{i1}$ and $X_{i2}$, while the second component encapsulates the remaining information, $X_{i3}$ and $X_{i4}$. Each target component contains information valuable for explaining either the outcome or the policy effect heterogeneity. Thus, when trained on the target components, the causal forest efficiently retrieves the correct dimensions of the policy effect heterogeneity even in small samples. Table (ref) in Appendix (ref) verifies that the variable importance for policy effect heterogeneity is similar for both methods. We further compare partial least squares with LASSO regression. Table (ref) in Appendix (ref) shows that, unlike partial least squares, LASSO drops the variables that are essential for explaining policy effect heterogeneity. Figures (ref) and (ref) in Appendix (ref) provide additional simulation designs and show the advantages of Forest-PLS over the conventional causal forest method.
In the second simulated experiment, we aim to assess the variation in policy effects in the presence of an unobservable latent variable, denoted as $U_i$. This latent factor impacts both the policy implementation and the outcome. Furthermore, the policy itself is influenced by two instrumental variables, namely $Z_{i1}$ and $Z_{i2}$:
The policy and outcome consist of errors $\varepsilon_{i1}$ and $\varepsilon_{i2}$, respectively. In (ref), the policy effect is represented by $0.5\times X_{i1}$. The framework in (ref) highlights the presence of a weak instrument and a strong confounder ($U_i$). Unlike the RCT design, in this framework, we have a single variable relevant to explaining the outcome and policy effect heterogeneity. We assume, the researcher does not have access to either the instruments $Z_{i1}$, $Z_{i2}$, or the latent variable $U_i$. Therefore, the algorithm receives the observable characteristics $X_{i1}, X_{i2}, X_{i3}, X_{i4}$ as the input.
Figure (ref) illustrates that the Forest-PLS and causal forest algorithms systematically and identically underestimate the mean and variance of the policy effects. We also find that the methods are comparable across a varying number of observations.
The Pennsylvania "Reemployment Bonus" Demonstration was a randomized controlled trial in 1988-89 that aimed to investigate the impact of financial incentives on the reemployment outcomes of unemployed individuals. The study population was divided into a control group, which received the usual benefits provided by the Unemployment Insurance System, and six treatment groups. Treated individuals were offered a cash bonus for fulfilling certain criteria related to finding and retaining employment. Specifically, participants in the treatment groups were required to accept a bonus that would be paid to them if they were able to secure a full-time job of at least 32 hours per week within a specified period (the qualification period) and maintain that employment for at least 16 weeks.
Two bonus levels were tested. These two levels were a low bonus and a high bonus, which were respectively three and six times the weekly benefit amount (WBA) received by the participants. The low bonus was on average $\$500$, while the high bonus was $\$997$. In addition to these two levels of bonus, the study also considered two different qualification periods, starting from the date on which the bonus offer was made. These periods were a short one of 6 weeks and a long one of 12 weeks.
In addition to testing the impact of financial incentives on reemployment outcomes, the Pennsylvania "Reemployment Bonus" Demonstration also aimed to investigate the effectiveness of providing job-search assistance to unemployed individuals. To this end, participants in the treatment groups were offered a workshop and an individualized assessment session as part of the treatment design. However, attendance at the workshop and completion of the assessment session were not mandatory for claimants.
In this article, we focus on treatment Group 4 which received a high bonus amount and a long qualification period, as well as an offer of a workshop. The primary outcome of interest is the logarithm of unemployment duration in weeks. The data include twenty different characteristics of the claimants, such as age, gender, the quarter of the experiment in which they enrolled, and unemployment rates in the local area. \footnote{The variables are described in detail at the following url: \url{http://qed.econ.queensu.ca/jae/2000-v15.6/bilias/readme.b.txt}.} Further information about the experiment and data can be found in the article by bilias2000sequential.
In this section, we identify and characterize policy-relevant target components. According to Figure (ref) in Appendix (ref), the optimal number of components equals two. To characterize and interpret the chosen scores, Table (ref) in Appendix (ref) illustrates the effect of the claimant characteristics on each target component. The negative value of a coefficient indicates that there is an inverse relationship between the characteristic in question and the outcome being measured. For instance, a black claimant is associated with a 1.3% lower score on average relative to a white claimant. Based on the sign of the coefficients in Table (ref), we can interpret the scores.
The lowest values of the target components identified in this article correspond to a subgroup of young, non-white male claimants in the non-durable manufacturing sector who enrolled in the experiment early on. On the other hand, the highest values of the components reflect a subgroup of middle-aged and older female individuals in the durable manufacturing sector. These claimants enrolled in the experiment late in the final quarter and tend to have a high number of dependents (as indicated by the positive coefficient for the "dep" variable in Table (ref) of Appendix (ref)). (ref) and (ref) summarize the characteristics of subgroups. \footnote{According to Table (ref) in Appendix (ref), these two components are almost identical in this setting, therefore, (ref) and (ref) hold for each. } Components are continuous, therefore, they characterize a full spectrum of individuals from one group to another.
In this section, we investigate the heterogeneity, or variability, in the effect of financial incentives on unemployment duration within and across different values of the components. This is done by examining various percentiles of the reemployment bonus effect on unemployment duration across the corresponding component vigintiles, as shown in Figure (ref).
The results depicted in Figure (ref) reveal considerable variation in the policy effects both within and between groups. In particular, financial incentives have been found to potentially motivate some young non-white individuals (the group represented by low score values in (ref) to enter the labor market. However, these incentives may also provide a temporary financial cushion for others, potentially dissuading them from actively seeking employment. In contrast, this variation is less pronounced among older white claimants (upper vigintiles of each component, the group corresponding to high score values in (ref).
To contrast our method with the causal forest algorithm, Figure (ref) in Appendix (ref) assesses the heterogeneity of policy effects based on the latter. Unlike Forest-PLS, the causal forest does not depict significant heterogeneity either across the target components, or the original independent characteristics.
Our analysis shows that the "Reemployment Bonus" policy has, on average, a negative effect on unemployment duration. However, significant variation is captured by different percentiles of policy effects. The difference between the 97.5th and 2.5th percentiles of policy effects is 92.8% in the first vigintile of Component 2, and this difference decreases to 22.1% in the 20th vigintile of the same component. These findings suggest the need for targeted, personalized measures for younger non-white male claimants.
Policymakers frequently seek to understand the impact of interventions on specific subgroups or segments of the population, defined by certain characteristics or attributes known as covariates. In this article, we present a method for analyzing the density of policy effects within these target segments, which are defined as linear combinations of the explanatory variables. To achieve this, we combine two existing techniques, partial least squares and causal forests, that allow us to identify and analyze policy effects for the full range of the target segments. This approach enables policymakers to understand how policy effects vary within and across these segments, providing valuable insights for personalized policy analysis.
We show that the method is consistent and leads to asymptotically normally distributed policy effects. Additionally, our approach generalizes beyond a single policy effect to multiple (plausibly) correlated policy effects. Our analysis based on data from Pennsylvania "Reemployment Bonus" Demonstration reveals a significant variation in the effect of financial incentives on the logarithm of unemployment duration. The findings highlight the need for targeted measures for young non-white male participants.
One potential extension of our method is to incorporate quantile regression forests meinshausen2006quantile, conditional on the target components. In addition to using randomized control trials, we also plan to explore the application of our method to observational data with an endogenous policy vella1999estimating, baiocchi2014instrumental. By applying the method to observational data, we hope to gain further insights into the heterogeneity of policy effects and inform the design of interventions for institutional settings.