Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
40,943 characters · 7 sections · 33 citation commands
Enabling Decision-Making with the Modified Causal Forest: Policy Trees for Treatment Assignment
\begingroup \let\relax \endgroup
\thispagestyle{empty}
Policymakers are keen to gain insights into policy impacts, aiming to facilitate future applications. Therefore, current econometrics research is dedicated to developing methods that accurately estimate policy effects, specifically focusing on understanding relevant heterogeneity across sub-populations. However, it is important to distinguish between estimating heterogeneity and providing guidance on the design of treatment assignment mechanisms. The latter pertains to the policy learning research area, which effectively uses information from randomised or observational studies to assign individuals to future treatments.
The seminal work of manski2004statistical pioneered the literature on policy learning, then further developed by kitagawa2018should, mbakop2021model, athey2021policy, kitagawa2021equality and zhou2023offline, among others. Specifically, kitagawa2018should introduce a policy learner that attains the $\sqrt{N}$ convergence rates in settings with a binary treatment and known propensity scores. athey2021policy propose a policy learner that achieves the same convergence rate even without known propensity scores, making it well suited for observational studies. Finally, zhou2023offline propose the Cross-fitted Augmented Inverse Propensity Weighted Learning (CAIPWL) policy learner, which achieves optimal convergence rates in multi-action settings with unknown propensity scores.
This work presents the mcf Policy Tree as implemented in the mcf package. The policy learner is based on the algorithm proposed by zhou2023offline but introduces three key modifications: policy scores calculation, implementation of constraints, and different handling of categorical and continuous variables. First, the consistent and asymptotically normally distributed policy scores (or potential outcomes) are computed by the mcf to calculate individualised average treatment effects (IATEs)\citep*{lechner2024comprehensive}. They differ from the doubly-robust scores used in the original algorithm since the mcf splitting criterion differs. In particular, the mcf algorithm minimises the estimation errors of the conditional mean responses of the treatments and the covariance of these errors while also taking into account propensity score heterogeneity to reduce selection bias. Additionally, the IATEs differ from doubly robust scores since they depend only on the features and not the outcome and treatment. The second modification concerns the implementation of constraints. They are integrated as treatment-specific cost values in terms of the outcome, which are then subtracted from the policy scores. This also applies to constraints that are only indirectly related to costs, such as constraints on treatment shares. Finally, the mcf Policy Tree differs from existing methods in the literature in handling categorical and continuous features. Traditional approaches in the literature often rely on one-hot encoding for categorical features, which may lead to rather extreme leaves with many different values when building shallow trees due to splitting one category against the remaining ones. In contrast, the mcf Policy Tree employs a distinct procedure that allows several categories on both sides of a split. For a detailed explanation of this approach, refer to Chapter 9.2.4 in hastie2009elements. Beyond this functionality, the \textit{mcf} algorithm provides flexibility for handling continuous and categorical, unordered features with many possible values by allowing users to specify an approximation parameter. This parameter determines the number of potential splitting points, offering finer granularity\footnote{4th level, i.e., at the top of the tree, A/8; 3rd level A/4; 2nd level A/2; and the 1st level, at the bottom of the tree, A; with A being the approximation parameter as used in zhou2023offline as one single global approximation level.} compared to the global approximation parameter implemented in zhou2023offline.
This paper contributes to the literature in various ways. First, we demonstrate the application of the Policy Tree algorithm across diverse fields, including medicine, epidemiology, and business, within both randomised and observational research settings. Second, this work provides new empirical results on causal effect heterogeneity and policy learning. In that respect, we evaluate well-known and used data in the respective scientific fields. Third, our results take cost-related constraints for decision-making into account. Fourth, all replication materials, including data and programming codes, can be accessed on Github. To run these codes, the open-source Python package mcf provides the classes ModifiedCausalForest() for estimation and OptimalPolicy() for decision-making.
In the remainder of this paper, Section (ref) introduces the framework for identification and estimation and package functionalities. Section (ref) shows the setups and findings of the empirical studies. Finally, Section (ref) discusses computational facets and concludes.
The mcf package is a forest-based causal machine learning algorithm that produces consistent and asymptotically normal treatment effect estimates for various levels of granularity in randomised control trials and selection-on-observable settings. The latter imposes several assumptions to identify the causal effects, namely ignorability, overlap, the stable unit treatment value assumption, and exogeneity of features; see ImWo2009 for more details on these identifying assumptions. The estimands of interests identify the causal effect at different aggregation levels, in particular, the average potential outcomes, the average treatment effect, and conditional average effects like IATEs and grouped aggregates of these effects. The IATEs are the average effects for the most granular sub-groups, identified by specific realisations of the available exogenous features. The evaluation of the IATEs makes it possible to detect potential heterogeneous effects across sub-groups of the population. If heterogeneity is observed, certain individuals may either benefit or not from a particular treatment. This is a crucial starting point for a policy learning analysis, indicating the potential for refining the treatment assignment mechanism and informing policy recommendations.
To address this, we introduce the OptimalPolicy() class of the mcf, which explores the space of all available trees and chooses the optimal one. The optimal tree maximises the value function (or welfare), computed as the sum of the individual policy scores, such as potential outcomes or IATEs, by assigning all observations in a terminal leaf node to a single treatment. For a fixed depth, the tree algorithm implements an exhaustive and recursive search across all pre-specified variables and values. It performs the splits when the policy scores within the leaves (or subsets) are maximised under a specific treatment.
Algorithm (ref) illustrates, in a simplified way, how the tree-search algorithm works. To introduce it, we can assume the following notation: For \( i \) observations, ${X_i}$ represents the features, specifically, \( p_1 \) is the number of ordered features, \( p_2 \) is the number of unordered features, $j\in\{0,\ldots, D \}$ is the treatments space, \( \hat{\Theta}_i \) is the vector (or matrix) of policy scores for each observation \( i \), \( \hat{\Theta}_i(j) \) is the potential outcome for each observation for each treatment \( j \), and \( L \) is an integer indicating the depth of the tree plus one.\footnote{For detailed reference of the algorithm \href{https://github.com/MCFpy/mcf/blob/ae13663d0a8e86c7d123471bd7741489a39401d6/mcf/optpolicy_pt_eff_functions.py}{optpolicy_pt_eff_functions.py}.}
Besides the Policy Tree, the OptimalPolicy() class provides a best-score method to create an assignment rule. This method simply assigns units to the treatment with the highest estimated potential outcome. Although computationally cheap, this best-score method lacks clear interpretability for the allocation rules, which may make it difficult for policymakers to adopt it.
The mcf package offers empirical researchers dual valuable functionalities: the possibility of conducting heterogeneity analyses, as shown in bodory2022high, and refining the decision-making process by optimising treatment allocation rules. This section studies three real-world applications from different scientific fields based on distinct research settings. In these empirical tasks, we first perform an effect heterogeneity evaluation calling the ModifiedCausalForest() class of the mcf algorithm. We then use the potential outcomes estimated by the mcf to optimise the treatment allocations generated by the Policy Trees using the mcf's OptimialPolicy() class. The samples from all three studies are split using the same approach. Specifically, 40% of the data is allocated for training the \textit{mcf}, and another 40% for estimating the effects and training the policy learning algorithm. Finally, the remaining 20% of the data are reserved for predicting out-of-sample policy rules and welfare. This allocation helps to estimate the treatment effect effectively and to mitigate the risk of overfitting, particularly for deeper trees.
In 2008, Oregon initiated a lottery with limited spots to provide low-income individuals access to its Medicaid program. This lottery offers an opportunity to examine the impact of expanding access to public health insurance within a randomised controlled setting. Researchers analysed various outcomes over two years following the experiment, including healthcare utilisation, financial strain, health status, labour market outcomes, and political participation. Extensive studies have been conducted by finkelstein2012oregon, baicker2013oregon, finkelstein2016effect, baicker2017effect,/content/paper/687686456188, baicker2018impact, finkelstein2019value, and baicker2014impact, among others. Overall findings suggest that access to Medicaid leads to an increase in healthcare utilisation and a decrease in financial strain and depression, albeit without statistically significant improvements in physical health or labour market outcomes. This analysis follows finkelstein2012oregon and specifically focuses on the impact of being selected in the lottery on primary care visits during the first year post-selection. We rely on survey data collected within the experimental setting, given that the hospital records are not publicly available. The end-line survey, conducted after 12 months, was completed by 23,777 individuals. The outcome is measured by a binary variable indicating if primary care has been utilised during the year. Among the several outcomes available (e.g. emergency room usage or drug prescription), we choose primary care utilisation because of its importance from a policy perspective. Indeed, primary care visits may lead to preventive care for more serious diseases and, consequently, prevention of more expensive medical care.
After dropping observations with missing values, the size of our dataset is 23,527 observations, with 50% of the individuals with Medicaid health insurance. As in finkelstein2012oregon, our intention-to-treat specification incorporates additional features, such as socio-demographic factors, which are valuable for exploring heterogeneity and informing policy, even though they are not essential for identifying causal effects. On the contrary, although assignment to the lottery is random, all family members of randomly selected individuals can apply for Medicaid. Therefore, the probability of receiving treatment depends on the number of family members, which we include as a control variable. Table (ref) in the Appendix describes the 11 variables used for data analysis.
We evaluate the distribution of the IATEs to detect potential effect heterogeneity. Figure (ref) displays a density plot of the IATEs, ranging from -0.13 to 0.37, with a mean value of 0.07 and an average standard error of 0.1. The variation in the effects justifies our following analysis for decision-making, aiming at expanding the number of primary care visits by optimising the treatment allocation process.
Table (ref) provides details on welfare, measured as utilisation of primary care visits and allocation shares of individuals for various policies. We compare the results of the Policy Trees with our two baseline policies, namely the observed and random treatment allocations. In this study, we examine both unconstrained and constrained Policy Trees. Given the lack of specific cost information for Medicaid insurance, we assume that only approximately half of the population (as in the empirical data) can be covered by this health insurance. The algorithm enforces constraints by finding a cost value to subtract from the policy score. It adjusts the treatment costs to ensure that the allocation of observations to treatments does not exceed the maximum treatment shares. We build our constrained Policy Trees by specifying either the depth of a first optimal tree (Constrained optimal Policy Tree) or, optionally, specifying, in addition, the depth of a second optimal policy tree (Constrained sequentially optimal Policy tree). In the latter case, the second tree is built within the strata obtained from the leaves of the first tree. The final Policy Tree will not be optimal. However, its performance in terms of welfare is comparable to that of an optimal tree with the same depth, with the advantage of reduced computational time.\footnote{The sequentially optimal policy tree is implemented as the hybrid policy tree within the policytree package (policytree)}
In Table (ref), all the Policy Trees report an increase in welfare with respect to the baseline policies. The unconstrained Policy Trees achieve the highest welfare, increasing the probability of primary care visits from 51.70% to 54.98%. They allocate almost all individuals to the treatment, which can be interpreted as access to Medicaid insurance for nearly everyone. Compared to the unconstrained Policy Trees, those with the capacity constraint always report a small decrease in the probability of primary care visits, allocating a significantly lower proportion of individuals to the treatment. Constrained optimal Policy Trees and Constrained sequentially optimal Policy Trees, with a last sequential optimal sub-tree of the same depth, show comparable and stable results in terms of welfare achieved. From a computational point of view, such findings may be relevant for large datasets since Policy Trees with the last sequential optimal layers reduce computation time by performing similarly well.
Figure (ref) presents the allocation rules of a constrained optimal Policy Tree with a depth of 2. Notably, even at this relatively shallow depth, the Policy Tree demonstrates improvements in welfare compared to the observed allocations and the simplest allocation rules. Individuals with age between 37 and 51 years old (38% of the sample) are assigned to Medicaid. Additionally, individuals older than 51 living in metropolitan areas, constituting 9% of our sample, are also allocated to the program. Conversely, individuals younger than 36 and older than 51 who do not live in metropolitan areas are excluded from the program. For reference, note that the age ranges from 20 to 63.
In a seminal study, connors1996effectiveness investigated the impact of Right Heart Catheterization (RHC) on various outcomes, including subsequent survival, healthcare costs, intensity of care, and length of hospital stay. Their findings revealed a positive association between RHC and mortality, as well as increased costs and prolonged length of stay. This conclusion was corroborated by subsequent research utilising different estimation methods, such as those employed by knaus1995support, ramsahai2011extending, keele2018pre, and keele2021comparing. In accordance with previous research, we also use the dataset originally collected from the SUPPORT prospective cohort study (Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments) as in connors1996effectiveness. This dataset includes 5,735 observations on ill and hospitalised individuals between 1989 and 1994 in five medical centres in the US. 38% of the patients received the surgery. bodory2022high utilise the same information to provide new results on effect heterogeneity by exploring the functionalities of the mcf package.
In this empirical application, we focus on finding an optimal allocation of RHC to patients to increase their probability of survival six months after the medical surgery. Previous work exploits expert opinion to identify eight features as relevant confounding factors ramsahai2011extending. These features include an index of activities of daily living two weeks before admission, age, the acute physiology and chronic health evaluation score (APACHE III), the Glasgow coma score, mean blood pressure, the probability of surviving two months (based on model estimations), an indicator for do not resuscitation status on the first day, and information on nine primary disease categories. Recognising the significance of these factors, we employ them as features in the Policy Tree to obtain assignment rules for decision-making. Age and mean blood pressure are continuous variables, and we allow the tree algorithm to automatically determine the optimal split points based on the default number of evaluation points in the mcf package.
Our analysis starts by estimating the policy scores and IATEs. Additionally to the high-priority factors, we include potential confounding features (e.g. information on having cancer, possessing health insurance, etc.) to achieve identification through a selection-on-observable approach. Table (ref) of the Appendix comprehensively describes all 55 confounders.
Figure (ref) displays the density function of the IATEs, the most granular treatment effects. While slightly right-skewed, the distribution shows positive and negative treatment effects, ranging from -0.12 to 0.08, and an average standard error of 0.06. This result underscores the (rather moderate) heterogeneous response to the surgery, indicating that while some individuals benefit from the intervention, others do not. Such heterogeneity emphasises the importance of conducting a treatment allocation analysis to investigate potential improvements in the allocations of individuals.
Table (ref) shows that simple allocation schemes may improve the overall survival rate while decreasing the number of surgeries at the same time. Again, the first two rows of Table (ref) present results using the empirically observed and random assignment of individuals to treatments, which are used as baseline comparison policies. The remaining rows show the results of Policy Trees aiming to optimise the treatment allocation and increase the survival rate. Given the relatively small sample size, this study focuses on optimal trees of depths two and three, alongside Policy Trees with depths 3 and 4, including a final sequential optimal sub-tree (+1).
Table (ref) first reveals that the observed treatment allocation outperforms a random treatment assignment in terms of the survival rate (68.23% vs. 67.88%). Considering the performance of the Policy Trees, the findings suggest treatment allocations that could further increase the average survival rate within six months after the intervention up to 69.99% while significantly diminishing the number of individuals receiving surgery by more than 50%. The maximum welfare gain, among the Policy Trees estimated, is already achieved with a Policy Tree of depth two. Deeper trees do not lead to a relevant increase in the survival rate. However, they reduce the share of individuals undergoing an operation. For example, compared to the depth-2 tree, the depth-3+1 tree indicates a reduction in survival rate by only 0.02 percentage points (69.99% vs. 69.97%) but decreases the share of patients having surgery by nearly three percentage points (19.21% vs. 16.22%).
Since the maximum welfare gain is already attained with a depth-2 tree, Figure (ref) displays the decision rules established by this tree, which is also the most straightforward to interpret. The Policy Tree performs the first split on age, while the second splitting variable is the estimated probability of surviving within two months, measured before the surgery. The decision tree allocates individuals to surgery with an age below 65 years and a survival probability below 48%, as well as with an age above 65 years and a survival probability below 40%. Note that tables representing all Policy Tree rules used are available in the Appendix.
In 2003, karlan2008credit conducted a randomised controlled trial to examine the impact of personalised pricing by a microfinance lender in South Africa. They randomised interest rates to study the effect on the decision to borrow (extensive margin) and, among other outcomes, on the lender's revenues (intensive margin). Considering the intensive margin, the overall findings suggest that lower interest rates only marginally decrease profits.
This study is particularly relevant because microfinance initiatives may serve broader purposes beyond profit maximisation. They can play a crucial role in promoting financial inclusion among specific population subgroups. Moreover, any initial losses incurred by the lender could be offset by a wider uptake of microfinance services.
Therefore, this study employs the same data and the intention-to-treat strategy of karlan2008credit to investigate the impact of receiving a mail invitation with different interest rates on the lender's returns. kallus2021fairness also leverages this dataset to propose policy allocations aimed at maximising the lender's revenue while considering fairness implications. This paper does not explicitly take a stance on fairness. Consequently, we include variables commonly regarded as sensitive in the literature.
The dataset includes 48,852 individuals who received an invitation letter to apply for a loan at the offered interest rate. Differently from the study of karlan2008credit, our specification incorporates socio-demographic information (see Table (ref) in the Appendix for the complete list) collected from the bank operator before the randomisation. While not crucial for treatment effects identification, these variables are important for exploring heterogeneity and conducting policy learning analysis. In addition, we discretise the treatment variable in three different interest rate groups: low, medium and high rates, based on the client's risk profile. The cut-offs are indicated in karlan2008credit and are based on clients' risk category pre-approved by the lender. The discretisation into three groups enhances the interpretability of results from a policy perspective, particularly when employing Policy Trees.
As for the other applications, our heterogeneity analysis is based on the distribution of the IATEs. Specifically, we compare the medium and high-interest rates treatment groups with respect to the low-rate group. As illustrated in Figure (ref), both distributions show positive and negative treatment effects. Looking in more detail at how the medium (high) interest rate group compares to the low-rate group, the effects range from South African Rand R1,520 (R1,656) to R1,610 (R2,327), with an average effect of R78 (R163) and an average standard error of R506 (R487). These findings indicate the existence of effect heterogeneity, which could be used to potentially enhance the allocation mechanism.
As for the previous studies, we use Policy Trees of different depths and compare the welfare produced by the different allocations with the welfare from random and observed allocations. Due to this application's relatively larger sample size, we also demonstrate the performance of both shallow and deeper trees. Additionally, we explore various combinations of trees: optimal, optimal with one final sequential optimal tree (+1), and optimal with two final sequential optimal trees (+2).
Table (ref) indicates that the observed and random allocations lead to the same welfare since the observed allocations are based on a stratified random assignment. The Table also shows that the Policy Trees increase welfare compared to the baseline policies, irrespective of the tree depth. Even implementing a Policy Tree of depth two demonstrates an increase in overall welfare by 31%, and deeper trees continue to yield improvements, although at a decreasing rate. The allocation pattern of the optimal Policy Trees shows that deeper trees tend to shift individuals away from the high-rate to a lower-risk group while increasing the average returns at the same time. The welfare estimated with the mixed trees (one or two sequential optimal final trees) is slightly lower than those obtained by the fully optimal tree. For example, the Optimal Policy Tree of depth four generates a welfare of R701, higher than R686 (R685) yielded by the depth-3+1 (depth 2+2) mixed Policy Tree.
Figure (ref) shows the allocation rule for an optimal Policy Tree of depth 4, which combines both a large welfare improvement and policy rules that are relatively easy to interpret. As indicated in Table (ref), the Policy Tree assigns roughly half of the sample to the high-rate treatment. Among this subgroup, the highest share (16% of the overall sample) falls between the ages of 35 and 43 and lacks higher education. Low-income individuals older than 53 also constitute a significant portion of the high-risk cohort (12% of the whole sample). The highest share of individuals assigned to the medium interest rate (14% of the sample) range from 28 to 32 years of age. The largest subgroup assigned to the low-rate group consists of high-income individuals older than 50 years (4.6% of the sample).
For detailed information on the treatment allocation rules for the other Policy Trees, we direct the reader to Section (ref) of the Appendix.
An optimal tree of arbitrary depth is computationally challenging due to its NP-hard nature. However, specifying a particular depth allows for polynomial-time resolution, making the depth of the tree a critical complexity parameter \citep*{Sverdrup2020}. Typically, estimating optimal trees beyond a depth of 4 (16 leaves) becomes challenging with moderate-sized training data. To improve computational performance, one option is to reduce the depth, sacrificing some efficiency for computational speed (and improved interpretability). Another option is to use sequential optimal trees. This means building a shallow initial tree, followed by a second shallow tree built from the sub-final leaves of the first one. Building the trees in sequence is much faster than constructing a single, deeper tree of the same overall depth. Alternatively, the minimum leaf size is crucial for performance optimisation. Setting leaf sizes too small can be impractical and increase computation times. Additionally, users can enhance speed by reducing the size of the training data at the cost of amplifying sampling noise. Another approach to enhancing speed is to limit the number of values considered during splitting. For continuous variables, the mcf algorithm evaluates by default only 100 equally spaced values, while it assesses 100 random combinations for categorical variables. Although this approximation improves computational efficiency, it may lead to some loss in accuracy, particularly as the data in the leaf nodes diminishes.
Furthermore, like predictive trees, policy trees can also exhibit instability regarding policy rules or welfare estimation. However, how to conduct inference about the stability of policy trees remains an open question in the literature. athey2021policy propose a form of cross-validation to evaluate the accuracy of policy learning procedures, and they assess improvements over a random baseline. Similarly, zhou2023offline propose an alternative form of cross-validation and test whether optimal policy rules perform significantly better than sending all units to the same treatment. While the assessment of policy tree stability is not currently tackled in the present paper, future research efforts may concentrate on developing methods to evaluate this aspect effectively.
Despite this, the mcf 's OptimalPolicy() class presents a notable advantage by providing a practical implementation of Policy Trees within multi-action settings, facilitating the creation of explainable assignment rules. Moreover, it offers best-scores and random allocation methods, which can be utilised for comparative welfare estimation. Notably, the integration of this class into the mcf package is advantageous. Indeed, it allows for the combination of causal machine learning estimates produced by the ModifiedCausalForest() as inputs for the policy tree, enhancing decision-making. The mcf package is an open-source Python package available on the Python Package Index (PyPI). The current work is based on version 0.5.1.