Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
44,678 characters · 12 sections · 37 citation commands
Hyperparameter Tuning for Causal Inference with Double Machine Learning: A Simulation Study
Double/Debiased machine learning (DML) is an estimation framework for causal parameters based on ML-estimated nuisance functions that has been established by chernozhukov2018double. DML combines the strengths of machine learning for prediction with estimation and inference of causal parameters. The major idea of the double machine learning framework is to make the estimation framework robust to inherent biases of ML estimation: To address the bias-variance tradeoff, ML methods introduce some regularization. Without any further adaption of the estimation procedure this will effectively translate into a bias of the causal parameter of interest. To overcome these shortcomings, the double machine learning approach combines three key ingredients bach2022, bach2021R: (i) Identification of causal parameters through Neyman-orthogonal moment conditions, (ii) high-quality machine learning estimators and (iii) sample splitting. Incorporating these ingredients makes it possible to establish $\sqrt{N}$ convergence and asymptotic normality of the causal estimator. In recent years, the DML framework has become popular in various disciplines, including econometrics Knaus_2022, reinforcement learning narita2020off and management science schacht2023. Whereas there is an extensive literature and ample benchmark studies on hyperparameter tuning approaches in predictive ML tasks, for example see bischl2021, there is a considerable gap on the interaction between the predictive performance and the quality in terms of causal estimation. Several studies investigate model selection strategies for different estimation approaches that are based on ML\textcolor{black}{, mostly with a focus on heterogeneous treatment effects} schuler2018comparison, mahajan2022empirical.
\textcolor{black}{Previous studies with a specific focus on DML compare different sample splitting schemes okasa2022metalearners or different ML learners knausMC for estimation of heterogeneous treatment effects based on synthetic or semi-synthetic data}. However, the question of how to select learners within the DML framework remains unclear in the existing literature. Few studies address the aspect of hyperparameter tuning in the context of double machine learning. The approach by cdml is based on a target metric that combines the predictive losses from two predictive tasks incorporated in partially linear regression models. chernozhukov2022riesznet provide a multitask framework for neural networks that is based on a general Riesz representation. In this work, we will perform a thorough empirical study to show that hyperparameter tuning is required when using DML in order to achieve optimal inference on the target parameter. We assess different data splitting schemes for the tuning process, draw connections between the predictive performance of the ML learners and the error of the causal estimate and give recommendations on the selection of the causal model.
In DML, the goal is estimation and inference on a target parameter $\theta_0$ in the presence of a high-dimensional nuisance parameter. Importantly, identification in DML is based on an orthogonal moment condition with a score function $\psi(W;\theta,\eta)$, namely
with data $W$, true value of the causal parameter $\theta_0$ is and nuisance function $\eta_0$. The general idea is to use ML methods to estimate each of the nuisance parameters, plug them into the score function and solve for the target parameter. A key condition for valid inference is so-called Neyman-orthogonality
$\partial_\eta$ denotes the pathwise Gateaux derivative operator. Neyman-orthogonality ensures that the moment condition identifying $\theta_0$ is insensitive to small perturbations of the nuisance function $\eta$ around $\eta_0$ under relatively general assumptions. This eliminates the first-order bias which might stem from regularization, when replacing $\eta_0$ by the ML estimate $\hat{\eta}_0$. Formally, the learner requirement applies to the rates of convergence of the employed ML prediction methods. If the used ML estimators do converge sufficiently fast, the bias in $\hat{\eta}_0$ will eventually vanish in the limit, giving rise to a $\sqrt{N}$-consistent and normally distributed estimate $\hat{\theta}_0$. The theoretical framework of chernozhukov2018double specifies structural assumptions that guides the choice of the ML learners. For example, under the assumption of sparsity, $\ell_1$ penalized estimators such as the lasso tibshirani1996 are known to satisfy the rate criterion for estimating $\eta_0$, such that it can be used for causal estimation of $\theta_0$. Generally, the theoretical criterion on the learners refers to the error in the composed nuisance term $\eta_0$, which collects multiple prediction problems in a so-called combined loss, which is specific to a particular causal model and orthogonal score formulation chernozhukov2018double.
\textcolor{black}{How to select among different ML learners for the nuisance part, is key for practical applications and understanding this better has been a main motivation for this study.}\\ To make the role of the learners in DML more tractable, we will briefly introduce two causal models that will also be used in our simulation study: The Partially Linear Regression (PLR) model and the Interactive Regression Model (IRM). Under common assumptions for identification such as unconfoundedness, the PLR can be used to estimate the Average Treatment Effect (ATE) of a binary or continuous treatment variable $D$ on an outcome $Y$. The PLR incorporates a partially linear additive structure, i.e., the causal effect is constant for all individuals
Using the so-called partialling out score, the corresponding nuisance component comprises two functions, $\eta_0 = (\ell_0, m_0) = \left(\mathbb{E}[Y|X], \mathbb{E}[D|X]\right)$. The IRM imposes weaker functional form assumptions and makes it possible to estimate heterogeneous treatment effects. Thus, the nuisance component $g_0$ can also model interactions between $D$ and $Y$.
Using the doubly robust score which identifies the ATE of a binary treatment variable $D$ on $Y$, the nuisance parameter comprises two functions $\hat{\eta}_0 = (g_0, m_0) = \left(\mathbb{E}[Y|D,X], \mathbb{E}[D|X]\right)$. \textcolor{black}{Double machine learning is closely related to “double robustness” and DML-based estimators build on doubly robust scores (Robins1994) and their augmented inverse probability weighting estimators. Doubly robust estimators, which rely on two nuisance parameters (outcome regression and propensity score), are robust against misspecification of one of the nuisance parameters. Hence, whenever it is possible to estimate one nuisance component at the parametric rate, DR estimators tolerate slower rates for the other nuisance component. DML generalizes this idea to a broader class of causal models and formulates the rate requirement in terms of a model-specific combined loss. For the PLR and IRM, their definition is provided in Table (ref)}. For a detailed discussion we refer to Knaus_2022 and Smucler2019. Finally, we would like to note that DML is also closely related to targeted maximum likelihood learning vanderLaanRubin+2006.\\ Figure (ref) serves as a simplified illustration of the role of an appropriate hyperparameter choice for the causal estimation quality. It shows the mean squared error with respect to the true causal parameter $\theta_0$ in Panel (a), \textcolor{black}{the empirical coverage in Panel (b) and the combined loss in the PLR in Panel (c) as obtained for a grid of lasso-penalty values}. The example is based on a high-dimensional sparse data generating process with $n=100$ observations and $p=200$ explanatory variables as presented in belloni2014, which we refer to as BCH. In a PLR model, the lasso can be used as an appropriate learner for the sparse nuisance relationships $\eta_0 = (\ell_0, m_0)$. The surface plots illustrate that an appropriate choice of the hyperparameter, i.e., the lasso penalty $\lambda$, is essential for a precise estimator of $\theta_0$. The figures show that it is important to have suitable estimation performance in both nuisance components $\ell_0$ and $m_0$ in order to obtain a low mean squared error for the causal parameter as well as to achieve the nominal coverage of the confidence intervals. \textcolor{black}{In line with the theoretical foundation of the DML approach, Panel (c) of Figure (ref) reveals that a lower combined loss for the nuisance component corresponds to a better estimation performance and higher coverage in Panel (a) and (b). An additional motivating example that highlights the importance of an appropriate hyperparameter choice in terms of the empirical distribution of the DML estimator is provided in Appendix (ref).}
In addition to using an orthogonal score function for identification and appropriate ML estimation of the nuisance part, DML incorporates sample-splitting as a third ingredient. Splitting the sample in different partitions makes it possible to abstract from (small) overfitting biases. Sample splitting guarantees the independence of the samples that are (i) used to learn the nuisance functions $\eta_0$ (training data) and (ii) used to solve the orthogonal score with regard to the causal parameter $\theta_0$ (holdout data). Cross-fitting is an efficient form of data splitting, that uses all the available data by swapping the roles of the training and holdout data in a cross-validated manner. Cross-fitting proceeds as chernozhukov2018double: (1) Take a $K$-fold random partition $(I_k)_{k=1}^K$ of observation indices $\{1,\dots,N\}$ such that the size of each fold $I_k$ is $n=N/K$. For each $\{1,\dots,K\}$, define $I^{\textrm{C}}_k\coloneqq\{1,\dots,N\}\setminus I_k$. (2) For each $k \in K$, construct an ML estimator $\hat{\eta}_{0,k}$ based on the observations $i \in I^{\textrm{C}}_k$. (3) For each $k \in K$, construct the estimator $\hat{\theta}_0$ using the empirical expectation of the estimator using the observations $i \in I_k$ and average over the $K$ folds. This procedure refers to fitting the nuisance learners and does not explicitly address hyperparameter tuning and similar considerations. Sample splitting and resampling is crucial to avoid overfitting in predictive machine learning bischl2023hyperparameter. There are different ways to combine the DML cross-fitting procedure with ML tuning cross-validation schemes in practice. In our simulation study, we investigate three sample splitting approaches when tuning the nuisance estimators, which are illustrated in Figure (ref). Full sample: We apply cross-validated tuning on the whole sample and find the best learner and the best hyperparameter configuration by minimizing a loss target for each of the nuisance prediction tasks. Causal estimation by DML is also performed on all observations according to the cross-fitting algorithm, using the same learner and set of hyperparameters on all folds. The splits in the cross-validation scheme for tuning and the splits in the DML cross-fitting algorithms are independent of each other. Split sample: Splitting the data into a tuning and an inference fold is frequently employed in predictive machine learning. We adapt it by splitting the sample in half and tune the hyperparameters using cross-validation on $50\%$ of the data only. We choose the optimal learner and hyperparameter set and use them for fitting the nuisance functions in the remaining $50\%$ observations using DML cross-fitting. In this schedule, the samples used for tuning and causal estimation do not overlap. Hence, fitting the ML learners only uses data that has not been used for tuning. On folds: We perform sample splitting as in the cross-fitting algorithm and perform cross-validated hyperparameter tuning within each of the cross-fitting folds: We take a $K$-fold random partition $(I_k)_{k=1}^K$ of observation indices $[N] =\{1,\dots,N\}$ such that the size of each fold $I_k$ is $n=N/K$. For each $k \in [K]$, we perform cross-validation based hyperparameter optimization with the observations $i \notin I_k$ directly on the same partition (or fold) which is used for predicting the nuisance components. This approach takes $K$-times the calculation power of the other two. \textcolor{black}{With our empirical results, we would like to investigate how the cross-fitting scheme of DML and the cross-validation splitting schemes for hyperparameter tuning interact, for example, comparing the split sample approach with non-overlapping subsamples to the overlapping, but independent sample splits of the full sample and on folds approach.}
The empirical assessment of causal estimators is generally limited by the fact that the ground truth (i.e., the true value of the causal parameter $\theta_0$) is not observable in real data sets. Hence, empirical studies have to rely on simulated data sets. To investigate the role of different sample splitting schemes, the choice of learners and hyperparameters as well as the scope to which predictive metrics can be used to motivate the choice of a causal model (PLR vs. IRM), we use two sources of data. Our major analysis will be based on data originally introduced for the 2019 Atlantic Causal Inference Conference (ACIC) data challenge.\footnote{For more info, see \url{https://sites.google.com/view/acic2019datachallenge/data-challenge}.} We choose this data \textcolor{black}{source} because it \textcolor{black}{generates designs with common and relatively complex empirical patterns} under unconfoundedness.\footnote{There are some scenarios, for which the unconfoundness assumption will not hold exactly, see Table (ref).} Moreover, we focused on an external resource for the DGP as this helps to abstract from biases that might stem from using self-defined simulation scenarios. To gain more detailed insights, we used the BCH DGP by belloni2014 as a second source. The advantage of this design is its clear structure, which makes it possible to match the empirical results with general theoretical reasoning.
The ACIC data consist of $1600$ data sets from $16$ data generating processes (DGP) with $n=1000$ or $n=2000$ observations \textcolor{black}{and} $p=200$ covariates, a binary treatment variable and a continuous outcome. Table (ref) contains a brief overview of the processes. A more detailed description can be found in Appendix (ref). The data generating processes are mostly in line with the unconfoundedness or conditional exogeneity assumption, $Y(d)\perp D \mid X$, with $Y(d)$ indicating the potential outcome under treatment $D=d$. Hence, these settings allow for identification of the average treatment effect, $\theta_0 = \mathbb{E}[Y(d=1)-Y(d=0)]$. Overall, the summary in Table (ref) makes clear that the data generating processes of the ACIC challenge are very disparate, including settings with homogeneous and heterogeneous treatment effects, sparse and non-sparse, linear and non-linear settings, different values for the signal-to-noise ratio and varying values for the ATE across settings. In DGP $5$ and $6$, the unconfoundedness assumption is violated, which complicates estimation of the ATE. The information on the effect of $D$ on $Y$ is informative for the assumptions in the causal models introduced in Section (ref). In cases of a linear and additive treatment effect of $D$ on $Y$, the assumptions of the PLR are satisfied, such that it should allow for consistent estimation of $\theta_0$. In case of interactions of $D$ with other covariates, $X$, the functional form assumption of the PLR is violated, such that the resulting estimator is likely to be affected by a misspecification bias. The IRM, however, is more general, such that it should basically be suitable for causal estimation of $\theta_0$ in both cases. As the PLR incorporates more structure, it is expected to lead to more efficient estimation in cases the underlying assumptions hold in the data generating process. \\ We study hyperparameter optimization with four different learners, including lasso, random forest breiman2001, extreme gradient boosting xgboost and the AutoML learning framework FLAML wang2019flaml. \textcolor{black}{We focus on these learners as they are widely used in empirical research and industry practice. Moreover, the learners differ substantially in terms of the modeling assumptions and the way they introduce regularization to avoid overfitting. For example, the linear lasso learner uses an explicit penalty, whereas the nonlinear random forests and gradient boosting learners are based on ensembles of tree learners. As a result, these learners have different hyperparameters, which in turn affect the estimation performance in a different way probst2019tunability. For random forests\footnote{For random forests, 100 trees were chosen for computational reasons. However, the performance did not significantly alter in trials with 300 or 500 trees.} and gradient boosting, the hyperparameter choice is based on cross-validated search over a grid of values presented in Table (ref). We use cross-validated lasso with the built-in penalty selection and the automated search for FLAML with a time budget of 60 seconds. As a tuning-free, but linear and low-dimensional benchmark, we compare our results to causal models that are based on linear and logistic regression.}
In this section we present results on different aggregation levels. Table (ref) in Appendix (ref) presents an overview on the different metrics\footnote{Full results and replication code are available at \url{https://github.com/DoubleML/DML-Hyperparameter-Tuning-Replication}.}.
\textcolor{black}{Tuning on the full sample or on folds exhibit similar performance, which is superior to the split sample approach in small samples.} The first set of our results refer to the role of the sample splitting scheme that is used for hyperparameter optimization as described in Section (ref). Figure (ref) provides a summary of our results across all ACIC DGPs as obtained from 100 repetitions of every DGP. It presents an evaluation of the performance for different learners and sampling schemes based on a selection of the causal model (PLR, IRM) according to the underlying assumptions of the DGPs presented in Table (ref).\footnote{If $\theta_0$ is additively separable, i.e., constant for all individuals, we use the PLR and the IRM otherwise. We select the results according to the causal model in order to abstract from model misspecification considerations.} \footnote{\textcolor{black}{Due to the considerable differences in the simulation designs, we would like to note that Figure (ref) is not entirely able to present a comprehensive picture of our extensive simulation results. We would like to refer readers to the more detailed results for every learner, sample splitting scheme and each causal model for each DGP in Appendix (ref) and (ref).}} Panel (a) in Figure (ref) shows the average root mean squared error relative to an oracle estimator for each of the employed learners and each sampling scheme as averaged over all DGPs. Panel (b) shows a boxplot of the bias of the corresponding causal estimates and the empirical coverage of $95\%$ confidence intervals is depicted in Panel (c). The results reflect the main empirical finding that the full sample and on folds scheme perform similarly well. The only exception from this is the result on the AutoML learner, for which the adjusted tuning time for the on-folds scheme might not have been sufficient. The causal estimates obtained from the split sample approach are on average less accurate (in terms of the RMSE relative to an oracle) and more variable (cf. the boxplots in Panel (b)). This observation holds for virtually all learners and simulation settings considered as can be recognized from the Figures in (ref) and (ref) as well. We interpret these results as evidence of a considerable efficiency loss in finite samples by using only $50\%$ of the observations for hyperparameter tuning. The empirical coverage of the confidence intervals are below the nominal level, which mainly reflects the differences in the underlying simulation settings. Due to the different structural assumptions across DGPs, it is unlikely that one learner will perform well in all settings.\\ \textcolor{black}{The efficiency loss of the split sample approach vanishes with increasing sample size.} We investigate to what extent the efficiency loss of the split sample approach persists with increasing samples size in Figure (ref), which is based on data from the BCH DGP with $p=200$ explanatory variables. In line with our previously reported results, the split sample scheme provides a higher RMSE on the causal parameter than the other two options. However, with growing number of observations, we observe that the RMSE of all tuning approaches converge with $\sqrt{n}$-rate. Thus, the difference between the schemes diminishes for larger samples.
\textcolor{black}{The performance of the causal estimation depends on structural assumptions and appropriate learner choice.} Comparing the four learners in the study, overall we find that the tuned AutoML and lasso learners perform very well across multiple settings (see Appendix (ref) and (ref)). In line with the theoretical formulation of the learner criterion in the DML approach, we observe that the causal estimation performance depends on the validity of structural form assumptions such as sparsity and linearity. For example, in linear settings (i.e., DGP 1, 3 or 16) lasso performs well in terms of the rRMSE for the causal estimate as well as the empirical coverage. We believe that the reason why the AutoML learner performs well is its ability to adapt to various settings.\\ \textcolor{black}{A lower combined nuisance loss is associated with better causal estimation. Low signal-to-noise ratios create challenges in small samples.} In our simulation study, we investigate the relationship of the predictive performance for the nuisance functions, $\eta_0$, and the quality of the resulting causal estimate. A practical strategy for selecting the learners which would be aligned with the theoretical reasoning would be to pick the learner with the smallest out-of-sample loss for the nuisance predictions. In our simulations, however, we noticed that greedily minimizing the combined loss might not always deliver the lowest error (bias) on the causal parameter. \textcolor{black}{We provide more insights on this in Appendix (ref).}\\ As we compare tuned algorithms, for most DGPs we only observe an area of combined loss, where the fit is sufficient for a good causal inference performance. Still, considering the plots in Appendix (ref), it is valid to say that the possibility of a high bias decreases with a low combined loss. Considering the different scenarios of the ACIC data, it is important to relate the predictive performance for the nuisance function, $\eta_0$, to the noise in the prediction problems. Whenever it is possible to approximate $\eta_0$ relatively well, it is possible to obtain an accurate causal estimator. However, the ability to find such a predictive model depends on the noisiness of the incorporated predictive tasks, see the information in Table (ref). This is further highlighted by Figure (ref), which illustrates the relationship of the average bias and the combined predictive loss for increasing samples based on the BCH DGP: The larger the $n/p$-ratio, the higher is the predictive quality of the nuisance estimators in terms of the combined loss. The scatter plots also include the comparison to the default settings for the parameters of lasso and xgboost, which basically implement untuned learners. With increasing sample size, it is easier to detect the default parameter learners by considering their higher combined nuisance loss. It can be observed that the estimation quality with respect to $\theta_0$ is worse for these learners. The results for the tuned estimators show a similar performance in terms of the combined nuisance loss as well as the \textcolor{black}{bias of the causal estimator}.\\ \textcolor{black}{Combined nuisance loss serves as a good learner selection metric for causal estimation.} We briefly evaluate practical selection strategies for the employed learners based on the ACIC data. To do so, we focus on the full sample scheme and pick the causal model with the lowest predictive loss on $Y$ (see Section (ref)). We investigate the following approaches: (1) Pick the single best learner for each data set based on the best predictive performance for $Y$, (2) Pick the single best learner for each iteration based on the combined out-of-sample nuisance loss, (3) Stack all tuned learners. We present average results for the rRMSE as well as a boxplot for the bias as aggregated over all DGP in Figure (ref). It is visible that using the learner with the lowest combined loss leads to the best overall rRMSE (mean). The strategies that select using the predictive error on $Y$ or always using FLAML appear to perform also well, especially concerning the median rRMSE. The performance of basic stacking wolpert1992 is overall competitive in terms of the median rRMSE, but is affected by a weak performance of random forests in several scenarios (high mean rRMSE). \textcolor{black}{The linear/logistic benchmark model exhibits an overall inferior performance as compared most other learners.}
In this section we assess the relationship of the structural assumptions encoded in the causal models (IRM and PLR) and the predictive performance of these models.\\ The performance of the causal estimators depends on the structural assumptions encoded in causal models. As described in Section (ref), the PLR model incorporates stronger structural assumptions than the IRM. Hence, in cases where these assumptions hold in the underlying DGP, we expect the PLR to benefit from that additional structure in terms of more precise estimation performance as compared to IRM estimates. However, in cases with a non-additive heterogeneous causal effect, the PLR estimate on $\theta_0$ will be biased due to model misspecification. For our investigation, we split the DGPs into two categories: DGPs with a linear, additive effect and in DGPs with a heterogenous effect.\footnote{We drop some DGPs, in which we observe both models struggle with the estimation, i.e. because model assumptions (conditional exogeneity) fail.}
The results in Figure (ref) are in line with our expectations, i.e., in linear additive DGPs, the PLR performs better than the IRM as it exhibits a lower rRMSE on average. Figure (ref) illustrates that violations of these assumptions lead to a bias and, thus, less precise estimation of the ATE with the PLR. In contrast, the IRM is found to still accurately estimate $\theta_0$. These results hold generally for all considered learners.\\ \textcolor{black}{The predictive performance of the causal models provides guidance on the choice of the causal model.} In practice, researchers are typically concerned with the question of which causal model to choose when multiple models are basically applicable, i.e., to what extent one is willing to make the assumption of a constant treatment effect. We investigate whether the predictive performance of the causal models with regard to the outcome variable $Y$ might be informative for that choice. To do so, we first estimate the causal model and then, generate predictions for the outcome variable $Y$ as based on the PLR model (i.e., plugging in the observed values for $D$ which are multiplied by the estimate $\hat{\theta_0}$) and the IRM (using predictions as generated by $g(D,X)$), see Table (ref). The results are displayed in Figure (ref). We find that in many of the settings, the relative advantage in the error predicting $Y$ is associated with a better estimation performance for the causal parameter $\theta_0$. However, as displayed in Table (ref) presented in Appendix (ref), this measure does not always prove reliable. In the group of linear, additive DGPs, only in $33.42\%$ of the iterations, the predictive loss using the PLR was lower, although it would yield the lower rRMSE on $\theta$ in overall nearly $60\%$ . In terms of the IRM, this is a bit more straight forward: In $75.65\%$ of the cases, the predictive loss accurately points towards the better causal model to use (see Table (ref)). \textcolor{black}{These results can be interpreted to reflect a conservative modeling approach. The prediction-based selection of the causal rather tends to recommend the more general IRM unless there is stronger evidence for validity of the PLR assumptions.}
We apply the tuning methodology to the infant health and development program (IHDP) semi-synthetic data, which was previously used as a benchmark of causal ML methods in chernozhukov2022riesznet and shi2019adapting, see Appendix (ref) for more information. We use the IRM model and the full sample tuning approach, as well as FLAML, as this combination has performed well in our ACIC analysis. The results are displayed in Table (ref). It is visible that the standard DML approach is very competitive with other state-of-the art appraoches such as RieszNets and DragonNet of shi2019adapting.
Our results underscore the importance of proper hyperparameter tuning and selection of the ML estimators \textcolor{black}{and the proper choice of DML related parameters}. Particularly, the use of default parameter estimators can lead to significant bias in the causal estimate. Hence, we would like to encourage analysts working with causal and double machine learning approaches to transparently communicate their tuning strategies as well as information on the nuisance prediction loss, \textcolor{black}{in particular when also employing several ML learners for nuisance estimation}. Our results show that in general, tuning on the full data or on the folds is preferable over splitting the sample. In larger samples the difference of these approaches is found to vanish. Moreover, the choice between PLR and IRM was crucial in our results to achieve a low causal estimation error. To find the proper causal model, monitoring the predictive performance of the nuisance learners on $Y$ can serve as a helpful guidance for the choice of the causal model. Finally, we generally find evidence supporting a relationship between the out-of-sample prediction error in the nuisance components and the error on the target parameter. This is in line with theoretic results. However, our results also suggest the measure of combined loss can be noisy, especially when dealing with small sample sizes and low signal-to-noise ratios. In our study, the lowest nuisance prediction error did not always translate in the optimal ML configuration for DML. As guidance for empirical applications is crucial, we would like to analyse the use of neural networks klaassen2024doublemldeep, more advanced stacking algorithms ahrens2022pystacked, van2007super, more elaborate tuning approaches, and extend the simulation framework to conditional average treatment effects (CATE).
\acks{We would like to thank the organizers of the 2019 ACIC data challenge, namely Susan Gruber, for sharing the data generating processes as well as the results of the competing teams. Our gratitude goes to the participants of the 2022 Causal Data Science Meeting as well as the 2023 YES Causal Inference Workshop for their feedback on the presentation of this paper.}