Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
30,572 characters · 19 sections · 16 citation commands
Double Machine Learning at Scale to Predict Causal Impact of Customer Actions
\authorrunning{S. More et al.}
Causal Impact (CI) is a measure of the incremental change in a customer's outcomes (usually spend or profit) from a customer event or action (e.g, signing up for a paid membership). CI values are used across the industry as important signals of long-term value for multiple decisions, such as marketing content ranking to long-term investment decisions.
Business teams are typically interested in calculating CI values for relevant actions that a customer participates in. Some examples include customer actions such as `first purchase in category $X$', `first $Y$ stream', or `sign up for program $Z$'\footnote{We use placeholder $X$,$Y$,$Z$ to maintain business confidentiality}. The CI values are leveraged by partner teams to understand and improve the value they generate. For many of these customer actions, we are unable to conduct A/B experiments due to practical or legal constraints. CI values are thus estimated off of observational data, effectively leveraging rich customer data to isolate causal relationships in the absence of a randomized experiment.
In this paper, we provide results for average treatment effects and conditional average treatment effects (i.e., customer-level CI values) estimated using a variant on the Double Machine Learning (DML) methodology Chernozukov. The paper is arranged as follows. In Sec. (ref), we give a brief overview of the use and scale of CI in the industry. In Sec. (ref), we introduce the traditional system used for calculating CI values. We also discuss the shortcomings of the traditional method and the advantages of moving to a DML-based method for calculating CI.
Sec. (ref) covers the details of our DML implementation for calculating CI values. Our contributions include improving the robustness of CI estimates through inverse propensity weighting, adding the ability to produce heterogeneous CI values, implementing customer-level confidence intervals with various assumptions, and making available the JSON Machine Learning interface to accelerate experimentation. We present results in Sec. (ref) for a few customer actions and conclude with the takeaways and ideas for future work in Sec. (ref).
Causal impact estimation drives a large number of business decisions across industry. This includes multiple organizations such as retail, search, devices, streaming services, and operations. To this end, most companies have invested in developing and deploying models that vend CI values for the cutsomer actions under consideration. In the next section, we give an overview of the traditional potential-outcome based model which is widely used in the industry for CI estimation. This will be the baseline model.
CI framework applies the principles of observational causal inference. We rely on it because A/B testing is not possible to evaluate the impact of certain treatments due to practical constraints (e.g., the treatment is not effectively assignable, or would be too expensive to assign at scale). Observational causal inference methods rely on eliminating potential confounders through adjustment on observed variables. Under a "selection on observables" assumption, we believe we can estimate the causal effect correctly on average. Applied to the customer's next 365 days of spending, for example, the CI value represents the incremental spending that a customer makes because of participating in a certain action compared to the counterfactual case where they didn’t take that particular action. The formal framework for this kind of counterfactual reasoning is the "potential outcomes" framework, sometimes known as the Neyman-Rubin causal framework Holland, Neyman, Rubin.
The potential outcome based CI model has two parts:
In addition, we require the CI model to be able to scale to the business use case. For instance, we may want generate CI values for hundreds of customer actions in an automated way. In the rest of the paper, we refer to the traditional potential outcome based CI model as “CI-PO” and the DML-based model as “CI-DML”.
Note that one of the challenges in validating the causal estimates is posed by the Fundamental Problem of Causal Inference Jasjeet. The lack of observable ground truth makes it difficult to validate the output of a causal model. We therefore prefer models with proven desirable theoretical properties such as $\sqrt{n}$ convergence.
The Double/Debiased Machine learning (DML) method proposed by Chernozhukov et al. Chernozukov leverages the predictive power of modern Machine Learning (ML) methods in a principled causal estimation framework that is free of regularization bias asymptotically.
For treatment $D$, features $X$, we express the outcome $Y$ as an additively separable function of $D$ and arbitrary function of features $X$:
DML’s estimation strategy is motivated by writing out the residualized representation of Eq. (ref) and its parts:
We use ML models to estimate $E(Y|X)$ and $E(D|X)$. The residuals from outcome equation (Eq. (ref)) are regressed on residuals from propensity equation (Eq. (ref)) to obtain the causal parameter $\beta$. We use ML models to predict $E(Y|X)$ and $E(D|X)$. We leverage K-fold sample splitting so that training and scoring of the ML models happens on different folds. We use a 3-fold sample split and follow the “DML2” approach Chernozukov where we pool the residuals outcome and propensity residuals across all the folds to fit a single, final regression of the residualized outcome on the residualized treatment (Eq. (ref)).
We use a weighted ordinary least squares to solve the residual regression equation (Eq. (ref)). where the weights are determined by the Inverse Propensity Treatment Weighting (IPTW or IPW) biometric. Our IPTW weights correspond to the Horvitz-Thompson (HT) weight Ruth, in which the weight for each unit is the inverse of the probability of that unit being assigned to the observed group. In Table (ref) we define the weights that balance the distributions of covariates between comparison groups for two widely used estimands, the Average Treatment Effect (ATE) and Average Treatment Effect on the Treated (ATT). Weighting helps achieve additional robustness, bringing us closer to a conventional Doubly Robust estimator. Applying these weights when conducting statistical tests or regression models helps reduce impact of confounders over and above what we get from the regression adjustment Austin. Secondly, the weights allow us to target the estimand; the ATT has long been the preferred estimand for CI values, since it represents the treatment effects for those customers actually treated historically. (We refer to the customer-level counterparts of ATE, ATT estimands as HTE and HTT respectively.)
For many treatments, propensity distribution has significant mass near 1 for the treated group and near 0 for the control group (see an example histogram in Fig. (ref)). Scores near the boundary can create instability in weighting methods. In addition, these scores often represent units for whom we cannot make an adequate treatment-control comparison. We limit analysis to the common support region, where propensity score distributions overlap between treated and untreated samples.
We also use trimming to exclude customers whose estimated propensity is outside of the range $[\alpha, 1-\alpha]$. We experimented with different thresholds on various customer actions and observed that $\alpha=0.001$ with rescaled propensity scores works the best.
When using the IPW, we normalize the weights by rescaling the propensity scores for each customer $i$ as in Eq. (ref).
$\overline{D}$ and $\overline{\hat{e}(X)}$ in Eq. (ref) are the averages of treatment assignment and propensity score respectively taken over both the treatment and control population combined. Propensity trimming and rescaling reduces variance, leads to more stable estimates, and tighter confidence intervals as seen in Fig. (ref).
CI-DML implements a version of the heterogenous effects modeling proposed in Cherno_goldman_semenova_taddy, by leveraging the treatment-feature interactions in the final stage of DML to identify heterogenous (customer-level) responses. The general form of Eq. (ref) can be written as
In fact, Eq. (ref) is a special case of Eq. (ref) with $h(X, \tilde{D}) = \tilde{D} \beta$. We interact treatment with the features and define $h(X, \tilde{D}) \equiv \psi(X) * \tilde{D} \, \beta$, where `$\ast$' represents element-wise multiplication. Thus, the heterogeneous residual regression becomes:
We want $\psi(X)$ to be low-dimensional so that we are able to extract the coefficient $\beta$ in Eq. (ref) reliably.
Let $N$ and $M$ be the number of customers and features respectively. If the dimension of $\psi(X)$ is $N\times K$, we want $K \ll M$. In our use case, $M$ is typically 2000 and $K$ is typically around 20. To get the low-dimensional representation, $\psi(X)$ we proceed as follows:
Once we have calculated the distance features $\psi(X)$ for each customer, we interact them with the propensity residuals and fit a linear regression model using IPW (refer sec. (ref)) to extract the coefficients $\beta$ in Eq. (ref). The heterogenous estimates are given by
E.g., for $K=3$, $h = \psi_1 \beta_1 +\psi_2 \beta_2 + \psi_3 \beta_3$.
A schematic of CI-DML workflow is shown in Fig. (ref).
One of the disadvantages of CI-PO is that generating confidence intervals requires bootstrapping around the multi-step process and is computationally expensive. Obtaining confidence intervals in CI-DML is straightforward. For a single ATT parameter estimate, we obtain the confidence interval simply by calculating the variance of the estimate of $\beta$ in Eq. (ref). We also estimate Huber-White heteroscedasticity consistent standard errors Huber, White. For the ATT case, the steps for calculating variance of the coefficient $\hat{\beta}$ are as follows:
For a customer, `$p$':
where $\hat{\beta}$ is the value of coefficient from solving Eq. (ref). Note that $\tilde{Y}_p$ and $\tilde{D}_p$ are scalars. $\Sigma$ in Eq. (ref) is a diagonal matrix with the squared prediction error $\hat{\sigma}_p^2$ for each customer on its diagonal and $H$ in Eq. (ref) is defined as
where $W$ are the IPW weights as defined in Sec. (ref).
We compute the confidence intervals on the causal estimate $\beta$ using $Var(\beta)$.
CI-DML also provides the ability to obtain customer-level confidence intervals. From Eq. (ref), we can write
We calculate variance of the heterogeneous coefficients following similar approach as in Eqs. (ref), (ref), and (ref). The only difference is we replace $\tilde{D}_p \rightarrow \psi(X_p) * \tilde{D}_p$ in Eq. (ref) and $\tilde{D} \rightarrow \psi(X)*\tilde{D}$ in Eq. (ref).
For the ATT case, $Var(\beta)$ is a scalar whereas for the HTT case, $Var(\beta)$ is a $K\times K$ matrix. The first and second terms in the summation in Eq. (ref) are the diagonal and the off-diagonal terms of the $Var(\beta)$ matrix respectively.
We developed a causal ML library with JSON driven modeling configuration (see Fig. (ref)). JSON ML Interpreter (JMI) translates JSON configuration to executable Python ML application.
The main advantages of JMI approach are:\\ Flexibility: Business questions from various domains cannot always be addressed through a single unified configuration of a causal model. We address this in CI-DML where users can invoke different causal analysis frameworks (DML, Causal Forests) and prediction algorithm type (regression, classification, clustering). \\ Scalability: CI-DML utilizes distributed implementation of algorithms and file system via Apache Spark which helps causal modeling at the big-data scale (100 millions customers, multiple targets, and time horizons) \\ Persistence: CI-DML inherits SparkML serialization and deserialization methods to persist and instantiate fitted models. \\ Compatibility: In addition to Spark, interfaces to adapt ML libraries from scikit-learn, tensorflow, MXNet and other communities can be onboarded using the configurable abstraction support by JMI.
In the CI-DML system, we dockerize the JMI Causal ML library which is platform agnostic which has the flexibility to extend and utilize different compute engines like AWS EMR, Sagemaker or AWS Batch based on the use case and will abstract the computation information from the user. Dockerization also helps version control and the build environment via standard software development tooling.
Next we present the results for CI-DML. The target variables we focus on is customer spending, but our framework on can be leveraged to obtain the causal impact on any other target variable of interest (e.g., net profit, units bought etc). For every CI run, we produce both the population-level ATT values (Eq. (ref)) and the customer-level HTT (Eq. (ref)) values.
We compared the CI-PO and CI-DML results for 100+ customer actions. As noted earlier, two major advantages of CI-DML are the availability of customer grain results (aka. HTT) and confidence intervals. In Fig. (ref), we present population-level (ATT) and customer-level values for selected representative customer actions \footnote{We anonymize actions to preserve business confidentiality}. The reported confidence intervals are for both homoscedastic and heteroscedastic error variances. To get a sense of the level of variance in customer-grain results, we report the percentage of customers where the customer-level confidence interval crosses zero. We also report the out-of-sample fit metrics for outcome and propensity model in DML.
Takeaways from analysis of 100+ actions are:
The hyperparameters (e.g., regularization strength) in the outcome and propensity model are chosen based on the out-of-sample performance. For the HTT estimates, the two main hyperparameters are the number of principal components and the number of clusters.
We choose the number of principal components (PC) based on the percentage of variance explained. We find that around 300 PC, about $80\%$ of the variance is explained (Fig. (ref)). The amount of variance explained grows much slowly as we add more number of PC. To avoid sparsity issues in the downstream K-means calculation, we choose the number of PC components to be 300.
Choosing the number of clusters is less straightforward. Standard tools such as elbow method and Silhouette score do not yield a clear answer for the optimal cluster choice. In the current work, we choose 20 clusters as we think its best suited for our data. One of our ongoing work, is to make this choice in a more data-driven way. We also find that the mean of HTT values is robust with respect to the choice for number of clusters.
So far, we have only looked at the mean of customer-level values in Fig (ref). In Fig. (ref), we look at the customer-level CI scores for a representative customer action. We see that most of the customers have CI value close to the average HTT value. We see that there are few customers with a low CI value which shifts the mean to the left. One of the industry application of CI is marketing optimization. Having access to distribution of customer-level CI scores as in Fig. (ref) can help personalization and aid finer decision making.
Placebo tests help us understand the relative ability of competing causal estimates to account for selection bias. Selection bias occurs when customers who take an action (e.g., stream video) have unobserved characteristics not included in the model that makes them systematically more or less likely to take the action (e.g., high income, low age etc.). In placebo tests, we take the treatment group customers and simulate as though they took the action a year before the actual date. This is achieved by shifting the event date by one year and recalculating the features based on the shifted event date. The CI estimated in this set up is the “placebo error”. Since, this is a fake event, a model with a lower placebo error than another suggests that it has a lower selection bias. Running placebo tests on all events is computationally expensive, so we selected a few events for placebo analysis. The results are shown in Fig. (ref).
The key takeaways from Fig. (ref) are:
One of the major wins in CI-DML is that we provide heteroskedasticity-consistent confidence intervals at both a customer and aggregate level for every CI analysis in a scalable and lower-cost fashion. We compare the uncertainty estimates (specifically the width of confidence intervals) from CI-DML with the bootstrap results in CI-PO for a few events in Fig. (ref).
We find confidence interval width to be comparable among the two approaches. On average, the CI-DML width (scaled with CI-PO point estimate) is 1.5% smaller when compared to bootstrap-based confidence interval. A bootstrap-enabled CI-PO run takes about 2.5X more time than a CI-DML run. Bootstrap also does not scale for events with large number of customers. As CI-DML approach for confidence intervals is based on a closed form implementation, we do not have any scalability issues. In addition, note that bootstrapping has theoretical limitations when used for matching estimators Abadie_Imbens.
In this work, we introduced a state-of-the-art methodology used for calculating CI values. We noted that a DML based framework eliminates bias, allows us to extract heterogeneity in CI values, and provides a scalable way to construct heteroscedastic confidence intervals. We also made a case for using IPW and common support to refine the CI estimates. We demonstrated how leveraging PCA followed by K-means clustering allowed us to introduce customer-level heterogeneity. Using JSON based config allows flexibility to experiment with a wide variety algorithms and can take us from experimentation to production in minimal steps.
We presented results for few anonymized customer actions across different domains. Both the population-level and customer-level results for the customer actions we have looked at so far are aligned with the CI-PO results and our expectations.
Note that estimation of heterogeneous or context-aware treatment effects is an active area of research with wide applications ranging from marketing to health care. Distribution of treatment effects across different subgroups, or as a function of specific individual-level characteristics provides researchers with additional insights about the treatment/ intervention analyzed. Our work showcases, a scalable real-world application for extracting average as well heterogeneous causal estimates which we believe will be of interest to the broader scientific community.
Validating the causal estimates is challenging due to lack of ground truth. In the current work, we relied on the model fit metrics in the DML steps, placebo tests, and on bridging the CI-DML and CI-PO outputs. In the future, we plan to include metrics which focus on the validation of heterogeneous treatment effects. Examples of these include metrics based on Generic Machine Learning GLM and empirically calibrated Monte Carlo resampling techniques MC. We also plan to experiment with doubly robust estimator in the DML stage.