Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
133,732 characters · 39 sections · 49 citation commands
Design and Evaluation of Personalized Free Trials
\thispagestyle{empty}
\thispagestyle{empty}
\setcounter{page}{1}
One of the major trends in the software industry over the last few years has been the migration of software firms from the perpetual licensing business model to the “Software as a Service” (SaaS) model. In the SaaS model, the software is sold as a service, i.e., consumers can subscribe to the software based on monthly or annual contracts. Global revenues for the SaaS industry now exceed 200 billion USD gartner_2019. This shift in the business model has fundamentally changed the marketing and promotional activities of software firms. At a high level, since the software is now a service, sales and trials play a more important role than traditional advertising and brand building activities. In particular, a new type of customer acquisition strategy that has become popular is free trial promotions, where new users are given a limited time to try the software for free.
While free trials are now almost universal in the SaaS industry, there is no consensus on how long these trials should be. We observe trial lengths ranging anywhere from one week to three months in the industry; e.g., Microsoft 365 offers a 30 days free trial whereas Google's G Suite offers a 14 days free trial. There are pros and cons associated with both long and short trials, and the optimal trial length can depend on the mechanism through which the free trial operates. A short trial period lowers the firm's acquisition costs since the length of time that the firm supports a non-paying user is short. It is also less likely to suffer free-riding concerns. For example, if consumers need the software for a specific/defined task which will take only a few days (e.g., set up a website or edit a document/image), a long free trial allows them use the product without paying for it whereas a short trial period will force the customer to purchase it for the period of use. On the other hand, a long trial period can create stickiness/engagement and increases switching-back costs. For example, if a consumer has already created or stored content in the software, it is costly to move it to the local disk or alternative software. Longer trial periods can also enhance consumer learning by giving them more time to learn how to use the software (e.g., product features, functionalities).
While the above arguments make a global case for shorter/longer trial periods, the exact mechanism at work and the magnitude of its effect can be heterogeneous across consumers. For instance, it may be better to give longer free trials to beginners because they need more time to learn the software and convert, whereas students may have higher conversion rates with shorter free trials because it prevents free-riding. In principle, if there is significant heterogeneity in consumers' response to the length of free trials, then SaaS firms may benefit from adopting a personalized free trial policy. Formally, a personalized targeting policy is defined as a mapping from the space of users' pre-treatment attributes to the treatment space. In the free trial context, a personalized policy is one that assigns each consumer a different trial length depending on her/his observed demographics and skills. Intuitively, SaaS firms can benefit from personalization if they can assign each consumer the trial length that maximizes her/his expected responsiveness.
Indeed, SaaS free trials are particularly well-suited to personalization because of a few good reasons. First, software services have zero marginal costs and there are no direct cost implications of offering different trial lengths to different consumers. Second, it is easy to implement a personalized free trial policy at scale for digital services, unlike physical products. Finally, consumers are less likely to react adversely to receiving different free trial lengths (unlike other marketing mix variables, e.g., prices). Thus, personalization of free trials promotions is not only feasible, but can also lead to substantial improvements in consumer acquisition rates and revenues in this industry.
In this paper, we use data from a large-scale free trial experiment conducted by a major SaaS firm to design and evaluate the effectiveness of personalized free trial policies. The firm sells a suite of related software products (e.g., Microsoft 365, Google G Suite, Adobe Creative Cloud) and is the leading player in its category, with close to monopolistic market power. At the time of this study, the firm used to give users a 30-day free trial for each of its software products, during which they had unlimited access to the product.
To evaluate the effectiveness of different trial lengths, the firm conducted a field experiment involving 337,724 consumers from six major geographic markets -- Australia and New Zealand, France, Germany, Japan, United Kingdom, and United States of America. During the experiment period, new users who started a free trial for one of the firm's products were randomly assigned to one of 7, 14, or 30-day trial length conditions. These three trial lengths were chosen because they are the most commonly used ones in the industry and represent a vast majority of the SaaS free trials. The subscription and retention decisions of users in the experiment were monitored for two years. The firm also collected data on users' pre-treatment characteristics (e.g., skill level and job) and post-treatment retention, revenue, and product usage behavior to examine the heterogeneity in users' responses to trial lengths.
Our goal is to use the data from this field experiment to understand how the length of free trials affects customer acquisition in the SaaS industry. In particular, we are interested in four key questions related to trial lengths and personalization in the SaaS industry. First, what is the best uniform free trial policy, i.e., if we want to give all users the same trial length, what is the best option -- 7, 14, or 30 days? Second, is there heterogeneity in users' responsiveness to trial lengths? If yes, what percentage of users respond the best to 30 day trials and what are their characteristics (i.e., demographics/skills), what percentage respond the best to 14 day trials, and so on. Related to this, we would like to shed light on the role of different mechanisms at play here (e.g., learning, free-riding) and present some evidence on the extent to which they are heterogeneous across consumers. Third, what are gains from adopting the optimal personalized free trial policy compared to uniform policies. Finally, how do personalized policies designed to maximize short-run outcomes (i.e., customer acquisition) perform on long-run metrics such as consumer loyalty and revenue?
To answer these questions, we seek to develop a framework that managers and researchers can use to design and evaluate personalized targeting policies, and identify the optimal policy. We face a few key challenges in this task. First, from a theoretical perspective, searching for the optimal policy is a non-trivial task because the cardinality of the policy space can be quite large. (In our setting there are $3^{987,840}$ possible policies.) Second, from an empirical perspective, the covariate space of consumers' attributes is high dimensional in most applications.\footnote{In our application, we have $987,840$ possible sub-regions.} So we will not have sufficient data in each sub-region of the covariate space to learn how well a policy performs there. We therefore require methods that can automatically pool sub-regions that show similar responsiveness to the firm's actions and learn the optimal policy at the right level of granularity.
A series of recent papers at the intersection of machine learning and causal inference have been developing methods to address these challenges and obtain individual-level treatment effects, which can then be used to personalize treatment assignment Athey_Imbens_2016, Wager_Athey_2018. Similarly, a series of papers in marketing have combined powerful predictive machine learning models with experimental (or quasi-experimental) data to develop personalized targeting policies Rafieian_Yoganarasimhan_2020, Rafieian_2019a, Rafieian_2019b, Simester_etal_2019a. At a high level, all these papers share the common goal of personalizing marketing interventions in order to maximize some measure of reward that is important to the firm. However, the optimal policy that each of these papers/methods arrive at in a given empirical context can be very different because they all impose very different assumptions on the data generating process, are trained to optimize different objective functions, make different types of bias-variance trade-offs in the learning process, and have vastly different compute times. Thus far, we have little to no understanding of how these methods compare to each other when it comes to designing the optimal targeting policy.
This brings us to the third challenge. In order to understand which of these methods produces the most effective personalized policy, we need to be able to evaluate the performance of each policy {\it offline} (without deploying it in the field). This is essential because deploying a policy in the field to estimate its effectiveness is costly in time and money. Moreover, given the size of the policy space, it is simply not feasible to test each policy in the field.
In this paper, we provide a three-pronged framework for personalized policy design and evaluation that addresses these challenges. In the first component, we define the optimal policy design problem and present the solution concepts. As discussed earlier, an unstructured search for the optimal policy is challenging because the policy space can be very high dimensional. We circumvent this problem by adopting a two-step approach. In the first stage, we learn flexible functions of either outcomes or pairwise heterogeneous treatment effects. Then, in the second stage, we use the function(s) learned in the first stage to assign the optimal treatment for each user. Thus, we can design personalized policies if we have consistent estimates of either outcomes or heterogeneous treatment effects.
In the second component, we design personalized policies based on five outcome estimators (linear regression, lasso, CART, random forest, and boosted regression trees) and two heterogeneous treatment effects estimators (causal tree and causal forest). All these methods impose different parametric or semi-parametric assumptions on users' responses or heterogeneous treatment effects. As such, while they should all give the same optimal policy when response functions are relatively simple and data are infinite, neither of these conditions is likely to be true in most applications.
The third component of our framework consists of offline policy evaluation. For this task, we use the Inverse Propensity Score (IPS) reward estimator, which has been used in the reinforcement learning and counterfactual policy evaluation literatures in computer science Horvitz_Thompson_1952, Dudik_etal_2011. Intuitively, for {\it any} given policy, the estimator takes all the users who happened to receive the policy-prescribed treatment and scales them up by their propensity of receiving that treatment. This scaling gives us a pseudo-population that has received the policy being considered. The observed reward of this pseudo-population is, therefore, a consistent estimate of the reward that we can expect from the policy if we were to implement it.
We apply this framework to data from the free trial experiment and present a set of substantive findings. At the outset, we find that the firm can do significantly better by simply assigning the 7-days trial to all consumers (which is the best uniform policy). In the test data, this leads to a 5.59% gain in subscriptions, over the baseline of 30 days for all policy. In contrast, 14-days for all policy does not significantly increase subscriptions. This suggests that, simply shortening the free trial to 7 days will lead to a significant increase in subscription.
Next, we present a series of results comparing the performance of seven personalized targeting policies designed based on five outcome prediction estimators (linear regression, lasso, CART, random forest, and boosted regression trees or XGBoost) and two heterogeneous treatment effects estimators (causal tree and causal forest). First, we find that the treatment allocations vary significantly depending on the estimator used to design the policy. For example, the policy based on lasso prescribes the 7 day trial to 69% of users whereas the policy based on causal forest prescribes the 7 day trial to 91% of users. Second, we find that the personalized policy based on lasso performs the best, with a $6.81\%$ gain over the baseline of 30 days for all, followed by the policy based on XGBoost ($6.17\%$ improvement). However, policies based on other outcome estimators (e.g., random forests, regressions) perform poorly. Interestingly, policies based on the recently popular heterogeneous treatment effects estimators (causal tree and causal forest) also perform poorly. Causal tree is unable to personalize the policy at all. Causal forest personalizes policy by a small amount, but the gains from doing so are marginal. This an important finding since these methods are gaining traction in the marketing literature Guo_etal_2017, Fong_etal_2019. However, it is not obvious that personalizing targeting interventions based on these methods is ideal or appropriate. Indeed, a key takeaway from our results is that managers may be better off simply going with the best uniform policy instead of investing resources in personalizing policies based on these methods.
We then adopt a descriptive approach to investigate the source of the difference in the performance of these policies. We find that methods like CART, causal tree, and casual forest perform poorly because they are unable to personalize the policy sufficiently. In contrast, methods like linear regression and random forest seem to infer excessive spurious heterogeneity, which is also problematic. Lasso and XGBoost lie somewhere in between these two groups -- these estimators are able to capture heterogeneity in users' responsiveness without over-fitting, and therefore perform the best when it comes personalized policy design. We then propose a new metric for quantifying the gains from implementing a specific policy compared to the baseline experiment, and demonstrate its usefulness in explaining policy performance.
Next, we focus on a substantive question of relevance to free trials in the SaaS industry: why do some users respond well to longer trials, while others respond better to shorter trials? To answer this question, we segment consumers into three groups based on their optimal trial length. We find that consumers for whom the 30-day trial is optimal are more likely to be {\it legacy users} (e.g., are more experienced and less likely to be students or hobbyists), consumers for whom the 14-day trial is optimal are more likely to be {\it learners} (e.g., have higher usage and subscription rate), and consumers for whom the 7-day trial is optimal are more likely to be {\it free-riders} (e.g., more likely to be beginners and students and have the lowest subscription rate).
Finally, we find that the best-personalized policy (the one based on lasso) also performs well on long-term metrics, with a $7.96\%$ increase in subscription length and $11.61\%$ increase in revenues. Note that the magnitude of gains on these outcomes is different from that on subscriptions. We show that this is because trial length can affect long-run outcomes through multiple channels (in addition to conversion). Overall, our findings suggest that personalized targeting policies designed to maximize short-run conversions also perform well on long-run outcomes.
Our research makes two broad contributions to the literature. First, from a substantive perspective, we quantify the role of trial lengths in customer acquisition in the SaaS industry. We find that personalized free trials can significantly improve both short-run conversions and long-run profitability. Second, from a methodological perspective, we present a framework that managers and researchers can use to design and evaluate the effectiveness of personalized targeting strategies. While our findings on the relative performance of different estimators are specific to our context, the framework itself is quite general and can be used to personalize other marketing mix variables.
The rest of the paper is organized as follows. In $\S$(ref) we discuss the related literature. In $\S$(ref), we describe the application setting and data, and we present a series of descriptive analysis in $\S$(ref). Next, in $\S$(ref), we describe the three components of our frameworks -- personalized policy design, estimators used to design policies, and policy evaluation. In $\S$(ref), we apply this framework to our data and discuss the performance of a series of uniform and personalized counterfactual policies. In $\S$(ref), we present some descriptives on consumer segmentation under the optimal personalized policy, and examine how customer loyalty and revenues evolve under this policy. We conclude in $\S$(ref).
First, our paper relates to the theoretical and empirical research on policy design and evaluation that spans marketing, economics, and computer science. In an early theoretical paper, Manski_2004 presents a method that finds the optimal treatment for each observation by maximizing a regret function. More recent papers in this area include Kitagawa_Tetenov_2018 and Athey_Wager_2017. However, the methods in these papers are constrained by the size of the policy space (which in our case is $3^{987,840}$). Therefore, we first estimate outcomes or heterogeneous treatment effects, and then use these estimates to develop and evaluate personalized policies.\footnote{There is another stream of literature within marketing that seeks to learn consumers preferences using active machine learning methods toubia_etal_2003, dzyabura_hauser_2011, dzyabura_hauser_2019. In this case, the goal is preference discovery, not personalization and it mainly relies on conjoint experiments to achieve this task.}
Recent empirical papers in this area include Lefortier_etal_2016, Swaminathan_etal_2017, and Simester_etal_2019a. In a particularly relevant paper, Simester_etal_2019b investigate how data from field experiments can be used to design targeting policies for new customers or new regimes. They find that model-based methods (e.g., lasso) offer the best performance, though this advantage vanishes if the setting and/or consumers change significantly. Our paper differs from theirs both substantively and methodologically. Methodologically, our paper differs from theirs in three key ways. First, they evaluate effectiveness of different policies by running a second field experiment, whereas we use the offline IPS estimator. So our framework avoids the costs associated with a running a second field experiment. Second, unlike them, we also consider policies based on the newly proposed heterogeneous treatment effects estimators (e.g., causal forest). Third, we present descriptive approaches and metrics that are designed to explain the difference in the performance of the different policies. Finally, from a substantive perspective, we focus on free trials in the SaaS domain, whereas they study mail promotions for a retail firm.
Our paper also adds to the small but growing marketing literature on applications of tree-based heterogeneous treatment effects estimators. Ascarza_2018 estimates the heterogeneous effect of retention interventions and shows that customers with the highest churn risk are not necessarily those who benefit most from retention interventions based on the method proposed by Guelman_etal_2015. More recently, Guo_etal_2017 investigates the heterogeneous effect of information transparency on payments between pharmaceutical firms and physicians using causal forest Wager_Athey_2018. Similarly, Fong_etal_2019 use the method to explore the heterogeneity in the returns to targeted promotions based on an individual's purchase history. Our paper contributes to this literature by demonstrating that causal forest, and its predecessor causal tree, are unable to fully capture the heterogeneity in users' responses, and therefore are not very effective for personalizing targeting interventions.
Substantively, our paper adds to two important areas in digital marketing -- personalization of marketing actions and free trials. We discuss both in detail below.
A recent stream of marketing papers has focused on the question of “how to personalize marketing actions using machine learning methods in real-time”: website design hauser_etal_2009, ranking of search engine results Yoganarasimhan_2020, mobile ads Rafieian_Yoganarasimhan_2020, the sequence of ads shown in a mobile in-app setting Rafieian_2019a, Rafieian_2019b, and promotions Hitsch_Misra_2018. We contribute to this research by presenting a framework for designing and evaluating personalized targeting policies and a case study on personalized free trial promotions.
Finally, our work relates to research that examines the effectiveness of free trials. Analytical papers in this area have explored mechanisms by which free trials can be effective. They have proposed a multitude of (often conflicting) explanations such as switching costs, software complexity, network effects, quality signaling, and consumer learning Cheng_Liu_2012, Dey_etal_2013, Wang_Ozkan_2018. However, there are only two empirical papers that focus on free trials. Foubert_Gijsbrechts_2016 build a model of consumer learning and show that while free trials can enhance adoption, ill-timed free trials can also suppress adoption. Using a bayesian learning approach, Sunada_2018 compares the profitability of different free trial configurations. However, neither of these papers explore the question of how to optimize the length of free trials because they have no variation in the length of the free trials offered in their data. In contrast, we use data from a large-scale field experiment with exogenous variation in the length of free trials to identify the optimal trial length for each user.
In this section, we describe our application setting and data, and present some simple descriptive analyses that support the idea that personalizing free trial lengths can improve firm-level outcomes.
Our data come from a major SaaS firm that sells a suite of software products. The suite includes a set of related software products (similar to Excel, Word, PowerPoint in Microsoft's MS-Office). The firm is the leading player in its category, with close to monopolistic market power. Users can either subscribe to single-product plans that allow them access to one software product or to bundled plans that allow them to use several products at the same time. Bundles are designed to target specific segments of consumers and consist of a set of complementary products. The prices of the plans vary significantly and depend on the bundle, the type of subscription (regular or educational), and the length of commitment (monthly or annual). Standard subscriptions run from $\$30$ to $\$140$ per month depending on the products in the bundle and come with a monthly renewal option. (To preserve the firm's anonymity, we have multiplied all the dollar values in the paper by a constant number.) If the user is willing to commit to an annual subscription, they receive over $30\%$ discount in price. However, users in annual contracts have to pay a sizable penalty to unsubscribe before the end of their commitment. The firm also offers educational licenses at a discounted rate to students and educational institutions, and these constitute $20.8\%$ of the subscriptions in our data.
At the time of this study, the firm used to give users a 30-day free trial for each of its software products, during which they had unlimited access to the product.\footnote{This free trial is at the software product level, i.e., users start a separate trial for each software product, and their trial for a given product expires 30 days from the point at which they started the free trial for it.} In order to access the product after the trial period, users need a subscription to a plan or bundle that includes that product.
To evaluate the effectiveness of different trial lengths, the firm conducted a large-scale field experiment that ran from December $1^{st} 2015$ to January $6^{th} 2016$ and spanned six major geographic markets -- Australia and New Zealand, France, Germany, Japan, United Kingdom, and United States of America. During the experiment period, users who started a free trial for any of the firm's four most popular products were randomly assigned to one of 7, 14 or 30 days free trial length buckets. The assignment was at user level, i.e., once a user was assigned to a treatment (trial length), her/his trial length for the other three popular products was also set at the same length. The length of the free trial for other products during this period remained unchanged at 30 days. The summary statistics for the treatment assignment and subscriptions are shown in Table (ref).
The experiment was carefully designed and implemented to rule out the possibility of self-selection into treatments, a common problem in field experiments. In our setting, if users can see which treatment (or free trial length) they are assigned to prior to starting their trial, then users who find their treatment undesirable may choose to not start the trial. In that case, the observed sample of users in each treatment condition would no longer be random, and this in turn would bias the estimated treatment effects. Moreover, since the experimenter cannot obtain data on those who choose to not to start their free trials, there is no way to address this problem econometrically.
To avoid these types of self-selection problems, the firm designed the experiment so that users were informed of their trial-length only after starting their trial. In order to try a software product, users had to take the following steps: (1) sign up with the firm by creating an ID, (2) download an app manager that manages the download and installation of all the firm's products, and (3) click on an embedded start trial button to start the trial for a given product. Only at this point in time, they are shown the length of their free trial as the time left before their trial expires (e.g., “Your free trial expires in 7 days”). While users can simply quit or choose to not use the product at this point, their identities and actions are nevertheless captured in our data and incorporated in our analysis.
In sum, the design satisfies the two main conditions necessary for the experiment to be deemed “clean” -- (1) unconfoundedness and (2) compliance Mutz_etal_2019.
We have data on 337,724 users, who started a free trial for at least one of the four major products included in the field experiment. For each user $i$, we observe the following information -- (1) Treatment assignment ($W_i$), (2) Pre-treatment demographic data ($X_i$), and (3) Post-treatment behavioral data ($Z_i$). The treatment assignment variable denotes the trial length that the user was assigned to -- 7, 14, or 30 days. The variables under the latter two are described in detail below.
Note that the last three variables are self-reported though not open-ended, i.e., the firm gives users a list and requests them to pick one option from that list. However, users may choose not to report these values, in which case, the missing values are recorded as “unknown”. We treat this as an additional category for each of these three variables in our analysis.\footnote{Only a small fraction of people chooses to not report these data. For example, the percentage of users with “unknown" Skill and Job is 7.4% and 21.9%, respectively.} The list of all the six pre-treatment variables and their summary statistics are shown in Table (ref).
For all the users in our data, we observe all product download and usage data for the duration of their trial period. Further, we observe their subscription and renewal decisions for approximately 24 months (from December 2015 till November 2017). Using these data, we extract the following post-treatment behavioral information:
The summary statistics of these post-treatment variables are presented in Table (ref). Both subscription length and revenue are shown for: (a) all users and (b) the subset of users who subscribed. There are a couple of points to note regarding the data on subscription length and revenues:
The usage data are also missing (at random) for a subset of users and we report the summary statistics for non-missing observations.
To implement and test any policy that we design, we first need to partition the data into two independent samples -- training data and test data. These are defined as follows:
We use 70% of the data for training (and validation) and 30% for test. See Table (ref) for a detailed breakdown of how the data are split across the two data-sets. Note that while the joint distributions of the variables in the two samples should be the same theoretically, there will be some minor differences between the two data-sets due to the randomness in splitting in a finite sample. It is important to keep this in mind when comparing results {\it across} the two data-sets.
We now present some model-free evidence on the effect of trial length on subscriptions and the heterogeneity in users' responsiveness (on subscription) to different trial lengths.
In a fully randomized experiment (such as ours), the average effect of a treatment can be estimated by simply comparing the average of the outcome of interest across treatments. We set the 30-day condition as the control and estimate the average effects of the 14 and 7-day treatments on subscriptions for training and test data. The results from this analysis are shown in Table (ref).
The 7-day trial increases the subscription rate by $4.34\%$ over the baseline of the 30-day condition in the training data and by $5.59\%$ in the test data. However, in both data sets, the effect of the 14-day trial is not significantly different from that of the 30-day trial. These results suggest that a uniform targeting policy that gives the 7-day treatment to all users can significantly increase subscriptions. We also see that the average treatment effect is fairly small compared to the outcome, which is either zero or one. This makes sense because it is natural that the effect of the length of the free trial is small compared to other factors that can affect customer acquisition, especially in a near monopolistic market. Finally, note that the gains and subscription rates in the training and test data are slightly different. As discussed earlier, this is due to the randomness in the splitting procedure.
We now examine whether the firm can do better by personalizing its treatment assignment based on a user's pre-treatment characteristics. In the top left panel of Figure (ref), we partition the data into six sub-groups based on the user's geographic region and present the average subscription rates for the three trial lengths for each region. The results suggest that there is some heterogeneity in response rates by region. For example, the 14-day trial is more effective in Germany while the 7-day trial is more effective in the United States of America. Next, we perform a similar exercise on skill-level and job (see the top right and bottom panels in Figure (ref)). Again, we find that users' responsiveness to the treatment is a function of their skill level and job. For instance, the 7-day trial is significantly better for Beginners, whereas the 14-day trial is more effective for Mixed-skill users.
We also find that users' responsiveness varies with the interaction of two or more pre-treatment variables. Figure (ref) shows the treatment effect for users whose job description is Business Professionals by their geographic region. Based on Figure (ref), we know that, on average, the 7-day trial is better than the 30-day trial for Business Professionals (Job $=2$). However, this effect varies by the user's geographic location. While Business Professionals in the US do better with shorter free trials, those in Japan respond better to the 30-day trial.
Together, these results suggest that users' responsiveness to trial lengths is heterogeneous along many pre-treatment variables and their higher-order interactions. If the firm can successfully exploit this heterogeneity and tailor its free trial assignment policy at the individual-level, then it may be able to achieve higher subscriptions than simply assigning all users to the 7-day trial.
We now present a three-pronged framework that allows us to design and evaluate personalized targeting polices. In $\S$(ref), we define the optimal personalized policy design problem and solution concept. Next, in $\S$(ref), we discuss the set of estimators that can be used to design the optimal personalized policy. Finally, in $\S$(ref), we discuss policy evaluation. Throughout this section, we describe the framework in general terms and discuss the application to our free trials data in $\S$(ref).
We define the optimal policy design problem in $\S$(ref) and discuss the solution concept in $\S$(ref).
Consider a setting with $i\in \{1, \ldots, N\}$ independent and identically distributed users, where each user is characterized by a pre-treatment covariate vector $X_i \in X$ of dimension $D$. Let $W_i \in \mathcal{W}$ denote the treatment or intervention that $i$ receives. $\mathcal{W} = \{0, \ldots, W-1\}$ refers to the set of treatments, and the total number of treatments is $W$. We restrict ourselves to settings where treatments are assigned based on a fully randomized experiment or those where the treatment propensities are a function of observable variables (e.g., an observational study). We use $e(W_i = w, X_i)$ to denote the probability that user $i$ with pre-treatment attributes $X_i$ is assigned treatment $w$. Finally, let $Y(X_i, W_i)$ denote the outcome for a user $i$ with pre-treatment variables $X_i$ when she is allocated treatment $W_i$.
We begin with a formal definition of a personalized treatment assignment policy.
$\pi(X_i) = W_i^{\pi}$ implies that the policy $\pi$ assigns treatment $W_i^{\pi}$ to a user $i$ with pre-treatment attributes $X_i$.
Next, we define the objective function that a firm or policy-maker wants to maximize. Let $Y$ denote an outcome of interest. Then:
We can now define the optimal personalized policy as follows:
where $\Pi$ is the set of all possible policies.
The problem of finding the optimal personalized policy is equivalent to one of finding the policy $\pi^*$ that maximizes the reward function $R(\pi, Y)$. As discussed in $\S$(ref), this is a non-trivial problem since the cardinality of the policy space can be quite large. The total number of possible policies is $W^{\prod_{d=1}^D c_d}$, when we have $D$ pre-treatment variables and the $d$-th variable can take $c_d$ different values.
Given the cardinality of the policy space, a direct search over the policy space to find the optimal policy is infeasible. So we discuss the two-step approaches available to find the optimal policy $\pi^*$ that avoid this problem. However, we first need to make three assumptions on the data generating process for these approaches to be valid.
The unconfoundedness assumption holds when we have a fully randomized experiment or when treatment assignment propensities are only function of observables.
SUTVA is a reasonable assumption in most marketing settings, and it is violated only when there are WOM or network effects.
This assumption is satisfied in randomized experiments and in observational studies where there is residual randomness in treatment assignment, conditional on observables. This assumption is necessary for both being able to design policy for the full distribution of $X$s over all $W$s, as well as for empirically evaluating the performance of a policy $\pi$ based on the data we have. Intuitively, if there are some combinations of covariates and treatments that have a zero probability of occurring in our data, then we cannot make inferences in this space nor can we evaluate how a counterfactual policy $\pi$ would perform in this space.
With these assumptions in place, we can design the optimal personalized policy if we either have estimates of the outcome of interest or pairwise treatment effects. We discuss both these approaches in detail below.
Intuitively, both these approaches try to put some structure on the relationship between a user's responsiveness and her treatment and demographic variables in the first step. Then, in the second step, they use this structure to assign the best policy. This enables them to avoid an unstructured search for the best policy. Both approaches are consistent and correct. With an infinitely large dataset and sufficiently flexible models (of outcomes and treatment effects), both should give the exact same optimal policy. However, in practice, data are finite, the covariate space is high dimensional, and different outcome/treatment effect estimators impose very different parametric/semi-parametric assumptions on users' response behavior. Therefore, the optimal policy in a given empirical setting can vary significantly based on the exact model used to estimate the outcome or heterogeneous treatment effects. Indeed, one of the main objectives of this work is to examine the efficacy of different types of estimators for designing policy.
We now present the second component of our framework -- the estimators used for policy design. In $\S$(ref), we discuss a series of outcome estimation methods, and in $\S$(ref), we discuss heterogeneous treatment effects estimators.
As discussed in $\S$(ref), an empirical challenge that we face when estimating outcomes or heterogeneous treatments effects is the dimensionality of the covariate space of consumers' attributes (which is equal to $\prod_{d=1}^D c_d$, and can be quite high even in simple settings). In most applications, we will therefore not have sufficient data to estimate a local model in each sub-region of the covariate space. We require methods that can automatically pool together observations from sub-regions that show similar responsiveness to the firm's actions. We face a standard bias-variance trade-off here -- models that impose more structure (e.g., regression) may have high in-sample bias if the structure does not match the true data generating process. But they will also have low out-of-sample variance since the model is more global. On the other hand, models that are more flexible (e.g., random forests) may have low in-sample bias (i.e., over-fit to the training data), but perform poorly out-of-sample. Thus, which of these estimators is the best for a given setting is an empirical question.
Outcome estimators seek to learn a model $f(x,w)=\operatorname{\mathbb{E}}[Y|X_i=x, W_i=w]$. We consider five commonly used outcome estimators: (1) linear regression, (2) lasso, (3) CART, (4) random forest, and (5) boosted regression trees. We focus on these because of a few key reasons. First, linear regression is the simplest and most commonly used method to model any outcome of interest. Second, lasso is worth exploring because it was designed to improves on the predictions from a linear regression by reducing the out-of-sample variance using variable selection. Third, CART is a semi-parametric way to model outcomes by partitioning the covariate space into areas with the highest within-group similarity in outcomes. Finally, both random forest and boosted regression trees are improvements of CART and have been shown to offer much higher predictive ability than simpler models such as CART, regression, or lasso. We discuss each of these briefly below.
Regression-based methods: A linear regression model with first-order interaction effects can be used to predict an individual $i$'s observed outcome $Y_i$ as a function of her pre-treatment variables, treatment variable, and the interaction of both, as follows:
$\beta_1$ is a vector that captures the effect of the pre-treatment variables ($X_i$) on the outcome. $\beta_2$ is a vector that captures the main effect of the different treatments on $Y$. Finally, the vector $\beta_3$ captures the interaction effect of treatment and pre-treatment variables. This interaction term is important because it helps us in personalizing the policy by capturing how the effectiveness of different treatments varies across individuals (as a function of their pre-treatment attributes).
However, in a high dimensional covariate space, linear regressions with first-order interaction effects will have a large number of explanatory variables. This usually leads to poor out-of-sample fit. That is, such regressions tend to have low bias, but high variance -- they tend to overfit on the training sample but perform poorly in new samples, especially if the data generating process is noisy. Since our goal is to design optimal policies for counterfactual situations, the out-of-sample fit is the only metric of importance.\footnote{We can also add higher-order interaction effects into the regression model. However, we refrain from doing so because it will increase model complexity significantly and exacerbate this problem.}
Lasso addresses the above problem by learning a simpler model that uses fewer variables Tibshirani_1996, Friedman_etal_2010. Practically, lasso estimates a linear regression that minimizes the MSE with an additional term to penalize model complexity as shown below:
where $||\beta_{i}||_1$ is the L1 norm of the vector $\beta_{i}$ and is equal to the sum of the absolute value of the elements of vector $\beta_{i}$. Intuitively, if there are multiple weak (and correlated) predictors, lasso will pick a subset of them and force the coefficients of others to zero, thereby estimating a simpler model. Model-selection in lasso is data-driven, i.e., $\lambda$ is a hyper-parameter that is learned from the data (and not assumed).\footnote{Another regression-based model that is worth considering is elastic net. Elastic net penalizes both L1 norm (like lasso) and L2 norm (like ridge regression). However, in our application setting, the cross-validation procedure chooses the L2 norm coefficient to be zero. So elastic net converges to lasso in our setting. So we do not discuss it in detail here.}
Tree-based methods: Next, we discuss tree-based models, starting with CART. CART recursively partitions the covariate space into sub-regions, and the average of $Y$ in a given region ($E(Y)$) is the predicted outcome for all observations in that region breiman_etal_1984. This type of partitioning can be represented by a tree structure, where each leaf of the tree represents an output region. A general CART model can be expressed as:
where $x, w$ is the set of explanatory variables, $R_m$ is the $m^{th}$ region of the $M$ regions used to partition the space, $\rho_m$ is the predicted value of $y$ in region $m$.
Trees are trained by specifying a cost function (e.g., mean squared error) that is minimized at each step of the tree-growing process using a greedy algorithm. To avoid over-fitting, usually a penalty term is added to the cost function, whose weight is a hyper-parameter learnt from the data using cross-validation.
While CART has some good properties\footnote{CART can accept both continuous and discrete explanatory variables, is not sensitive to the scale of the variables, and allows any number of interactions between features murphy_2012. It therefore has ability to capture rich nonlinear patterns in the data. Further, CART can do automatic variable selection, i.e., it uses only those variables that provide better accuracy in the prediction task for splitting.}, it often has poor predictive accuracy because it is discontinuous in nature and is sensitive to outliers. Therefore, even with regularization, CART models tend to over-fit on the training data and under-perform on out-of-sample data. We can address these problems using two different techniques -- (1) Bagging and (2) Boosting. We discuss both of these techniques and the resulting estimators below.
In general, deep trees have high in-sample fit (low bias), but high out-of-sample variance because of over-fitting. However, we can improve the variance problem by bagging or averaging deep trees using bootstrapping. Ho_1995 formalized this idea and proposed random forests. Random forest usually consists of hundreds or thousands of trees, each of which is trained on a random sub-sample of columns and rows. Each tree is thus different from other trees and the average of these random trees is a better predictor than one single tree trained on the full data.
Another approach is to start with shallow trees. Shallow trees have poor in-sample fit (high bias), but low out-of-sample variance. Additively, adding a series of weak trees that minimize the residual error at each step by a process known as boosting can improve the bias problem, while retaining the low variance. This gives us boosted regression trees. Conceptually, boosting can be thought of as performing gradient descent in function space using shallow trees as the underlying weak learners Breiman_1998, Friedman_2001.\footnote{It should be noted that bagging is not a modeling technique; it is simply a variance reduction technique. Boosting, however, is a method to infer the underlying model $y = f(x,w)$. Thus, they are conceptually completely different.} In this paper, we use XGBoost, a version of boosted trees proposed by Chen_Guestrin_2016 because it is superior to earlier implementations both in terms of accuracy and scalability.
Please see Appendix $\S$(ref) for the set of hyper-parameters that need to be tuned in CART, random forest, and XGBoost.
These methods are based on the potential outcomes framework and estimate the treatment effect for any pair of treatments ($w_j, w_{j'}$) at each point $x$ as shown in Equation (ref). However, this equation is not very useful in practical settings (where the covariate space is high-dimensional and data are finite) because we would not have sufficient observations at each $X_i$ to estimate precise treatment effects. Therefore, the general idea behind modern heterogeneous treatment effects estimators is to pool observations that are close in the covariate space and estimate the conditional average treatment effect for sub-populations instead of estimating the treatment effect at each point in the covariate space. That is, we can modify Equation (ref) as:
where $l(x)$ is the set of covariates that are fairly similar to $x$. Intuitively, for each point $x$, we use the observations in $l(x)$ to estimate treatment effects.
The main question in these methods then becomes how to find the optimal $l(x)$ around each $x$. On the one hand, if $l(x)$ is too small, then we will not have sufficient observations within $l(x)$, which would result in noisy estimates of treatment effects. On the other hand, if $l(x)$ is too large, then we will not capture all the heterogeneity in the treatment effects, which is essential for personalizing policy. Indeed, if $l(x)$ is the entire data, then we simply have one Average Treatment Effect for all users (which will give us one global policy). Thus, finding the optimal $l(x)$ involves the classic bias-variance trade-off.
Starting with Rzepakowski_Jaroszewicz_2012, a growing stream of literature has focused on developing data-driven approaches to finding the optimal $l(x)$ based on ideas from the machine learning literature. Among these methods, the recently developed causal tree and causal forest (or Generalized Random Forest) have been shown to have superior performance. So we focus on them.
Causal tree builds on CART. The only difference is that instead of partitioning the space to maximize predictive ability, the objective here is to identify partitions with similar within-partition treatment effect. Athey_Imbens_2016 show that maximizing the variation in the estimated treatment effects (with a regularization term added to control complexity) achieves this objective. Their algorithm consists of two steps. In the first step, it recursively splits the covariate space into partitions. In the second step, it estimates the treatment effect within each partition ($l(x)$) using Equation (ref). Intuitively, this algorithm pools observations with similar treatment effects into the same partition because splitting observations that have similar treatment effects does not increase the objective function (i.e., variation in the post-split treatment effect).
The causal tree algorithm nevertheless suffers from weaknesses that mirror those of CART. To resolve these issues, Wager_Athey_2018 proposes causal forest, which builds on random forests Breiman_2001. More broadly, Athey_etal_2019 show that the intuition from random forests can be used to flexibly estimate any heterogeneous quantity, $\theta(x)$, from the data, including heterogeneous treatment effects. They suggest a Generalized Random Forest (GRF) algorithm that learns problem-specific kernels for estimating any quantity of interest at a given point in the covariate space. For treatment estimation, the method proceeds in two steps. In the first step, it builds trees whose objective is to increase the variation in the estimated treatment effects. Each tree is built on a random sub-sample of the data and random sub-sample of covariates. In the second step, the algorithm uses the idea of a weighted kernel regression to calculate the treatment effect at each point $x$ using weights from the first step.
Please see Appendix (ref) for a detailed discussion both causal tree and causal forest and Appendix $\S$(ref) for details on hyper-parameter optimization in these two methods.\footnote{Athey_Imbens_2016 propose an additional approach to avoid over-fitting in causal tree and causal forest -- honest splitting. The main idea behind honesty is to split the data into two parts and use one part for growing each tree (i.e., generating the partitions) and the other part for estimating the treatment effects given a partition. Since the two data-sets are independent of one another, there is a lower likelihood of over-fitting. However, honest splitting comes with its own costs -- it reduces the amount of data we have for learning in both stages by half. In settings where the magnitude of the treatment effects is small (such as ours), honest splitting can adversely affect the performance of models. Indeed, we found that models based on honest splitting lead to worse policies compared to models without honest splitting in our setting. So we do not employ honesty in our analysis.}
We now discuss the final component of our three-pronged framework -- offline policy evaluation.
To identify the optimal policy among the set of all possible policies ($\Pi$), it is important to be able to consistently evaluate the gain from implementing any policy $\pi$ {\it without} actually deploying it. As discussed in $\S$(ref), this is important because deploying a policy in the field is costly in time and money. Moreover, when the policy space is large, it is simply not feasible to test each policy in the field. So we turn to off-policy estimators proposed in the reinforcement learning literature Sutton_Barto_2018, where the goal is to estimate the performance of a given policy $\pi$ using data generated by an experiment or another policy.
In any setting where Assumption (ref) is satisfied, there is a non-zero probability that a user $i$ will be assigned to the treatment prescribed by a given policy $\pi$. Thus, as shown in Figure (ref), we have two types of observations in the data:
These two sets of observations form the basis of our evaluation metric: the Inverse Propensity Score (IPS) Estimator for Reward. This estimator has been extensively used in the off-policy evaluation literature Horvitz_Thompson_1952, Dudik_etal_2011, and can be defined as:
The IPS estimator is unbiased as long as Assumption (ref) is satisfied. The intuition behind this estimator can be easily understood using Figure (ref). Essentially, it takes all the observations in the policy congruent cells and scales them up by their propensity of receiving the treatment assigned to them. This scaling gives us a pseudo-population that received the policy-prescribed treatment. Thus, the average of the outcome for this pseudo-population gives us an estimate of the reward for the true population, if we were to implement the proposed policy in the field.
We now apply the framework presented above to our free trial experiment and discuss the main findings. We first train the outcome and heterogeneous treatment effects models and discuss their performance in $\S$(ref). Next, in $\S$(ref), we define and derive a set of counterfactual targeting policies, and present the fraction of users assigned the three treatments under each of these policies. Then, we quantify the gains in subscription from adopting different counterfactual policies in $\S$(ref), and we finally discuss the source of these differences in $\S$(ref).
We start by briefly describing our implementation of the estimators discussed in $\S$(ref). First, in our setting, all the pre-treatment variables, $X_i$s, are categorical. So we transform each categorical variable into a set of dummy variables for each sub-category (also referred to as one-hot encoding in the machine learning literature). After this transformation, we have 82 dummies for the pre-treatment variables, three treatment dummies, and 246 first-order interaction variables. This gives us a total of 331 explanatory variables for the linear regression and lasso models. For the rest of the tree-based models, we directly feed in the 82 dummy variables for the pre-treatment characteristics and the three treatment dummies.\footnote{The GRF algorithm estimates outcomes and propensities in the first step (see Appendix (ref)). However, in our setting, we do not need to estimate propensity scores since they are known and constant for each observation since treatment assignment is fully randomized. Our qualitative results remain the same if we instead estimate them from the data.} Next, we use five-fold cross-validation on the training data to tune hyper-parameters for all the estimators (except regression which does not require hyper-parameter tuning). See Appendix $\S$(ref) for more details on model selection.
The main result of interest for outcome prediction models is their predictive performance. So in Table $\S$(ref), we present the MSE for the five outcome estimators on both training and test data. Two main findings emerge. First, we find that linear and lasso are the worst in terms of model-fit, with XGBoost performing the best. This finding is consistent with earlier papers that have found XGBoost to be the best outcome prediction method in tasks involving prediction of human behavior Rafieian_Yoganarasimhan_2020, Rafieian_2019a. However, we also find that all the tree-based models -- CART, Random Forest, and XGBoost -- suffer from overfitting in spite of hyper-parameter tuning. That is, their performance on the test data is worse than that on training data. This is common in most machine learning models and stems from the differences in the test and training data.
Next, we discuss the results from the heterogeneous treatment effects estimators. In this case, we cannot compare the estimates of treatment effects with any ground truth since we (as researchers and managers) do not know the true treatment effects. So we simply present the distributions of treatment-effects in Appendix (ref).
We start by designing the following three uniform (one length for all) policies:
Next, we use the framework described in $\S$(ref) to design a series of personalized policies. However, before doing so, we need to ensure that Assumptions (ref) -- (ref) are satisfied. So we discuss their applicability to our context below:
We design seven personalized policies based on the estimators discussed in $\S$(ref): $\pi_{reg}$, $\pi_{lasso}$, $\pi_{cart}$, $\pi_{r\_forest}$, $\pi_{xgboost}$, $\pi_{c\_tree}$, and $\pi_{c\_forest}$. To design these policies, we follow the two optimal policy design methods discussed in $\S$(ref). As discussed there, in reality, there is only one optimal policy. In theory, with sufficiently large data and a fully flexible model, we should be able to correctly identify this reward-maximizing optimal policy. However, in practice, each of these methods imposes different semi-parametric restrictions on the functional form of the model and make different bias-variance trade-offs. Thus, with finite data, these policies can vary substantially.
Table (ref) presents the fraction of users who are given each of the three treatments (7, 14, and 30 days) under the policies described above. There are a few points worthy of note here. First, policies based on CART and causal tree do not personalize treatment assignment and end up giving the 7-day treatment to all users. Thus, they are equivalent to the best uniform policy: $\pi_{7} \equiv \pi_{cart} \equiv \pi_{c\_tree}$. Second, the two policies based on the two outcome estimators -- lasso and XGBoost -- are somewhat similar in their treatment assignment. Both prescribe the 7-day treatment to $\approx 70\%$ of users, the 14-day treatment to $\approx 20\%$ of users, and the 30-day treatment to $\approx 10\%$ of users. In contrast, $\pi_{reg}$ and $\pi_{r\_forest}$ prescribe the 7-day treatment to the least number of users while $\pi_{c\_forest}$ prescribes the 7-day treatment to 91% of users (and the 30-day treatment to no one). Thus, a main takeaway here is that each of these policies behaves quite differently when it comes to treatment assignment.
We can evaluate the expected reward from each policy described using Equation (ref). In theory, in a randomized experiment, the propensity of treatment assignment is orthogonal to the treatment prescribed by any policy $\pi$. Thus, $e(W_i = w, X_i) = e(W_i = w) \; \forall \; w \; \in \mathcal{W}$ is known and constant for all observations. However, in practice, within the set of users for whom policy $\pi$ prescribes $w$, the empirical treatment propensities ($\hat{e}_{\pi(X_i)}(W_i=w)$) might not be the same as that in the full data. Thus, to correctly estimate the reward under a given policy, we need to use the empirical treatment propensities within each assigned trial length. We therefore modify Equation (ref) as follows:
where ${\hat{e}_{\pi(X_i)}(W_i)}$ is the probability that a user whom the policy prescribes treatment $\pi(X_i)$ is given $W_i$. Formally, ${\hat{e}_{\pi(X_i)}(W_i)}= \frac{\frac{1}{N}\sum_{j=1}^N 1[W_j=W_i,\pi(X_j)=\pi(X_i)]}{\frac{1}{N}\sum_{j=1}^N 1[\pi(X_j)=\pi(X_i)]}$. Thus, instead of calculating the treatment propensities on the entire data, we now do it on the subset of users who are prescribed $\pi(X_i)$ by the policy $\pi$. Intuitively, if we look at Figure (ref), instead of calculating treatment propensities on the full square (i.e., all the data), we calculate them separately for each column (i.e., set of users for whom the policy prescribes a treatment).
We present the expected reward from all the policies in Table (ref). In all our reward calculations, we use subscription as the outcome of interest. Thus, the expected rewards shown in columns 3 and 4 are the estimated subscription rates (in %) for the training and test data, respectively.\footnote{Estimates of the subscription rates (or reward) with theoretical propensities based on Equation (ref) are very similar to those shown in Table (ref).}
We clarify a few points before discussing the results. First, all the results will look more promising on the training data since the policies developed are based on models trained and validated on this data. So, the test data provide a more neutral ground for model comparisons and we focus on it in our discussions. That said, we cannot directly compare the estimated subscription rates (or expected reward) for different datasets Yi_etal_2013. In general, the performance of a given policy or model on a dataset is a function of: (a) the model used to form the policy, and (b) the joint distribution of the covariates and outcomes in that dataset. Simply put, in Table (ref), comparisons within a column are valid but comparisons across columns are less meaningful.
The top four policies based on Table (ref) are: $\pi_{lasso}$, $\pi_{xgboost}$, $\pi_{c\_forest}$, and $\pi_{7}$, in that order. However, these differences may not be significant. Therefore, in Table (ref), we present results from paired t-tests comparing the IPS-reward estimates for the top four methods with each other and show that all these differences are significant. We refer readers to Appendix $\S$(ref) for details on these tests.
We now discuss the main findings of interest from Tables (ref) and (ref). First, we find that 7-days for all is the best uniform policy (5.59% increase over the baseline of 30-days for all).
Next, we find that among the personalized policies, the policy based on lasso is the most profitable (6.81% improvement in subscription rate), followed by the policy based on XGBoost (6.17% improvement in subscription rate). Moreover, the differences between these policies and others are significant, i.e., the reward from $\pi_{lasso}$ is significantly better than that from other policies (see column 2 in Table (ref)). Similarly, $\pi_{xgboost}$ is significantly better than all other policies, except $\pi_{lasso}$ (see column 3 in Table (ref)).
However, not all outcome estimation methods do well when it comes to policy design. Linear regression and random forest overfit to the training data (high estimated subscription in-sample), but perform poorly on test data.\footnote{While this could be due to differences in the data itself, the magnitude of this difference suggests overfitting.} Interestingly, we do not find much correlation between an outcome estimator's predictive ability and its efficacy in policy design (recall Table (ref)). This is expected since the objective function in outcome estimation methods is predictive ability, which is different from policy design or performance.
An important point of note here is that poorly designed personalized policies (e.g., those based on regression and random forest) can actually do worse than the best uniform policy based on average treatment effects. As shown in the last column of Table (ref), the reward from $\pi_7$ is significantly better than that from the personalized policies, $\pi_{reg}$ and $\pi_{r\_forest}$. This problem becomes obvious only when we look at the evaluation metric (IPS reward estimate) on the test data. Simply looking at the relative performance of these policies in the training data can lead to the (incorrect) conclusion that $\pi_{reg}$ and $\pi_{r\_forest}$ outperform the uniform policy $\pi_7$. We, as policy-makers, should therefore be careful in both designing and evaluating personalized policies. It is essential that: (1) we do not conflate a model's predictive ability with its ability to form policy, and (2) we evaluate the performance of each policy on an independent test data with appropriate policy evaluation metrics.
Finally, we find that the recently proposed heterogeneous treatment effects estimators, causal tree and causal forest, perform poorly when it comes to personalized policy design. As discussed in $\S$(ref), causal tree does not personalize the policy at all since it assigns the 7-day trial to all users. While $\pi_{c\_forest}$ does personalize treatment assignment to some degree, the gain over the simple 7-days for all policy is very marginal. (As shown in column 4 of Table (ref), $\Delta \hat{R} (\pi_{c\_forest}, \pi_7)$ is positive and significant, but very small compared to the gain from other methods such as lasso and XGBoost.) Thus we have: $\hat{R}_{IPS}(\pi_{c\_tree}, Y) = \hat{R}_{IPS}(\pi_{7}, Y) \approx \hat{R}_{IPS}(\pi_{c\_forest}, Y)$.
Our results thus suggest that managers may be better off adopting the best uniform policy instead of investing resources in personalizing policies based on these methods. This is a useful finding since these methods are gaining traction in the marketing literature and researchers are starting to use them (e.g., Guo_etal_2017, Fong_etal_2019). However, both in marketing research and practice, the end goal is often policy design and not estimation of treatment effects (e.g., decide who should get a price promotion and who should not, or who should be targeted with advertising and who should not be). In such cases, relying on heterogeneous treatment effects estimators, such as causal forest, can be sub-optimal. Further, the causal forest model suffers from a lack of interpretability and is more susceptible to hyper-parameter tuning issues. In contrast, lasso is not only easy to specify, estimate, and interpret but also offers the best performance for policy design (at least in our setting).
We now examine if there is any relationship between the Conditional Average Treatment Effects (or CATEs) estimated based on a specific method and the performance of the policy based on that method. In Figure (ref), we show the CDFs of CATE estimates for the three pairs of treatments ($\tau_{7,30}$, $\tau_{14, 30}$, and $\tau_{7,14}$) based on each of the methods discussed in $\S$(ref). For expositional purposes, we focus on the top panel in the rest of this section. This panel depicts the CATE estimates for the 7 and 30-day treatments, i.e., $\tau_{7,30}$. The discussions for the middle and bottom panels are similar.
The first pattern that stands out is that CATE estimates based on CART, causal tree, and casual forest show very little heterogeneity (see the three vertical lines to the right of zero). This explains why policies based on these methods perform poorly -- they are unable to personalize the policy sufficiently to optimize users' response at the individual level. Recall that $\pi_{cart}$ and $\pi_{c\_tree}$ give the same treatment to all users and $\pi_{c\_forest}$ gives the 7-day treatment to over 91% of users (Table (ref)).
In contrast, CATE estimates based on linear regression and random forest show the maximum amount of heterogeneity (see the two rightmost curves). This pattern, in combination with the poor performance of $\pi_{reg}$ and $\pi_{r\_forest}$ on test data (and their extremely good performance on training data) suggests serious overfitting problems. That is, these models seem to infer much more heterogeneity than is true in the data. This, in turn, leads them to personalize policies to a larger extent than is ideal. Recall that $\pi_{reg}$ and $\pi_{r\_forest}$ show the maximum dispersion (or variation) in treatment assignment compared to other policies (Table (ref)).
Finally, we see that the CDFs of treatment effects based on lasso and XGBoost lie somewhere in between the above two groups. They show sufficient heterogeneity, but not too much. Hence, policies based on these methods are able to personalize the treatment sufficiently, but also avoid overfitting at the same time. As shown in Table (ref), the dispersion in treatment assignment for these two policies is higher than that in $\pi_{cart}$, $\pi_{c\_tree}$, and $\pi_{c\_forest}$, but lower than that in $\pi_{reg}$ and $\pi_{r\_forest}$.
Thus, the ideal estimators for policy design are those that are able to capture sufficient heterogeneity to personalize effectively without overfitting (i.e., capture spurious heterogeneity).
We now examine why some policies perform better than others. First, in $\S$(ref), we derive a general mathematical expression to quantify the gains from any policy over the current experiment. Next, in $\S$(ref), we use these mathematical expressions to explain the difference in the performance of the various counterfactual policies.
We now formalize the intuition behind this simple example for a general case. In a fully randomized experiment, the treatment propensities are the same for all observations i.e., $e(W_i = w, X_i) = e_w$. So we can write the IPS reward function for any policy $\pi$ as:
where $\upsilon_w=\sum_{i=1}^N\frac{1[{\pi(X_i)=w}]}{N}$ is the fraction of observations for whom the the policy $\pi$ assigns treatment $w$ and $\bar{Y}_{[\pi(X_i) = W_i = w]}$ is the average outcome for policy congruent observations that are assigned treatment $w$ by policy $\pi$.
Next, let $\Upsilon(\pi, Y)$ denote the change or improvement in reward from implementing policy $\pi$ compared to the current experiment. We can formally write $\Upsilon(\pi, Y)$ as:
Note that $\frac{1}{N}\sum_{i=1}^N Y_i$ is not a function of the policy $\pi$ being evaluated because it is simply the reward of the randomized experiment currently being implemented. It is therefore constant across different policies. Hence, the policy improvement function $\Upsilon(\pi, Y)$ can be used as a metric to compare the performance of different policies and explain why some policies do better than others.
According to Equation (ref), gains from a policy $\pi$ depend on two quantities: (1) For each treatment $w$, the fraction of observations for which that treatment is assigned ($\upsilon_w$), and (2) For these observations, the difference in expected outcomes for policy congruent and policy in-congruent users, i.e., $\left(\bar{Y}_{[\pi(X_i) = W_i = w]} -\bar{Y}_{[\pi(X_i) = w, W_i = w']}\right) \; \forall \; w'$. A good policy is one that ensures that -- (a) the difference between policy congruent and in-congruent users is positive and large for each $w$, and (b) a higher fraction of users ($\upsilon_w$s) get the more positive and higher differences.
We now use the expressions derived above to explain the difference in performances of the counterfactual policies discussed earlier. In Table (ref), for each policy $\pi$, we show: (1) the fraction of users for whom the policy prescribes the three treatments (7, 14, and 30 days), and (2) the subscription rate (or average outcome) of both policy congruent and in-congruent users (e.g., for users for whom the policy prescribes 7 days, the subscription rate of all three sets of users -- policy congruent users who received the 7-day treatment, policy in-congruent users who received the 14 and 30 day treatments). Note that the first sub-column within each policy simply shows the fraction of users assigned 7, 14, and 30 days by that policy. It is useful to recognize that these numbers in the bottom panel (for test data) are exactly the same as that in Table (ref) from $\S$(ref).
We now discuss the performance of the different counterfactual policies in the training data (top panel of Table (ref)). First, consider $\pi_{reg}$. Here, the difference between policy congruent and policy in-congruent cells for each treatment is always positive, e.g., for the set of users for whom the policy assigns 7 days, the average outcome for policy congruent users is $14.224$ (grey cell), which is greater than both $12.36$ and $12.569$ (the white cells). The same pattern is true for the rest of the policies in the training data -- for any treatment $w$, the average outcome of policy congruent users is always greater than that of policy in-congruent users.
However, things look very different when we look at the performance of these policies in the test data (bottom panel). In the case of $\pi_{reg}$, we see that the average outcome for policy congruent users is not always the highest. For example, the average outcome for policy congruent users assigned the 30-day trial is $11.761$. However, if these users had instead received the 7 or 14-day trials (which the policy deemed sub-optimal for them), they would have done better (with average subscription rates of $11.809$ and $11.993$, respectively)! $\pi_{reg}$ is similarly sub-optimal for users for whom it prescribes the 14-day treatment. Together, these two sets of users constitute $47.9\%$ ($0.157 + 0.322 = 0.479$) of the population. Thus, the policy $\pi_{reg}$ is sub-optimal for a large fraction of users and the extent of sub-optimality (the differences in the outcomes for policy congruent and in-congruent users) is high. Hence, the overall performance of this policy is poor.
In contrast, if we look at $\pi_{lasso}$, we see that for users whom it assigns to the 7 and 14-day treatments, the average outcome for the policy congruent cells is higher than that of policy in-congruent cells. More importantly, these differences are substantial and happen for a large fraction of users ($0.689 + 0.232 = 0.921$). We do find that this policy is somewhat sub-optimal for the users it assigns to the 30-day treatment ($16.914$ is less than both $17.172$ and $17.033$). However, these differences are quite small ($16.914 - 17.172 \approx -0.26$ and $16.914 - 17.033 \approx 0.12$) and the fraction of users for whom it makes this mistake is also small ($0.079$). Overall, this is the reason behind the superior performance of $\pi_{lasso}$: it is able to improve the response of a vast majority of users (over 92%) by a substantial amount; and when it does make a mistake in treatment allocation, it is for a small set of users and the magnitude of this mistake is quite small.
Next, we can see why $\pi_{xgboost}$ is the second-most effective policy after $\pi_{lasso}$: it is sub-optimal for users whom it prescribes the 14-day treatment. While the difference between the expected outcomes for policy congruent and in-congruent users is not very high, the fraction of users for whom this mistake happens is higher than that in the case of lasso ($18.5\%$).
It is also apparent why the policy based on random forest, $\pi_{r\_forest}$, performs so poorly -- it assigns sub-optimal treatments to all the users. Notice that in no condition (or treatment) is the grey cell the highest for that condition. Finally, we look at the policy based on causal forest, $\pi_{c\_forest}$. Interestingly, this is the only policy that does not make sub-optimal allocations. For both sets of users (those assigned to the 7-day and 14-day treatments), the policy congruent cells are always highest. However, the problem with this policy is that these differences are minor (e.g., in the case of users whom it assigns to the 7-day treatment, the gain of going from 14-days to 7-days is $0.381$). Thus, $\pi_{c\_forest}$ is not able to sufficiently personalize treatments. It is thus similar to the $\pi_7$, which also does not make sub-optimal allocations (but also does not personalize treatment assignment).
In this section, we conduct a series analysis based on the best personalized policy ($\pi_{lasso}$) and derive a set of substantive findings of relevance to SaaS firms in the free trial context. In $\S$(ref), we present some descriptive analysis to explain why/how free trial assignment varies across consumers under the optimal personalized policy. Then, in $\S$(ref), we discuss the improvements in long-term outcomes like customer loyalty and revenue improvements under different counterfactual free trial policies.
So far, we focused on the question of “Who (should get a treatment)”. We now examine the question of “Why (should s/he get a specific treatment)”. Understanding why some users respond well to longer trials while others respond better to shorter trials can give us insight into consumers' preferences and decision-making process. These insights are valuable for two reasons. First, from the firm's perspective, they can be leveraged to improve other marketing interventions such as advertising and pricing. Second, from a research perspective, this gives us a better understanding of the heterogeneity in the effectiveness of trial length on subscription, which can be extrapolated to other firms and settings.
We now segment consumers based on their prescribed treatment (according to $\pi_{lasso}$, since it is the best policy) and correlate their prescribed treatment with pre-treatment demographic and post-treatment behavioral variables. Since these variables are not randomized across individuals, we cannot make any causal inference on their effect on users' responsiveness. Rather, the goal is to present some suggestive evidence for the underlying mechanisms driving users' behavior.
In Appendix (ref), we present Tables (ref) and Table (ref), that show the distributions of pre-treatment demographic variables and post-treatment behaviors for three segments:
We now discuss the attributes and behavior of these three segments.
It worth mentioning that our findings disprove some of the previous theories on the relationship between users' experience and the effectiveness of free trials. For example, Dey_etal_2013 argue that beginners should be given longer free trials because longer trials allow them to learn about the product, which increases their likelihood of subscription. However, we find the opposite -- shorter trials are more effective for beginners and longer trials are better for experienced users. This is because more experienced users are likely to be familiar with the functionalities in the legacy software and need more time to evaluate these functionalities in the new cloud service and compare it with the legacy version. On the other hand, beginners are just starting to get to know the software, and are unlikely to have enough information to evaluate the software. Given that this firm is the dominant player in this market and sets the standards in this industry, a beginner's decision to subscribe or not is likely driven by the brand name rather than their experience during the free trial.
So far we have focused on designing and evaluating policies that maximize subscriptions. However, a policy that maximizes subscriptions (or short-run conversions) may not be the best long-run policy if it brings in users who are less “profitable” or less “loyal”. For example, a policy that increases subscriptions among students (who get a significant educational discount and hence pay lower prices) and/or users who subscribe to lower-end products/bundles (that are priced much lower than the all-inclusive software suite) at the expense of high-end users can lead to lower revenues. Similarly, some policies may increase subscriptions, but do so at the expense of long-term retention, i.e., they may bring in the less loyal consumers who churn within a short period. Thus, a subscription-optimal policy may in fact be sub-optimal from the perspective of long-run outcomes Gupta_etal_2006, Fader_Hardie_2009, Mccarthy_etal_2017.
In this section, we therefore examine the performance of the policies discussed earlier on two important post-subscription outcomes of interest for the firm.
Recall that the summary statistics of subscription length and revenues were shown in Table (ref).
We first describe the procedure for deriving the IPS estimates of the average subscription length and revenue under a given policy. For each policy $\pi$:
The IPS estimates of subscription length and revenues for the counterfactual policies are shown in Table (ref). (By definition, these IPS estimates are for the full population, and not just subscribers.) The main takeaway is that $\pi_{lasso}$ performs the best here also, followed by $\pi_{xgboost}$.\footnote{Tables (ref) and (ref) in Appendix $\S$(ref) show the results for paired t-tests comparing the gains in subscription length and revenue across policies.} Thus, there are no concerns about this policy bringing in more subscriptions at the expense of long-run consumer loyalty or revenue.\footnote{Note that $\pi_{14}$ outperforms $\pi_7$ in the test data (on revenue) even though $\pi_7$ is the best policy in the training data. We present a brief explanation for this discrepancy now. In general, estimates from one data set are valid in another data set only when the joint distribution of outcomes and covariates are similar in both data sets. This is usually true when the data set is very large or the signal-to-noise ratio is low. However, as discussed in $\S$(ref), neither of these are true in our case; so there are some minor differences in our training and test data due to the randomness in the splitting procedure. The main difference is this -- the distributions of subscription length for the 14-day condition in the training and test data are different. This is however not the case for the 7- or 30-day conditions; see Table (ref) in Appendix (ref). Thus, the estimate of subscription length from training data does not translate well to test data, and this leads to the large difference in the subscription length and revenue estimates across the training and test data sets.}
An interesting empirical pattern here is that the gains in subscription length and revenues are quantitatively different from the gain in subscription (comparing the percentage increases in Tables (ref) and (ref)). We now discuss the source of this difference.
We can expand the expected subscription length (denoted by $Y_i^l$) conditional on treatment $W_i$ for the population of users as follows:
where $\textrm{Pr}(Y^s_i|W_i)$ is the probability that user $i$ will subscribe conditional on receiving treatment $W_i$ and $\operatorname{\mathbb{E}} \left[T_{end} - T_{start}|W_i, Y^s_i=1\right]$ is $i$'s expected length of subscription conditional on receiving treatment $W_i$ and subscribing ($Y_i^s = 1$). The reason for the discrepancy in the gains on the two outcomes -- subscription and subscription length -- becomes apparent from Equation (ref). If trial length affects not just subscription, but also how long a subscriber will remain loyal to the firm, then the gains in $Y_i^l$ will be naturally different from the gains in subscription. To examine if this is true in our data, we show the summary statistics for $\operatorname{\mathbb{E}} \left[T_{end} - T_{start}|W_i, Y^s_i=1\right]$ for the three trial lengths in Table (ref) in Appendix (ref). We see that there are some small differences in this metric across the three trial lengths, which account for the differences between the gains in subscription and gains in subscription length.
Similarly, we can write the expected revenue (denoted by $Y_i^r$) conditional on treatment $W_i$ for the population as:
This is similar to Equation (ref), with the additional $\operatorname{\mathbb{E}} \left[\textrm{Price}_i|W_i, X_i, Y^s_i=1\right]$ term. It suggests that trial length can influence revenues through three channels -- (1) subscriptions, (2) length of subscription, and (3) price of the product subscribed. The first two were already discussed in the paragraph above. We now examine whether the products that consumers subscribe to and the prices that they pay are also a function of trial length. That is, we examine whether $\operatorname{\mathbb{E}} \left[\textrm{Price}_i|W_i, X_i, Y^s_i=1\right]$ is indeed a function of $W_i$ in our data. Note that the price that a subscriber pays is a function of both the product that s/he subscribes to (e.g., single product, all-inclusive bundle) as well her/his demographics (e.g., students pay lower prices for the same product). In Table (ref) in Appendix (ref), we present the distribution of products and subscription type (educational, commercial, or government) by trial length for all the subscribers in our data. Again, we see that there are some minor differences in product and subscription types across trial lengths. However, the main reason why revenue gains are significantly different from subscription gains is likely due to the difference in the samples used to calculate the two gains. Recall that revenue gains are calculated on a sample of $334,223$ users for whom revenue data are available, whereas subscription gains are calculated on the full sample of $337,724$ users.
In sum, there are two main takeaways from this section. First, personalized policies designed to optimize short-term outcomes (such as conversion/subscription) also perform well on long-term outcomes (at least in our setting). Second, the magnitude of gains can be different across outcomes since the treatment can affect long-run outcomes through multiple channels (in addition to conversion). In our setting, we find that trial length only plays a marginal role on other factors that affect revenue or loyalty; so the magnitude of the gains across outcomes are similar (controlling for the sample). However, these effects can vary across settings and policy-makers should keep them in mind when designing and evaluating personalized policies.
Free trials promotions are now a commonly used acquisition strategy in the SaaS industry. In this paper, we examine -- (1) how a firm can personalize the length of free trials based on a customer's observed demographics or skill level, and (2) document the returns to personalized free trials on both short-term conversions and long-run outcomes such as customer loyalty and revenues.
We first propose a general framework for personalized targeting policy design and evaluation that is compatible with a high-dimensional covariate space. Our framework consists of three components and is applicable to any setting where researchers have data from a well-designed experiment that satisfies the two important properties of unconfoundedness and compliance. In the first component, we define the optimal policy design problem and present a two-step solution concept. In the first stage, we learn flexible functions of either outcomes or pairwise heterogeneous treatment effects. Then, in the second stage, we use the function(s) learned in the first stage to assign the optimal treatment for each user. In the second component, we design personalized policies based on five outcome estimators (linear regression, lasso, CART, random forest, and boosted regression trees) and two heterogeneous treatment effects estimators (causal tree and causal forest). In the third component, we use the Inverse Propensity Score (IPS) reward estimator to evaluate the reward or gain from any targeting policy.
We apply our framework to data from a large-scale field experiment on free trials conducted by a leading SaaS firm. The firm manipulated the length of free trials such that new users were randomly assigned to 7, 14, or 30 days of free trial. In this setting, the 7-day trial improves subscription by $5.59\%$ over the baseline of 30 days for all policy. We also find that the two outcome estimators -- lasso and XGBoost -- perform the best in our setting, i.e., maximize conversions. In contrast, the two heterogeneous treatment effects estimators -- causal tree and causal forest -- perform poorly because they are unable to personalize the policy sufficiently. Indeed, a key takeaway of our research is that while personalization can lead to substantial benefits, many commonly used methods to personalize are often ineffective. Hence, researchers and managers need to use appropriate policy design and evaluation methods for effective personalization. Finally, we show that, personalized free trial policies designed to maximize short-run conversions also perform well on long-run outcomes such as consumer loyalty and profitability in SaaS settings. We expect our substantive findings to be of relevance to SaaS firms and managers who want to optimize their free trial promotions.
Today's technology and digital marketing firms have unprecedented access to individual-level data on consumers' attributes, the ability to experiment at scale, and customize marketing actions in real-time. This has led to significant interest in the question of how to personalize marketing interventions. However, so far researchers and practitioners have little theoretical or empirical guidance in this task. We hope that the general framework presented in the paper combined with the empirical lessons comparing the performance of a series of estimators will help managers to effectively personalize marketing-mix variables in other settings.