Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
70,066 characters · 17 sections · 71 citation commands
A Causal Perspective on Loan Pricing Investigating the Impacts of Selection Bias on Identifying Bid-Response Functions
\setcounter{page}{1}
Pricing loans is a challenging task for banks. A pricing policy must take into account and balance several risks to be profitable. Prices must be both high enough to account for operating costs and losses and low enough to match price sensitivities of customers and to be competitive in the overall market. If a price is too high, customers may reject it and seek a loan at a competitor. However, if a price is too low, the bank may not turn the loan into a profit and possibly suffer losses.
As customers and their preferences are heterogeneous, an optimal price in terms of revenue or profit maximization is a personalized one Varian.1989. In reality, many pricing policies are lacking systematic and data-driven personalization. Instead, prices are typically derived from rigid pricing grids that segment customers into large groups. Individual components, such as a discount, are typically applied at the discretion of a bank clerk incentivized by corporate policies Phillips.2015.
Developing a personalized pricing policy to optimize loan prices is challenging since a bank must know the individual preferences and characteristics of a customer. Typically, these characteristics, such as price sensitivity and willingness to pay, are hidden. Banks must assume or approximate those characteristics to build well-functioning models Phillips.2021. These approximations prove complicated as they rely on observational data of past loan negotiations, introducing several challenges: while the problems of endogeneity and hidden confoundedness are generally prominently studied in the established pricing literature Berry.1994, Villas.1999, selection bias Heckman.1990 is not. Selection bias (or “bid selection bias”, as in the context of this work) persists in observational pricing data due to bias in the established pricing policy. Certain customer groups might receive preferential treatment in accordance with an established pricing grid or regulatory requirements Jain.2016, bank clerks might suffer from unconscious bias in handling customers and applying discounts, and customer groups might self-select in choosing which financial products to purchase.
Our work aims to study the impact of selection bias on learning personalized pricing models from observational data and builds on identifying methodologies that overcome potential adverse effects of selection bias. In order to do so, we propose to frame loan pricing as a problem of causal inference and look at pricing as a case of treatment and effect. The offered loan price, the “bid”, is considered to be a continuously-valued treatment, whereas the treatment effect is the probability of a customer accepting this bid, i.e., the “bid response”. A model of the individual treatment effect, that is, the individual bid response, can be used to optimize and personalize pricing. Applications include decision support for bank clerks, data-driven allocation of personalized prices, and improved customer segmentation in existing pricing policies.
To study the effects of bid selection bias, we work with a semi-synthetic dataset created in collaboration with a bank in Belgium. We apply a set of machine learning techniques, including novel methods from causal machine learning, to estimate bid-response curves and evaluate their robustness to varying levels of selection bias in the training data.
To the best of our knowledge, our study is the first of its kind and adds to the literature in the following ways:
The remainder of our work is organized as follows: in Section (ref), we provide an overview of the established literature on customized pricing. Section (ref) defines pricing as a problem of causal inference and discusses reasons for selection bias, as well as methods for bid-response estimation. Section (ref) describes our experimental evaluation, including the generation of our dataset. We conclude in Section (ref), in which we reflect on results and discuss potential future research.
Customized pricing: Loan pricing is a case of customized pricing Phillips.2012, Phillips.2013. Contrary to the pricing of uniform goods Varian.2014, in customized pricing, neither goods nor prices are homogeneous across customers Phillips.2021. In loan pricing, customized products and prices can better meet customers' financing demands and offer banks a way to manage and mitigate risks. Customization of the product occurs by varying the duration, amount, and starting principal of the loan. Additionally, there typically is no price transparency, as customers cannot observe prices offered to other individuals. This changes the objective from setting a single best price for a customer group to setting the optimal price per individual. As a result, customized pricing is often considered an optimization problem Oliver.2014, Sundararajan.2011, Phillips.2012 that aims to optimize a certain business objective. We summarize the customized pricing process as such: after a customer has approached a seller, they request a price for a product, commonly referred to as a bid. The seller will try to determine the bid that maximizes a certain objective, for example, revenue or profit generation per customer, which is facilitated by estimating the probability that a bid is accepted. The probability of acceptance as a function of the height of the bid is referred to as the bid-response function. It differs from the demand curve in microeconomics by representing a probability concerning a single individual instead of an aggregated demand over a customer group. Hereby, customized pricing is related to the wider field of market response modeling, to which an introduction can be found in, e.g., Hanssens.2003. For the case of loans, we describe an exemplary loan pricing and sales process in Section (ref).\\
Endogeneity in pricing: Traditionally, estimation of the bid response is approached with parametric and semiparametric methods, assuming a certain functional form of the bid response Phillips.2021, vanRyzin.2012. This estimation is subject to various challenges in terms of, for example, the available information and infrastructure, model complexity, and legal and regulatory requirements Phillips.2012. The most prominent challenge discussed in the established pricing literature is “endogeneity” in observational training data (or “hidden confoundedness” in the wider sense). Endogeneity is present if variables critical to explaining customer preferences or their actions are unobserved (or not available for modeling) Phillips.2012a. Reasons for endogeneity are plentiful, stemming, for example, from interactions between customers and sales representatives that are not or insufficiently recorded, due to internal and external regulations. There is a wide body of literature on identifying and handling endogeneity, both in pricing and more generally in economics and statistics Heckman.1978, Newey.1987, Rivers.1988, Kuksov.2008. De facto standard methods are instrumental variable regression Hausman.2012, and BLP procedures named after the authors of Berry.1995. More recently, machine learning has been proposed to finding bid-response models Lawrence.2003, Agrawal.2007, Arevalillo.2019, Guelman.2014.\\
Selection bias in pricing: The impact of selection bias (cf. Section (ref)) on pricing has received little attention in the literature. There is, however, an established body of research from other domains on the issue, such as medicine Berrevoets.2020, economics Varian.2016, market modeling Ferkingstad.2011, and maintenance Vanderschueren.2022.
To showcase the impact of selection bias on customized pricing, we propose a methodology in four steps. In Section (ref), we describe the general loan pricing and sales process and frame it as a problem of causal inference. In Section (ref), we build on this framing and introduce selection bias in observational data and its relevance towards pricing. In Section (ref), we discuss the necessary assumptions for the causal identification of bid responses. Finally, in Section (ref), we present the methods that we apply to learn bid-response functions from observational data. Our experimental setup and results follow in Section (ref).
The general process of a customer taking out a loan is formalized as follows:
For the purpose of our study, we simplify the process and look at it from the perspective of a single bank. The process reduces to three steps: a customer approaches the bank, the bank provides a loan offer including a bid for the price of the loan, and finally a customer decides on acceptance or refusal of the offer. The simplified process is depicted in Figure (ref).\\
Loan price decomposition: In practice, a loan price often consists of multiple parts, such as a base price and a customer-specific component. The base price serves to cover risks and costs to the bank and is determined via risk-based pricing. For risk-based pricing, banks cluster customers into a finite number of risk bands and determine a price per risk band that covers running costs of the bank, as well as expected losses Phillips.2013. Risk models are typically proprietary, resulting in price dispersion in the market. The individual component to a loan price is given by a bank clerk during the sales process based on their information and personal assessment of the customer. Discounts might be driven by incentive schemes resulting from corporate strategy. The impact of such incentive schemes Garrow.2006, Phillips.2015 and the risk of principal-agent problems Berhold.1971,Ross.1973 is out of scope in this work. Additionally, we do not take into account the impacts of negotiations between customers and bank clerks that might occur.\\
Loan pricing as a problem of causal inference: We assume that a bank has records on their historical sales processes available, which are formalized as such: after a customer $i$ has decided about a specific loan offer, the result is summarized as a tuple $(\mathbf{x}_{i},b_{i,f},y_{i,f})$. All values are realizations of random variables $(\mathbf{X},B,Y)$. $\mathbf{X} \in \mathbb{R}^{d}$ describes the characteristics of the customer, as well as the specifications of the loan, excluding the bid, and information on competitor offers\footnote{We assume that all information relevant to the customer decision is known and in $\mathbf{X}$ to satisfy necessary assumptions (cf. Section (ref)).}. $B \in \mathbb{R}$ is the bid made to the customer for the loan price, i.e., the credit rate in percent of interest per year. $Y \in \{0, 1\}$ is the decision to accept the offer and take up the loan or not. We assume $Y$ to be a Bernoulli random variable that is dependent on $P \in [0,1]$, the probability of accepting a loan offer; hence, $Y \sim Bernoulli(P)$.
In framing loan pricing as a problem of causal inference, we adopt the Neyman-Rubin potential outcome framework Rubin.2004, Rubin.2005 and assume that for every realization of a bid $b$ we find a potential outcome $P(b)$ of the probability of accepting the loan offer and a realization $Y(b) \in \{0,1\}$. All observed data and corresponding probabilities are referred to as “factual”, including the factual bid $B_{f}$, the factual acceptance probability $P_{f} = P(B_{f})$, and the factual acceptance $Y_{f} \sim Bernoulli(P_{f})$. Any potential outcomes to unobserved combinations of a customer characteristic and a bid are referred to as “counterfactual outcomes”.\\
Bid response as a treatment effect: For our study, we assume that a bank is interested in identifying the individual response of a customer to a certain bid conditional on the customer's characteristics and the loan they have applied for. Such a model could be used for data-driven bid allocations, improved customer segmentation for risk-based pricing, or to support bank clerks in defining the optimal discretionary discount to give to a customer to make a winning bid. To win a customer, two requirements must be met: on the one hand, a bid cannot exceed the overall willingness-to-pay of the customer, that is, even without competition, a customer is not willing to pay any price for the loan. On the other hand, the loan conditions offered (including the bid) must be more favorable to the customer than those of any of the other competing banks. We note that there might be situations in which a customer might not just pick a bank based on loan conditions but based on personal relations or past experiences. Modeling such a behavior is out of the scope in our study.
In our experiments, we aim to find an unbiased estimate of the individual bid-response function, i.e., the probability of accepting a loan conditional on the bid and customer characteristics:
To evaluate how a bank could use information about $\mu(b,\mathbf{x})$, we assume that a bank wants to determine the optimal bid $b^{*}_{i}$ per customer that maximizes the expected revenue from the loan pricing and sales process. We expect the revenue $R$ from a loan offer to be proportional to both the height of the bid $b$ and the decision to take out the loan with respect to the bid $Y(b)$:
The optimal bid $b_{i}^{*}$ for an individual $i$ is defined as the bid that maximizes the expected revenue for the customer characteristics $\mathbf{x}_i$:
Our formalization of the loan pricing and sales process makes two simplifications that we want to motivate: first, in reality, a bank is interested in profit over revenue maximization. Yet, to maximize profit, further assessments are needed, such as those of risk. These assessments are beyond the scope of this work. In addition, the bid response might be a component of assessing risk, thereby motivating this simplification. Second, we do not take loan negotiation rounds into account. In several markets, loan pricing and sales processes involve multiple rounds of iterative bidding. We simplify this process and assume that only one bid is made by the bank, as stated before.
We are interested in learning $\mu(b,\mathbf{x})$ from observational data. This, in itself, is an intricate task, as for every observation, one observes the outcome of a loan pricing and sales process to only the factual bid $b_{f}$ and not to any other. This is known as the “fundamental problem of causal inference” Holland.1986. Furthermore, for a single customer, there is only a small number of observations available, as a single individual takes out only a low number of loans in their lifetime and not necessarily all at the same bank.
The task is further complicated as the observational data that can be used to learn $\mu$ are expected to be collected under a specific pricing policy, in contrast to data collected using predefined experiments, for example, randomized controlled trials Rubin.1974, Angrist.2009, Deaton.2018. This established pricing policy introduces selection bias in the data. Selection bias describes the relationship between the factually assigned bid (or any intervention more generally) and the pretreatment covariates of an observation. Figure (ref) shows the dependency of the factual bid and the pretreatment covariates of an observation, as assumed in the remainder of this manuscript. Running randomized control trials in loan pricing is infeasible, for example, due to regulation, compliance, ethical concerns, and costs.
The problem of learning effects of an intervention under the presence of selection bias has been studied for both the case of binary interventions Rosenbaum.1983, Rosenbaum.1984, Yoon.2018 and continuously-valued interventions Imbens.2000, Imai.2004, Hirano.2004, Bica.2020. Adverse effects of selection bias on traditional approaches to estimating intervention effects have been studied across fields Schwab.2020, Vanderschueren.2022.
We elaborate on three exemplary reasons for bid selection bias in pricing data:
In the literature, we find three assumptions that typically underly the estimation of intervention effects from observational data Imbens.2000, Lechner.2001. We will discuss these assumptions and their applicability in pricing:
Overlap requires that for every possible customer characteristic $\mathbf{x}$, every possible bid $b$ has a nonzero possibility of being offered. In reality, an established pricing policy might assign specific bids with little to no deviation to customer groups, resulting in a violation of the overlap assumption. Overlap might be ensured, however, due to bank clerks assigning discretionary discounts of sufficient size to their customers. The dispersion of bids ensures that the true bid response can be estimated for all possible bid levels for a certain customer.
Consistency means that the observed outcome is the true potential outcome for the applied treatment. As Vanderschueren.2022 note, this appears to be a straightforward assumption. To ensure consistency in pricing, we must carefully observe any changes in economic sentiment and correct or retrain models if, e.g., prime rates change. For example, if monetary policy tightens, the response of a customer to a certain bid $b$ will likely change as well. For our experiments, we assume that the economic sentiment is stable and that consistency can be assumed.
Unconfoundedness requires that all information that determined the factually assigned bid is known and available for modeling. In pricing, this assumption is easily violated, e.g., due to unrecorded interactions between bank clerks and customers Phillips.2012a, legal bounds on the use of personal data, and missing information about competitor offers, commonly referred to as “competitive uncertainty”.\\
For the purpose of our work, we do not further evaluate the impacts of violating these assumptions on estimating bid responses. Instead, we assume the assumptions are met, to isolate the effect of selection bias.
We evaluate a total of seven methods to model individual bid responses, covering parametric models, nonparametric models, and causal and noncausal methods, as depicted in Table (ref). Each method is discussed further below:
We consider four noncausal methods for bid-response learning to test their robustness to selection bias and appropriateness to modeling bid-response functions: first, na\"ive pricing is a strategy in which we do not estimate any dose response but assume that the best bid $b^{*}$ per customer is the factually assigned bid under the established pricing policy. Second, logistic regression is a classification method that is widely adopted across scientific domains and industry applications, such as fraud detection, credit risk modeling, and churn prediction. It is also the de facto standard in estimating bid-response models vanRyzin.2012 due to its well-suited hypothesis space. Logistic regression is fully parametric and easy to interpret. Third, we consider a random forest classifier Breiman.2001. The random forest classifier combines multiple decision trees into an ensemble to improve the accuracy and reduce the variance of the final model. Random forests can handle a variety of input variables and are robust to overfitting. Random forests, and tree-based methods more generally, have found wide adoption in industry applications of machine learning, motivating our choice to adopt them in our experiments. Fourth, we fit an artificial neural network, i.e., a multilayer perceptron (MLP). MLPs are universal function approximators Hornik.1989 and are powerful methods for a multitude of prediction tasks, making them an important benchmark for our experiments.
Additionally, we implement three causal methods that are meant to handle selection bias by design, expecting that these methods will show improved robustness to increasing levels of bid selection bias in pricing data. First, we implement the Hirano Imbens estimator Hirano.2004, which builds on the idea of the generalized propensity score, as proposed by Imbens.2000. The HIE is an imputation-type estimator. Initially, a model of the treatment assignment is built. Based on that model, the propensity for every observation and their factual bid is calculated. Finally, the bid response is modeled as a function of the bid level and the propensity score. We follow Schwab.2020 and Bica.2020 in considering a linear relationship between bid and pretreatment covariates with normally distributed error terms, as well as a logistic regression on the second-order polynomial of bid level and propensity score as the final model of the bid response. Second, we implement DRNets Schwab.2020. DRNet extends TARNet Shalit.2017 for estimating effects of binary interventions to the continuous case, by training a neural network with $\rho$ individual prediction heads for observations with different levels of a continuous intervention. DRNets learn a shared representation of the training observations, which implicitly have a regularizing effect on overfitting the training data. Third, we implement VCNets. Nie.2021 propose VCNets as an extension to DRNets, by proposing a varying coefficient neural network, where coefficients of the neural network are a function of the continuously valued intervention, enforcing a continuous dose response function, over the tendency of DRNets to estimate discontinuous functions. We use the architecture proposed in Nie.2021. We illustrate the architectures of DRNets and VCNets in Figure (ref). Finally, we would like to motivate our choice of not including SCIGAN Bica.2020, another powerful causal machine learning method for the estimation of continuously valued interventions, in the experiments. Even though SCIGAN has shown high performance in previous studies Bica.2020, Vanderschueren.2022, the set-up of using generative models for the generation of counterfactuals to learn unbiased responses is not suited for the case of pricing, as the training data are not continuous. In our own experiments, SCIGAN performed worse than both DRNets and all other methods considered.
We use a real-life dataset on mortgage loan offers from a bank in Belgium. The data cover over 12,000 loan offers. Each offer relates to precisely one customer. In collaboration with the bank, we selected and processed an anonymized subset of 13 variables for training. The selection of variables was based on their availability for all customers to ensure no missing data, an expert assessment of their relevance for estimating price sensitivity. Categorical variables have been dummy encoded, and all variables have been standardized. These variables include information on terms and conditions of the loan (5 variables), financial background of the customer (3 variables), the socioeconomic background of the customer (4 variables), and the existing relationship of the customer with the bank (1 variable). The precise definition of the variables is confidential information that cannot be shared.
As highlighted in Section (ref), evaluating the correctness of predicted potential outcomes is challenging. First, we never observe the true bid-response curve but only the outcome to one bid per customer due to the fundamental problem of causal inference. Second, estimating the overall effectiveness of one pricing policy over another over a group of customers is complicated due to the observational data. Methods designed to estimate such an aggregate effect, such as the Qini-coefficient Radcliffe.2007, require randomized controlled trial (RCT) data Angrist.2009 and are negatively affected by selection bias. Due to compliance, regulation, and the risks of losses and reputation, such trials cannot be implemented reasonably in many real-life settings, including banking operations.
To overcome this issue, we make use of semisynthetic data, a common approach in evaluating potential outcome estimation and to avoid the fundamental problem of causal inference Yoon.2018, Berrevoets.2020, Schwab.2020, Qian.2023. In such a setup, typically, the explanatory variables in the data are kept as-is, whereas both treatment variables $B$ and outcomes $P$ and $Y$ are generated from a well-defined and known ground truth to control factors such as the strength of the selection bias and the functional form of the potential outcome in relation to $\mathbf{X}$. We deem the use of semisynthetic data in pricing as particularly interesting to assess the impacts of selection bias on modeling bid responses.
In the following two subsections, we discuss how we model the underlying ground-truth bid response (Section (ref)) and assignment of factual bids (Section (ref)). For our experiments, we use 70% of the total data for training, 10% for validation and parameter tuning and 20% for testing.
Based on vanRyzin.2012 and Phillips.2021, we define four requirements with regards to a synthetic ground-truth bid-response function. In our experiments, we standardize bids to be in the domain $[0,1]$, with $b=0$ the lowest observed bid and $b=1$ the highest observed bid:
Furthermore, we consider two different ground-truth bid-response curves: a Richards curve and a stacked sigmoid curve (Figure (ref) provides a comparative visualization of both curves).\\
Richards curve: The generalized logistic function or Richards curve Richards.1959 is defined as:
with $1 - \alpha$, the left asymptote (or maximum level of acceptance at the lowest observed bid $b_{min}$), $1 - \beta$, the right asymptote (or minimum level of acceptance at the highest observed price $b_{max}$), $\gamma$ the steepness of the curve and $\delta$ the position of the turning point of the curve. The logistic function has seen various applications across domains and is often used as a modeling tool for classification tasks Cramer.2002.
To condition $p(b,\mathbf{x})_{RC}$ on individual customer characteristics, we parameterize the function by means of linear combinations of the input features in $\mathbf{x}$ via:
For all $i \in \{1,2,3,4\}$, $\mathbf{w}_{i} \sim \mathcal{U}((0,1)^{d \times 1})$, as proposed in Bica.2020.\\
Stacked sigmoid: To test the performance of models in the case of highly nonlinear bid-response curves, we evaluate them considering the bid response as a mixture of two sigmoid curves. We define:
$\sigma(x)$ is defined as a sigmoid curve:
As before, we condition $p(b,\mathbf{x})_{SS}$ on individual customer characteristics via:
For all $i \in \{1,2,3,4\}$, $\mathbf{w}_{i} \sim \mathcal{U}((0,1)^{d \times 1})$.
Next to the simulation of the bid response, we control the levels of selection bias by sampling the factual bid $b_{f}$ from a beta distribution. The approach is motivated and detailed in Bica.2020. We model varying levels of selection bias by assigning to every observation $\mathbf{x}$ a factual bid:
where $\theta \geq 1$ controls the level of selection bias. $\theta = 0$ results in no selection bias. $\mu$ defines the modal rate, i.e., the most likely assigned price, and is defined as
with $\mathbf{w}_{5} \sim \mathcal{U}((0,1)^{d \times 1})$. Hence, the distribution of the assigned bids in the dataset depends on $\mathbf{X}$, thereby introducing selection bias. Different levels of bias are illustrated in Figure (ref).
Finally, in addition to directly controlling the level of selection bias, we use the observed bid levels from the original dataset to assess the impact of levels of bid selection bias observed in real-world scenarios.
We evaluate our method using three performance metrics:\\
Mean Integrated Squared Error (MISE): The MISE evaluates the potential of a method to predict the true bid response per customer along all observed bid levels:
with $n$ the number of test observations, $b_{min}$ the lowest bid, and $b_{max}$ the highest bid observed in the training data\footnote{Due to the overlap assumption (cf. Section (ref)), we only evaluate bids between $b_{min}$ and $b_{max}$}. The MISE was initially proposed for individual continuous treatment effect estimation by Schwab.2020\footnote{In addition to to evaluating the bid response, we evaluate the MISE over the expected revenue in accordance with Equation (ref), further referred to as the MISE R. The results can be found in the Appendix.}.\\
Policy error (PE): The PE evaluates how well a method is able to identify the optimal bid $b^{*}$ and was proposed by Schwab.2020:
with $n$ the number of test observations, $b_{i}^{*}$ the actually optimal bid of observation $i$, and $\Hat{b_{i}^{*}}$ the estimated optimal bid.\\
Brier score (BS): Finally, we evaluate each model in terms of their Brier score Brier.1950 to assess the potential to estimate the factual outcome of a loan pricing and sales process. The Brier score informs about a model's capacity to estimate the factual response:
with $y_{i,f}$ the factual outcome of observation $i$ and $\hat{\mu}(b_{i,f},\mathbf{x}_{i}$ the estimated probability of bid acceptance of observation $i$ for their factually assigned bid.
We implement our experiments in Python 3.9. Neural network-based methods are implemented in PyTorch Paszke.2017. Random forest models and logistic regression are implemented using the SciKit-Learn library Scikit.2011. HIE is implemented using the statsmodels library Seabold.2010. For VCNets, we use the implementation provided in the original work of Nie.2021. For a list of hyperparameters used to train each method, we refer to Section (ref) in the Appendix. All code used for this project is available online at:
We apply our data generating process to the data presented in Section (ref). We run the process a total of 10 times for each of seven bias strengths increasing from no bias ($\theta = 0$) to heavy bias ($\theta = 20)$, as well as the factual bids. The stated performance metrics are averaged over each of the 10 randomly initialized runs.
The metrics introduced in Section (ref) enable evaluation of each model in terms of its capacity to predict the bid response along all observed bids (via the MISE metric) and its appropriateness for operational decision-making to set an optimal bid $b^{*}$ (via the PE metric). The Brier score, moreover, evaluates the ability to estimate the factual outcome of an observation.
Note that the stated performances depend on both our data generation process (cf. Section (ref)) and the operational objective we have set (cf. Section (ref)).\\
Dose response prediction (MISE, Tables (ref) and (ref)): If the true dose-response function assumes the shape of a Richards curve (cf. Table (ref)), the standard neural network (MLP) achieves the best performance. Second best are standard logistic regression and VCNets. Contrarily to what we had initially assumed, logistic regression and the MLP are seemingly least affected by the increase in selection bias, whereas, e.g., DRNets (a causal method) sees an increase of almost 100% in MISE from no bias to the strongest bias.
The strong performance of logistic regression is likely explained by its hypothesis space being closely related to the ground-truth bid response in the simulation. As the hypothesis spaces are comparable, standard logistic regression is likely able to find a well-performing model by averaging over the observed data. Likewise, DRNet is not able to adequately fit the dose responses. We relate this to the criticism of Nie.2021 on DRNets being prone to fitting discontinuous functions due to estimating separate head networks for different bid levels. This can explain why VCNets improve upon DRNets in the case of the Richard's curve, as their estimated bid-response curve is continuous. HIE does not improve upon the nonparametric causal methods, which might be due to the functional form chosen for modeling the treatment assignment or bid response.
If we assume higher degrees of nonlinearity in the dose response, i.e., in the case of a stacked sigmoid ground truth (cf. Table (ref)), we observe an increasingly adverse effect of selection bias on estimating the bid response. The logistic regression model is not able to model the functional shape of the bid response with an increasing MISE for higher levels of selection bias. The random forest classifier is again most affected by selection bias, performing worst of the methods we have applied. As with the Richard's curve, the MLP achieves superior performance when no selection bias is present. This time, when bias increases, the MLP is significantly affected by the selection bias in the data. In this setting, DRNets exceed the performance of other methods, generating second-best results for low levels of bias and achieving best results for bias levels larger than $\theta = 2.5$, as well as for the bid levels observed in the original dataset. DRNets even outperform VCNets, indicating their applicability to causal effect estimation when the training data are binary. VCNets appear to be not well suited in these kinds of settings, as soon as there is a larger degree of nonlinear relations in the data.\\
Optimal bid prediction (PE, Tables (ref) and (ref)): In terms of policy error, we observe similar results as with the MISE. If the true dose response is a Richards curve (cf. Tabel (ref)), the best-performing models in terms of PE are the MLP and logistic regression. The added flexibility of nonparametric methods, or the use of causal methods, is not sufficient to outperform these two baseline approaches. Again, the method seemingly most affected by selection bias is the random forest classifier.
With increased nonlinearity, the causal methods again score better and best. For moderate levels of bias, VCNets outperform DRNets, likely due to their continuous estimate of the bid response over the discountinuous function of DRNets. If the bias increases further, this benefit is not sufficient to overcome the performance of DRNets, seemingly a more robust estimator.\\
Factual outcome estimation (BS, Tables (ref) and (ref)): Evaluating each of the methods in terms of the Brier score illustrates the need to look at pricing from a causal perspective. If models are solely tested on their predictive performance within the established pricing mechanism, the results are robust across all levels of selection bias. Most noteworthy, the random forest classifier achieves second-best performance under the highest level of selection bias (cf. Table (ref)), even though it achieves the lowest performance in terms of MISE and PE. This finding stresses that the Brier score cannot be used to evaluate how well a pricing model can generalize across bid levels, i.e., its generalization power. Hence, testing and selecting models based on the Brier score might even introduce vicious cycles Mansoury.2020, Adam.2020, an important threat that must be further investigated in the case of banking and lending operations.
In this paper, we explore the use of (causal) machine learning to estimate models of individual bid-response curves from observational loan pricing data. We investigate the adverse effects of bid selection bias. In a series of experiments, we evaluate the impact of varying levels of bid selection bias using a semisynthetic dataset on mortgage loan applications and test several data-driven approaches in learning individual bid-response models. Thereby, we frame pricing as problem of causal inference and, more specifically, of individual treatment effect estimation.
The presented results show that ensuring robustness to bid selection bias is crucial in developing a data-driven pricing solution, which potentially is an overlooked issue and threat both in literature and industry. Failing to adjust for selection bias may significantly impact bid-response models that are obtained from observational data and, as such, introduce systematic errors. This, in turn, might induce vicious feedback cycles in which the effectiveness of a pricing policy is both overestimated and suboptimal. Tree-based models, such as random forests, seem especially unable to accurately predict counterfactual outcomes, even when their performance on estimating the outcomes of an existing pricing policy might be convincing. The wide adoption of tree-based methods in the industry makes this a substantial threat that calls for further research. Finally, we show that approaches from causal machine learning prove potent in overcoming issues that result from selection bias, at least when the underlying assumptions discussed in this paper are satisfied.
We draw two key conclusions from our seminal work:
Having adopted a semisynthetic set of experiments, we want to highlight the assumptions and limitations of our work: in our data simulation (cf. Section (ref)), we purposefully satisfy both overlap and unconfoundedness, two assumptions that are deemed critical for the application of causal inference. Both assumptions might be violated in real-life data. We currently assume that the decision to accept a certain bid is a function of only the observed characteristics $\mathbf{X}$ and the bid $B$. This does not hold in reality, where hidden confounding is likely to occur, as explained in Section (ref) and prominently discussed in previous studies on endogeneity in pricing (cf. Section (ref)). We see two interlinking components: first, customers compare bid offers obtained from different banks, which are typically not observed. Second, some available data are not permitted to be used for business decision making, for example, under regulation 2016/679 of the European Parliament\footnote{Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) [2016] OJ L 119/1.} (GDPR). Finally, our experiments do not consider any temporal component in the training data. If economic sentiment changes and average credit rates decrease, the bid response, $\mu(b, \mathbf{x})$, ceteris paribus, will certainly shift, but possibly change in form as well.
We conclude with an outlook on future work. First, we believe that developing and testing causal machine learning methods for business processes such as pricing is crucial but has not yet been tackled in the established literature. Specifically, we call for the development of methods that learn an individual probability estimate as an effect on a continuously valued intervention from binary training data. This research might advance the application of causal machine learning in many other fields, such as risk assessment and policy evaluation. Second, further research should evaluate to what extent violations of the assumptions in our study impact the performance of causal methods and to what extent assumptions such as overlap and unconfoundedness are violated in practice, potentially requiring the development of alternative solutions. Such research might have critical importance to the adoption of causal machine learning in practice. We link this stream of future research to Bertsimas.2022 and the notion that for operational decision-making, a true, unbiased causal model (by the standards of meeting all requirements discussed in Section (ref)) might not always be needed. Third, we highlight the need for research in identifying the strength of selection bias in data. While counterfactual data will never be available for modeling, the strength of selection bias could be assessed. Such a methodology would be able to inform decision makers on whether a causal approach must be considered. Finally, our performance metrics rely on relative performance per customer and do not take absolute costs and revenues into account. We believe that pricing would therefore be a promising application of cost-sensitive approaches in causal inference, such as introduced in Verbeke.2020 and Verbeke.2023.
The authors want to thank AXA Bank Belgium, Crelan, and the involved project team who have supported this project throughout by providing feedback, know-how, and access to the data used for this research project.\\
This work was supported by the FWO research project G015020N.
The authors state no conflict of interest.
\setcounter{table}{0} \setcounter{figure}{0}