Estimating High-Dimensional Discrete Choice Model of Differentiated Products with Random Coefficients

Estimating High-Dimensional Discrete Choice Model of Differentiated Products with Random Coefficients

Introduction

There are many occasions in which high-dimensional product attributes are available for estimating and predicting demands for differentiated products, especially in context where machine learning techniques such as pattern recognition and natural language processing can generate high-dimensional representation of product attributes. For example, we can consider a consumer choice over clothes. In addition to the typical characteristics of prices, countries of origin, and materials, there are numerous varieties of design patterns that can influence a consumer's choice. The choice over books is dependent on their contents. With a natural language processing technique, we can potentially represent their contents with a high-dimensional semantic vectors. In this paper, we investigate a framework that enables us to integrate these potentially informative but not-yet-fully-used product attributes in estimation, inference, and prediction of demands for differentiated products.

We are not the first considering the estimation of high-dimensional discrete choice model. gillenBLP2LASSOAggregateDiscrete2019 studied an estimation of a model that extends berryAutomobilePricesMarket1995's random-coefficients discrete-choice model with high-dimensional attributes and applied to the study of political campaign for elections. In their model, the product attributes can grow exponentially relative to the sample size; however, random coefficient is only allowed for the price. Under this assumption, the high-dimensional attributes can only affect the mean indirect utility. Thus, they can apply a $l_1$-regularized least squares method to the inverted mean indirect utility to select relevant variables.

Recent methodological developments in statistics and econometrics allows us to apply a high-dimensional estimation procedure to a broader class of models. In particular, the recent work of belloniHighdimensionalEconometricsRegularized2018 provides a set of useful results for $l_1$-regularized minimum distance estimation following the techniques developed by frankStatisticalViewChemometrics1993 and tibshiraniRegressionShrinkageSelection1996. We derive the property of our $l_1$-regularaized minimum distance estimator and its debiased version based on their results. Because the objective function of the BLP model is non-linear in the parameters, we use the contraction inequality theorem of ledouxProbabilityBanachSpaces1991 to control the tail probability of the estimation error. This contraction inequality exploits the Lipschitz continuity of the objective function with respect to a single index. In the context of the BLP discrete choice model with non-linear but smooth moment conditions, we show that the analogue principle can be applied to the case with multiple indices. This allows us to have fixed number of random coefficients on the indices of potentially high-dimensional attributes.

The rest of the paper proceeds in the following manner. In the next section, we describe our model and introduce the regularized GMM (RGMM) problem for our model. In section 3, we show the probability bound for the estimation error from the regularized GMM problem. In section 4, we consider the de-biased procedure for the proper inference. The last section concludes.

Demand estimation as a regularlized GMM

Model

Consider there are $J$ products in each market $i \in \{1,\ldots,n\}$. Let us denote $[\cdot]$ for an integer indicates index set $\{1,\ldots, \cdot\}$. Each product $j$ shares a non-zero demand $S_{ij} \in (0,1)$ in each market $i$. Each product $j$ in a market $i$ has observed $L$ attributes $\{x_{ijl}\}_{l \in [L]}$ including cost of attaining the product $j$, $-p_{ij}$, and an unobserved attribute $\xi_{ij}$.

In this paper, we consider the high-dimensionality in the product attributes $x_{ijl}$. The key restriction is that we assume there are $G$ known finite partitions of the high-dimensional product characteristics $[L]$. In particular, we consider the following indirect utility for a product $j$ in a market $i$: \[ u_{ij} = \sum_{g \in [G]} \sum_{l \in L_g} x_{ijl}(\beta_{l} + \gamma_l \tilde{\beta}_{g}) + \xi_{ij} + \epsilon_{ij} \] where $\epsilon_{ij}$ is an idiosyncratic error term and $L_g$ is mutually exclusive subset of $[L]$ for each $g \in [G]$ such that $\cup_{g \in [G]} [L_g] = [L]$. Note that $\sum_{g \in [G]} \sum_{l \in L_g} x_{ijl}\beta_{l} + \xi_{ij} = x_{ij}'\beta + \xi_{ij}$ with $\beta = (\beta_1,\ldots,\beta_L)'$ captures the mean utility of the product $j$ in the market $i$. Also, $\sum_{g \in [G]} \sum_{l \in L_g} x_{ijl}\gamma_{l} \tilde{\beta}_{g} = \sum_{g \in [G]} x_{ijg}'\gamma_g \tilde{\beta}_{g}$ with $x_{ijg} = [x_{ijl}]_{l \in L_g}'$ and $\gamma_g = [\gamma_l]_{l \in L_g}'$ and $\tilde{\beta}_{g} \sim_{iid} N(0,1)$ capture the individual heterogeneity in the utility for the product $j$ in the market $i$. We assume $\tilde{\beta}_{g}$ is independent of $\tilde{\beta}_{g'}$, and the group specific variance of the random coefficient $\tilde{\beta}_g$ is normalized to 1 for each corresponding vector $\gamma_g$. Therefore, the variance of the indirect utility of the product $j$ in the market $i$ is $\sum_{g \in [G]} (\sum_{l \in [L_g]} x_{ijl} \gamma_l)^2$. This finite indices restriction allows us to apply the contraction inequality theorem, ledouxProbabilityBanachSpaces1991, which is in principle applied to a Lipschitz transformation of a single index. If the random coefficients are not restricted, then the number of indices grows to infinity as the number of attributes grows infinity, and we cannot apply the contraction inequality to bound the estimation error. For the reminder of the paper, let $g(l)$ represents the group $g \in [G]$ that the attribute $l$ belongs to.

The mean utility is the same as usual differentiated product demand model such as the BLP model. The heterogeneity term is different from the usual random coefficient model. This model can be seen as a special case of the usual random coefficient model that the product attributes $x_{ijl}$ and $x_{ijl'}$ share the same individual preference shock $\beta_{g}$ if $l, l' \in L_g$. In other words, consumers observe the set of characteristics $\{x_{ijl}\}_{l \in L_g}$ as a common index characteristics of $x_{ijg}'\gamma_g$, but individuals may have different preference over the index $x_{ijg}'\gamma_g$. In the example of the design pattern, we share the same objective descriptions of the design, but the subjective preference over the descriptions differ across people. An important feature is that we do not restrict $\beta_l = \gamma_l$. Therefore, the mean $\sum_{l \in L_g} x_{ijl}\beta_l$ and the variance $(\sum_{l \in L_g} x_{ijl}\gamma_l)^2$ of each utility load from a group of characteristics $L_g$ may be unrelated each other.

Moment condition

Suppose that $\epsilon_{ij}$ follows iid Type-I extreme value distribution, then the parameters $\theta \equiv \{\beta',\gamma'\}'$ and observed characteristics $x_{ij}$ pin down the share of the product $j$ in the market $i$ as the function of mean utility vector $x_{i}'\beta + \xi_i$ such that \[ s_j(x_{i},x_{i}'\beta + \xi_{i};\theta) = \int_{\tilde{\beta}} \frac{\exp(x_{ij}'\beta + \xi_{ij} + \sum_g x_{ijg}'\gamma_g \tilde{\beta}_{g})}{1 + \sum_{j' \in [J]} \exp(x_{ij'}'\beta + \xi_{ij'} + \sum_g x_{ij'g}'\gamma_g\tilde{\beta}_{g})} dF_{\tilde{\beta}} = S_{ij} \] where $F \sim N_{[G]}(0,I_{[G]})$ and $S_{ij}$ denotes the population share of the product $j$ in the market $i$. \footnote{To simplify the argument, we ignore the measurement error issue of the observed market share from the population share for now.}

Here we assume that the unobserved product type $\xi_{ij}$ is mean independent of a vector of instruments $w_{ij}$, \[ E[\xi_{ij}|w_{ij}] = 0 \] where the expectation is taken over the markets $i$ for each $j \in [J]$. This conditional moment restriction leads to a set of unconditional moment conditions with transformed vector of $K$ instruments for each product $j$, $h_{jk}(w_{ij}), k \in [K]$, such that \[ E[\xi_{ij} h_{jk}(w_{ij})] = 0 \] for each $j \in [J]$. berryEstimatingDiscreteChoiceModels1994 shows that there exists unique inverse functions of the share $s_j(x,\cdot;\theta)$, such that

Therefore, the moment condition is now \[ E[(s_j^{-1}(x_{i},S_i;\theta) - x_{i}'\beta)h_{jk}(w_{ij})] = 0, \forall j \in [J], k \in [K]. \] Now let \[ \xi_{j}(\tilde{x};\theta) \equiv s_j^{-1}(x,S;\theta) - x_{ij}'\beta \] where $\tilde{x} \equiv (x',S',w')'$.

Regularized GMM problem

Following belloniHighdimensionalEconometricsRegularized2018, we consider a regularized GMM approach for the moment condition above. In particular, let $f(\tilde{x};\theta)$ be the score function vector of $(J,K)$ elements with \[ f_{jk}(\tilde{X};\theta) \equiv \xi_{j}(\tilde{X};\theta) h_{jk}(W), \] for each $j \in [J]$ and $k \in [K]$ where $\tilde{X} \equiv (X',S',W)'$, and let \[ f(\theta) \equiv A E[f(\tilde{X};\theta)] \] and \[ \hat{f}(\theta) \equiv \hat{A} \mathbb{E}_n [f(\tilde{X};\theta)] \] with some weight matrix $A$ and its estimate $\hat{A}$, where $\mathbb{E}_n[\cdot]$ represents sample mean of a random vector. For now, let $A = \hat{A} = I$.

The regularized GMM estimator $\hat{\theta}$ solves the following optimization problem \[ \min_{\theta \in \Theta} \|\theta\|_1 : \|\hat{f}(\theta)\|_{\infty} \leq \lambda \] for some regularization parameter $\lambda$.

Bounds on the estimation error

belloniHighdimensionalEconometricsRegularized2018 show the rate of convergence for the estimation error under two conditions in addition to the regularization condition which is the constraint of the optimization problem shown above. Below we cite their statement under three high-level conditions

Identifiability condition

For the second condition of the identifiability, belloniHighdimensionalEconometricsRegularized2018 offers the following sufficient condition

The last condition in the above assumption (ref) is specific to the non-linear problem. Nevertheless, this assumption does not bind in our model because our target moment function is continuously differentiable everywhere.

The second condition in assumption (ref) regulates the modulus of continuity $k(\theta_0,l)$. Lemma 3.1 of Belloni et al. (2018) offers a sufficient condition for the second condition in exactly sparse model. For the linear IV regression model, for any sub-vector of covariates $X$, we need some sub-vector of instruments $W$ such that $E[W'X]$ is non-singular. In other words, there exists some instruments that are strong for any sub-vector of endogenous covariates. In our context of the BLP model, the Jacobian matrix is $JK \times 2L$ matrix with each $l$ entry for $jk$ element as \[ G_{jk,l}(\tilde{X},\theta) =

\] where $s(\tilde{\beta};\tilde{X},\theta)$ is $J\times 1$ vector of $s_j(\tilde{\beta};\tilde{X},\theta) \equiv \frac{\exp(X'_j \beta + \xi_j(\tilde{X};\theta) + \sum_g X_{jg}'\gamma_g \tilde{\beta}_{g})}{1 + \sum_{j' \in [J]} \exp(X'_{j'} \beta + \xi_{j'}(\tilde{X};\theta) + \sum_g X_{j'g}'\gamma_g\tilde{\beta}_{g})}$ and $D_{j}(\theta)$ is the $j$-th row of the inverse matrix of the Jacobian matrix of $s(\tilde{X};\theta) \equiv s(X,X'\beta + \xi(\tilde{X};\theta);\theta)$ vector with respect to the mean utility vector. Therefore, the modulus of continuity condition requires that the variables $h_{jk}(W)$ serve as the strong instruments for the attribute $l$ of the product $j$, $X_{jl}$, as well as the weighted sum of the attributes $l$ of the products $j'$ across the market $X_{j'l}$. This is not a strong restriction for the most of the attributes as we often assume the attributes are exogenous. For the endogenous attributes, we need to be cautious on the restriction as the instruments are not necessarily strong in particular when the asymptotic is considered for the size of markets $J$ rather than the number of markets $n$. See armstrongLargeMarketAsymptotics2016 for the relevant discussion.

Tail probability bound

Now, let $\Delta \theta \equiv \theta - \theta_0$ for any $\theta, \theta_0 \in \Theta$, where $\Theta$ is a subset of $\mathbb{R}^{2L}$ defined such that $\Delta \theta \in \Theta$.

Next, we consider the bound for the tail-probability of the estimation error process: \[ \sup_{\Delta \theta \in \Theta, j \in [J], k \in [K]} \left| \mathbb{G}_n (f_{jk}(\tilde{X};\theta_0 + \Delta \theta) - f_{jk}(\tilde{X};\theta_0)) \right| \] where $\mathbb{G}_n X$ is the empirical process of the sequence $\{X_i\}_{i \in [n]}$.

For the rest of the discussion, we introduce the following indices. For each $g \in \{0,1,\ldots, G\}$ and $j \in [J]$, let $X_{j,g}$ be a sub-vector of $X_j$ corresponding $g$-th partition of $[L]$ with $X_{j,0} \equiv X_j$, and let $\theta_g$ and $\nu_{jg}$ be defined as \[ \nu_{jg}\equiv X_{j,g}\theta_{g} \equiv

\] and \[ \xi_{j}(\nu;\tilde{X}) \equiv \xi(\tilde{X};\theta) \] such that $\nu_{j'g} = X_{j',g}\theta_g$ for every $j' \in [J]$ and $g \in \{0,1,\ldots, G\}$.

Also, let \[ \nu_{j'g0}\equiv X_{j',g}\theta_{g0} \equiv

\] In lemma (ref) in the Appendix, we show that the score functions $f_{jk}$ are Lipschitz continuous in $\nu_{j'g}$ uniformly for every $\nu_{-j',-g}$ with the Lipschitz constant $J$ times some universal constant. Using this property, we employ the Ledaux-Talagrand contraction inequality as follows:

Then the following statement shows the error rate of RGMM BLP estimator:

De-biased RGMM

Given the RGMM estimator $\hat{\theta}$, it is recommended that we update the estimate in order to make a proper inference. De-biased Lasso, or De-biased RGMM procedure in belloniHighdimensionalEconometricsRegularized2018 takes the following steps

First of all, we need to provide maximal inequalities for the auxiliary estimators $\hat{\gamma}$ and $\hat{\mu}$. The strategy follows the parallel argument of the maximal inequality for $\hat{\theta}$. Therefore, we need the following modulus of continuity conditions for $\gamma$ and $\mu$.

Note that the first condition requires that the variance matrices constructed from any elements of the score functions $f_{jk}(X,\theta_0)$ is non-singular, and the second condition follows if all eigenvalues of $G'\Omega G$ are bounded in absolute values from zero uniformly over $n$.

Then given a choice of penalty parameters $\lambda_l^{\gamma}$ and $\lambda_j^{\mu}$, we achieve the maximal inequality for $L^1$-norm of $\hat{\gamma}_l$ and $\hat{\mu}_j$

Next, we need maximal inequalities for the norms $\|\hat{G} - G\|_{\infty}, \|\hat{G} - \tilde{G}\|_{\infty},$ and $\|\hat{\Omega} - \Omega\|_{\infty}$ where $\tilde{G} \equiv -\hat{G}(\tilde{\theta})$ with $\tilde{\theta}$ as the intermediate value of $\hat{\theta}$ and $\theta_0$. Unlike lemma 3.7 of belloniHighdimensionalEconometricsRegularized2018 which assumes the tail probability bound for the process $\hat{\Omega} - \Omega$ and $\hat{G} - G$, we need certain modification of lemma as we do for theorem (ref).

Combining these results, we attain the asymptotic linearlity

Conclusion

In this paper, we propose a $l_1$-penalized estimation for random coefficient logit model of differentiated product demands. Unlike the existing approach, our procedure allows for random coefficients on possibly high-dimensional attributes. Therefore, both of the mean and variance of the indirect utilities may be determined by high-dimensional but sparse set of attributes.

Our strategy bases on the contraction inequality by ledouxProbabilityBanachSpaces1991 as is used in belloniHighdimensionalEconometricsRegularized2018 for the GMM procedure with a single index. We show that the $l_1$-regularized GMM estimation and its de-biased procedure are valid for the BLP model with fixed number of indices generated out of high-dimensional but sparse set of attributes.

Unfortunately, the contraction inequality principle does not apply to a fully flexible random coefficient BLP model as the number of indices grows in the same rate as the number of the attributes. Also, our current result does not accommodate the models with the number of products growing exponentially. These challenges are left for the future work.