Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
14,752 characters · 7 sections · 23 citation commands
An MPEC Estimator for the Sequential Search Model
The consumer search process, through which individuals gather information about choices, is essential to understanding decision-making behavior. This process has become increasingly observable to researchers through browsing data, which reveals the options considered before a final choice. The availability of such data has allowed for the estimation of structural models of consumer behavior, as noted by ursu2023sequential. The current benchmark model solves implicit functions related to reservation prices using a fixed-point approach, which is computationally demanding.
To address this, we propose an estimator based on the Mathematical Program with Equilibrium Constraints (MPEC) approach su2012constrained. MPEC is a constrained optimization problem subject to equilibrium conditions, that avoids iterative solutions to the fixed-point problem and removes approximation and estimation errors — a key issue in demand estimation dube2012improving, dynamic programming su2012constrained, and misclassification models lu2014mpec.
Monte Carlo simulations show that MPEC performs comparably, with lower bias and root-mean-squared error (RMSE) in small samples but higher bias and RMSE in larger samples, relative to the common estimation method using an ad hoc look-up table. The MPEC approach is particularly valuable for identifying parameters comparable to the benchmark, eliminating the need for the look-up table. MPEC can effectively construct this table by requiring the equilibrium constraints to be satisfied during estimation. We conclude that MPEC is a useful alternative for obtaining benchmark estimates before advancing to more complex models, especially when approximation and estimation errors from an ad hoc look-up table are unknown to researchers.
We construct the sequential search model based on weitzman1979optimal. A decision maker $i$ faces a set of boxes $\mathcal{J}=\{1,\cdots,J\}$ and box $j$ gives a potential reward $u_{ij}\in \mathbb{R}$ independently drawn from a known distribution $F_{ij}(u)$. Opening box $j$ takes cost $c_{ij}\in \mathbb{R}^{++}$. An outside option is denoted as $j = 0$ with a known reward $u_{i0}$ is available at no cost. The decision maker opens boxes via sequential search steps and her goal is to maximize her expected reward net of total costs.
Suppose that the decision maker has opened a set \(S_{i}\) of boxes, which revealed a maximum reward value of \(u_{i}^{*}=\max _{j \in S_{i} \cup 0} u_{i j}\), and \(\bar{S}_{i}\) unopened boxes can still be opened. Her dynamic programming problem choosing whether to stop opening boxes and get payoff \(u_{i}^{*}\), or to continue opening boxes is described by the following Bellman equation:
where \(W_{j}\left(\bar{S}_{i}, u_{i}^{*}\right)\) is the expected value of continuing to open boxes and is defined as
The reservation utility of a product \(z_{i j}\) is the utility level defined as
A set of optimal decision rules, developed by weitzman1979optimal, is used to characterize consumers' optimal search and choice strategies. The rules are as follows:
Then, the optimal search and choice decision rules are expressed as follows:
Empirical economists often assume consumer $i$'s utility defined as
where \(\delta_{i j}\) is utility which is known by the consumer prior to search (“pre-search utility") and \(\varepsilon_{i j}\) is utility that is only known by the consumer after search (“post-search taste shock"). We assume that the pre-search utility \(\delta_{i j}\) consists of a component \(\xi_{i j}\) that can be observed by the researcher and a pre-search taste shock \(\mu_{i j}\) that cannot be observed by the researcher. According to ursu2023sequential, we need to further normalize their variance by setting $\sigma_{\mu}= \sigma_{\varepsilon} =1$.
Under the assumption of normally distributed post-search taste shocks, we can derive the following expression for the reservation utility:
where \(m\left(c_{i j} \right)\) is the implicit function that solves the following equation (see kim2010online):
with \(\phi\) and \(\Phi\) denoting the standard normal pdf and cdf, respectively. weitzman1979optimal shows the existence and uniqueness of the solution of (ref).
There are four primary methods to solve (ref). The first method, proposed by kim2010online, involves pre-computing the mapping between \( m \) and \( c \) and storing it in a look-up table. The second method, suggested by jiang2021consumer, employs Newton's method to compute reservation utilities by iteratively improving approximations to the root of the function:
\[ q(m) = (1-\Phi(m))\left(\frac{\phi(m)}{1-\Phi(m)} - m\right) - c = 0. \]
The third approach, introduced by elberg2019dynamic, uses a contraction mapping defined as:
\[ \Gamma(m) = -c + \phi(m) + m \times \Phi(m). \]
The fourth method, proposed by morozov2023measuring, directly estimates \( m(c_{ij}) \).
ursu2023sequential highlight limitations in each method: (1) the first method introduces errors due to linear interpolation for search cost values that do not align with grid points; (2) the second and third methods avoid interpolation errors but require iterative computation of \( m \) and a convergence threshold, which can cause numerical errors if the threshold is too loose; and (3) the fourth method involves estimation errors for \( m(c_{ij}) \). In practice, the second and third methods typically converge quickly, allowing for tight convergence thresholds that minimize numerical issues ursu2023sequential. Similar challenges in demand estimation are addressed by the MPEC approach.
As a fifth method, we propose a straightforward estimator for the sequential search model, utilizing the Mathematical Programming with Equilibrium Constraints (MPEC) approach introduced by su2012constrained. The MPEC estimator bypasses the need for iterative computations to find the fixed point by treating the equilibrium equations as constraints.
Let \(\theta\) represent the set of parameters. The MPEC estimator solves the following constrained optimization problem:
where individual likelihood $L_{i}(\theta,(z_{ij})_{j\in \mathcal{J}},(u_{ij})_{j\in \mathcal{J}})$ is derived as
A remarkable advantage of MPEC is that it does not need an ad hoc look-up table which is unknown to researchers and does not incorporate approximation and estimation error of equilibrium constraints (ref), in addition to the main advantage of MPEC that it does not need to solve the fixed point problem iteratively. Detailed construction is provided in the Appendix.
For comparison, we follow the parameter settings for \(\xi_{ij}\) and \(c_{ij}\) in (ref) as described in Appendix B of ursu2023sequential. We evaluate the MPEC approach against their kernel-smoothed frequency estimator (benchmark), which employs a look-up table—commonly used in empirical research. The same ad hoc table from ursu2023sequential, with a grid fineness of 0.001, is utilized. We generate 50 simulated datasets, each representing \(N\in\{500,1000\}\) consumers who make sequential search and purchase decisions across four brands and an outside option (with the mean utility of the outside option normalized to zero). The utility function includes only brand intercepts, specified as \((\beta_1,\beta_2,\beta_3,\beta_4) = (1.0, 0.7, 0.5, 0.3)\). The search cost logarithm is set at \(\log c = -3.0\), and \(D = 100\) draws are used for the error terms. All estimations start from an initial vector of zeros. The replication code, written in Julia for fair comparison, is available on the authors' GitHub.
Table (ref) presents the bias and RMSE of the estimated coefficients. Panel (a) shows that the MPEC estimator, though still biased, achieves a smaller bias and RMSE than the benchmark in small samples, aligning with the MPEC misclassification model lu2014mpec. However, Panel (b) reveals worse performance in both bias and RMSE for larger samples, except for the search cost. MPEC also requires over four times the computational time and struggles with finding local optima.
Despite these seemingly discouraging results, we argue that the MPEC approach remains useful for identifying candidate parameters comparable to those from the benchmark method, which relies on an ad hoc look-up table. Furthermore, MPEC can construct this table dynamically by solving the equilibrium constraints during the estimation process.
The optimal sequential search model, based on weitzman1979optimal, has been widely used in empirical research ursu2023sequential. However, caution is needed regarding estimation and approximation accuracy, as the commonly used approach relies on an ad hoc look-up table.
To address these issues, we propose an MPEC estimator that bypasses the need for approximations and estimation of equilibrium constraints. Despite certain limitations, the MPEC approach proves useful for identifying parameters comparable to the benchmark while dynamically generating the look-up table during the estimation process.
\paragraph{Acknowledgments} This work was supported by JST ERATO Grant Number JPMJER2301, Japan.