Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
103,844 characters · 20 sections · 92 citation commands
Consider or Choose? The Role and Power of Consideration Sets
\RUNAUTHOR{Akchen and Mitrofanov}
\RUNTITLE{Consider or Choose?}
\TITLE{Consider or Choose? The Role and Power of Consideration Sets}
\ARTICLEAUTHORS{ \AUTHOR{Yi-Chun Akchen} \AFF{School of Management, University College London, London E14 5AB, United Kingdom, \EMAIL{\tt [email removed]}} \AUTHOR{Dmitry Mitrofanov} \AFF{Carroll School of Management, Boston College, \EMAIL{\tt [email removed]}} }
\ABSTRACT{Consideration sets play a crucial role in discrete choice modeling, where customers often form consideration sets in the first stage and then use a second-stage choice mechanism to select the product with the highest utility. While many recent studies aim to improve choice models by incorporating more sophisticated second-stage choice mechanisms, this paper takes a step back and goes into the opposite extreme. We simplify the second-stage choice mechanism to its most basic form and instead focus on modeling customer choice by emphasizing the role and power of the first-stage consideration set formation. To this end, we study a model that is parameterized solely by a distribution over consideration sets with a bounded rationality interpretation. Intriguingly, we show that this model is characterized by the axiom of symmetric demand cannibalization, enabling complete statistical identification. The latter finding highlights the critical role of consideration sets in the identifiability of two-stage choice models. We also examine the model’s implications for assortment planning, proving that the optimal assortment is revenue-ordered within each partition block created by consideration sets. Despite this compelling structure, we establish that the assortment problem under this model is NP-hard even to approximate, highlighting how consideration sets contribute to nontractability, even under the simplest uniform second-stage choice mechanism. Finally, using real-world data, we show that the model achieves prediction performance comparable to other advanced choice models. Given the simplicity of the model's second-stage phase, this result showcases the enormous power of first-stage consideration set formation in capturing customers' decision-making processes. }
\KEYWORDS{discrete choice, consideration sets, symmetric cannibalization, assortment optimization, bounded rationality, identification} \HISTORY{First version: February 8, 2023; Second version: June 12, 2024; This version: February 19, 2025}
The rise of the digital economy and technological advancements has fundamentally changed how we shop and make purchasing decisions. With an overwhelming variety of products and services available online, consumers, when making a choice, must navigate an enormous amount of information, including detailed product descriptions, customer reviews, and ratings. In an ideal scenario, fully rational individuals would make their purchase decisions by thoroughly assessing the features of each alternative, calculating its utility, and selecting the option with the highest utility, provided they have unlimited time and resources to perform such an evaluation. In reality, individuals have physical and cognitive limitations and thus only consider a subset of the available alternatives. Economists and psychologists term such smaller sets of alternatives as “consideration sets” wright1977phased or “evoked sets” howard1969theory,brisoux1981evoked. The notion of consideration sets has been well-documented in the marketing literature when studying consumer behavior hauser2014consideration and various heuristics have been proposed to model the formation of the consideration sets, such as screening rules based on product prices or other features pras1975comparison,gilbride2004choice,jedidi2005probabilistic. The concept of consideration sets is further supported by the psychology literature, which questions consumers' ability to consistently evaluate every product within the offer set miller1956magic,hauser1990evaluation,iyengar2000choice.
In fact, the concept of consideration set formation goes beyond merely interpreting consumer behavior; it also has substantial practical implications, particularly in developing more comprehensive and accurate choice models. To this end, the seminal work by hauser1978testing uses a goodness-of-fit statistic to demonstrate that consideration sets can explain nearly three-quarters of the variation in choice data, whereas a logit model, which is based solely on consumer preferences, explains only a quarter. This insight has contributed to the widespread adoption of two-stage choice models that integrate consideration sets into consumers' decision-making processes and many papers provide evidence of their superior prediction performance silk1978pre,hauser1984application,gensch1987two. Within this two-stage framework, consumers initially form consideration sets, often using screening rules and simple heuristics hutchinson2005simple. In the second stage, they apply a choice mechanism to select and purchase the product that maximizes their utility from the options in the consideration set ben1995discrete,shocker1991consideration. Specifically, we can illustrate two-stage choice models with an example of online hotel booking, where customers face an abundance of alternatives and information. On a travel website, they encounter details such as the hotel's location, services, amenities, quality, view, ratings, photos, and reviews, among other characteristics. Analyzing all this information for every hotel within a limited time is impractical. Instead, customers might use simple heuristics to narrow their options. Figure (ref) shows a customer applying screening criteria -- rating (7+), price (\$150-\$200 per night), and three-star quality -- to reduce five hundred options to a consideration set of three alternatives. A customer then is expected to use a choice mechanism to evaluate these three considered alternatives and select the one offering the highest utility.
Given the evidence showing the importance of capturing customers' consideration set formation, it is unsurprising that two-stage choice models incorporating this process have gained significant popularity over the past several decades. Many of these models leverage advanced choice mechanisms to further enhance their predictive accuracy and data-fitting capabilities aouad2021assortment, jagabathula2022demand. A common feature of these models is their foundation in nonparametric choice modeling principles, drawing inspiration from machine learning models and algorithms. Their primary characteristic is an advanced and refined second-stage choice mechanism which can be as sophisticated as the choice mechanisms proposed by farias2013nonparametric and chen2022decision. In this paper, we adopt an opposing approach: instead of exploring more complex {second-stage choice mechanisms}, we take a step back to examine the fundamental properties of decision-making processes driven solely by consideration sets, where the second-stage choice mechanism is kept as simple as possible. More specifically, we examine a class of nonparametric models defined solely by a distribution over consideration sets, which we refer to as the consideration set model (CSM). Through the lens of this generic consideration-based model, we examine the role and power of consideration sets in choice model representation, demand cannibalization, choice prediction, and assortment optimization. We also discuss the implications for other choice models that incorporate consideration set structures. Specifically, we make the following contributions:
The concept of consideration sets originated in the fields of marketing and psychology and has been studied extensively for several decades, with numerous studies exploring how consumers form and use consideration sets in their purchasing decisions. For a more detailed overview, we refer readers to the survey papers by roberts1997consideration and hauser2014consideration. Overall, it is widely accepted in the literature that consumers generally make decisions through a two-stage process swait1987incorporating. Specifically, consumers first consider a subset of products (i.e., form a consideration set) and then select one item from that set to purchase. In fact, abundant empirical evidence supports the concept of consideration sets. For example, hauser1978testing adapts an information-theoretic viewpoint and shows that a model which incorporates the consideration set concept can explain up to 78% of the variation in the customer choice. Then, hauser1990evaluation empirically investigate the consideration set size for various product categories and report that the mean consideration set size, which is the number of brands that a customer considers before making a choice, is relatively small: 3.9 for deodorant, 4.0 for coffee, and 3.3 for frozen dinner. To this end, our numerical analysis in Section (ref), examining the size of consideration sets across various product categories using real-world grocery sales data, is consistent with the finding that consideration set sizes tend to be relatively small.
Over the past few decades, numerous theories and models have been developed to explain how consumers form their consideration sets. From a decision theory perspective, consumers are expected to continue searching for new products as long as the utility (or satisfaction) gained from finding a better option exceeds the cost of the search ratchford1982cost,roberts1991development. Additionally, cognitive limitations may lead consumers to rely on simple heuristic rules to construct their consideration sets. These rules include elimination by aspects tversky1972elimination, conjunctive and disjunctive rules pras1975comparison,brisoux1981evoked,laroche2003decision,gilbride2004choice,jedidi2005probabilistic, and compensatory rules keeney1993decisions,hogarth2005simple. In this paper, we take a more general and flexible perspective on the formation of consideration sets. Rather than focusing on specific rules or heuristics that consumers use to construct their consideration sets, we model the possibility that a consideration set sampled by a customer could be any subset of products. To this end, our paper is related to the work of jagabathula2022demand, where the authors model customer choice using the consider-then-choose (CTC) model, which is parameterized by the joint distribution over consideration sets and rankings. Different from the CTC model, which uses a mixture of rankings as its second-stage choice mechanism, our model employs the simplest possible choice mechanism. Consequently, it can be viewed as a special case of the model proposed by jagabathula2022demand. Furthermore, our model is also a special case of the ranking-based model block1959random,farias2013nonparametric and the mixed MNL model train2009discrete as discussed in Section (ref).
More recently, the concept of a consideration set has been gaining a lot of attention in the field of operations management. feldman2019assortment propose an algorithm for determining the optimal assortment, incorporating the unique features of the ranking-based model and the assumption of small consideration sets. Similarly, aouad2021assortment examine assortment optimization under the CTC choice model, where customers select the product with the highest rank from the intersection of their consideration set and the offered assortment. In wang2018impact, the authors present a mathematical model that represents the trade-off between a product's expected utility and the search cost associated with it. mitrofanov2024choice investigate the assortment optimization problem under a two-stage choice model, where customers initially use non-parametric dominance relationships to narrow down their options and then make a selection from the shortlisted products using the multinomial logit (MNL) model. chitla2023customers use the consideration set concept in order to build the structural model and study the multihoming behavior of users in the ride-hailing industry. Interestingly, it was shown in the paper by jagabathula2022demand that the consideration set approach is particularly advantageous in the online platform setting or in the grocery retail setting where we might expect the noise in the sales transaction data because of the stockouts. To this end, there is a lot of evidence in the literature that online platforms might not have real-time information on the product availability in the grocery retail stores which could be the major cause for the stockouts knight2022disclosing despite the AI-enabled technology to alleviate this problem knight2023impact,kim2024ai.
In this section, we begin with an overview of choice modeling before introducing the consideration set model. Then, we connect this model to other choice models in the economics, marketing, and operations literature.
We begin this subsection by introducing several notations. We consider a universe $N \equiv \{ 1,2,\ldots,n \}$ of $n$ products. Each assortment or offer set $S$ is a subset of $N$, i.e., $S \subseteq N$. When a set of products $S$ is offered, a customer or an agent chooses either a product in $S$ or the “default” option $0$. Depending on the context, the default option can be the no-purchase option or any product outside the universe $N$. Following the standard assumption in the literature, the default option is always assumed to be available to customers. Then, to simplify notation, we denote $N^+ = N \cup \{ 0 \}$ and $S^+ = S \cup \{ 0 \}$ for any $S \subseteq N$. Throughout the paper, $\mathbb{I} \left[ A \right]$ or $\mathbb{I}_{A}$ denotes the indicator function {\color{black} that} is equal to 1 if condition $A$ is satisfied and 0, otherwise.
A choice model can be described by a mapping or function $\mathbb{P}$ that takes as input an assortment $S$ and outputs a probability distribution over the elements of $S^+$. This probability distribution represents the likelihood of each product in the assortment being chosen by a consumer. Specifically, $\mathbb{P}_j(S)$ represents the probability that a customer selects product $j$ from the offer set $S$, while $\mathbb{P}_0(S)$ represents the probability of choosing the default option. Note that when the offer set is empty, the default option is always chosen, i.e., $\mathbb{P}_0(\emptyset)=1$. More formally, a choice model specifies the choice probabilities $\{\mathbb{P}_j(S) \colon j \in S^+, S \subseteq N\}$ that satisfy the standard probability laws: $\mathbb{P}_j(S) \geq 0$ for all $j \in S^+$ and $\sum_{j \in S^+} \mathbb{P}_j(S) = 1$, for all $S \subseteq N$. Note that discrete choice models are widely used to predict consumers' purchase decisions in operations, marketing, and economics research. For further details, we refer readers to ben1985discrete and train2009discrete. While early research on choice modeling primarily focused on parametric models such as the multinomial logit (MNL) model, the mixed MNL model, and the nested logit model train2009discrete, recent years have seen a rapid surge in consumer choice data, driving the development of nonparametric choice models. These models aim to enhance the accuracy of consumer decision predictions and provide greater flexibility in fitting the data block1959random,farias2013nonparametric,aouad2021assortment,jagabathula2022demand, chen2022decision.
More specifically, recent research in nonparametric choice modeling has primarily focused on developing more advanced and sophisticated second-stage choice mechanisms, with or without incorporating a consideration set formation stage, to better fit consumer choice data. In contrast, our paper takes the opposite approach by employing the simplest possible second-stage choice mechanism. This approach enables us to explore the role and power of first-stage consideration set formation and examine how incorporating consideration sets influences the characteristics and applicability of choice models. To this end, we introduce and analyze a model, referred to as the consideration set model, which is described in detail below.
In what follows, we introduce the consideration set model. As elaborated later in Section (ref), this choice model is not based on entirely new principles but is instead nested within several well-established choice models. This structure enables us to extend our findings to other choice models that implicitly or explicitly incorporate consideration set structures. In essence, the consideration set model represents a probability distribution over sets of products, with each set $C \subseteq N$ corresponding to a consideration set. Specifically, let $\mathcal{C}$ be a collection of subsets of $N$ and $\boldsymbol{\lambda}$ is a probability distribution over $\mathcal{C}$ such that $\lambda_C \geq 0$ for all $C \in \mathcal{C}$ and $\sum_{C \in \mathcal{C}} \lambda_C = 1$. Each $C \in \mathcal{C}$ denotes a consideration set, which is a subset of $N$. Furthermore, we assume that each consideration set $C$ specifies a set of preference relations between elements in $N^+$ as follows.
In Definition (ref), following the convention, we use $i \succ j$ to denote “$i$ is preferred to $j$” and use $i \sim j$ to denote that $i$ and $j$ are equally preferred. Note that we do not need to specify preference relations between the items outside of the consideration set $C$, since those items are dominated by the default (i.e., no-purchase) option $0$ which is always available. However, without loss of generality, one can still assume that $i \sim j$ for all $i, j \notin C$.
Following the standard interpretation in choice modeling, the consideration set model $(\mathcal{C},\boldsymbol{\lambda})$ can be viewed either as an individual customer's stochastic decision rule or as a representation of customer segments in the market. In the former case, we assume that with probability $\lambda_C$, a customer samples a consideration set $C$ according to distribution $\boldsymbol{\lambda}$ before making a final choice. In the latter case, each consideration set $C \in \mathcal{C}$ is associated with a customer type, and $\lambda_C$ represents the proportion of customers of type $C$ in the market. Before we show how to compute the choice probabilities under the consideration set model, we further provide an alternative parametrization of the model, which specifies a conditional distribution $\boldsymbol{\lambda}_{\cdot | S}$ given an offered assortment $S$.
In fact, Equation (ref) specifies the distribution over the sets within the offered assortment $S$. For example, assume that $N = \{ 1,2,3,4,5 \}$ and an assortment $S = \{ 1,2,3 \}$ is offered. Then the consideration set $C = \{ 2,3,4 \}$ is equivalent to the consideration set $C' = C \cap S = \{ 2,3 \}$. Put differently, if a customer who is considering buying products $C = \{ 2,3,4\}$ enters a store that only offers products $S = \{1,2,3\}$, then the customer would only consider buying product $C \cap S = \{ 2,3 \}$, as product $4$, despite being considered, is not offered. Intuitively, Definition (ref) specifies that the probability that the customer samples a conditional set $C'$ from the offer set $S$ is the sum of the probabilities of sampling consideration sets $C$ such that $C \cap S = C'$. With the conditional set distribution defined in Definition (ref), there are two ways to describe a consumer's decision-making process when selecting from an assortment $S$. In the first approach, the customer is assumed to sample a consideration set $C \subseteq N$ according to the distribution $\boldsymbol{\lambda}$ and then make a final choice from the intersection $C \cap S$. Alternatively, the customer can be assumed to sample a conditional set $C' \subseteq S$ directly, based on the conditional set distribution $\boldsymbol{\lambda}_{\cdot | S}$, and then make a final choice from $C'$. Both approaches lead to identical choice probabilities, making them equivalent.
It is straightforward to verify that the conditional set distribution $\boldsymbol{\lambda}_{\cdot | S}$ satisfies the standard conditions such as $\lambda_{C'|S} \geq 0$ for all $C' \subseteq N$ and $\sum_{C' \subseteq N} \lambda_{C'|S}=1$. In addition, $\boldsymbol{\lambda}_{\cdot|S}$ is consistent with the “monotonicity property” where for all $S_2 \subseteq S_1 \subseteq N$, we have that $\lambda_{C'|S_2} \ge \lambda_{C'|S_1}$ for all $C' \subseteq S_2$. More specifically, for any $C \subseteq N$, such that $C \cap S_1 = C'$ is satisfied for $C' \subseteq S_2$, we have $C \cap S_2 = C'$. Next, we formally define the probability of choosing an item $j$ from the assortment $S \subseteq N$ under the consideration set model $(\mathcal{C},\boldsymbol{\lambda})$ by means of the conditional set distribution $\boldsymbol{\lambda}_{\cdot | S}$.
In other words, Equation (ref) implies that when an assortment $S$ is offered, a customer forms a consideration set $C'$ with probability $\lambda_{C' \mid S}$ and, within this set, assigns equal preference to all products (as outlined in preference set (a) in Definition (ref)) while considering every product preferable to the default (no-purchase) option (as specified in preference set (b) in Definition (ref)). To this end, we assume that a customer does not rely on a specific second-stage choice mechanism to determine the “best” product from $C'$ and therefore would select any product from $C'$ with equal likelihood which leads to a factor $\lambda_{C' \mid S} / |C'|$ in the summation. Recall that we intentionally have this assumption to make the second-stage choice mechanism as simple as possible, allowing us to focus exclusively on analyzing the role and power of the consideration set formation layer in choice modeling. Alternatively, this assumption can be justified by the bounded rationality of customers who, constrained by cognitive and physical limitations, are unable to differentiate and rank all options within the consideration set simon1955behavioral.
As mentioned above, we can also model the customer's decision-making process by first sampling $C$ from $\boldsymbol{\lambda}$ and then choosing a product from $C \cap S$. In this case, we have the following expression to compute the probability of choosing item $j$ from offer set $S$:
if $j \in S$, and $0$, otherwise. Similar to Equation (ref), the factor $|C \cap S|$ comes from the fact that a customer equally prefers to buy a product from $C \cap S$ after she/he forms the consideration set $C \subseteq N$ and thus chooses one from $C \cap S$ uniformly at random. According to Definition (ref), the choice probabilities $\mathbb{P}^{(\mathcal{C},\boldsymbol{\lambda})}_j(S)$ in Equations (ref) and (ref) are equivalent (by rearranging the sums). We next present a simple example to illustrate the calculation of choice probabilities $\mathbb{P}^{(\mathcal{C},\boldsymbol{\lambda})}_j(S)$.
Finally, we highlight that although we simplify the choice mechanism by assuming customers select items from the considered set of products uniformly at random, this does not imply that customers have equal probabilities of purchasing any two offered products that appear in the same consideration set. Moreover, our model can readily accommodate scenarios where a customer decides not to purchase any product from the offered assortment. Following the previous example, consider a model specification $(\mathcal{C},\boldsymbol{\lambda})$ in which a representative customer is characterized by $\mathcal{C} = \{ C_4, C_5, \emptyset \}$, with $C_4 = \{ 1 \}$, $C_5 = \{ 1,2 \}$, and $(\lambda_{C_4},\lambda_{C_5},\lambda_{\emptyset}) = (0.3,0.4,0.3)$. When an assortment $S = \{ 1,2 \}$ is offered, a customer would choose product $1$ with a probability of $0.5$, product $2$ with a probability of $0.2$, and opt not to purchase any offered product with a probability of $0.3$. It is important to note that, in this scenario, the customer does not show indifference between products $1$ and $2$ appearing in the same consideration set $C_5 = \{1, 2\}$. In fact, the customer is more likely to choose product $1$ due to the presence of a smaller, nested consideration set $C_4$ within $C_5$ and also has the option not to purchase any product from the assortment.
Upon closer examination, it becomes clear that the consideration set model, while not built on an entirely new foundation, is both sophisticated and highly flexible. This nonparametric model offers up to $2^n-1$ degrees of freedom, precisely matching the number of parameters required to define the distribution $\boldsymbol{\lambda}$ over the $2^n$ subsets in $N$. Moreover, our model is a special case of several well-established choice models.
The consideration set model, in fact, can be viewed as a special case of the mixed MNL model. To recap, the mixed MNL model is characterized by $k$ segments, with each segment $\ell \in \{ 1,2,\ldots,k \}$ accounting for a probability mass $\lambda_\ell$ of the market. Customers in segment $\ell$ make purchasing decisions based on an MNL model with parameters $(w_{\ell 1}, w_{ \ell 2}, \ldots, w_{ \ell n})$. For a given assortment $S$, the choice probability under the mixed MNL model is calculated as $\mathbb{P}_j(S) = \sum_{\ell =1}^k \lambda_\ell \cdot \frac{w_{{\color{black} \ell j}}}{1 + \sum_{i \in S} w_{{\color{black} \ell i}}}$. This formulation demonstrates that the consideration set model can be effectively represented within the structure of the mixed MNL model. Specifically, suppose $\mathcal{C} = \{ C_1, C_2, \ldots, C_k \}$ and $\boldsymbol{\lambda} = (\lambda_{C_1}, \ldots, \lambda_{C_k})$. We can then construct a mixed MNL model as follows. Let $\mathcal{M}$ be a sufficiently large constant. For each $C_\ell$, $\ell = 1,2,\ldots,k$, we define a customer segment in the mixed MNL model with population weight $\lambda_\ell \equiv \lambda_{C_\ell}$, and set $w_{ \ell i} = \mathcal{M}$ for $i \in C_\ell $ and $w_{\ell i} = 0$, otherwise. As $\mathcal{M}$ approaches infinity, it becomes evident that the corresponding choice probabilities in this mixed MNL model converge to those in Equation (ref). Furthermore, the consideration set model can also be viewed as a special case of the ranking-based model. This relationship is formally stated in the following lemma.
We formally prove Lemma (ref) in Section (ref). Herein, we illustrate the idea of the proof with a straightforward example. Suppose the consideration set model consists of a single consideration set, $C_1 = \{ 1, 2 \}$. This model can be represented as a ranking-based model comprising two rankings, $\sigma_1 = \{ 1 \succ 2 \succ 0 \}$ and $\sigma_2 = \{ 2 \succ 1 \succ 0 \}$, each assigned a weight of $0.5$. In other words, to construct an equivalent ranking-based model, we interpret each consideration set $C$ as an equal-weighted average of $|C|!$ rankings, where each ranking is a permutation of the elements in $C$ followed by the no-purchase option $0$. By averaging over all permutations of $C$, we ensure that customers assign equal preference to each product in $C$. It is worth noting that Lemma (ref) also establishes that the consideration set model is a member of the random utility maximization (RUM) class thurstone1927law,mcfadden1973conditional. In the RUM framework, each alternative is associated with a random utility, and customers select the alternative with the highest utility once the randomness is revealed, effectively maximizing their utility. It is well-known that the RUM class is equivalent to the ranking-based model class, as the revealed utilities of products can be sorted to form a ranking over the products block1959random,farias2013nonparametric. Finally, we note that our general consideration set model, where $\boldsymbol{\lambda}$ is a distribution over consideration sets, subsumes the more restrictive consideration set distribution parameterized by independent attention parameters manzini2014stochastic. This directly follows from the parameterization of the distribution $\boldsymbol{\lambda}$ in the CSM model: $\lambda_{C} = \prod_{i \in C} \gamma_i \cdot \prod_{i \in N \backslash C} (1 - \gamma_i)$, where $\boldsymbol{\gamma}$ is the vector of attention parameters specifying the independent consideration set formation in the paper by manzini2014stochastic. We formally state this result in the following lemma.
In this section, we establish the identifiability of the consideration set model and present its axiomatic characterization. Additionally, we explore the implications of these findings for general two-stage choice models, highlighting how both the “consider" and “choose" steps influence the representational power of these models. For ease of notation, we let $N_j \equiv \{ S \subseteq N: j \in S \}$ denote the collection of assortments that include product $j$. {\color{black} Also, throughout this section, we let choice data refer to the collection of the ground-truth choice probabilities $\{ \mathbb{P}_j(S) : j \in S^+, S \subseteq N \}$.} For now, we assume that the exact value of choice probabilities is provided to us, that is, we ignore potential finite sample issues. This is a reasonable assumption if the number of transactions under each assortment is large. We relax this assumption in Section (ref) when estimating the consideration set model from real-world grocery retail data.
Recall that the consideration set model is fully characterized by the distribution $\boldsymbol{\lambda}$ over subsets. Therefore, the identification of the consideration set model reduces to the identification of the distribution $\boldsymbol{\lambda}$. For simplicity, we refer to $\boldsymbol{\lambda}$ as the consideration set model throughout this section. In what follows, we present a collection of results that provide different ways to obtain the distribution $\boldsymbol{\lambda}$ in closed form. The first result requires the specification of the choice probabilities for selecting an item $j$ across the assortments in $N_j$ to compute $\lambda_C$ for each $C \in N_j$.
Proof sketch: For each $C \in N_j$, we first define three functions as follows: $\chi_C(X) ={1}/{\abs{C \cap X}}$, $\psi_C(X)=\mathbb{I}_{\abs{C \cup X} = n-1} \cdot (-1)^{\abs{C}+\abs{X}-n+1}$, and $\varphi_C(X) =\mathbb{I}_{\abs{C \cup X} = n} \cdot n \cdot (-1)^{\abs{C}+\abs{X}-n+1}$. The first function $\chi_C(X)$ is used to account for the size of the intersection $C \cap X$ between an assortment $X$ and a consideration set $C$. The remaining two functions $\psi_C(X)$ and $\varphi_C(X)$ jointly form the orthonormal basis to $\chi_C(X)$. By varying the assortment $X$ and observing the corresponding changes in the choice probability $\mathbb{P}_j(X)$, we can infer the probability weight $\lambda_C$ of a consideration set $C$ that includes product $j$. With these three functions in hand, we rewrite the choice probability, defined in Equation (ref), by means of the function $\chi_C(X)$ as follows:
We also notice that $\psi_C(X)$ and $\varphi_C(X)$ share the same factor $(-1)^{|C| + |X| - n + 1}$ and, therefore, their summation can be simplified as follows:
Next, we claim that for all $C,C' \in N_j$,
Invoking the claim, Equation (ref) in the theorem follows immediately, since
Therefore, in order to complete the proof of Equation (ref), it is sufficient to prove the claim in Equation (ref). Since the proof of the claim is quite involved, we relegate it to Section (ref) in the e-companion.
In what follows next, we complete the proof of Theorem (ref) by showing that the distribution $\boldsymbol{\lambda}$ is unique. First, note that Equation (ref) relates probability distribution $\boldsymbol{\lambda}$ over the consideration sets to the choice probability $\mathbb{P} _j\big( X\big)$ through the system of linear equations which can be represented as $\boldsymbol{y}=A \cdot \boldsymbol{ \lambda}$, where ${\boldsymbol{y}} = \left(\mathbb{P}_j(X) \right)_{X: j \in X}$ and $\boldsymbol{\lambda} = \left( \lambda_C \right)_{C: C \in N_j}$ are two vectors of length $2^{n-1}$. Then, Equation (ref) provides another relationship between choice frequencies $ \mathbb{P}_j(X)$ and the model parameters $\boldsymbol{\lambda}$ in a linear form as $\boldsymbol{ \lambda}=B \cdot \boldsymbol{y}$. Given that $A$ is a $2^{n-1} \times 2^{n-1}$ dimensional matrix, proving the uniqueness of $\boldsymbol{\lambda}$ distribution reduces to showing that $\det(A) \ne 0$. This is easy to see: as we have $\boldsymbol{ \lambda} =B \cdot \boldsymbol{y}= B \cdot A \cdot \boldsymbol{\lambda}$ for any $\boldsymbol{\lambda}$, it implies that $I=B \cdot A$ and $1 = \det(I)=\det(B) \cdot \det(A)$. Consequently, $\det(A)\ne 0$. We refer the readers to Section (ref) in the e-companion for the complete proof of the theorem. $\square$
Thus, Theorem (ref) allows us to compute the probability mass of each consideration set $C \subseteq N$ such that $C \ne \emptyset$. We can then find $\lambda_{\emptyset}$ directly by using the equation $\sum_{C \subseteq N} \lambda_C =1$. It follows from Theorem (ref) that the identification of $\lambda_C$ relies only on the access to the probabilities of choosing an arbitrary item $j$ in $C$ under various assortments, i.e, $j$ can be any item in $C$. As a result, for each $C$, there are at least $\abs{C}$ ways to compute $\lambda_C$. We can also formulate the corollary below that specifies the necessary conditions for the data generation process to be consistent with the consideration set model.
Note that this corollary follows directly from the Theorem (ref). Alternatively, we can recover the underlying parameters of the consideration set model from the collection of choice probabilities $\{{\mathbb{P}}_0(S) \colon S \subseteq N\}$, where we only need to have access to the probability to choose the default option under assortments $S \subseteq N$. In particular, we have the following result:
In fact, Lemma (ref) follows from a particular form of the inclusion-exclusion principle stated in graham1995handbook. For any finite set $Z$, if $f \colon 2^Z \to \mathbb{R}$ and $g \colon 2^Z \to \mathbb{R}$ are two real-valued set functions defined on the subsets of $Z$ such that $g(X) = \sum_{Y \subseteq X} f(Y)$, then the inclusion-exclusion principle states that $f(Y) = \sum_{X \subseteq Y} (-1)^{\abs{Y} - \abs{X}} g(X)$. Our result then follows from setting $f(Y)$ to $\lambda_C$ and defining $g(X)=\mathbb{P}_0(N \setminus X) = \sum_{C \subseteq X} \lambda_C$, where the second equality holds since a customer of type $C$ would choose the outside option $0$ from $N \backslash X$ if and only if $C \subseteq X$. {\color{black} Importantly, while recovering the distribution $\boldsymbol{\lambda}$ over consideration sets in the most general case would require observing the default choice probabilities for $2^n$ different assortments, practical evidence suggests that consideration sets typically have limited cardinality hauser1990evaluation, hauser2014consideration, which significantly reduces the number of required assortments. Specifically, if the size of consideration sets is bounded by a finite number $m$, then, by Lemma (ref), identifying the distribution over consideration sets $\{ C \subseteq N \mid |C| \leq m \}$ only requires computing the default choice probability $\mathbb{P}_0(N \backslash X)$ for sets $X$ where $|X| \leq m$. This involves only $\sum_{i=0}^m {{n}\choose{n-i}} = O(n^m)$ different assortments.}
Also, note that in real-world retail settings, it can be challenging to observe the probability that customers choose the default option. Consequently, Theorem (ref) has higher practical importance in identifying parameters of the consideration set model than Lemma (ref), although Equation (ref) is less involved than Equation (ref). Lemma (ref) also connects to the classic decision theory findings of block1959random and falmagne1978representation, as it recognizes that the sum in Equation (ref) can be reformulated as a Block-Marshak polynomial. We will revisit these classical results in Section (ref). Finally, after combining together Theorem (ref) and Lemma (ref), we can formulate another set of necessary conditions imposed on the choice data for the data generation process to be consistent with the consideration set model.
We first highlight that most nonparametric choice models are not identifiable from the choice data $\{ \mathbb{P}_j (S) : j \in S^+, S \subseteq N \}$ alone. The examples include the ranking-based model farias2013nonparametric and the decision forest model chen2022decision, among others. In particular, sher2011partial show that the ranking-based model cannot be identified when $n\ge 4$. Intuitively, the nonidentifiability of the aforementioned nonparametric models can be explained by the fact that the parameter space grows much faster with $n$ than the number of available equations that match predicted and actual choice probabilities $\{ \mathbb{P}_j (S) \}_{j \in S^+, S \subseteq N}$ to identify model parameters. Specifically, these equations can be written in the form $\mathbb{P}^{\texttt{MD}}_j(S) = \mathbb{P}_j(S)$, for $j \in S^+$ and $S \subseteq N$, where $\texttt{MD}$ is a choice model and $\mathbb{P}^{\texttt{MD}}_j(S)$ is the predicted probability to choose item $j$ from assortment $S$ under the choice model MD.
For instance, a ranking-based model requires the estimation of $O\left(n!\right)$ parameters, which is the number of all possible rankings over $N^+$. Meanwhile, the number of available equations to identify model parameters is upper bounded by $O \left(n \cdot 2^n\right)$, where the factor $2^n$ is the total number of assortments $S \subseteq N$ and the factor $O(n)$ is the number of choices $j \in S^+$ under an assortment $S$. As $n! \gg n 2^n$, the ranking-based model is not identifiable. In contrast, the number of parameters in the consideration set model is $2^n-1$, which is the number of parameters to specify a distribution $\boldsymbol{\lambda} = (\lambda_C)_{C \in 2^N}$ over consideration sets. As $2^n < n \cdot 2^n$, it is not very surprising that the consideration set model does not suffer from the overparameterization faced by the ranking-based model. In fact, to the best of our knowledge, the consideration set model is one of the most flexible choice rules (i.e., it has the highest degrees of freedom) which are still identifiable from the choice data alone. Note that identifiability can benefit the downstream applications of the choice model. We empirically demonstrate the value of identifiability in Section (ref) in the e-companion.
Given the complete identifiability of the first-stage consideration set formation, Theorem (ref) suggests that if a two-stage choice model is non-identifiable, the first-stage consideration set formation itself is not responsible for that. {\color{black} Additionally, given that the consideration set model has $2^n-1$ parameters and there are at most $O(n \cdot 2^n)$ equations available to identify a choice model, Theorem (ref) indicates that any two-stage choice model characterized both by the general distribution over consideration sets as well as by the second-stage choice mechanism is at high risk of being non-identifiable.} Specifically, any choice mechanism that models a purchase decision based on the first-stage consideration set could significantly increase the degree of freedom by at least $\Omega(n)$ factor, resulting in a choice model with $\Omega(n \cdot 2^n)$ parameters. For example, jagabathula2022demand demonstrates that the choice model, characterized by a joint distribution function over consideration sets and complete rankings, requires estimating $n! \cdot 2^n$ parameters and cannot be identified solely from sales transaction data.
Finally, we note that Theorem (ref) is not only about identifiability but also provides a closed-form expression to compute all the parameters of the consideration set model. This further distinguishes the consideration set model from other parametric and non-parametric choice models. The MNL model is one of the very few models that have a similar property. Another example is a consider-then-choose (CTC) model studied by jagabathula2022demand where the authors show that the model is partially identifiable as the marginal distribution over consideration sets can be identified from the choice data. jagabathula2022demand also investigate a special case of the CTC model, named the GCC model, where all customers follow the same single ranking $\sigma$ to make decisions after sampling a consideration set. They provide closed-form expressions to estimate the parameters of the GCC model. However, this model is quite restrictive and suffers from one-directional cannibalization which implies that all customers have to follow the same preference order jagabathula2022demand.
In the previous section, we outlined various necessary conditions that choice data must satisfy to be consistent with the consideration set model (see Corollaries (ref) and (ref)). While these necessary conditions are valuable for ensuring consistency with the consideration set model, Corollaries (ref) and (ref) are primarily algebraic in nature and lack significant intuitive interpretation. This encourages us to develop axioms based on customers' revealed preferences that could characterize the consideration set model and provide more intuition. To this end, we propose two axioms, called default regularity and symmetric demand cannibalization.
The axiom of default regularity relates to the classic decision theory developed by block1959random and falmagne1978representation. Specifically, block1959random define the Block-Marshak polynomial of the choice probabilities as follows:
block1959random and falmagne1978representation jointly show that the choice data belongs to the RUM class, i.e., the choice data is consistent with a ranking-based model, if and only if the inequality $H(i,S) \geq 0$ holds for all alternatives $i \in S^+$ and assortments $S \subseteq N$. Our first axiom, the default regularity, is specifically equivalent to the nonnegativeness of the Block-Marshak polynomial $H(0,S)$ of only the default (i.e., no-purchase) option $0$. {\color{black} Thus, the constraints imposed on the choice probabilities by default regularity axiom is no more restrictive than the constraints imposed by the RUM class.} In what follows we formally state the default regularity condition.
In other words, the axiom of default regularity imposes the RUM type of restriction only to the default choice probabilities in the choice data. Our second axiom, symmetric cannibalization, imposes restrictions on the choice data related to purchasing a specific product $j \in N$.
The axiom of symmetric demand cannibalization relates to the concept of demand cannibalization, where the sales or market share of one product decreases due to the presence of a competing product. This axiom states that for any pair of products, $j$ and $k$ in $S$, the influence of product $k$ on demand for product $j$ is equal to the influence of product $j$ on demand for product $k$ across all product assortments $S \subseteq N$. This indicates a symmetric pattern in how products cannibalize each other's demand. In what follows below, we present our main theorem which characterizes the consideration set model through the axioms defined above.
The proof is relegated to the e-companion (see Section (ref)). As it can be seen therein, establishing necessity is rather straightforward, but establishing sufficiency is more involved as it requires two auxiliary lemmas. We prove sufficiency by constructing the distribution $\boldsymbol{\lambda}$ and then the uniqueness of $\boldsymbol{\lambda}$ follows directly from the Theorem (ref). From the proof, we can also notice that the non-negativity of the Block-Marshak polynomial $H(0,S)$ of the no-purchase option also ensures that the parameters of the consideration set model computed from the choice probabilities, as described by Lemma (ref), are well-defined, i.e., $\lambda_C = \sum_{X \subseteq C} (-1)^{|C| - |X|} \mathbb{P}_0(N \backslash X) = \sum_{\bar{C} \subseteq \bar{X}} (-1)^{|\bar{X}| - |\bar{C}|} \mathbb{P}_0(\bar{X}) = H(0,\bar{C})\ge 0 $, where $\bar{X} = N \backslash X$ denotes the set complement for a set $X \subseteq N$. This is because the functional form in the axiom of default regularity resembles the formula used to calculate the probability mass of the consideration set distribution presented in Lemma (ref).
Interestingly, Theorem (ref) suggests that in a general two-stage choice model, it is the choice mechanism, rather than the consideration set formation, that is responsible for capturing the asymmetric demand cannibalization among products. The theorem reveals a fundamental limitation of the consideration set formation in the first stage. While the consideration set formation can explain the heterogeneity of customers' preferences (as illustrated in Example 1), it falls short of completely capturing demand substitution in the way that the ranking-based model or other general RUM models can. In other words, in order to capture complex inter-product substitution using a two-stage choice model more accurately, one should focus on developing the second-stage choice mechanism. Although the symmetric demand cannibalization can be considered as a limitation, in Section (ref) we demonstrate that the consideration set model remains competitive in prediction performance compared to the ranking-based model when tested on real-world data. Thus, the effect of demand cannibalization in real-world settings may not be that asymmetric.
We also note that Theorem (ref) can be used to verify if the choice generation process is indeed consistent with a consideration set model. It can also be observed that the default regularity axiom when combined with the symmetric cannibalization axiom ensures a well-known regularity property (or weak rationality), i.e, $\mathbb{P}_j(S_1) \geq \mathbb{P}_j(S_2)$ whenever $S_1 \subseteq S_2$. which plays an important role in the economics literature rieskamp2006extending. We state it formally as follows.
Thus, Theorem (ref) and Lemma (ref) jointly imply that the only restriction that the consideration set model imposes on top of the general RUM class of choice models, such as the ranking-based model, is the symmetric demand cannibalization property. Finally, we formulate a lemma that states that if the two axioms are satisfied, then the cannibalization effect of item $k$ on item $j$ diminishes when we enlarge the assortment.
In other words, it follows from the lemma that if item $k$ cannibalizes item $j$ in assortment $S_1$, then item $k$ also cannibalizes item $j$ in assortment $S_2 \subseteq S_1$. Equivalently, if item $k$ does not cannibalize item $j$ in assortment $S_1$ then item $k$ also does not cannibalize item $j$ in assortment $S_2 \supseteq S_1$. We omit the proofs of Lemmas (ref) and (ref), as they follow straightforwardly.
We finish this section by adding two additional remarks. First, we note that the axiomatic characterization of choice models in the economics literature is usually established for parametric models. Luce's independence of irrelevant alternatives (IIA) axiom luce2012individual,hausman1984specification is one of the most popular examples and is used to demonstrate the limitation of the MNL model both in the economics and operation fields. A more recent example is provided by echenique2019general where the authors extend the Luce model by proposing a set of axioms that relax the IIA property. Among the nonparametric models, the ranking-based model is one of a few models that are characterized by axioms. As we discussed above, the choice data under the ranking-based model can be characterized by the Block-Marshal polynomial $H(i,S)$ such that $H(i,S) \geq 0$ for all $i \in S^+$ and $S \subseteq N$; see barbera1986falmagne and mcfadden1990stochastic. Consequently, our paper contributes to the rare examples of axiomatic nonparametric choice models.
We also note that Theorem (ref) indicates that neither the MNL nor the consideration set model subsumes each other. It is straightforward to check that the MNL model does not satisfy the symmetric cannibalization property unless its attraction parameters are the same for all items, meaning that all items have equal market shares. It is also obvious to see that the MNL model does not subsume the consideration set model, as the latter model is with a much higher degree of freedom. Symmetric cannibalization property also differentiates the consideration set model from the broad class of choice models with a single preference order manzini2014stochastic,jagabathula2022demand. In practice, as we will see in Section (ref), the assumption of symmetric demand cannibalization does not seriously impair the predictive performance of the consideration set model, as it performs closely to the mixed MNL model and the ranking-based model.
In this section, we investigate how accounting for consideration sets in choice modeling might influence the design and complexity of operations strategies in downstream applications. Specifically, we focus on the assortment optimization problem, which aims to identify the optimal set of products to offer to customers to maximize revenue. Throughout this section, let $r_i$ denote the revenue associated with product $i \in N$. Without loss of generality, we assume the products are ordered such that $r_1 \geq r_2 \geq \ldots \geq r_n > 0$. In this context, the default option generally refers to either a customer leaving without making a purchase (i.e., the no-purchase option) or choosing a product outside the designated set (i.e., the outside option), both of which result in zero revenue. For ease of notation, we let $(\mathcal{C},\boldsymbol{\lambda})$ denote a consideration set model where $\mathcal{C} = \{ C_1,C_2,\ldots,C_k \}$ and $\lambda_{C_j} \equiv \lambda_j > 0$ for $j \in K \equiv \{1,2,\ldots,k\}$. Additionally, for a customer type associated with a consideration set $C$, we denote the expected revenue under assortment $S$ as $\text{Rev}_C(S)$:
Hence, $\sum_{j \in K} \lambda_j \cdot \text{Rev}_{C_j}(S)$ is the expected revenue from all the customer segments in the population. We can thus state the assortment optimization problem under the consideration set model as follows:
In addition, without loss of generality, we assume that $\cup_{j \in K} C_j = N$. Specifically, if a product $i \notin \cup_{j \in K} C_j$, then $i \notin C_j$ for any $j \in K$, meaning no customer in the market considers purchasing it. As a result, the product has zero demand across all assortments and does not affect the choice probabilities of other products, making it irrelevant to the assortment decision. Therefore, it can be excluded from the product universe $N$.
Finally, note that assortment optimization is an important application of choice modeling, widely used in practical tasks such as menu design and product recommendation. For a comprehensive overview, we refer readers to the monograph by kok2008assortment. In this section, we first establish foundational results regarding the optimal solution to Problem (ref), followed by an analysis of its computational complexity.
In this subsection, we provide an exact characterization of the optimal assortment structure. We begin with some definitions. First, we let $b \in \{ 0,1 \}$ be a binary variable. Next, we define $\eta_{b}(C)$ as an operator over a set $C \subseteq N$, such that $\eta_{1}(C) = C$ and $\eta_{0}(C) = \bar{C} = N \backslash C$. In other words, when $b = 1$, the operator $\eta_{b} \left( \cdot \right)$ functions as the identity mapping, whereas for $b = 0$, it returns the complement of the set. Then, given a binary vector $\mathbf{b} \in \{ 0,1 \}^k$ of length $k$, we define a block $I_{\mathbf{b}}$ in the following way:
For example, if $\mathbf{b} = (0,1,0)$, then $I_\mathbf{b} = \bar{C}_1 \cap C_2 \cap \bar{C}_3$. We also let $\mathcal{B} = \{ 0,1 \}^k$ and define $\mathcal{I}$ as the collection of all blocks such that $\mathcal{I} = \{ I_{\mathbf{b}} \mid \mathbf{b} \in \mathcal{B} \}$. {\color{black} Consequently, as demonstrated in the proof of the upcoming theorem, the non-empty sets in $\mathcal{I} = \{ I_{\mathbf{b}} \mid \mathbf{b} \in \mathcal{B} \}$ form a partition of $N$.}
Second, we define a set of products $S_1$ as revenue-ordered within its superset $S_2$ if there exists a threshold $r \in \mathbb{R}$ such that $S_1 = \{ i \in S_2 \mid r_i > r \}$. In other words, a revenue-ordered set $S_1$ consists only of the highest-revenue products in $S_2$. Notably, under this definition, the empty set is also considered revenue-ordered, as it can be obtained by setting the threshold $r$ arbitrarily high. In fact, the notion of a revenue-ordered structure is well-established in the assortment optimization literature. In particular, the seminar work by talluri2004revenue shows that the optimal assortment $S^*$ under the MNL model is revenue-ordered within the product universe $N$. With these definitions in place, we can now present the main theorem that characterizes the optimal assortment for Problem (ref).
{\color{black} While the formal proof of this theorem is relegated to Section (ref) in the e-companion, herein we discuss several intuitive insights derived from it. First, note that the products within a block $I_{\mathbf{b}}$ are always considered together by customers -- if one product in the block is considered by a customer, all other products in that block are considered by the same customer as well. In addition, due to the uniform choice mechanism in the second stage of the consideration set model, each product within the block generates the same demand if offered. Therefore, once deciding to include $n_{\mathbf{b}}$ products from block $I_{\mathbf{b}}$ into the assortment, it is optimal to select the $n_{\mathbf{b}}$ most expensive products. This strategy ensures that the demand is concentrated on the highest-revenue products in each block, maximizing overall revenue.}
Overall, these insights demonstrate that optimal assortments under the consideration set model (CSM) exhibit a clear and well-defined structure -- a characteristic often absent when solving assortment problems under other choice models, such as the ranking-based model or the mixed MNL model. Moreover, leveraging the intuition outlined earlier, Theorem (ref) enables a polynomial-time algorithm to solve the assortment problem (ref), provided the number of consideration sets, $k$, is bounded by a constant. We formally state this result below.
{\it Proof:} We prove this proposition by invoking Theorem (ref). First, we express each block $I_{\mathbf{b}}$ as $\{ i^{\mathbf{b}}_1,i^{\mathbf{b}}_2,\ldots,i^{\mathbf{b}}_{|I_{\mathbf{b}}|} \}$, where $i^{\mathbf{b}}_j$ is the $j$th most expensive product in block $I_{\mathbf{b}}$. Then, by invoking Theorem (ref), the optimal assortment $S^*$ must belong to the following collection of assortments:
Therefore, we can find the optimal solution by enumerating all assortments in $\mathcal{S}_{\text{OPT}}$ and calculating each assortment's expected revenue. Given that $| \mathcal{B}| = 2^k$, the number of assortments in $\mathcal{S}_{\text{OPT}}$ is bounded as follows:
Furthermore, computing the expected revenue for each assortment requires a runtime of $O(nk)$. Consequently, the total runtime of the algorithm is $O (nk) \cdot (n+1)^{2^k} = O(k \cdot n^{2^k + 1})$, which remains polynomial in $n$ if $k$ is upper bounded by a constant. $\square$
In what follows, we discuss several implications of Proposition (ref). First, this result can be contrasted with the complexity of assortment optimization under the mixed MNL model. To begin with, recall that the CSM model is a special case of the mixed MNL model, where each customer type follows an MNL model that assigns “infinite” weights to considered products (see Section (ref) for details). In the mixed MNL setting, it is well-established that the assortment optimization problem is generally intractable. Specifically, no polynomial-time optimal algorithm exists, even for the case of just two customer segments, i.e., $k=2$ rusmevichientong2014assortment. To address this, desir2022capacitated proposed a fully polynomial-time approximation scheme (FPTAS) that provides a $(1-\epsilon)$-optimal solution for the mixed MNL model when the number of customer segments $k$ is bounded by a constant. In contrast, Proposition (ref) demonstrates that for the CSM model -- a special case of the mixed MNL model -- the assortment optimization problem becomes more tractable. Specifically, under the same assumption of a constant number of customer types as in desir2022capacitated, the optimal solution under the CSM model can be found using a polynomial-time optimal algorithm. This highlights that the CSM model, compared to the general mixed MNL model, results in a more computationally tractable assortment optimization problem.
In the interest of practical implementation, we also demonstrate how the assortment problem (ref) can be formulated and solved as a mixed-integer linear program (MILP) in a simpler and more efficient manner. Note that the objective function in the assortment problem (ref) takes the form of a linear-fractional sum, enabling the application of standard linearization techniques charnes1962programming,csen2018conic to reformulate it as follows:
where $\mathbf{x}$ is a binary decision vector such that $x_i = 1$ if and only if product $i$ is included in the assortment. Importantly, the four sets of constraints in the optimization problem jointly ensure that $h_{ij}=x_i / \sum_{i \in C_j} x_i$ when $\sum_{i \in C_j} x_i \ne 0$. Also, note that the aforementioned constraints naturally ensure that $h_{ij}=0$ if $\sum_{i \in C_j} x_i = 0$. In the interest of space, we evaluate the scalability and effectiveness of the MILP (ref) in Section (ref) of the e-companion.
We further demonstrate that relaxing the assumption of a bounded number of customer segments makes the assortment problem (ref) computationally hard, even if the size of each consideration set is restricted. This result is formally stated as follows.
We prove this hardness result by constructing a reduction from the vertex cover problem, a well-known NP-hard problem garey1979computers. The proof, provided in Section (ref) of the e-companion, shows that the assortment problem remains NP-hard even under the restricted condition that each consideration set contains at most two products (i.e., $|C| \leq 2$ for all $C \in \mathcal{C}$). Interestingly, Propositions (ref) and (ref) jointly imply that it is the variety of customer types, represented by distinct consideration sets, rather than the size of the consideration sets themselves, that fundamentally drives the computational complexity of the assortment problem under the CSM model.
Furthermore, in the general case where neither the size of the consideration sets nor the number of customer types is bounded by a constant, the assortment problem (ref) can be shown to be NP-hard even to approximate. We formally state this result as follows\footnote{We sincerely thank Danny Segev for helping us develop this theorem.}.
We prove this theorem by constructing a reduction that transforms any instance of the maximum independent set problem on an $n$-vertex graph, known to be NP-hard to approximate within an $O(n^{1-\epsilon})$ factor haastad1999clique, into an instance of the assortment problem (ref) of $n$ products and $n$ consideration sets. We refer the readers to Section (ref) of the e-companion for details. Note that aouad2018approximability use a reduction from the maximum independent set problem to show that the assortment optimization problem under the ranking-based model is NP-hard to approximate within factor $O(n^{1-\epsilon})$. While our construction of the problem instances resembles that of aouad2018approximability, our retrieval procedure to construct an independent set from an assortment solution is quite different and involved, resulting in the $O( n^{\frac{1}{2}-\epsilon} )$ factor (see the proof of Claim EC.2 in Section (ref) of the e-companion).
{\color{black} In addition, it is worth noting that Theorem (ref) establishes a lower bound of $O\left( \sqrt{n} \right)$ for the inapproximability factor of the assortment problem (ref). An upper bound of $O(n)$ -- matching the inapproximability for the ranking-based model -- can be easily obtained by approximating Problem (ref) with an assortment of all products (i.e., when $S = N$). While the exact inapproximability factor of Problem (ref) remains an open question, it is striking that the assortment optimization problem under the CSM model already exhibits an inapproximability factor of at least $O\left( \sqrt{n} \right)$. }
From an operational perspective, Theorem (ref) highlights the challenges of incorporating consideration sets into choice models. Even with the simplest possible second-stage mechanism, the CSM model leads to an assortment optimization problem that is hard to approximate. This underscores the crucial role of consideration sets in the tractability of assortment optimization problems: any choice model that explicitly accounts for consideration set formation is likely to be computationally intractable unless additional structural assumptions are imposed to simplify the consideration set formation process.
In this section, we compare the predictive performance of the consideration set model against state-of-the-art benchmark models using the IRI Academic dataset bronnenberg2008database.
The IRI Academic Dataset consists of consumer packaged goods (CPG) purchase transaction data over a chain of grocery stores in two large Behavior Scan markets in the USA. In this dataset, each item is represented by its universal product code (UPC) and we aggregate all the items with the same vendor code (comprising digits 3 through 7 in a 13-digit-long UPC code) into a unique “product”. Next, to alleviate data sparsity, we first include in our analyses only the products with a relatively high market share (i.e., products with at least 1% market share) and then aggregate the remaining products into the outside option. In order to streamline our case study, we focus on the top fifteen product categories out of thirty-one that have the highest number of unique products (see Table (ref)) and consider the first four weeks in the year 2007 for our analyses.
We represent our sales transactions with the set of the tuples $\{ (S_t,i_t) \}_{t \in \mathcal{T}}$, where $i_t$ is the purchased product, $S_t$ is the offered assortment and $\mathcal{T}$ denotes the collection of all transactions. For every purchase instance in the dataset which is characterized by a tuple $(S_t,i_t)$, we have the week and the store ID of the purchase which allows us to approximately construct the offer set $S_t$ by taking the union of all the products that were purchased within the same category as $i_t$, during the same week, and at the same store.
Next, we split our sales transaction data into the training set, which consists of the first two weeks of our data and is used for the calibration of the choice models, and the test/hold-out set, which consists of the last two weeks of our data and is used to compute the prediction performance scores defined below. In what follows, we use the mean absolute percentage error (MAPE) to measure the predictive performance of the choice models:
where $\bar{p}_{i,S}$ is the empirical choice frequency computed directly from the sales transaction data, i.e., $\bar{p}_{i,S} = \tau_o({S,i}) / (\sum_{i \in N^+} \tau_o({S,i}) )$ and $\tau_o({S,i})$ is the number of times alternative $i \in S^+$ was chosen under assortment $S$ in the test dataset, $\hat{p}_{i,S}$ is the predictive probability of choosing item $i \in S^+$ from the offer set $S$ by a specific choice model, $\mathcal{S}$ is the set of unique assortments in the test dataset, and $\tau_o(S) = \sum_{i \in S^+} \tau_o(S,i) $ is the number of observed transactions under assortment $S$. In the interest of space, we report the predictive outcome based on out-of-sample KL-divergence in Section (ref) of the e-companion. For both metrics, a lower score indicates better performance.
We begin by discussing the insights gained from calibrating the CSM model. For brevity, a detailed description of the CSM calibration method is provided in the e-companion. Specifically, Section (ref) introduces an estimation approach based on the maximum likelihood estimation (MLE) framework. The core idea is to reformulate the MLE problem as a large-scale concave maximization problem, where the objective is the log-likelihood function and the linear constraints map the distribution of the consideration sets, $\boldsymbol{\lambda}$, to the choice probabilities. To solve this problem optimally, we employ the column generation technique. This approach has also been used in previous studies, including van2014market for estimating ranking-based models and chen2022decision for estimating decision forest models from sales data. Additional details can be found in Section (ref) of the e-companion.
After estimating the consideration set model, we can emphasize several key observations. First, the fifth column of Table (ref) reports the number of unique customer types (i.e., $|\mathcal{C}|$) in the estimated model. This column reveals that the number of consideration sets is moderate, ranging from 36 to 108 across all product categories, which suggests sparsity in the number of customer types. Second, the last column of Table (ref) presents the weighted average size of the consideration sets in the estimated model $(\mathcal{C}, \boldsymbol{\lambda})$, with each weight corresponding to the probability $\lambda_C$ of a consideration set $C \in \mathcal{C}$. From this column, we observe that the typical customer considers a relatively small number of products, with consideration set sizes ranging from 1.3 to 2.8 across all categories, despite some categories featuring as many as 18–21 products. This finding aligns with prior empirical research in behavioral economics and marketing, which consistently shows that consumers tend to consider only a limited number of alternatives before making their final choice hauser1990evaluation,hauser2014consideration.
In this subsection, we compare the predictive performance of the CSM model against six benchmark models. The first two benchmarks are the independent demand model and the MNL model. The independent model, though widely used in practice, does not capture substitution effects. The MNL model, widely used in both academic research and practical applications, is calibrated using a maximum likelihood estimation (MLE) approach in a straightforward way. The third benchmark, the mixed MNL model, is estimated using the expectation-maximization (EM) algorithm train2009discrete with $K=10$ latent classes. The fourth benchmark is the ranking-based model, which is prominent in the operations management literature farias2013nonparametric,van2014market. Like the mixed MNL model, the ranking-based model is estimated via the MLE framework van2014market,van2017expectation. Notably, both the mixed MNL and the ranking-based models subsume the CSM model studied in this paper as they are equivalent to the RUM class, and thus they are highly competitive benchmarks. The fifth model is the Markov chain model studied by blanchet2016markov, which captures demand substitution by Markov chains. We estimate this model by the EM algorithm csimcsek2018expectation. While the Markov chain model also belongs to the RUM class, berbeglia2022comparative empirically demonstrate that this model has superior predictive performance relative to the mixed MNL and ranking-based models across several datasets. The last benchmark model is the decision forest model proposed by chen2022decision, which can also be estimated by an MLE approach. The decision forest model is outside of the RUM class as it can subsume any discrete choice model. To alleviate the computational effort required for cross-validation, we estimate the decision forest model using trees with a depth of three.
Table (ref) summarizes the out-of-sample predictive performance of the consideration set model and the benchmark models, evaluated using the MAPE score on test data. The models are denoted as follows: ID (independent demand), MNL (MNL model), MMNL (mixed MNL), RBM (ranking-based model), MC (Markov chain), DF (decision forest), and CSM (consideration set model). As expected, the independent demand and MNL models exhibit significantly worse predictive performance compared to the CSM. Although the MNL model is not subsumed by the CSM (see Section (ref)), the latter consistently outperforms it in prediction accuracy.
The consideration set model also achieves comparable predictive performance to the mixed MNL and ranking-based models, both of which represent the general RUM class. Furthermore, the CSM remains competitive with the Markov chain model, which has been shown to have an edge over other RUM-based models berbeglia2022comparative. Of all the benchmarks, the decision forest model achieves the best predictive accuracy on real-world transaction data, as measured by the MAPE score, and also performs well in terms of KL-divergence (see Section (ref)). This result is expected, given that the decision forest model lies outside the RUM class and offers the greatest flexibility in capturing complex customer preferences. While the nonparametric nature and high flexibility of the decision forest model make it powerful for fine-grained predictions, they come at the cost of significantly increased computational complexity. Specifically, its large degree of freedom makes downstream operational tasks, such as assortment optimization, much more challenging compared to the CSM model akchen2021assortment.
The key takeaway from this study is the effectiveness of the CSM model in accurately predicting customer choices, even with the simplest uniformly random second-stage choice mechanism. This underscores the pivotal role of first-stage consideration set formation within the two-stage choice framework and highlights its substantial influence on choice modeling.
In Table (ref), we also present two variations of the consideration set model. The first, CSM2, includes only consideration sets with at most two products, i.e., $|C| \leq 2$ for all $C$ in $\mathcal{C}$. Interestingly, CSM2 only slightly underperforms the general CSM in predictive accuracy. This result is consistent with earlier findings on small consideration set sizes (see Table (ref)) and aligns with empirical studies in the literature hauser1990evaluation,hauser2014consideration. The second variant, {\color{black} the CSM model blended with rankings} (CSMR), is a mixture of the CSM model and the ranking-based model, enhancing the latter's predictive performance by accounting for ties between products (see Lemma (ref)). While CSMR remains within the RUM class, it outperforms the standard ranking-based model, which cannot explicitly handle product ties, and performs comparably to the Markov chain model. These findings are consistent with the study by desir2021mallows, which demonstrates that integrating a smoothed mixture of rankings can substantially enhance predictive performance.
{\color{blue}
}
In the interest of space, we relegate additional analyses and experiments to the e-companion. In Section (ref), we compare the CSM model with benchmark models using an additional performance metric, KL-divergence. Our findings confirm that the insights from Table (ref) remain consistent when evaluated with alternative metrics, highlighting the robustness of our results. In Section (ref), we examine the computational efficiency of the CSM model compared to the ranking-based model in the estimation process. Specifically, we analyze how the in-sample log-likelihood of both models evolves over a finite runtime. The results show that the estimation algorithm for the CSM model converges significantly faster to a near-optimal solution, requiring much less time than the ranking-based model.
In Section (ref), we explore the role of the symmetric cannibalization property introduced in Section (ref) in the predictive performance of the CSM model relative to the mixed MNL model. As noted earlier, symmetric cannibalization is a defining feature of the CSM model that enhances its tractability compared to other models in the RUM class. However, this property may also limit its ability to fully capture customer purchasing behavior. Our analysis identifies a correlation between the mixed MNL model’s predictive performance over the CSM model and the degree of demand cannibalization asymmetry, suggesting that deviations from symmetric cannibalization contribute to the CSM model’s occasional underperformance.
Finally, in Section (ref), we demonstrate the operational value of model identifiability in choice modeling. Using assortment planning as a revenue management application, we show that non-identifiable choice models can lead to significant variability in the optimal assortments they produce, resulting in reduced average revenue performance. To illustrate this, we compare the CSM model, which is identifiable, with the ranking-based model, which is non-identifiable, using the IRI dataset. The results demonstrate the benefits of choice model identifiability in achieving stable and reliable operational outcomes.
In this paper, we explore a class of consideration-based choice models that are fully defined by the distribution over consideration sets (i.e., the consideration set model) and examine the fundamental role and power of consideration sets in discrete choice modeling. We first prove that the consideration set model is identifiable from choice data in closed form and results in symmetric demand cannibalization. Then, we demonstrate the operational significance of consideration sets in choice modeling through the emphasis on assortment planning. To this end, we show that the optimal assortment is blockwise revenue-ordered under the consideration set model, leading to a polynomial-time optimal algorithm for the assortment problem if the number of consideration sets in the model is bounded by a constant. However, in general, the assortment optimization problem under the consideration set model is computationally hard even to approximate, although the model has the simplest possible second-stage choice mechanism. Finally, we empirically highlight the competitive predictive performance of the consideration set model despite its symmetric demand cannibalization property. To conclude, this paper examined the role and power of accounting for consideration sets in choice-based demand modeling, with the hope of motivating further research on consideration-set-based choice models and their applications in operations management.
We sincerely thank the department editor, the associate editor, and the three anonymous referees for their thoughtful comments that helped significantly improve this work. We are also grateful to Jacob Feldman and Danny Segev for their valuable feedback on the results in the assortment optimization section, which has greatly strengthened our analysis in that part of the study.
{1.0pt}
\ECSwitch
\ECHead{Electronic Companion}