Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
68,775 characters · 28 sections · 44 citation commands
\thispagestyle{empty}
\pagenumbering{arabic}
Crowding valuation of subway riders is an important input to various supply-side decisions of transit operators. The crowding cost perceived by a transit rider is generally estimated by capturing the trade-off that the rider makes between crowding and travel time while choosing a route. However, existing studies rely on static compensatory choice models and fail to account for inertia and the learning behaviour of riders. To address these challenges, we propose a new dynamic latent class model (DLCM) which (i) assigns riders to latent compensatory and inertia/habit classes based on different decision rules, (ii) enables transitions between these classes over time, and (iii) adopts instance-based learning theory to account for the learning behaviour of riders. We use the expectation-maximisation algorithm to estimate DLCM, and the most probable sequence of latent classes for each rider is retrieved using the Viterbi algorithm. The proposed DLCM can be applied in any choice context to capture the dynamics of decision rules used by a decision-maker. We demonstrate its practical advantages in estimating the crowding valuation of an Asian metro's riders. To calibrate the model, we recover the daily route preferences and in-vehicle crowding experiences of regular metro riders using a two-month-long smart card and vehicle location data. The results indicate that the average rider follows the compensatory rule on only 25.5% of route choice occasions. DLCM estimates also show an increase of 47% in metro riders' valuation of travel time under extremely crowded conditions relative to that under uncrowded conditions. \\
Keywords: Dynamic preferences; Crowding valuation; Inertia; Habit; Expectation-maximization.
Quantitative measurement of the user valuation of key attributes of the public transport trip, including in-vehicle travel time and crowding, is important in investment appraisal, demand modelling, and supply-side decisions such as fare optimisation. Traditionally, most of the research articles and consultancy reports use a stated preference (SP) survey and estimate the traveller's perceived value of crowding in terms of a crowding multiplier -- the ratio of value-of-travel-time under crowded and uncrowded conditions wardman2011twenty. SP studies generally elicit preferences of riders in a hypothetical route choice experiment and estimate discrete choice models (DCMs) to obtain the crowding multiplier bansal2019flexible.
Whereas the hypothetical bias is a major limitation of the SP data, the required information to estimate DCMs (riders' route preferences and attributes of all available routes) is difficult to obtain using conventional revealed preference (RP) surveys tirachini2016valuation. Due to these challenges, early crowding valuation studies relying on the RP data either deviated from DCMs kroes2014value or complemented the RP data with SP data batarce2015use. However, the emerging use of smart cards for fare collection provides an alternative way to collect the required RP data. tirachini2016valuation first illustrate how smart card data can be used to estimate the standing penalty of Mass Rapid Transit users in Singapore, i.e.\,the disutility of standing measured in the equivalent travel time loss. horcher2017crowding integrate smart card data with automated vehicle location (AVL) data to estimate the crowding multiplier of Hong Kong Mass Transit Railway (MTR) users.
We identify two research gaps in the crowding valuation literature. First, whereas dynamic route preferences and learning behaviours are hard to capture in the SP experiments, previous RP studies also rely on static choice models. This implies that strong assumptions had to be made on how users form expectations about attribute levels on alternative routes. For example, horcher2017crowding assume that the observed travellers are experienced enough to know the average train occupancy on available routes, at the time of day when their journeys begin. Second, even if expectations in the choice situation are correctly recovered, a regular subway user, e.g. a daily commuter, might not actively make a compensatory route choice before every trip. In fact, a rider can adhere to the same route until a bad experience occurs, but none of the previous public transport studies model such non-compensatory behaviour. The riders who choose routes based on inertia or habit should not contribute toward the crowding cost valuation and therefore, modelling such behaviour is crucial to accurately estimate the crowding multiplier.
A dynamic choice model that can identify the temporal variation in the decision rules (compensatory versus non-compensatory) used by a rider can address both research gaps. To explore if any such model exists, we succinctly review the relevant literature on modelling of different decision rules and dynamic choice models in sections (ref) and (ref), respectively.
Most of the previous studies modify the systematic part of the indirect utility to incorporate different decision rules elrod2004new, swait2001non. A few studies have used the latent class specification to simultaneously model multiple decision rules considering one-to-one correspondence between a decision rule and a latent class dey2018accommodating,hess2012allowing,swait2001influence. However, these models are static, i.e. they assume that the decision-maker uses the same decision rule across all choice occasions. Moreover, these studies test their latent class models using rather simplistic stated preference datasets.
In travel behaviour modelling, cumulative prospect theory (CPT), instance-based learning theory (IBLT), and hidden Markov models (HMMs) are popular approaches to understand the dynamics of travel preferences.
CPT is particularly used in eliciting day-to-day travel mode yang2017experimental,ghader2019modeling and route choices jou2013application, yang2014development. These choices fit in the framework of decision-making under uncertainty because travellers do not have perfect information of routes' or modes' characteristics, and thus are prone to violate the rationality assumption of the expected utility-maximization theory. Despite the success of CPT in understanding travellers' dynamic preferences, there are two practical concerns in using it. First, the results are highly sensitive to the reference point avineri2006effect and the parameters of gain and loss functions jou2013application. Ideally, a stated preference survey is required to estimate these parameters. Second, travel choices are experience-based, but CPT is applicable to the description-based decision-making erev2010choice, jou2013application.
Unlike CPT, IBLT is appropriate for experience-based learning situations. This psychological theory relies on the power law of forgetting gonzalez2003instance. tang2017exploratory first illustrated the application of IBLT in dynamic route choice models by integrating it into the econometric framework of the mixed logit model. The proposed IBLT-based model also accounts for the hot stove effect (i.e., bad outcomes have a lasting effect) and the pay-off variability effect (i.e., a larger variability in payoffs leads to random choices).
HMMs were originally developed in the machine learning literature, but choice modellers also find them appealing because they offer a flexible econometric framework to model dynamic choices. However, similar to IBLT, only a handful of studies have adopted HMMs to estimate and forecast dynamic choices of car ownership yang2017hidden, xiong2018high and travel modes xiong2015analysis, xiong2017dynamic, xiong2018measuring, zarwi2017modeling. We identify four main limitations of these studies. First, previous studies relying on HMMs do not account for the non-compensatory choice process (e.g., inertia or habit). cantillo2007modeling, cherchi2011accounting, and gonzalez2017testing model inertia and habit dynamics of decision-makers by including the lagged utilities in traditional choice models, but those studies miss the benefits of HMMs. Second, the state-specific distribution ignores the learning of travellers from previous trips, i.e. state-specific distribution is specified using attributes of the current period. Third, previous HMM-based studies use datasets with ten or fewer periods, which is not sufficient to exploit the utility of such dynamic models. This is perhaps the main reason that previous studies do not fully leverage the ability of HMMs to associate a behavioural interpretation to the hidden states (or latent classes). Fourth, the state-specific choice model in existing HMM studies ignores cross-consumer heterogeneity. This constrained specification could cause confusion between heterogeneity and dynamics because some states may capture heterogeneity along with dynamics netzer2017hidden. Due to all these challenges, we cannot use off-the-shelf HMMs to formulate the dynamic choice process of subway riders.
We also note that, except zarwi2017modeling, all aforementioned studies estimate HMMs using the Markov Chain Monte Carlo (MCMC) simulation, but none of them discusses the prevalent issue of label switching and subsequent remedies spezia2009reversible.
Our review suggests that, ironically, the existing choice models accounting for preference dynamics assume a fully-compensatory decision rule and the models incorporating heterogeneity in decision rules are not dynamic.
In this study, we propose a dynamic latent class model (DLCM) which incorporates the learning behaviour of riders using the IBLT, specifies compensatory and non-compensatory (i.e., inertia/habit) choice processes of riders as latent classes, and allows them to dynamically transition between these classes based on the differences between the expected and the experienced level of services and other historical attributes. Our model can also account for the unobserved heterogeneity in preferences of riders. Thus, the proposed DLCM provides a comprehensive and general framework to model dynamic choices while accounting for heterogeneity in decision rules. The resulting model turns out to be a new variant of the heterogeneous HMM where a rider's choice at any instance not only depends on the rider's current class (i.e., state), but is also influenced by the rider's lagged choice. To circumvent the label switching issue of MCMC, we extend the expectation-maximization (EM) algorithm for HMMs to estimate the proposed DLCM. We also adapt the Viterbi algorithm to predict the most likely sequence of latent classes of a rider, conditional on her observed route choices arulampalam2002tutorial.
We illustrate the applicability of the proposed DLCM in addressing the empirical research gaps (as discussed in section (ref)) by estimating the crowding cost of an Asian metro's riders. In doing so, we calibrate DLCM using a two-month-long dynamic panel dataset on riders' revealed route preferences. This is the first such application in the crowding valuation literature.
The remainder of this paper is organized as follows: Section (ref) formulates DLCM; Section (ref) derives the EM algorithm to estimate DLCM and provides inference procedure; Section (ref) describes the simulation setup and discusses results of the Monte Carlo study to validate the model formulation and estimation. Sections (ref)-(ref) present an empirical application of DCLM: Section (ref) discusses the longitudinal data from which route choices and trip experiences of riders are derived, together with the corresponding RP experiment design; Section (ref) illustrates the importance of DLCM by investigating the results of the empirical study. Conclusions and future work are discussed in Section (ref).
Assume that the researcher records observations of the route choice and trip experience of regular commuters, and these observations can be linked to each other through unique passenger (smart card) identifiers, thus documenting a sequence of repeated choices. The proposed DLCM has three components -- initialisation model, transition model, and choice model. The long panel data allows us to utilize the first few observations of riders to identify their initial latent classes. We consider that a rider can choose to be in any of two latent classes (or hidden states) at a choice occasion: 1) compensatory, 2) non-compensatory (i.e. inclined to make choices due to habit or inertia). In the choice model, conditional on the latent class and the lagged choice, a rider chooses a route from a set of two available routes. In what follows, we formulate DLCM for two alternatives and two latent classes, but without loss of generality, it can be extended to any number of alternatives and latent classes. For simplicity, we first describe transition and initialisation models, followed by the choice model.
A rider's class transition probabilities are likely to depend on the difference between the expected and experienced level of service on the route chosen in the previous period. Moreover, a choice sequence of a rider also provides information about the class of the rider. For example, a consistent route choice across several occasions indicate a rider's inclination towards being in the non-compensatory class.
If rider $i$ is in class $s$ at time $t$, the utility $M_{its}$ derived by her due to a mismatch between the expected and the experienced level of service at the chosen route $j_t$ is:
where $\bm{X}_{itj_{t}}$ is a vector of attributes (e.g., crowding level) experienced by rider $i$ on chosen route $j_{t}$ at time $t$. $\bm{X}_{it}^{C}$ is derived from a sequence of choices made by rider $i$. A proportion of choice transitions made by rider $i$ in the choice sequence observed until time $t$ is one such attribute. We define expected values of $\bm{X}_{itj_{t}}$: $\mathbb{E}(\bm{X}_{itj_{t}})$ using IBLT tang2017exploratory:
where $\mu$ is a memory decay parameter that captures the rate of forgetting the past experiences. Assuming Gumbel distributed $\epsilon_{its}$, transition probability expressions are:
If $\bm{X}_{itj_{t}}$ includes the level-of-service attributes for which “less is better" (e.g., travel time, crowding), we expect $\bm{\zeta}_{s}$ to be positive. Intuitively, if the experienced level of service is poorer than the prior expectation at time $t$, a rider is more likely to remain in or switch to the compensatory class (class 1) at $t+1$.
Consider that we observe a rider for $T_{I}+T$ periods. Since we only observe the attributes of the route chosen by a rider, we select $T_I$ in such a way that the rider at least chooses both routes once by the time $T_I$ (see Section (ref) for details on how the length of this phase is determined in our empirical application). We do not include the first $T_{I}$ choices of a rider in the choice model because a choice model cannot be estimated in the absence of attributes of both routes. If we shift the time clock by $T_{I}$ periods, the latent class and the choice at $t=T_{I}+1$ correspond to those at $t=1$. We thus consider the latent class probabilities after the choice made at time $T_{I}$ (i.e., at time $t=T_{I}+1$) as the initial latent class probabilities.
Similar to the transition model, based on the differences between the experienced and expected level of service on the chosen route at $t=T_{I}$, we can obtain the latent class probabilities of a rider after the choice made at $t=T_{I}$ .
Note that the first $T_{I}-1$ choices of a rider, if at all, are utilized to compute the expected level of service $\mathbb{E}(\bm{Z}_{iT_{I}j_{T_{I}}})$ for the route chosen by the rider at $t=T_{I}$ and creating other attributes $\bm{Z}_{iT_{I}}^{C}$. The expectation is computed using equation (ref).
If rider $i$ is in the compensatory class at time $t$ (i.e., $s_{it}=1$), her utility from choosing route $j$ at time $t$ is:
We consider that the marginal utility associated with attributes $\bm{F}_{itj}$ do not vary across riders, but preference heterogeneity is present for attributes $\bm{G}_{itj}$. $\bm{\chi}_{i}$ is assumed to follow normal distribution, but any parametric or semi-parametric mixing distribution can be specified depending on the context. The expected value of attributes is obtained using equation (ref). If $y_{it}$ is the route chosen by rider $i$ at time $t$ and $\nu_{itj}$ is Gumbel-distributed idiosyncratic error term, the route choice probabilities of passenger $i$, conditional on being in the compensatory class at the beginning of time $t$, is:
If a rider is in the non-compensatory class at time $t$, she is more likely to choose the same route at time $t$ as chosen at $t-1$. Based on this observation, we now define the route choice probabilities if rider $i$ is in non-compensatory class at $t$ (i.e., $s_{it}=2$):
where $\mathbb{1}[.]$ is an indicator function. We would expect $\lambda_1$ and $\lambda_2$ to be highly positive because the passenger is likely to make the same choice in two consecutive scenarios due to inertia or habit. Some route-specific attributes derived from the historical choice sequence of a rider can also be incorporated in the systematic utility. We choose this specific form of choice probabilities for the non-compensatory class based on the context of the empirical study but there is a flexibility to modify this function in other empirical contexts.
By combining all three components of the model, we write the conditional likelihood of the model:
where $q_{its}$ is 1 if the passenger $i$ belongs to class $s$ at time $t$, else it is zero. The model parameters are $\bm{\Theta} = \left\{\mu, \bm{\zeta}_{0}, \bm{\zeta}_{1},\bm{\zeta}_{2}, \bm{\zeta}_{0}^{C}, \bm{\zeta}_{1}^{C},\bm{\zeta}_{2}^{C}, \bm{\gamma}, \bm{\varrho}, \bm{\Psi}, \lambda_1, \lambda_2 \right\}$. Figure (ref) shows the schematic diagram of the proposed DLCM. This specification can be viewed as a variant of the traditional heterogeneous hidden Markov models where conditional on the latent class, choice probabilities also depend on the lagged choice.
Direct maximization of the likelihood is challenging due to a well-known risk of underflow (i.e., the product of probabilities is too small to be represented by the CPU). Moreover, the analytical gradient expression of the likelihood is complex and maximization using numerical gradients can result in prohibitively large computation time netzer2017hidden. To decompose the likelihood maximisation into simplified optimisation problems, we estimate DLCM using the EM algorithm. Readers can refer bansal2018minorization, bhat1997endogenous,sohn2017expectation, and zarwi2017modeling to know more about applications of the EM algorithm in estimating choice models. We extend the existing EM algorithm for the heterogeneous HMMs to account for the auto-correlated choices, preference heterogeneity in the choice model, and riders' learning behaviour.
The EM algorithm was originally developed to deal with the missing data problem. The DLCM likelihood maximisation problem also falls under the same category because latent classes can be treated as the missing data. The EM algorithm is a two-step iterative procedure where the conditional expectation of the missing data is obtained in the E-step and then the complete loglikelihood is maximised in the M-step to update the model parameters. The convergence criterion is defined based on the difference in parameter estimates or loglikelihood values of two consecutive iterations.
Assuming the latent class assignment as the missing variable, we write the complete likelihood $L_{c}$ and the complete loglikelihood $\log L_{c}$ of the model:
Based on the complete loglikelihood $\log L_{c}$ expression, the E-step in $(k+1)^{th}$ iteration requires computing the following expectations:
To compute expectations in the E-step efficiently, we define forward $(\alpha_{its})$ and backward $(\beta_{its})$ variables:
We then compute the $\pi_{its}^{k+1}$ and $\omega_{itrs}^{k+1}$ in terms of forward and backward variables using the Bayes theorem.
From now onward, we omit $\bm{\Theta}_{k}$ for brevity.
The computation details of forward $\alpha_{its}$ and backward $\beta_{its}$ variables are provided in the Appendix (ref).
After computing $\pi_{its}^{k}$ and $\omega_{itrs}^{k}$ in the E-step, the complete loglikelihood is maximised to obtain the parameters for $(k+1)^{\text{th}}$ iteration .
We adopt the one-dimensional grid search approach to select the value of the IBLT parameter $\mu$ and all other parameters are estimated using the EM algorithm at a given value $\mu$. There are two challenges in estimating the value of $\mu$ using the EM algorithm. First, since all three components of the model include the learning parameter $\mu$, the entire objective function is required to be optimized at once. This defies the purpose of the EM algorithm to decompose a complex optimisation problem into simpler ones. Second, as the value of $\mu$ changes at each iteration of the algorithm, we also need to iteratively update attributes because the expected value of attributes in each component of DLCM depends on $\mu$. Both challenges make the estimation computationally expensive and numerical issues also prevail. However, in our one-dimension grid search strategy, the expected value of attributes are computed only once at the beginning of the algorithm and parameters in the M-step are updated by solving the simpler optimization problems:
Whereas the update for $\{\bm{\gamma}, \bm{\varrho}, \bm{\Psi}\}$ is equivalent to estimating a weighted mixed multinomial logit model, other updates are analogous to the estimation of weighted multinomial logit models. Even after this simplification, the estimation is computationally expensive. For instance, if the EM takes $1000$ iterations to converge, the estimation of DLCM involves the estimation of $1000$ mixed logit models. For this reason, we rely on the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm with analytical gradients to optimise these functions in Python bansal2019flexible.
In the EM estimation of discrete choice models, the information matrix is generally obtained by taking cross-product of M-step score vectors at convergence train2008algorithms. However, bansal2018minorization illustrate that standard errors estimated using this procedure can be biased. Therefore, we only use the EM algorithm to get the point estimates of parameters and obtain standard errors by numerically computing the Hessian of true conditional likelihood at these parameter estimates. We use numdifftools library in python to numerically compute the Hessian.
Previous transport studies relying on HMMs did not focus on estimating the most probable sequence of latent class (i.e., hidden states) for a decision-maker, perhaps due to a lack of behavioural interpretation of classes and short panel datasets. However, the estimation of latent classes is meaningful in this study because a rider's choice behaviour (compensatory vs. non-compensatory) is characterised by these classes. This is even more important for supply-side transit policies because crowding multiplier identified from compensatory choices can be scaled down with the knowledge about the extent of a rider's non-compensatory behaviour.
Conditional on the sequence of observed route choices and parameter estimates, we estimate the most likely sequence of a rider's latent classes. To this end, we use adapt the Viterbi algorithm, which uses forward-backward recursion arulampalam2002tutorial, forney1973viterbi, he1988extended. Once we condition on the lagged choices in the recursion, the Viterbi algorithm for the heterogeneous HMMs can be used for the proposed DLCM.
To assess the recovery of parameters and the potential convergence issues in the EM estimation, we present a Monte Carlo study. Since standard errors are calculated using the regular asymptotic theory, we are certain that standard deviation of the parameter estimates across resamples would be close to their asymptotic standard errors and therefore, one sample of the data generating process (DGP) is sufficient for our purposes. We consider a DGP with preference heterogeneity in compensatory class.
We generate each component of explanatory variables $\{\bm{X}, \bm{Z}, \bm{F}, \bm{G}\}$ by taking draws from a normally-distributed random variable with mean $1.5$ and standard deviation of $0.3$. We utilize the first ten choices (i.e., $T_{I} = 10$) of riders to compute initial latent class probabilities and assume that riders develop an expectation for the level of service on a route based on their past three trips on that route. In both DGPs, we consider the memory decay parameter of the IBLT $\mu$ to be 1. We consider $2000$ riders (i.e., $N=2000$) and $30$ choice occasions per rider (i.e., $T+T_I =30$). A diagonal variance-covariance matrix is assumed on random parameters. The algorithm terminates when the absolute difference between the loglikelihood values of two consecutive iterations is below $10^{-6}$. We also try tighter convergence criteria, but results remain consistent.
Table (ref) presents the estimation results where superscripts on variables relate to the component number of the vector or matrix. For example, $\zeta_{0}^{2}$ denotes the second element of the vector $\bm{\zeta}_{0}$ and $\Psi^{22}$ implies variance of the second random parameter. A comparison of true and estimated values of parameters indicate that all model parameters are recovered well. Similar values of the loglikelihood at convergence and true loglikelihood further validate the estimation procedure. Gradient values at convergence are also close to zero for all parameters, which ensures the convergence of the EM to a local optimal and discard possibilities of any numerical or identification issues. Since true latent classes (or hidden states) of riders are known in the DGP, we could analyse the performance of the Viterbi algorithm in predicting latent classes. The results indicate that the Viterbi algorithm could predict latent classes correctly at 80.53% accuracy. The estimation code for this simulation study is provided as supplementary material.
We implement the proposed DLCM using the data from a RP-based route choice experiment. The experiment exploits a unique feature of the network of an Asian metro. Four urban metro lines form an inner-city loop and thus an excellent laboratory for revealed preference route choice data collection. We select 32 origin-destination (OD) pairs between Line 1 and Line 2. Passengers on the selected OD pairs can reach to destinations using exactly two competitive paths, either by transferring to line 3, or to line 4 (see Figure (ref) for a visual illustration). These paths have enough relative variation in travel time and crowding, circumventing the concern of the dominant alternative.
The routes chosen by passengers and attributes of routes are obtained by passing the day-to-day data on automated fare collection (AFC) and vehicle location (AVL) through a passenger-to-train assignment algorithm. To recover the crowding experience of observed travellers throughout the entire length of their journeys, we run a full network assignment and derive in-vehicle crowding densities by aggregating the number of individual passengers travelling on each train. The assignment algorithm follows the methodology introduced by horcher2017crowding. However, for the purpose of this experiment, we realise a series of efficiency improvements in the R implementation of the assignment algorithm, in order to make the assignment feasible for the two-month period. This is achieved by making the recovery of trip-level feasible train itineraries quicker using the Fast Overlap Joins function available from version 1.12.8 (released in December 2019) of the data.table package of R, and by assigning passengers simply to the most probable itinerary instead of the original stochastic assignment of horcher2017crowding. With these amendments, the computation time of a one-day assignment decreases from the original two days to just around 25 minutes.
Key trip attributes include travel time, the density of standing passengers, and the probability of standing. Our algorithm infers the probability of standing on the level of origin-destination pairs. The algorithm is detailed in Section 3.4.3 of horcher2017crowding. Both the density of crowding and the probability of standing are recovered for each inter-station section of an experimental passenger's trip. As fares are not differentiated based on the route chosen, we derive crowding cost valuations in terms of the equivalent travel time loss wardman2011twenty. This implies that the trip attributes in DLCM are interactions between travel time and crowding characteristics, and therefore we aggregate the link-level products of train movement times and crowding density or standing probability estimates. Our dataset covers two months, thus allowing for numerous repeated route choice observations from uniquely identified (but otherwise anonymised) smart card holders.
The observation unit in our analysis is a rider who travels between a specific origin-destination pair. The smart card number helps us in keeping track of the rider's route choices at several instances. Similar to other revealed preference datasets, we only observe attributes (e.g., travel time and crowding level) of the chosen route. We consider that a rider uses IBLT to develop the expectation of attributes on a route based on her previous experiences on that route. A modeller can only know the rider's expectation of attributes on both routes, if she has chosen them at least once in the past. In the absence of knowledge about (expectation of) attributes of both routes, estimation of a compensatory choice model is infeasible. Therefore, we define the initialisation period $T_{I}$ for each rider based on the time until both alternatives are chosen by a rider. For example, if we observe a rider at ten occasions with the following route choice vector $\{1,1,2,1,1,2,1,1,2,1\}$, we use four occasions for initialisation $(T_{I}=4)$ and the remaining six occasions for choice and transition models $(T=6)$. We follow four sequential screening steps to obtain an appropriate sample for the analysis.
To evaluate whether the riders following inertia/habit-based decision rules (i.e., the ones eliminated in step 3) travel more frequently as compared to those in our final sample, we create a kernel density plot of total choice occasions for both in Figure (ref). Similarities in density plots contradict such hypothesis; in fact, the riders making choices entirely based on habit/inertia appear to be less frequent. In the same figure, we also add a kernel density plot of the available choice occasions in the final sample. The added density plot is shifted to the left due to the removal of initial choice situations.
We consider crowding density (measured in passengers per m$^{2}$) and standing probability as dummy (or indicator) covariates in all models. We discretize the parameter space of these covariates because their marginal utilities, when entered in continuous form, do not exhibit statistical significance. Since empirical cumulative density function (CDF) of crowding density has values of 0.39, 0.76, and 0.97 at crowding density of 1, 2, and 3 riders/m$^{2}$,respectively, we create two indicators -- crowding density between 1 and 2 riders/me$^{2}$ (crowding1) and above 2 riders/m$^{2}$ (crowding2), with the base crowding density below 1 rider/m$^{2}$. Similarly, since empirical CDF of standing probability has values of 0.245 and 0.687 at standing probability of 0.4 and 0.7, we create two indicators -- standing probability between 0.4 and 0.7 (SP1) and above 0.7 (SP2), with the base standing probability below 0.4.
To highlight the challenges in modelling a dynamic choice process using static models, we estimate them and present their results.
We first estimate the multinomial logit (MNL) model using the data created to estimate the choice component of DLCM. The MNL estimates for memories 1 and 2 are presented in Table (ref). While computing expectation of route-specific attributes using IBLT, the memory decay parameter $\mu$ is considered to be 1. For both samples, the coefficient of travel time has an intuitive sign, but the sign of its interaction with SP2 is counter-intuitive -- positive with very high z-value. The results remain virtually the same for different values of $\mu$. The sample also includes riders who follow non-compensatory (inertia-based) decision rules and modelling their choices using a compensatory model might have resulted in this discrepancy.
With the possibility to segment riders following different decision rules, we estimate the latent class MNL (LC-MNL) model and results are presented in Table (ref). The improved likelihood suggests that LC-MNL explains the choice process slightly better, but both sign and magnitude of marginal utilities of interaction terms remain counter-intuitive.
We estimate the Dynamic Latent Class Model (DLCM) for both memory values. In the final specification, we keep covariates with z-value greater than 1 and set the EM convergence criterion to $10^{-8}$. Since parameter estimates are not very sensitive to the value of memory decay parameter $\mu$, we first set $\mu$ to 1 and find the model specification. Conditional on this specification, we obtain the optimal value of $\mu$ through grid search.
The parameter estimates of the proposed DLCM for $\mu=1$ are shown in Table (ref). Gradient values of all parameters at convergence are close to zero in both specifications, ensuring convergence to a local optimal. Whereas both specifications have the same number of parameters, the model with the memory of two previous choice occasions explains choices slightly better, i.e. converged to a better loglikelihood (-3734.9 vs. -3740.6) and that too with the rather lower number of available choice occasions ($T=13282$ vs. $T=13447$). Therefore, we find the value of $\mu$ through grid search for DLCM with the memory of two previous choice occasions.
To find the optimal value of $\mu$, we create a one-dimension grid between 0.5 to 1.5, at an increment of 0.1. The resulting loglikelihood values at each grid point are presented in Figure (ref). The results indicate that the optimal value of $\mu$ is 1 and therefore, the specification presented in Table (ref) remains the final specification.
We first discuss the results of the compensatory class. In our specification, Travel Time implies travel time at the base crowding density (< 1 riders/metre$^2$) and the base standing probability ($< 0.4$). Unlike MNL and LC-MNL, sign and magnitude of parameter estimates of the level-of-service attributes are intuitive in the compensatory class of DLCM (see Table (ref)). Values of travel time under crowding1-SP1, crowding1-SP2, crowding2-SP1, and crowding2-SP2 are 1.06, 1.09, 1.16, and 1.19 times that of the value in the base condition, respectively. Since only 3% of riders in our sample have experienced crowding levels above 3 riders/metre$^2$, the value of travel time at crowding density greater than 2 riders/metre$^2$ (crowding2) can be considered as the value of travel time when crowding density is between 2 and 3 riders/metre$^2$. If we further linearly extrapolate the values of travel time under crowding1-SP2 and crowding2-SP2, metro riders' valuation of travel time appears to increase by around 47% in extremely crowded condition (crowding levels between 5 and 6 riders/metre$^2$ and standing probability above 0.7) relative to the one obtained under uncrowded conditions (crowding density less than 1 riders/metre$^2$ and standing probability less than 0.4). We also explore the heterogeneity in crowding cost by specifying a lognormal distribution on the marginal disutility of travel time, but the standard deviation does not turn out be statistically significant.
Since studies based on SP experiments are likely to over-estimate the crowding cost, we compare our crowding cost estimates with those of RP and RP-SP studies tirachini2017estimation. The comparison indicates that the crowding cost estimates of DLCM are lower than the crowding multipliers reported by previous studies. In the RP study of Hong Kong MTR, horcher2017crowding find that increase in the disutility of travel time due to an additional rider per square metre is 0.12 times of the base disutility. Similar results are reported based on SP-RP experiments in Santiago batarce2015use and Paris kroes2014value.
There are two potential reasons why our results are not directly comparable to those of previous studies. First, we are using a non-random sample of regular riders to estimate the model -- our sample excludes more than 80% of riders who follow fully non-compensatory choices during our study period (see step 3 in Section (ref)). These riders might have compensated travel time with crowding when they started using metro, but do not update their route preferences during this experiment. Note that this does not imply that they are insensitive to crowding. Their actual crowding valuation can be lower or higher than our estimates but is not empirically identified due to lack of data on their initial preferences. Whereas earlier studies could identify the crowding cost of this subgroup of riders by making a strong assumption that these riders also make compensatory choices at all occasions, we do not make any such assumption and our crowding valuation estimates are therefore identified using only choice occasions when riders make compensatory choices. Second, the specification of the choice model in DLCM is different as compared to earlier experiments because they control for other trip attributes beside travel time and crowding levels.
We now discuss the results of the non-compensatory class. Conditional on being in the non-compensatory class, route choice in the last occasion and the historical frequency of chosen routes could explain the future route choices of a rider. Positive signs on coefficients of both covariates indicate that a rider is more likely to choose the route that she has chosen on the last choice occasion and the one that she has chosen more frequently in the past. The non-compensatory class thus captures inertia/habit-based decision rules.
We first discuss the results of the initialisation model. A rider who keeps using the same route and makes fewer transitions over the study period is more likely to be in class 2 (non-compensatory class) at the beginning. These results are aligned with the intuition. We also consider mismatch between the expected and experienced level-of-service, but those covariates do not turn out to be statistically significant.
In the transition model for both classes, the same covariates are statistically significant. Among the level-of-service attributes, standing probability has a statistically significant association with class transition probabilities. Loosely speaking, positive sign on standing probability indicates that if a rider expects to sit but she has to stand, she is likely to switch to the compensatory class if she is in the non-compensatory class, otherwise is likely to remain in the compensatory class. The proportion of historical transitions at time $t$ and total transitions in the study period also determine the class transition probabilities. Both covariates have different signs and high magnitude. This observation seems counter-intuitive at first because both variables appear to capture similar behaviour. However, it is not and is rather just a consequence of a structural relationship between these covariates. Keeping the proportion of total transitions constant, if a rider has a higher proportion of historical transitions, she is more likely to choose the same route on the next choice occasion, i.e. she is more likely to be in the non-compensatory class. Therefore, a negative sign on the proportion of historical transitions is intuitive and sensible. A positive sign of the proportion of total transitions can be interpreted similarly.
We also retrieve the latent class of each rider at different choice occasions using the Viterbi algorithm and then we compute the proportion of all choice occasions when riders are in the compensatory class (class 1). On average, riders follow compensatory decision rules only on 25.5% of choice occasions. This proportion is 35.1%, 26.7%, and 14.4% for visitors ($T+T_I <10$), less regular commuters ($10 \leq T+T_I \leq 20$), and regular commuters ($T+T_I > 20$), respectively. In essence, frequent riders are more likely to be in the non-compensatory (inertia/habit) state.
The crowding disutility of urban rail users is often computed by eliciting how riders trade travel time for crowding while making a route choice. Previous crowding valuation studies rely on static choice models, overlook learning of riders, and assume that riders always follow a fully compensatory decision rule while ignoring inertia/habit-based choice behaviour. No existing choice model can capture such dynamic semi-compensatory behaviour of riders.
In this study, we propose a dynamic latent class model (DLCM) which considers heterogeneity in decision rules of a rider by characterising latent classes with decision rules, allows the rider to transition between latent classes over time, and specifies learning behaviour of the rider using instance-based learning theory. We estimate this model by adapting the expectation-maximization (EM) algorithm and recover the most probable sequence of latent classes for each rider using the Viterbi algorithm. We apply the proposed DLCM to estimate the crowding cost of an Asian metro's riders using a two-month-long panel data on their revealed route preferences.
The empirical data indicate that more than 80% of riders keep using the same route or shift to another route only once over two months. This observation further strengthens the importance of incorporating inertia/habits in dynamic choice models. The results of DLCM indicate that marginal disutility of travel time of a rider under extremely crowded conditions (crowding density between 5 and 6 riders/meter$^2$ and standing probability above 0.7) is around 47% higher than the one obtained under uncrowded conditions (crowding density less than 1 riders/meter$^2$ and standing probability less than 0.4). This crowding cost estimate is lower than those reported by previous revealed preference studies. This is perhaps a consequence of an appropriate segmentation of compensatory and inertia-based choice processes. After aggregating the recovered sequence of latent classes for each rider, we find that an average rider follows a compensatory decision rule only on 25.5% of choice occasions and this proportion further decreases to 14.4% for regular commuters.
We note that the crowding cost estimates in this study are obtained from only choice occasions when riders make compensatory choices. However, this raises a natural question of how the consumer benefit of crowding related interventions should be evaluated for those making non-compensatory choices. Intuition suggests that travellers who are making non-compensatory route choices might also perceive the benefits of crowding reduction. However, the welfare calculation approaches for compensatory choices cannot be directly adopted for other non-compensatory decision rules. The subject of benefit calculation under heterogeneous decision rules thus opens up avenues for future research.
We derive the proposed model for two latent classes, but without loss of generality, the model can be extended to multiple classes representing distinct non-compensatory decision rules while accounting for temporal transitions between classes. The potential of DLCM can be explored in understanding dynamic travel behaviour (e.g., preferences for mobility-on-demand services) and various other types of consumer behaviour (e.g., food and healthcare preferences), specially in situations when the longitudinal preference data are easily accessible.