The exact contents of citations.db main_text.text for this paper — one flattened LaTeX string, title through conclusion, appendix excluded, unmodified except for removing email addresses. This is what our citation measures are computed over.
156,022 characters
Learning Sequential Mobility Choice: A Review of Route and Activity Choice through Inverse Reinforcement and Imitation Learning
\maketitle
\begin{abstract}
Route choice and activity choice are distinct instances of a common problem: learning purposeful, feasible sequences of mobility decisions on spatial, temporal, and resource-constrained networks. This review argues that scalable and policy-useful behavioral models should combine the structure of choice modeling with the learning capabilities of inverse reinforcement learning (IRL) and imitation learning (IL), rather than replace one tradition with the other. We develop a \emph{sequential mobility choice} framework with four layers--environment, behavioral objective, stochastic choice mechanism, and observation process--and use it to make the connections precise. Under explicit assumptions, recursive logit, logit dynamic discrete choice, and maximum-entropy IRL share a soft Bellman representation; trajectory occupancies and network flows obey corresponding conservation laws. These equivalences are computationally valuable, but they do not by themselves identify the same behavioral quantities: utility, reward, policy, occupancy, constraints, and observation errors support different interpretations and counterfactuals. We therefore organize route and activity applications by what is learned and what claim the evidence can support. The review assesses constrained MDPs and inverse constraint learning, occupancy-ratio and DICE methods, incomplete and mixed-quality demonstrations, graph and sequence learning, transfer, multi-source data fusion, strategic multi-agent choice, and large language models. The resulting design principle is a behaviorally disciplined hybrid: exact network, schedule, and resource transitions enforce feasibility; structured rewards retain interpretable trade-offs; learned components capture context and heterogeneity; source-aware observation models fuse sparse semantic surveys with dense passive traces; and behavioral solvers convert learned quantities into coherent paths, schedules, flows, and policy counterfactuals. The central message is that machine learning creates the most value in mobility choice when it expands scale, representation, and data integration while preserving behavioral identification and transport-system structure.
\end{abstract}
\noindent\textbf{Keywords:} sequential mobility choice; route choice; activity-based travel demand; recursive logit; dynamic discrete choice; inverse reinforcement learning; constrained MDP; inverse constraint learning; imitation learning; offline reinforcement learning; data fusion; trajectory modeling
\clearpage
\setcounter{tocdepth}{2}
\tableofcontents
\clearpage
\section{Introduction}
Travel behavior is rarely an isolated selection from a menu. A driver traverses a sequence of connected roads; a transit passenger combines services and transfers; and a household arranges activities, locations, modes, and durations under time, money, and coordination constraints. Each decision changes the opportunities available later. A useful behavioral model must therefore explain not only which alternative is selected now, but how feasible and purposeful sequences emerge--and how those sequences change when networks, prices, schedules, information, or constraints change.
Transportation and machine learning have developed different languages for this problem. Random-utility models, recursive route choice, activity-based microsimulation, dynamic discrete choice, and traffic assignment emphasize utility, substitution, feasibility, equilibrium, and policy interpretation. Markov decision processes (MDPs), inverse reinforcement learning (IRL), imitation learning (IL), offline reinforcement learning (RL), and sequence models emphasize reward, policies, occupancy, representation, and scalable prediction. The two traditions increasingly study the same empirical object--observed trajectories--but often estimate different objects and accept different evidence. This gap matters: matching observed paths is not the same as recovering preferences, and a reward that reproduces current behavior need not support a credible toll, closure, or accessibility counterfactual.
We use \emph{sequential mobility choice} for the common model class: a context-conditioned sequence of feasible travel and activity decisions whose consequences accumulate through a network and through time. Route choice is its spatial, usually destination-conditioned instance. Activity choice enlarges the state to include purposes, timing and duration, remaining resources, prior commitments, and household coordination. The common formulation exposes reusable mathematics and learning tools without pretending that the applications are identical. A next-link model can be adequate for routing and still be invalid for activity duration, endogenous destination formation, or coordination among household members.
\begin{keybox}
\textbf{Core message.} Route and activity choice are connected levels of the same sequential mobility process: activity choice organizes what people do, where, and when, while route choice governs how they move between those activities. IRL and IL provide scalable ways to learn rewards, policies, occupancies, and representations from observed trajectories, but these objects are not interchangeable with preferences and do not by themselves guarantee valid policy counterfactuals. The promising direction is therefore a behaviorally disciplined hybrid--use ML for scale, rich context, and data integration while keeping feasible choice sets, behavioral trade-offs, constraints, observation processes, and network feedback explicit.
\end{keybox}
The formal bridge is clearest for route choice. Recursive logit assigns utility to an outgoing link plus the continuation value of the remaining trip and converts these values into local choice probabilities \citep{fosgerau2013link,mai2015nested}. Logit dynamic discrete choice obtains the same soft recursion from independent extreme-value reward shocks, while entropy-regularized MDPs obtain it from an entropy term. Mai and Jaillet establish equivalence between the latter two formulations from both forward-control and inverse-learning perspectives \citep{mai2020relation}. On a network with an absorbing destination, recursive logit supplies the stochastic-shortest-path specialization; maximum-entropy IRL supplies a trajectory-distribution and reward-learning interpretation \citep{ziebart2008maxent,zimmermann2020tutorial}. Thus the connection among recursive choice, dynamic discrete choice, and maximum-entropy IRL is not merely terminological: under explicit assumptions, they share a soft Bellman structure.
The same perspective extends from paths to daily schedules. Activity-based demand starts from the principle that travel is derived from activity participation. A schedule joins activity type, timing, duration, location, mode, and trip chaining \citep{bhat1999activity,bowman2001activity}. Represented on a time-space-resource network, that schedule is a path through an augmented state space: actions travel, wait, begin or continue an activity, and end the day. Recursive choice, dynamic discrete choice, IRL, and IL can then operate on the larger graph \citep{zimmermann2018activity,song2024stateirl}. Activity choice is therefore a first-class sequential-learning problem, but a harder one than route choice because its goals may be endogenous, its horizon is longer, durations matter, and resource and coordination constraints bind across episodes.
The unification stops where behavioral interpretation begins. Equivalent soft policies can be described through utility shocks, entropy regularization, or rewards, but those descriptions do not automatically identify the same behavioral mechanism. Structural choice models typically target utility parameters, elasticities, welfare, and intervention response. IRL targets a reward representation, which is generally non-unique without scale, shaping, state, and constraint conventions \citep{ng1999shaping,train2009discrete}. IL targets a policy or occupancy and can reproduce behavior without explaining it. Offline RL uses logged behavior to improve a policy, making it prescriptive rather than automatically descriptive. Likewise, observation error, missing trips, platform selection, congestion, and unavailable alternatives can be absorbed incorrectly into a learned reward. The estimand--utility, reward, policy, occupancy, constraint, or trajectory distribution--must therefore be chosen from the scientific question, not from the preferred algorithm.
This leads to the paper's main thesis: \emph{a scalable behavioral mobility model should be a structured learning system, not a choice between econometrics and machine learning}. Four roles should remain distinct. First, exact network, schedule, and resource transitions define what can happen. Second, structured utility or reward terms define interpretable behavioral trade-offs, with explicit normalization and constraints. Third, flexible learned components represent high-dimensional context, heterogeneity, residual utility, latent trajectories, and amortized value functions where classical specifications or repeated solvers become limiting. Fourth, source-aware observation and selection models connect sparse semantic surveys with dense passive traces, while planners or equilibrium layers convert learned individual behavior into coherent system outcomes. This division of labor preserves feasibility and behavioral meaning while using ML where it contributes scale, representation, and data integration.
Under this view, modern ML methods are valuable for concrete behavioral bottlenecks rather than for architectural novelty alone. Occupancy-ratio and DICE methods can learn from fixed off-policy data; graph and sequence models can share information over large networks and long activity histories; constrained and inverse-constrained learning can separate cumulative limits from preferences; transfer and multi-source fusion can connect cities and data regimes; multi-agent learning can represent congestion, coordination, and strategic response; and large language models can assist data processing, semantic representation, synthetic scenario construction, and simulation interfaces. None of these capabilities by itself establishes preference recovery or policy validity. Their value must be demonstrated through identification checks, domain transfer, intervention response, system effects, robustness, equity, privacy, and computational performance.
The paper makes six contributions. First, it defines sequential mobility choice and provides a formal crosswalk among route choice, activity choice, dynamic discrete choice, IRL, and IL. Second, it separates exact equivalences from useful analogies and unresolved identification claims. Third, it organizes representative evidence by the object learned and the behavioral claim supported. Fourth, it evaluates modern ML families by the transportation bottleneck they address. Fifth, it proposes a hybrid architecture that preserves behavioral quantities while exploiting graph, sequence, occupancy, generative, and language models. Sixth, it specifies evaluation and reporting standards for prediction, transfer, counterfactuals, system impacts, and computation. Together, these contributions support one practical message: machine learning makes mobility choice models more scalable and useful when it strengthens--rather than obscures--behavioral structure.
The remainder of the paper is organized as follows. Section~2 defines the scope and review strategy, and Section~3 reviews the transportation foundations of route and activity choice. Section~4 develops the unified sequential-mobility-choice framework and introduces the running commute example. Sections~5 and~6 map method families by what they learn and explain where ML adds value, including occupancy methods, constraints, data fusion, multi-agent learning, and LLMs. Section~7 translates the synthesis into a behaviorally disciplined hybrid architecture, Section~8 specifies evaluation and benchmarking requirements, and Section~9 sets out the research agenda. Sections~10 and~11 discuss the broader implications and conclude; a final disclosure states how AI-assisted tools were used in preparing the review.
\section{Scope and Review Strategy}
This is a critical integrative review, not a systematic review with claimed exhaustive coverage. The focus is individual or household behavior represented as a sequence on a physical, temporal, or activity network. The core domains are road and transit route choice, pedestrian and bicycle routing, multimodal journeys, daily activity scheduling, and related dynamic discrete choices. Traffic control, autonomous driving, and vehicle routing are discussed only when their learning methods directly inform behavior-from-demonstration. The review also draws selectively on structural econometrics, operations research, and modern generative modeling.
The literature was organized using three concept blocks: (i) route, path, trajectory, activity, schedule, and travel behavior; (ii) recursive, dynamic, sequential, network, or flow choice; and (iii) IRL, IL, occupancy matching, offline RL, graph learning, transformer, diffusion, or flow matching. Seminal work before 2000 is retained where it defines random utility, dynamic discrete choice, activity-based demand, or IRL. Recent transportation applications are used to assess what has been demonstrated empirically, while survey and methodological papers support broader comparisons \citep{prato2009route,arora2021survey,hussein2017imitation,levine2020offline}.
The search was updated through 15 August 2026 using Crossref and TRID records, publisher indexes, broad scholarly-web searches, and backward and forward citation tracing from foundational papers. Queries combined at least one transportation term (route, path, activity, schedule, mobility, or travel behavior) with one sequential-model term (recursive, dynamic, Markov, network, trajectory, or occupancy) and one learning term (inverse reinforcement, imitation, offline, transfer, graph, transformer, diffusion, or language model). Titles and abstracts were screened first; full texts or publisher records were then checked for the learned object, data regime, transportation task, evaluation design, and publication status. Preprints are retained only where they supply distinctive scale or frontier evidence and are labeled as such in the synthesis. Because this is purposive rather than exhaustive coverage, the protocol supports transparency but not PRISMA-style prevalence claims.
Three inclusion rules guide the synthesis. A paper must learn or estimate a utility, reward, policy, occupancy, or trajectory distribution from observed behavior; or it must establish an identification, computational, or evaluation result directly relevant to that task. Prediction-only models are included when they illuminate a frontier, but they are not labeled behavioral merely because their inputs are travel traces. Finally, claims are weighted by what the evidence supports: in-sample fit supports representation, held-out trajectories support prediction, domain shifts support transfer, and interventions or known synthetic truths are needed for structural recovery.
Each application is read along five evidence dimensions: whether it establishes predictive performance, behavioral interpretation, reward or parameter recovery, transfer across contexts, and system or intervention validity. This coding is deliberately conservative. A large dataset supports a scale claim only when network size, destinations, runtime, and solver tolerance are reported; an interpretable surrogate supports description of a learned policy but does not by itself identify the underlying reward.
\section{Sequential Mobility Choice: Transportation Foundations}
This section establishes the transportation models that any learning-based alternative should be compared against. It proceeds from complete-path random utility to recursive link choice, flow-based choice, and activity-schedule models. These formulations do more than provide prediction baselines: they specify feasible alternatives, behavioral quantities, substitution patterns, and policy counterfactuals. At the same time, each has a characteristic bottleneck--path enumeration, repeated dynamic programs, aggregation, or combinatorial state. Accordingly, each subsection uses the same comparison: the model's behavioral advantage, its statistical or computational limitation, and the role in which ML can add value. The relevant question is not whether a neural model fits better in the abstract, but whether learned components expand representation, scale, or data coverage while retaining the choice set, normalization, constraints, and counterfactual meaning that the application requires.
\subsection{Path-based random utility}
Let $\mathcal{P}_{od}$ denote feasible paths between origin $o$ and destination $d$. A path-based random-utility model writes
\begin{equation}
U_{in}=V_{in}+\varepsilon_{in}, \qquad
V_{in}=\boldsymbol{\beta}^{\top}\mathbf{x}_{in},
\label{eq:pathrum}
\end{equation}
where $\mathbf{x}_{in}$ contains travel time, cost, turns, road class, reliability, or other attributes. Under independent type-I extreme-value errors, the probability of choosing path $i$ from choice set $C_n$ is multinomial logit. This formulation supports interpretable marginal utilities, elasticities, values of time, and welfare analysis \citep{mcfadden1974conditional,benakiva1985discrete,train2009discrete}.
The difficulty is the alternative space. Real networks contain enormous numbers of feasible, overlapping paths, and the analyst does not observe which paths a traveler considered. Generated choice sets can produce inconsistent inference unless their sampling mechanism is handled correctly \citep{frejinger2009sampling}. Shared links also induce correlation across path utilities, motivating path-size corrections, subnetworks, generalized extreme-value models, mixed logit, and other structures \citep{frejinger2007subnetworks,prato2009route}. Static path utilities are transparent, but complete-path enumeration and time-varying adaptation remain awkward.
Relative to a black-box path or sequence predictor, path-based random utility offers a transparent likelihood, interpretable marginal utilities, explicit substitution, and direct welfare calculations. Its limitations are a hand-specified utility surface and a sampled or generated alternative set whose omissions can bias estimation. ML can enrich path attributes, learn nonlinear residual utility and traveler heterogeneity, improve map matching, or propose informative alternatives; hybrid discrete-choice models demonstrate how learned representations or taste functions can be embedded inside a normalized utility model \citep{sifringer2020representation,han2022tastenet}. Those components should enter a sampling-corrected choice likelihood; otherwise improved prediction may come at the cost of unknown choice-set support, unnormalized scores, and uninterpretable elasticities \citep{frejinger2009sampling,train2009discrete}.
\subsection{Recursive route choice}
Recursive models replace one path choice with a sequence of link choices. At state $s$--a node or current link--the traveler selects a feasible outgoing action $a$. The action yields instantaneous utility $v_\theta(s,a)$ and moves the traveler toward an absorbing destination. For i.i.d. extreme-value shocks with scale $\mu$, expected maximum utility satisfies a log-sum recursion. The induced local probabilities define a distribution over all feasible paths without enumerating them \citep{fosgerau2013link}.
Nested recursive logit allows state- or link-specific scales and richer substitution patterns \citep{mai2015nested}. Generalized recursive models embed multivariate extreme-value correlation structures in the value recursion, and dynamic-programming algorithms make large network-based MEV specifications estimable \citep{mai2016correlation,mai2017mev}. Decomposition and linear-algebra methods reduce repeated solution cost during estimation, while convergence analysis clarifies when undiscounted cyclic networks admit a well-defined value function \citep{mai2018decomposition,mai2022undiscounted}. Recursive models also extend to stochastic, time-dependent networks in which the traveler chooses a routing policy conditional on evolving information rather than committing to one deterministic path \citep{mai2021stochasticroute}. These developments matter for IRL because correlated shocks, changing transition conditions, and adaptive policies alter what a recovered reward can explain. Recursive models are already parametric MDPs estimated from demonstrations. In machine-learning language, recursive route choice is a structured, entropy-regularized inverse control model with a known transition graph.
This formulation solves path enumeration, but not every behavioral problem. The Markov state must contain all information required for continuation decisions. Omitted memory, habitual plans, unobserved information, and route-level attributes can violate that condition. Linkwise independent shocks also impose substitution restrictions, while cyclic networks require attention to improper looping. These are modeling assumptions, not implementation details \citep{puterman1994mdp,prato2009route,mai2022undiscounted}.
Correlation has several noninterchangeable meanings in sequential mobility choice. Shared links create correlation among complete-path utilities; persistent traveler effects create serial correlation across repeated trips; nested or generalized extreme-value shocks alter local substitution; and congestion or coordination creates strategic dependence across decision makers. A neural embedding can improve prediction while leaving these sources unidentified. A behavioral application should state which correlation is represented, whether it enters utility, scale, latent classes, transitions, or equilibrium, and what substitution test distinguishes the specification from an independence model.
Relative to unconstrained next-link prediction, recursive choice provides exact action feasibility, destination-conditioned full-path probabilities, and a value recursion that supports behavioral parameters and network counterfactuals. Its main limitations are state sufficiency, restrictive utility or shock specifications, and the repeated solution of destination-specific dynamic programs at scale. Graph encoders and neural residual rewards can represent spatial context and heterogeneity, while amortized value functions, graph compression, and warm-started solvers can reduce computation \citep{zhao2023deepirl,barnes2023maps,battaglia2018relational}. The action mask, terminal condition, probability normalization, and exact or certified Bellman layer should remain explicit; a locally accurate neural policy need not reach the destination or preserve substitution under a closure \citep{fosgerau2013link,mai2022undiscounted}.
\subsection{Flow-based route choice}
Perturbed utility route choice (PURC) represents behavior through network flows rather than individual paths. A stylized formulation is
\begin{equation}
\max_{\mathbf{x}\in\mathcal{F}_{od}}
\left\{\mathbf{u}^{\top}\mathbf{x}-\Omega(\mathbf{x})\right\},
\label{eq:purc}
\end{equation}
where $\mathcal{F}_{od}$ is the flow-conservation polytope and $\Omega$ is a convex perturbation. The perturbation spreads flow across plausible alternatives and creates overlap-sensitive substitution without a generated path set. Estimation and prediction can exploit convex optimization, yielding an attractive full-network model \citep{fosgerau2022purc}. Its stochastic-assignment extension connects individual regularized choice to congestion feedback \citep{yao2024pusta}.
The flow view is important because an expected state-action occupancy in an MDP also satisfies conservation. This shared feasible set creates a direct bridge between network assignment and occupancy-based IL. Nevertheless, a flow optimum is not automatically an individual-level probabilistic model, and aggregation may hide heterogeneity. The objective, regularizer, and interpretation of fractional flow must therefore be stated.
Flow-based choice is attractive when the target is network-wide demand: conservation is exact, path enumeration disappears, and convex structure can support stable, large-scale computation. The cost is behavioral resolution. Aggregate flows need not identify individual path probabilities, taste heterogeneity, or the reward that generated them, especially when congestion changes the feasible equilibrium. ML can learn nonlinear edge utilities, demand segmentation, residual flow corrections, or differentiable equilibrium surrogates, and occupancy-ratio methods can reweight logged flows across regimes \citep{nachum2019dualdice,yao2024pusta}. Conservation and congestion feedback should remain solver constraints rather than training penalties; otherwise a low-error flow forecast may be infeasible or behaviorally uninterpretable \citep{fosgerau2022purc,yao2024pusta}.
\subsection{Activity-based and schedule choice}
Activity-based demand models replace independent trips with the organization of a day. Classical systems combine activity-pattern, tour, destination, mode, and time-of-day models; later microsimulators use rules, hazards, optimization, or network representations to construct schedules \citep{bhat1999activity,bowman2001activity,miller2003tasha}. Their behavioral advantage is substantive: trips are linked because they serve activities, household members coordinate, and time spent in one episode changes later opportunities.
The connection to sequential learning becomes exact on a time-space network. A state may be written
\begin{equation}
s_t=(\ell_t,t,\mathbf{b}_t,\mathbf{h}_t,\mathbf{c}_n),
\label{eq:activitystate}
\end{equation}
where $\ell_t$ is location, $\mathbf{b}_t$ records remaining time, money, or energy budgets, $\mathbf{h}_t$ summarizes completed or required activities, and $\mathbf{c}_n$ contains personal and household context. Actions include staying, traveling, selecting a mode or destination, starting an activity, and ending the day. A complete schedule becomes a path through this augmented network. Mixed recursive logit has been estimated on such networks for joint activity, location, timing, and mode choice \citep{zimmermann2018activity}; recent IRL applications learn activity-travel rewards and policies from mobile traces or travel surveys \citep{song2024stateirl,liang2026interpretable}.
Activity choice also exposes the limits of a naive MDP. An activity episode has a type, location, start time, and duration, and its continuation changes the opportunity set for the rest of the day. A semi-Markov representation makes duration explicit: choosing activity $a_t$ and duration $\Delta_t$ moves clock time from $t$ to $t+\Delta_t$ and accumulates activity benefit, schedule delay, and travel cost over that interval. Without such a representation, a model may reproduce the next activity while misrepresenting time allocation. Long-term commitments, household negotiation, and flexible timing may further require partially observed beliefs or multi-agent models \citep{puterman1994mdp,zimmermann2018activity}.
The empirical base is expanding but remains thinner than for route choice. Data-driven activity schedulers have generated regional populations with machine-learning components while retaining activity-based validation targets \citep{drchal2019scheduler}. State-based IRL has been evaluated on cellular signaling data with reward recovery and transfer tasks \citep{song2024stateirl}; context-aware IRL has generated daily schedules from a large resident travel survey \citep{liu2025contextirl}; and interpretable deep IRL has used policy distillation and reward analysis on Singapore travel-survey sequences \citep{liang2026interpretable}. These studies demonstrate prediction and representation, but they do not yet establish that learned rewards support welfare analysis or remain invariant under major land-use, household, or accessibility changes.
Activity-schedule models preserve the derived nature of travel, whole-day feasibility, time and resource accounting, and linkages among activity, location, mode, and duration choices. Their limitation is the combinatorial state and alternative space, compounded by sparse diaries, partial observation, long memory, heterogeneous constraints, and household interaction. Representation learning can compress histories and context; latent-variable and data-fusion models can combine surveys with passive traces; and sequence or IRL methods can learn flexible schedule rewards and policies \citep{drchal2019scheduler,song2024stateirl,liang2026interpretable}. These gains are credible only when semi-Markov duration, resource transitions, mandatory episodes, household coupling, and end-of-day completion remain explicit \citep{puterman1994mdp,zimmermann2018activity}. Next-activity accuracy alone is not evidence of a feasible daily schedule or a transferable behavioral reward.
\begin{longtable}{@{}p{0.18\textwidth}p{0.34\textwidth}p{0.38\textwidth}@{}}
\caption{Route and activity choice as distinct instances of sequential mobility choice.}\label{tab:instances}\\
\toprule
\textbf{Dimension} & \textbf{Route-choice instance} & \textbf{Activity-choice instance} \\
\midrule
\endfirsthead
\toprule
\textbf{Dimension} & \textbf{Route-choice instance} & \textbf{Activity-choice instance} \\
\midrule
\endhead
\endfoot
\bottomrule
\endlastfoot
State & Current node or link, destination, time, information, traveler context & Location, clock time, activity history, remaining commitments and resources, household context \\
\specialrule{0.25pt}{0.12em}{0.12em}
Action & Feasible outgoing link, maneuver, service, or local routing decision & Travel, wait, start or continue an activity, choose location/mode/duration, coordinate with others \\
\specialrule{0.25pt}{0.12em}{0.12em}
Horizon & Usually one trip with an absorbing destination & Day or week with endogenous goals, episode durations, and multiple terminal obligations \\
\specialrule{0.25pt}{0.12em}{0.12em}
Dependence & Path overlap, persistent tastes, information, congestion & Duration dependence, trip chaining, household coupling, accessibility, congestion \\
\specialrule{0.25pt}{0.12em}{0.12em}
Primary observations & GPS traces, map matches, probe trajectories, stated routes & Diaries, time use, smart cards, cellular traces, household and census data \\
\specialrule{0.25pt}{0.12em}{0.12em}
Core validation & Path distribution, substitution, destination completion, closure/toll response & Complete-day feasibility, timing and duration distributions, joint schedules, policy response \\
\end{longtable}
\section{A Unified Sequential-Mobility-Choice Framework}
This section supplies the mathematical hinge of the review. Rather than claiming that route choice, activity choice, dynamic discrete choice, IRL, and IL are interchangeable, it identifies three bridges that make their relationships precise: a soft-Bellman bridge between recursive choice and entropy-regularized decision policies; an occupancy-flow bridge between trajectory visitation and network flow conservation; and an inverse-problem bridge between observed choices and latent utility or reward. The construction begins with a four-layer generative model so that environment dynamics, behavioral objectives, stochastic choice, and data recording remain distinct. It then states the assumptions under which the bridges hold and the identification boundaries beyond which predictive equivalence does not support a behavioral interpretation. This organization draws on standard MDP and dynamic-choice formulations, recursive route choice, maximum-entropy inverse decision making, and occupancy-based imitation \citep{puterman1994mdp,rust1987replacement,fosgerau2013link,ziebart2008maxent,ziebart2010causal,mai2020relation,ho2016gail}.
\subsection{Four layers}
Let a context $c$ contain the destination, traveler attributes, trip purpose, time period, information regime, and policy environment. A latent trajectory is $\tau=(s_0,a_0,s_1,\ldots,s_T)$. Its model probability can be factored as
\begin{equation}
p(y,\tau\mid c)=p(s_0\mid c)
\prod_{t=0}^{T-1}\pi_\theta(a_t\mid s_t,c)
P(s_{t+1}\mid s_t,a_t,c)\,p_\psi(y\mid\tau,c),
\label{eq:fourlayer}
\end{equation}
where $y$ is the recorded GPS, diary, smart-card, or cellular observation. Equation~\eqref{eq:fourlayer} combines the standard controlled Markov factorization with an explicit observation model, thereby separating latent behavior from the process that records it \citep{puterman1994mdp,ziebart2010causal,mai2023incomplete}.
\begin{longtable}{@{}p{0.17\textwidth}p{0.25\textwidth}p{0.25\textwidth}p{0.23\textwidth}@{}}
\caption{Four layers of sequential mobility choice.}\label{tab:fourlayers}\\
\toprule
\textbf{Layer} & \textbf{Transportation terms} & \textbf{ML terms} & \textbf{Question} \\
\midrule
\endfirsthead
\toprule
\textbf{Layer} & \textbf{Transportation terms} & \textbf{ML terms} & \textbf{Question} \\
\midrule
\endhead
\endfoot
\bottomrule
\endlastfoot
Environment & Network, schedules, time, congestion, feasible modes & State, action, transition kernel, horizon & What can happen after each decision? \\
\specialrule{0.25pt}{0.12em}{0.12em}
Objective & Utility, generalized cost, activity benefit, regret, risk & Reward, return, cost, constraint cost & What makes the sequence purposeful? \\
\specialrule{0.25pt}{0.12em}{0.12em}
Choice mechanism & Random utility, scale, nesting, perturbation & Policy, planner, entropy, occupancy & How are values converted to stochastic behavior? \\
\specialrule{0.25pt}{0.12em}{0.12em}
Observation & GPS, diary, map matching, sampling, missing trips & Demonstrations, latent states, behavior policy, sensor model & Which part of behavior was actually recorded? \\
\end{longtable}
This decomposition prevents common category errors. Road connectivity belongs to the environment, not the reward. A legal prohibition is not necessarily a large negative preference. GPS error is not behavioral randomness. Entropy can represent unobserved utility variation, bounded precision, or a deliberate regularizer, but those interpretations imply different counterfactuals. Heterogeneity may enter rewards, beliefs, constraints, or noise scale; placing it indiscriminately in a neural embedding weakens behavioral meaning \citep{benakiva1985discrete,train2009discrete,altman1999cmdp,ziebart2010causal}.
\begin{figure}[t]
\centering
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize\small{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
\begin{tabular}{c@{\;$\longrightarrow$\;}c@{\;$\longrightarrow$\;}c}
\fbox{\parbox[c][3.1em][c]{0.24\textwidth}{\centering Context and\\heterogeneous sources}} &
\fbox{\parbox[c][3.1em][c]{0.24\textwidth}{\centering Observation and\\selection operators}} &
\fbox{\parbox[c][3.1em][c]{0.24\textwidth}{\centering Latent feasible\\trajectory or schedule}} \\
\multicolumn{3}{c}{$\Downarrow$} \\[0.35em]
\fbox{\parbox[c][3.1em][c]{0.24\textwidth}{\centering Learned object:\\utility, reward, policy, occupancy}} &
\fbox{\parbox[c][3.1em][c]{0.24\textwidth}{\centering Network solver, planner,\\or equilibrium layer}} &
\fbox{\parbox[c][3.1em][c]{0.24\textwidth}{\centering Paths, schedules, flows,\\and counterfactuals}}
\end{tabular}
\caption{The sequential-mobility-choice pipeline. Source-specific observations inform a latent feasible behavioral process; the scientific objective determines what is learned and which solver converts it into individual and system outcomes.}
\label{fig:pipeline}
\end{figure}
\subsection{The soft Bellman bridge}
For reward $r_\theta(s,a,c)$, transition kernel $P$, discount factor $\gamma$, and entropy scale $\mu>0$, define
\begin{align}
Q_\theta(s,a,c) &= r_\theta(s,a,c)+\gamma
\mathbb{E}_{s'\sim P(\cdot\mid s,a,c)}[V_\theta(s',c)], \label{eq:softq}\\
V_\theta(s,c) &= \mu\log\sum_{a\in\mathcal{A}(s,c)}
\exp\{Q_\theta(s,a,c)/\mu\}, \label{eq:softv}\\
\pi_\theta(a\mid s,c) &=
\exp\{[Q_\theta(s,a,c)-V_\theta(s,c)]/\mu\}. \label{eq:softpolicy}
\end{align}
In dynamic logit, Equation~\eqref{eq:softv} is the expected maximum utility generated by extreme-value shocks. In entropy-regularized RL it is a soft value function. In maximum-causal-entropy IRL, the likelihood of demonstrations is induced by Equation~\eqref{eq:softpolicy}. More precisely, entropy-regularized MDPs and stochastic MDPs with independent Gumbel reward shocks--the logit dynamic-discrete-choice formulation--are equivalent in both forward-control and inverse-learning perspectives \citep{mai2020relation}. Recursive logit is the network specialization in which actions are outgoing links, transitions are usually deterministic, the destination is absorbing, and the horizon is an undiscounted stochastic-shortest-path problem \citep{rust1987replacement,fosgerau2013link,ziebart2010causal,zimmermann2020tutorial}.
For the undiscounted episodic case ($\gamma=1$), consider a feasible path $\sigma=(s_0,a_0,\ldots,s_T=d)$ with deterministic transitions, additive reward, and $V(d)=0$. The product of local policies telescopes:
\begin{equation}
p_\theta(\sigma\mid s_0,d)=
\exp\left\{\frac{1}{\mu}\sum_{t=0}^{T-1}r_\theta(s_t,a_t,d)
-\frac{1}{\mu}V_\theta(s_0,d)\right\}.
\label{eq:telescoping}
\end{equation}
Thus local soft choices induce a globally normalized exponential distribution over feasible paths. This is the strongest formal connection among recursive logit, logit dynamic discrete choice, and maximum-entropy route IRL: recursive logit supplies the network state--action structure, while the entropy-regularized/stochastic-MDP equivalence supplies the common soft policy and its forward and inverse interpretations \citep{mai2020relation}. It requires a proper stochastic-shortest-path model: the destination must be reached with probability one and rewards must rule out infinitely attractive cycles so that the partition function is finite. With discounting, stochastic transitions, finite-horizon time indexing, nested scales, non-Markovian attributes, or alternative entropy regularizers, the corresponding trajectory distribution changes and the displayed telescoping form does not apply unchanged. Maximum trajectory entropy and maximum causal entropy must likewise be distinguished when transitions are stochastic \citep{ziebart2008maxent,ziebart2010causal,mai2022undiscounted,zimmermann2020tutorial}.
\subsection{The occupancy-flow bridge}
For initial distribution $\rho_0$ and policy $\pi$, define the discounted state-action occupancy
\begin{equation}
d_\pi(s,a)=(1-\gamma)\sum_{t=0}^{\infty}\gamma^t
\Pr_\pi(s_t=s,a_t=a).
\end{equation}
It satisfies
\begin{equation}
\sum_a d_\pi(s,a)=(1-\gamma)\rho_0(s)
+\gamma\sum_{s',a'}P(s\mid s',a')d_\pi(s',a').
\label{eq:occupancy}
\end{equation}
This is flow conservation with source terms. Route link flows can be constructed from occupancy counts; activity-state occupancies record visits to combinations of location and activity; and adversarial IL attempts to match the demonstrator's occupancy measure \citep{puterman1994mdp,ho2016gail}. PURC, entropy-regularized assignment, and occupancy matching therefore operate on closely related feasible polytopes \citep{fosgerau2022purc}. Their objectives differ: one may maximize perturbed utility, another minimize a divergence from expert occupancy, and another recover rewards whose optimal occupancy matches data.
The analogy becomes exact only after matching units. For a finite trip, define the unnormalized episodic visitation measure
\begin{equation}
\bar d_\pi(s,a\mid c)=
\mathbb{E}_\pi\!\left[\sum_{t=0}^{T-1}
\mathbf{1}\{s_t=s,a_t=a\}\mid c\right].
\label{eq:episodicoccupancy}
\end{equation}
If $q_c$ travelers share context $c$, then $q_c\bar d_\pi(s,a\mid c)$ is an expected link-turn flow. The normalized discounted measure in Equation~\eqref{eq:occupancy} is instead a probability distribution useful for continuing tasks and density-ratio estimation. For activity models, decision-epoch occupancy counts visits; it represents time spent only after weighting by episode duration or using a semi-Markov occupancy. These distinctions prevent probability mass, person trips, vehicles, and person-time from being conflated \citep{puterman1994mdp,zimmermann2018activity,nachum2019dualdice}.
\subsection{The inverse-problem bridge--and its boundary}
For linear reward $r_\theta(s,a)=\theta^{\top}f(s,a)$, the maximum-entropy log-likelihood gradient has the familiar form
\begin{equation}
\nabla_\theta\ell(\theta)=
\widehat{\mathbb{E}}_{E}\!\left[\sum_t f(s_t,a_t)\right]
-\mathbb{E}_{\pi_\theta}\!\left[\sum_t f(s_t,a_t)\right],
\label{eq:featurematching}
\end{equation}
which matches demonstrated and model feature counts \citep{ziebart2008maxent}. Structural dynamic discrete choice likewise infers flow utility from conditional choice probabilities and continuation values \citep{hotz1993ccp,aguirregabiria2010dynamic}. Both are inverse problems: choices reveal payoffs only relative to dynamics, opportunity sets, information, and shock assumptions.
The boundary is identification. Rewards are not generally unique. Adding constants, rescaling reward with choice noise, potential-based shaping, or reallocating payoff between immediate and terminal components can preserve behavior \citep{ng1999shaping,ng2000algorithms}. Omitted states can cause a learned reward to absorb beliefs, constraints, selection, or equilibrium conditions. In route choice, time and cost coefficients support a value of time only after scale normalization and under exogeneity \citep{train2009discrete}. In activity choice, an inferred preference for staying home may instead reflect childcare, inaccessible modes, or missing activities. IRL does not remove these structural problems; it makes their sequential form visible \citep{malik2021icrl}.
\begin{keybox}
Formal equivalence does not imply inferential equivalence. Two methods may generate the same policy while assigning different meanings to reward, shocks, regularization, and unobserved constraints. Behavioral claims require normalizations and invariance assumptions beyond predictive fit.
\end{keybox}
\subsection{Running example: a constrained commute and activity chain}
\label{sec:runningexample}
Consider a stylized weekday decision that will serve as a running example. A traveler begins at home, must drop a child at school before a fixed deadline, must reach work within a preferred arrival window, may stop for groceries after work, and then returns home. The realized chain is therefore not just a route but a linked sequence such as home $\rightarrow$ school $\rightarrow$ work $\rightarrow$ grocery $\rightarrow$ home. At each episode the traveler chooses whether to start or end an activity, when to depart, which mode to use, and which road or transit link to take. A policy experiment introduces a cordon toll and a temporary road closure; because many travelers respond, travel times and crowding change endogenously.
A unified state records location, time, current activity, remaining commitments, available modes and vehicles, accumulated expenditure, and relevant network information. Connectivity, operating hours, one-way rules, and unavailable modes define the feasible action set. The school deadline, daily time budget, fare or toll budget, and vehicle range are resource or CMDP constraints. Travel time, monetary cost, reliability, schedule delay, activity duration, safety, and comfort enter a structural reward, while a graph or sequence representation may capture nonlinear neighborhood context, history, and heterogeneous perceptions. This separation matters: bypassing a tolled link could reveal price sensitivity, a school deadline, lack of vehicle access, incomplete information, a GPS gap, or congestion caused by other travelers. An unrestricted reward model can absorb all six explanations.
The observation record is also layered. A household survey may reveal trip purpose, the school commitment, available vehicles, and socioeconomic context but cover only one or two days. GPS provides repeated fine-grained routes with missing segments and weak semantics; a smart card reveals transit transactions; traffic counts and probes observe aggregate flow and speed; and the network supplies legal transitions and tolls. In Equation~\eqref{eq:fourlayer}, these sources should inform one latent activity--travel trajectory through separate observation and selection models rather than being concatenated as equally reliable rows \citep{mai2023incomplete,graellsgarrido2023fusion,vo2026fusion}.
Table~\ref{tab:runningexample} should be read from left to right. Each row starts from a different question about the same observed day, identifies the object returned by the method, and then limits the conclusion that can be drawn from that object. The rows are therefore alternative scientific uses of the data, not competing names for one estimator.
\begin{longtable}{@{}p{0.15\textwidth}p{0.24\textwidth}p{0.22\textwidth}p{0.27\textwidth}@{}}
\caption{Different learning questions applied to the running commute example.}\label{tab:runningexample}\\
\toprule
\textbf{Method family} & \textbf{Question posed in the example} & \textbf{Output} & \textbf{What it supports--and does not} \\
\midrule
\endfirsthead
\toprule
\textbf{Method family} & \textbf{Question posed in the example} & \textbf{Output} & \textbf{What it supports--and does not} \\
\midrule
\endhead
\endfoot
\bottomrule
\endlastfoot
Recursive / dynamic choice & How do time, toll, reliability, and schedule delay change the probability of each feasible decision? & Normalized utility coefficients and recursive choice probabilities. & Can support elasticities, value of time, and welfare under scale, availability, exogeneity, observation, and equilibrium assumptions; good fit alone is insufficient. \\
\specialrule{0.25pt}{0.12em}{0.12em}
IRL & What reward makes the complete school--work--shopping sequence probable, and can it be replanned after a disruption? & Reward or soft value function, followed by an induced policy. & Supports goal or environment transfer if reward meaning is stable; shaping, omitted states, and hidden constraints prevent automatic taste interpretation. \\
\specialrule{0.25pt}{0.12em}{0.12em}
Behavior cloning / IL & Given the observed history, what activity, mode, or link will the traveler choose next? & Conditional policy over the next feasible action or sequence. & Supports fast prediction and behavioral simulation near the data distribution; it does not identify utility, value of time, or welfare. \\
\specialrule{0.25pt}{0.12em}{0.12em}
Occupancy / DICE & Can logged activity and link visits be reproduced or corrected for a different supported logging or recommendation regime? & Occupancy measure, density ratio, or corrected target policy. & Supports flow matching and some offline evaluation under coverage; density ratios are not taste coefficients and do not recover a transferable reward. \\
\specialrule{0.25pt}{0.12em}{0.12em}
Offline RL & Which recommendation best reduces emissions or delay while respecting safety, budget, and data-support constraints? & Prescriptive policy for an explicitly supplied operator reward and constraints. & Supports constrained recommendation under coverage and acceptance assumptions; it describes what should be recommended, not necessarily what travelers prefer. \\
\specialrule{0.25pt}{0.12em}{0.12em}
Multi-agent / equilibrium model & How do toll and closure responses alter congestion, crowding, and therefore every traveler's incentives? & Type-specific policies plus a policy--flow or game equilibrium. & Supports system counterfactuals when the solution concept and equilibrium are credible; a fixed-regime single-agent policy cannot capture this feedback. \\
\end{longtable}
To see the distinction, suppose the GPS trace shows that the traveler bypasses the newly tolled link and still reaches school on time. A structural choice model treats this as information about cost and time trade-offs conditional on the stated choice set. IRL asks which trajectory reward makes the entire detour and subsequent schedule probable. Behavior cloning learns the detour as a conditional action pattern. DICE asks how much the observed transition should be reweighted under a supported target regime. Offline RL may recommend a different detour because its operator reward values emissions or safety, while a multi-agent model asks whether widespread detouring changes the travel times that motivated the response. These outputs may predict the same maneuver, but they attach different scientific meanings to it.
The same observed chain can therefore support several valid analyses, but not the same conclusion. After a toll or closure, a policy counterfactual must additionally recompute congestion and acceptance. The scientific question--explanation, prediction, transfer, simulation, or prescription--must determine which object is learned and which assumptions are tested \citep{train2009discrete,arora2021survey,levine2020offline}.
\begin{keybox}
The running example makes the paper's central message concrete: one mobility trajectory can be represented as sequential choice, but utility, reward, policy, occupancy, and optimized recommendation remain different estimands. ML adds value when it solves a declared representation, data, or computation bottleneck without erasing feasibility, identification, or system feedback.
\end{keybox}
\section{What Is Learned? A Map of Method Families}
This section classifies methods by their learned object rather than by algorithmic label. The same set of GPS traces or activity diaries can be used to estimate utility coefficients, recover a reward, imitate a conditional policy, match an occupancy measure, correct an off-policy distribution, infer constraints, or generate a trajectory directly. These targets are not interchangeable: they imply different computational costs, data requirements, interpretations, and counterfactual claims. The method map therefore asks four questions for each family: what is learned, what scalability or data problem it solves, which behavioral quantities it preserves, and which failure mode limits its use as a choice model. This distinction prepares the later discussion of where ML adds value by separating gains in prediction and computation from evidence of preference recovery or structural validity. Table~\ref{tab:runningexample} illustrates the distinction on the constrained commute introduced in Section~\ref{sec:runningexample}.
\begin{longtable}{@{}p{0.17\textwidth}p{0.17\textwidth}p{0.22\textwidth}p{0.27\textwidth}@{}}
\caption{Method families classified by their primary estimand.}\label{tab:methods}\\
\toprule
\textbf{Family} & \textbf{Learned object} & \textbf{Main value} & \textbf{Main limitation for behavior} \\
\midrule
\endfirsthead
\toprule
\textbf{Family} & \textbf{Learned object} & \textbf{Main value} & \textbf{Main limitation for behavior} \\
\midrule
\endhead
\endfoot
\bottomrule
\endlastfoot
Path-based choice & Utility coefficients & Interpretation, elasticities, welfare & Generated alternatives and overlap \\
\specialrule{0.25pt}{0.12em}{0.12em}
Recursive choice / DDC & Flow utility and scale & Full path support and structural recursion & Repeated value solves; state design \\
\specialrule{0.25pt}{0.12em}{0.12em}
PURC / flow choice & Utility and aggregate flow & Convex, network-wide prediction & Individual heterogeneity needs structure \\
\specialrule{0.25pt}{0.12em}{0.12em}
Behavior cloning & Conditional policy & Simple, scalable prediction & Compounding error and weak extrapolation \\
\specialrule{0.25pt}{0.12em}{0.12em}
IRL & Reward, then policy & Goal and environment transfer & Reward ambiguity and planning cost \\
\specialrule{0.25pt}{0.12em}{0.12em}
Constrained IRL / ICRL & Reward under known constraints, or latent constraint costs & Separates preferences from feasibility and cumulative limits & Joint reward--constraint ambiguity and weak negative evidence \\
\specialrule{0.25pt}{0.12em}{0.12em}
Occupancy IL & Policy or implicit reward & Matches sequential visitation & Interpretation and domain transfer \\
\specialrule{0.25pt}{0.12em}{0.12em}
DICE / density-ratio learning & Occupancy correction or target policy & Offline evaluation, imitation, and constrained optimization & Requires support overlap and a correctly indexed flow model \\
\specialrule{0.25pt}{0.12em}{0.12em}
Observation-aware / state-only learning & Latent trajectory, state occupancy, or implicit policy & Uses incomplete, irregular, or action-free mobility records & Observation-model dependence and latent-path ambiguity \\
\specialrule{0.25pt}{0.12em}{0.12em}
Mixed-quality / avoidance learning & Quality-weighted reward or policy & Combines expert, suboptimal, unlabeled, or negative demonstrations & Quality-label assumptions and possible imitation of undesirable behavior \\
\specialrule{0.25pt}{0.12em}{0.12em}
Offline RL & Improved policy & Prescriptive optimization from logs & Support violations; not a descriptive model \\
\specialrule{0.25pt}{0.12em}{0.12em}
Sequence / generative model & Conditional trajectory distribution & Long context and multimodality & Feasibility and counterfactual meaning \\
\specialrule{0.25pt}{0.12em}{0.12em}
LLM / generative agent & Semantic representation, high-level plan, or prompted choice & Low-data adaptation and natural-language context & Prompt sensitivity, cost, hallucination, and weak identification \\
\end{longtable}
The advantages of ML appear along three distinct dimensions. Computational scalability comes from amortized policies and value functions, shared graph representations, minibatch optimization, and distributed training, although inverse methods may still contain an expensive planner or equilibrium loop. Statistical scalability comes from flexible nonlinear functions and partial pooling across travelers, destinations, networks, and contexts. Observational scalability comes from learning with passive traces that are incomplete, irregular, action-free, or mixed in quality: latent-path likelihoods, state-only occupancy methods, and quality-aware imitation can use records that a fully observed choice likelihood would discard \citep{barnes2023maps,mai2023incomplete,pham2025iostom,hoang2024sprinql}. Hybrid choice models further show that learned representations or taste functions can improve specification while retaining a normalized probabilistic core \citep{sifringer2020representation,han2022tastenet}. These are genuine benefits for route and activity modeling, especially when the network, feature space, and data volume exceed what manual specification and repeated exact estimation can handle.
The disadvantages concern what those gains do not guarantee. A fast policy can compound local errors and fail outside expert-state support; an occupancy match can reproduce current flows without identifying preferences; an offline optimizer can exploit unsupported actions; and a deep reward can absorb observation error, availability, constraints, congestion, or selection. Greater function capacity can therefore improve held-out prediction while making utility scale, substitution, welfare, and intervention response less identifiable. IRL also inherits reward shaping and normalization ambiguity, while generative sequence models can produce plausible but infeasible paths or schedules \citep{ross2011dagger,ho2016gail,ng1999shaping,levine2020offline,arora2021survey}. Post-hoc feature importance or a readable neural surrogate is useful for diagnosis, but it is not equivalent to recovering a behaviorally identified utility or reward.
This last distinction clarifies the meaning of \emph{structural learning}. In ML, structure may refer to graph topology, an architectural constraint, or a latent representation. In choice modeling, a structural claim ties learned quantities to preferences, feasible opportunities, information, and equilibrium so that a specified intervention has a defensible interpretation. An ML component becomes structurally useful when it is embedded in that generative system, respects normalization and feasibility, and is tested for invariance under the counterfactual it is meant to support \citep{train2009discrete,sifringer2020representation,han2022tastenet}. Accordingly, the method families below should not be ranked on one accuracy scale. Their value depends on whether the scientific target is prediction, behavioral explanation, reward transfer, population simulation, missing-data recovery, or policy optimization; Section~6 returns to these bottlenecks and identifies the ML role appropriate to each.
\subsection{Direct policy learning}
Behavior cloning estimates $\pi(a\mid s,c)$ by supervised learning. In route choice it predicts the next link, mode, or maneuver; in activity choice it predicts the next episode or schedule event. It is attractive because it avoids repeated planning and can use large neural architectures. Its central weakness is distribution shift: a small error can move the model to a state absent from training, after which subsequent predictions deteriorate. Interactive data aggregation such as DAgger addresses this by querying an expert in learner-visited states \citep{ross2011dagger}, but retrospective mobility datasets rarely permit such interaction.
Behavior cloning is appropriate when the target is near-term prediction within the observed regime, actions are well labeled, and the deployment distribution is close to the data. It is not a reward-recovery method and should not be used for welfare analysis. For offline travel data, support diagnostics, action masks, uncertainty, and destination- or plan-level losses are essential additions to local cross-entropy \citep{hussein2017imitation,arora2021survey}.
\subsection{Reward learning through IRL}
IRL assumes demonstrations are approximately optimal for an unknown reward. Early approaches used margin separation and feature-expectation matching \citep{ng2000algorithms,abbeel2004apprenticeship,ratliff2006margin}. Maximum-entropy IRL defines a normalized trajectory distribution and estimates reward parameters by feature matching \citep{ziebart2008maxent}. Adversarial IRL uses a discriminator to learn a reward while optimizing a policy, and methods such as IQ-Learn estimate soft values directly from demonstrations \citep{fu2018airl,garg2021iqlearn}.
Reward learning is especially attractive when preferences should transfer while destinations, networks, or constraints change. A road-type or safety reward can be replanned for a new destination; an activity reward can be applied to a changed transport system. Recent route-choice work uses context-dependent deep rewards and demonstrates prediction to unseen destinations \citep{zhao2023deepirl}. Large-scale industrial work uses graph compression, spatial mixtures of experts, and receding-horizon inverse planning on road graphs with hundreds of millions of states \citep{barnes2023maps}.
The promise should be stated carefully. Transfer occurs only if the learned reward captures a stable preference rather than city-specific opportunity, platform selection, congestion, or sensor artifacts. Raw neural reward magnitudes are not economic utility coefficients. A responsible application reports which reward contrasts are identified, fixes scale and shaping conventions, tests invariance across environments, and separates reward interpretation from policy interpretation \citep{ng1999shaping,arora2021survey,train2009discrete}.
\subsection{Occupancy matching and adversarial imitation}
Generative adversarial imitation learning matches expert state-action occupancies through a discriminator and policy optimization \citep{ho2016gail}. Occupancy methods reduce compounding error by matching the distribution induced by complete rollouts rather than only expert-state conditionals. For transportation, this naturally aligns with observed link flows, time-location profiles, and activity frequencies.
However, occupancy matching can reproduce the current demand regime without recovering preferences that remain stable under new origin-destination matrices or capacities. It also inherits simulator dependence: policy rollouts require a credible environment. The method is most compelling when the desired output is a realistic agent population for a known scenario and less compelling when the goal is value-of-time inference or welfare analysis \citep{ho2016gail,hussein2017imitation,arora2021survey}.
\subsection{The DICE family: occupancy and density-ratio correction}
The DIstribution Correction Estimation (DICE) family makes the occupancy-flow bridge operational for fixed, off-policy datasets. Depending on the formulation, the target is a discounted episodic occupancy or a stationary distribution. Let $d_\beta(s,a)$ be the occupancy of the behavior policy that generated the observations and $d_\pi(s,a)$ the corresponding occupancy of a target policy. DICE methods estimate the marginalized density ratio
\begin{equation}
w_\pi(s,a)=\frac{d_\pi(s,a)}{d_\beta(s,a)}
\label{eq:diceratio}
\end{equation}
by enforcing Bellman-flow identities, rather than multiplying a long sequence of action-level importance ratios. The target policy can then be recovered from reweighted behavior frequencies,
\begin{equation}
\pi(a\mid s)=
\frac{w_\pi(s,a)d_\beta(s,a)}{\sum_{a'}w_\pi(s,a')d_\beta(s,a')}.
\label{eq:dicepolicy}
\end{equation}
DualDICE estimates ratios for behavior-agnostic off-policy evaluation, while GenDICE extends stationary-value estimation across discounted and average-reward settings \citep{nachum2019dualdice,zhang2020gendice}. ValueDICE turns off-policy distribution matching into a direct imitation objective \citep{kostrikov2020valuedice}. The later family branches by target: OptiDICE seeks the return-maximizing offline policy; COptiDICE adds conservative cost constraints; DemoDICE combines limited expert data with supplementary imperfect demonstrations; SMODICE matches state occupancies when expert actions are unavailable; LobsDICE instead matches state-transition distributions from observations; and RelaxDICE weakens exact behavior-distribution matching when supplementary data are highly suboptimal \citep{lee2021optidice,lee2022coptidice,kim2022demodice,ma2022smodice,kim2022lobsdice,yu2023relaxdice}. MisoDICE extends this logic to cooperative multi-agent imitation with mixed-quality data through value decomposition and a learned mixing architecture \citep{bui2025misodice}. These are related density-ratio constructions, not one common behavioral estimator.
\begin{longtable}{@{}p{0.16\textwidth}p{0.27\textwidth}p{0.46\textwidth}@{}}
\caption{Representative DICE variants and their transportation interpretation.}
\label{tab:dice}\\
\toprule
\textbf{Method} & \textbf{Primary problem} & \textbf{Route/activity-choice interpretation} \\
\midrule
\endfirsthead
\toprule
\textbf{Method} & \textbf{Primary problem} & \textbf{Route/activity-choice interpretation} \\
\midrule
\endhead
\endfoot
\bottomrule
\endlastfoot
DualDICE / GenDICE & Off-policy evaluation & Reweight logged link or activity transitions to estimate a fixed counterfactual policy under coverage assumptions. \\
\specialrule{0.25pt}{0.12em}{0.12em}
ValueDICE & Off-policy imitation & Match an expert trajectory distribution without requiring fresh simulator rollouts. \\
\specialrule{0.25pt}{0.12em}{0.12em}
DemoDICE & Offline IL with imperfect support data & Use a small trusted travel sample plus broader, mixed-quality traces while limiting departure from observed support. \\
\specialrule{0.25pt}{0.12em}{0.12em}
SMODICE & State-only offline imitation & Learn from location or activity observations when maneuver or action labels are absent. \\
\specialrule{0.25pt}{0.12em}{0.12em}
LobsDICE & Transition-only offline imitation & Match observed state transitions when expert actions are unavailable and separate action-labeled support data exist. \\
\specialrule{0.25pt}{0.12em}{0.12em}
RelaxDICE & Offline IL with highly suboptimal support data & Preserve support without forcing the target flow to reproduce an undesirable behavior distribution exactly. \\
\specialrule{0.25pt}{0.12em}{0.12em}
OptiDICE & Offline policy optimization & Learn a prescriptive routing or recommendation policy from logs; this is not automatically a model of traveler preference. \\
\specialrule{0.25pt}{0.12em}{0.12em}
COptiDICE & Constrained offline optimization & Optimize recommendations subject to expected safety, emissions, accessibility, budget, or service constraints. \\
\specialrule{0.25pt}{0.12em}{0.12em}
MisoDICE & Cooperative multi-agent offline IL & Learn coordinated local policies from unlabeled, mixed-quality joint demonstrations using decomposed values. \\
\end{longtable}
For an origin--destination, departure-time, and traveler-type context $c$, an episodic route occupancy becomes an expected link-turn flow after multiplication by the demand $q_c$. Equation~\eqref{eq:diceratio} therefore becomes a target-to-observed occupancy correction, and Equation~\eqref{eq:dicepolicy} converts corrected occupancies into local recursive choices. This provides three concrete uses. First, a fitted choice policy can be evaluated on logs collected under another information or routing regime. Second, a demonstrator flow pattern can be reproduced by a scalable weighted policy without repeated adversarial interaction. Third, a constrained recommendation policy can be optimized while remaining close to the observed support. Activity choice has the same interpretation on a time--space--resource graph, with duration-weighted occupancies when person-time is the target.
The connection has strict boundaries. DICE weights are not taste coefficients, and matching corrected flows does not identify value of time, welfare, or a transferable reward. Coverage must hold within each relevant destination, time, information, and traveler-type stratum; a road link observed in the pooled data may still be unsupported for a particular decision maker. Absorbing destinations and finite trips require episodic flow constraints rather than blindly applying an infinite-horizon stationary formula \citep{nachum2019dualdice,lee2021optidice}. Congestion also makes $d_\pi$ endogenous to the policies of other travelers. In that case a single-agent ratio is a conditional correction at a fixed traffic regime, not a strategic counterfactual; equilibrium or multi-agent extensions are required.
\subsection{Learning from observations and incomplete trajectories}
Many mobility records contain locations but no explicit actions, irregular sampling, missing trip segments, or ambiguous map matches. Learning from observation reconstructs inverse dynamics or matches state-only occupancies \citep{torabi2018behavioral,sun2019observation}. In road networks, actions may become inferable after map matching, but treating a single matched path as truth understates uncertainty.
A stronger formulation marginalizes latent paths in Equation~\eqref{eq:fourlayer}. Missing route segments can be integrated over all feasible connections using dynamic programming or linear systems rather than filled with one shortest path. For recursive route choice, the probabilities of disconnected observations can be evaluated exactly and accelerated by decomposition--composition, offering an alternative to expectation--maximization with sampled completions \citep{mai2023incomplete}. For activity choice, missing short stops, unobserved in-home episodes, and coarse cell locations require a joint latent schedule and observation model. When expert states are observed but expert actions are unavailable, state-transition occupancy matching can learn an implicit transition policy and infer actions without a separately trained inverse-dynamics model \citep{pham2025iostom}. These methods show how probabilistic sequence learning can address incomplete data without changing the behavioral core or declaring one imputed trajectory to be truth.
\subsection{Learning from suboptimal, mixed-quality, and negative data}
Observed travel is not uniformly expert. Drivers make wrong turns, travelers explore, habits persist after conditions change, and recorded routes may be undesirable from a safety or emissions perspective. Standard IL can overfit a small expert sample or imitate the larger mass of suboptimal behavior. SPRINQL uses expert and suboptimal demonstrations jointly, learning an inverse soft-$Q$ representation that gives different alignment weights to the two sources and admits a convex formulation over $Q$ functions \citep{hoang2024sprinql}. A complementary line asks what can be learned when only undesirable demonstrations are labeled: maximum-entropy avoidance learning can use negative and unlabeled examples without pretending the undesirable data are expert behavior \citep{hoang2025avoidance}.
Transportation data naturally fit this spectrum. Demonstrations can be labeled by incident exposure, rule violations, excessive detours, accessibility failures, or user feedback, while unlabeled data supply network coverage. Good/bad labels should be tied to an explicit objective or constraint rather than a vague expert score. In multi-agent settings, MisoDICE combines preference-based trajectory labeling with occupancy learning to exploit unlabeled, mixed-quality demonstrations \citep{bui2025misodice}. These methods are relevant both descriptively--to avoid treating every observation as optimal--and prescriptively--to learn safer policies while retaining information about dynamics and feasible alternatives.
\subsection{Offline reinforcement learning}
Offline RL learns a policy with high expected return from a fixed dataset. Its target differs fundamentally from observed behavior. If demonstrations contain unsafe, costly, or environmentally damaging decisions, the desired recommender may deliberately depart from them. Yet value maximization outside the data support can exploit extrapolation error; conservative methods therefore penalize unsupported actions or values \citep{levine2020offline,kumar2020cql}.
Transportation applications should separate three layers: an IRL or choice model describes traveler preferences; a system objective values safety, emissions, reliability, or welfare; and an acceptance model predicts whether recommendations will be followed. Combining these into one reward obscures whose objective is optimized. Offline RL is suitable for prescription only when coverage, constraints, uncertainty, and human response are evaluated explicitly \citep{levine2020offline,lee2022coptidice}.
\subsection{Learning regime: offline, interactive, continual, and transfer}
``Online'' is used ambiguously in transportation: it can mean real-time prediction from a fixed model, sequential parameter updating as data arrive, or active interaction that changes which observations are collected. Only the last two are online learning. Most revealed mobility data support offline estimation because untested routing or activity policies cannot ethically be deployed merely to acquire demonstrations. Interactive IL such as DAgger queries an expert in learner-visited states \citep{ross2011dagger}; active preference learning selects informative trajectory comparisons \citep{biyik2018active}; and active IRL explores to reduce uncertainty about rewards and dynamics \citep{lindner2022active}. In mobility applications, comparable interaction can use stated-choice queries, opt-in recommendation experiments, or staged field trials, with safety, burden, selection, and interference reported explicitly.
Continual learning updates a descriptive model as networks, services, information, and habits change. It requires drift detection and versioned parameters so adaptation is not mistaken for stable preference. Transfer learning instead asks which component should remain invariant across destinations, travelers, cities, or policies. Destination-conditioned deep route IRL has shown prediction to unseen destinations \citep{zhao2023deepirl}, while recursive-logit-based meta-IRL shares information across sparse routing tasks \citep{zhang2024metairl}. Transfer is credible only when the split removes leakage through link identities, nearby destinations, repeated travelers, and common time periods.
\begin{longtable}{@{}p{0.18\textwidth}p{0.25\textwidth}p{0.26\textwidth}p{0.22\textwidth}@{}}
\caption{Learning regimes for sequential mobility choice.}\label{tab:regimes}\\
\toprule
\textbf{Regime} & \textbf{Information available} & \textbf{Mobility use} & \textbf{Principal validity risk} \\
\midrule
\endfirsthead
\toprule
\textbf{Regime} & \textbf{Information available} & \textbf{Mobility use} & \textbf{Principal validity risk} \\
\midrule
\endhead
\endfoot
\bottomrule
\endlastfoot
Offline estimation / IL & Fixed logged trajectories, surveys, or transitions & Behavioral description and supported prediction & Coverage, selection, and confounding \\
\specialrule{0.25pt}{0.12em}{0.12em}
Offline RL / DICE & Fixed logs plus a target reward, cost, or policy & Evaluation and constrained recommendation & Extrapolation beyond behavioral support \\
\specialrule{0.25pt}{0.12em}{0.12em}
Interactive IL / active IRL & Expert corrections, stated preferences, or controlled feedback & Efficient elicitation and model correction & Query burden, intervention effects, safety \\
\specialrule{0.25pt}{0.12em}{0.12em}
Continual learning & Time-ordered data with repeated updates & Adaptation to incidents, services, and changing habits & Forgetting and confusion of drift with taste \\
\specialrule{0.25pt}{0.12em}{0.12em}
Transfer / meta-learning & Related tasks, populations, destinations, or networks & Cold-start prediction and sparse-context estimation & Leakage and unstable invariance assumptions \\
\end{longtable}
\subsection{What transportation applications currently establish}
Table~\ref{tab:evidence} summarizes representative evidence rather than ranking methods. It makes visible the imbalance in the literature: route applications currently support stronger computational and destination-transfer claims, while activity applications increasingly support schedule generation and reward analysis but have fewer intervention tests.
\begin{longtable}{@{}p{0.18\textwidth}p{0.24\textwidth}p{0.27\textwidth}p{0.22\textwidth}@{}}
\caption{Representative transportation evidence for learned sequential choice.}\label{tab:evidence}\\
\toprule
\textbf{Study and domain} & \textbf{Data or setting} & \textbf{Strongest evidence} & \textbf{Important boundary} \\
\midrule
\endfirsthead
\toprule
\textbf{Study and domain} & \textbf{Data or setting} & \textbf{Strongest evidence} & \textbf{Important boundary} \\
\midrule
\endhead
\endfoot
\bottomrule
\endlastfoot
\citet{fosgerau2013link}, road routes & Network observations and recursive likelihood & Full feasible-path support without explicit path enumeration & Parametric utility and Markov-state assumptions \\
\specialrule{0.25pt}{0.12em}{0.12em}
\citet{zhao2023deepirl}, road routes & Shanghai taxi GPS trajectories & Nonlinear context-dependent reward and unseen-destination prediction & Prediction does not establish welfare-valid reward \\
\specialrule{0.25pt}{0.12em}{0.12em}
\citet{barnes2023maps}, road routing & Industrial road graphs and large-scale inverse planning & Computation over graphs with hundreds of millions of states & Preprint evidence and platform-selected behavior \\
\specialrule{0.25pt}{0.12em}{0.12em}
\citet{mai2023incomplete}, incomplete routes & Disconnected route observations & Exact marginal likelihood over missing route segments & Requires specified observation and route-choice model \\
\specialrule{0.25pt}{0.12em}{0.12em}
\citet{drchal2019scheduler}, activity schedules & Regional synthetic population and travel evidence & Data-driven daily scheduling with activity-model validation & Supervised schedule reproduction, not reward recovery \\
\specialrule{0.25pt}{0.12em}{0.12em}
\citet{song2024stateirl}, activity travel & Cellular signaling trajectories & State-based IRL, reward recovery, and transfer experiments & Coarse semantic state and limited policy interventions \\
\specialrule{0.25pt}{0.12em}{0.12em}
\citet{liu2025contextirl}, activity schedules & Large resident travel survey & Context-aware schedule generation and generalization tests & One-day reported behavior and assumed MDP structure \\
\specialrule{0.25pt}{0.12em}{0.12em}
\citet{liang2026interpretable}, activity travel & Singapore travel-survey sequences & Policy distillation and short-/long-run reward diagnostics & Surrogate interpretability is not structural identification \\
\end{longtable}
\section{Where Machine Learning Adds Value: Core and Emerging Roles}
Machine learning adds value to sequential choice modeling through more than flexible function approximation. It brings an integrated computational and empirical toolkit: hardware-accelerated tensor operations, automatic differentiation, stochastic and distributed optimization, reusable representations, data-efficient adaptation, uncertainty estimation, and disciplined evaluation under distribution shift. These tools can make route and activity models feasible at larger spatial, temporal, and population scales, and can extract information from data that are too heterogeneous or incomplete for a single conventional likelihood. Their value is nevertheless conditional. Faster estimation or better held-out prediction does not by itself identify preferences, preserve welfare meaning, respect network feasibility, or support a policy counterfactual. This section therefore treats each ML contribution as a pairing of an operational gain with a behavioral requirement and a corresponding research risk. In the running commute, for example, GPU batching can scale link and schedule likelihoods and a graph encoder can represent context, but neither determines whether toll avoidance is preference, constraint, observation error, or equilibrium response.
The evidence is most mature for scalable solvers, structured nonlinear rewards, incomplete observations, and graph or sequence representations. Accelerator-aware computation, modern evaluation practice, and sample-efficient transfer are enabling capabilities that cut across these roles. Generative models, strategic multi-agent learning, and LLM agents are best read as emerging extensions unless they are coupled to transport-specific feasibility, identification, and intervention tests.
\subsection{Scalable, accelerator-aware computation}
Scale has four dimensions: graph size, number of destinations or activity goals, number of trajectories, and complexity of the reward or state. Recursive choice removes path enumeration but can require one value solution per destination and parameter update. Activity networks add time and resource dimensions. IRL creates an outer inverse problem around a forward planner.
Many of these operations can be written as batched tensor programs and differentiated end to end. GPUs are especially useful for dense neural encoders, parallel trajectory likelihoods, Monte Carlo samples, batched Bellman backups, and repeated origin--destination or traveler-context computations; automatic-differentiation frameworks also remove much of the manual derivative work required by nested likelihood, planning, and equilibrium components \citep{baydin2018autodiff,paszke2019pytorch}. The speedup is not automatic. Irregular sparse graphs, destination-specific linear systems, dynamic choice sets, and frequent host--device transfers can leave an accelerator underused or exhaust its memory. A practical design may therefore keep graph indexing and sparse factorizations on the CPU while batching rewards, value updates, and gradients on the GPU, or use accelerator-native sparse kernels when their numerical behavior is verified.
Choice modeling can also borrow mature algorithmic engineering from ML: vectorized data loaders, minibatching by destination or schedule context, warm starts across nearby parameter values, cached graph features, importance-corrected alternative or subgraph sampling, gradient accumulation, checkpointing, mixed precision, just-in-time compilation, and data-parallel training. These devices can change an infeasible estimator into a usable one, but some also change the numerical or statistical problem. Uncorrected negative sampling changes probability normalization; truncated planning changes the implied continuation value; stale caches break gradient consistency; and low precision can corrupt small probability differences or convergence tests. Approximate training should therefore retain exact feasibility masks, sampling corrections, and periodic likelihood, Bellman-residual, flow-conservation, and equilibrium checks.
Promising model-level responses include destination batching, sparse linear algebra, graph compression, contraction hierarchies, decomposition, sampled subgraphs with corrected objectives, low-rank value functions, and amortized planning. An amortized value network $\widehat V_\phi(s,c)$ can warm-start or approximate repeated Bellman solves, while periodic exact residual checks prevent silent drift. Spatial mixtures of experts let different models specialize while sharing parameters \citep{barnes2023maps}. The important reporting unit is not ``runs on a large network'' but runtime, memory, planner calls, and energy as functions of edges, destinations, contexts, batch size, hardware, precision, and solver tolerance.
\subsection{Sample efficiency and reusable learning}
Mobility datasets can contain billions of link transitions while still providing little independent information about rare travelers, new destinations, disruptions, or policy changes. Sample efficiency should therefore be defined by the scarce unit that identifies the target: labeled trips, distinct travelers and days, cities, interventions, expert comparisons, or simulator calls, rather than by the raw number of state transitions. ML offers several useful mechanisms. Self-supervised pretraining can learn network and schedule representations from unlabeled traces; hierarchical learning and partial pooling share strength across traveler types; transfer and meta-learning adapt a common model to a new destination or city; active learning selects informative stated-choice or trajectory queries; and replay or importance weighting can reuse logged experience \citep{finn2017maml,biyik2018active,zhang2024metairl}.
The correct test is a learning curve, not a single full-data score. Studies should report performance against the number of independent people, observed days, labels, queries, and forward-planner calls, and should compare pretraining and transfer with a from-scratch model given the same target-domain information. Gains may otherwise be pseudo-efficiency caused by repeated observations from the same person, leakage through destination or link identifiers, or synthetic trajectories generated from the evaluation environment. Negative transfer is also behavioral: pooling cities or traveler groups can erase genuine taste, scale, availability, and information differences. A sample-efficient choice model must state what is shared, what remains group-specific, and whether the shared representation stays invariant under the counterfactual of interest.
\subsection{Evaluation infrastructure and metrics}
ML has developed useful experimental infrastructure that choice modeling can adopt: versioned train--validation--test splits, automated hyperparameter search with fixed budgets, strong baselines, component ablations, repeated random seeds, uncertainty estimation, calibration diagnostics, stress tests, and out-of-distribution evaluation. These practices are important because flexible models can be miscalibrated and stochastic learning pipelines can produce material variation across runs \citep{guo2017calibration,lakshminarayanan2017deepensembles,henderson2018matters}. Shared pipelines also make it easier to test a classical choice model and an ML-augmented model with the same information, feasible alternatives, and observation process.
No single ML metric is sufficient for a behavioral choice model. Local prediction requires proper probabilistic scores such as negative log likelihood or the Brier score, calibration by traveler and context, and top-$k$ coverage where useful. Sequential coherence requires destination completion, valid transitions, schedule feasibility, and trajectory or occupancy fidelity. Structural validity requires plausible signs, elasticities, substitution patterns, scale normalization, and stability of welfare-relevant coefficients. Policy use additionally requires held-out intervention response, support diagnostics, uncertainty, equilibrium consistency, and system outcomes such as congestion, accessibility, emissions, and distributional effects. Computational reporting should include runtime, peak memory, energy, inference latency, and learning curves. Section~8 turns these layers into a benchmark suite; the central principle is that an accuracy gain cannot compensate for failure on the behavioral or counterfactual layer required by the application.
\subsection{Flexible yet structured reward functions}
Linear-in-parameters utility is interpretable but can miss nonlinear interactions among time, road environment, weather, trip purpose, schedule pressure, and personal constraints. A useful hybrid is
\begin{equation}
r(s,a,c)=\boldsymbol{\beta}^{\top}\mathbf{x}(s,a,c)
+g_\phi(\mathbf{z}(s,a,c)),
\label{eq:hybridreward}
\end{equation}
where $\mathbf{x}$ contains policy-relevant attributes with sign, scale, or monotonicity restrictions, and $g_\phi$ is a regularized learned residual. The structural term supports values of time and elasticities; the residual captures interactions and perceptual context. Orthogonality penalties, residual centering, or staged estimation can reduce the tendency of $g_\phi$ to absorb the interpretable component \citep{zhao2023deepirl,liang2026interpretable}.
Heterogeneity should also be structured. Hierarchical or latent-class rewards can distinguish stable taste variation; embeddings can capture remaining high-dimensional variation; and random scale should not be confused with random preferences. Partial pooling is preferable to one unconstrained model per traveler, especially when personal histories are short \citep{train2009discrete}.
\subsection{Graph representation learning}
Road, transit, and activity networks provide relational inductive bias. Graph neural networks can encode topology, land use, turns, transfer relations, and neighborhood context \citep{battaglia2018relational}. Their most defensible role is to produce edge features, rewards, or value corrections while an exact action mask and network solver enforce connectivity. This preserves feasibility and allows the same encoder to operate on different networks.
Generalization is not automatic. A model can memorize node identities, spatial coordinates, or frequently observed destinations. Evaluation should therefore include unseen regions and networks, permutation or identifier checks, and topology changes. Graph learning earns its complexity when it improves transfer or represents context unavailable to hand-designed attributes, not merely when it improves a random-split accuracy score.
\subsection{Long-context sequence models}
Transformers can represent long histories, activity chains, trip purpose, household context, and non-Markovian dependence. Decision Transformer recasts offline control as sequence modeling over returns, states, and actions \citep{chen2021decision}. For activity choice, an autoregressive model can condition each episode on earlier activities and remaining day structure. For routes, it can capture planned-path persistence and dependence beyond the current link.
The design should remain transport-aware. Feasible-action masks prevent disconnected moves; time and resource tokens enforce schedule logic; graph positional encodings replace arbitrary link identifiers; and a terminal-consistency loss penalizes failure to reach a destination or complete mandatory activities. A transformer policy predicts sequences, but return conditioning alone does not identify utility or justify welfare claims.
\subsection{Generative trajectory models}
Diffusion and flow-matching models can represent multimodal distributions and generate diverse trajectories \citep{janner2022diffuser,chi2023diffusion,lipman2023flow}. Potential transportation uses include probabilistic map matching, completion of missing route segments, generation of plausible activity schedules, and scenario-conditioned simulation. They may capture multiple route modes better than a unimodal decoder.
Discrete networks make unconstrained generation risky. A sequence that is geometrically smooth can still be disconnected, violate a one-way restriction, or make an impossible transfer. Generative models should operate over feasible path representations, use constrained decoding, or generate latent preferences that are passed to a solver. Likelihood calibration, rare-event coverage, controllability, and intervention response remain open research questions; diffusion is not a substitute for a behavioral estimand.
\subsection{Constrained MDPs and inverse constraint learning}
Travel behavior contains restrictions that should not all be represented as large negative utilities. A context-specific constrained MDP (CMDP) augments the reward with constraint-cost functions $\kappa_j(s,a,c)$ and budgets $b_j(c)$. For an episodic or discounted mobility process, it solves
\begin{equation}
\begin{aligned}
\max_{\pi}\quad
J_r(\pi\mid c)
&=\mathbb{E}_{\pi,P}\!\left[\sum_{t=0}^{T-1}\gamma^t r(s_t,a_t,c)\right],\\
\text{s.t.}\quad
J_{\kappa_j}(\pi\mid c)
&=\mathbb{E}_{\pi,P}\!\left[\sum_{t=0}^{T-1}\gamma^t
\kappa_j(s_t,a_t,c)\right]\le b_j(c),
\quad j=1,\ldots,m .
\end{aligned}
\label{eq:cmdp}
\end{equation}
CMDPs formalize cumulative limits \citep{altman1999cmdp}, while safe RL supplies tools such as shields, robust backup, and primal--dual optimization \citep{garcia2015safe}. A complementary equivalence result shows that, for a fixed reward, regularized or stochastic MDP policies can be represented by suitable convex statewise policy constraints; this connects entropy and random-utility interpretations to a particular constrained-MDP representation, but not to arbitrary trajectory-level resource constraints \citep{mai2020relation}. In mobility, $\kappa_j$ can measure collision exposure, emissions, monetary expenditure, accessibility violations, missed commitments, or battery depletion. The budget may be traveler-specific, household-specific, or imposed by an operator. Hard, known rules--network connectivity, one-way restrictions, closed stations, vehicle eligibility, or impossible time transitions--belong instead in the feasible action set $\mathcal{A}(s,c)$ or transition kernel. Encoding a known impossibility as a finite penalty incorrectly leaves it available with small probability and invites the reward model to relearn network law.
This distinction has a direct random-utility interpretation. The feasible set defines which alternatives are available; reward or utility ranks the available alternatives; and a CMDP couples present choices through a trajectory-level budget. In route choice, avoiding a link may reveal a dislike of tolls, an access prohibition, insufficient vehicle range, or a cumulative risk limit. In activity choice, staying home may reveal preference, childcare or mobility restrictions, a binding time budget, or an unobserved mandatory episode. Treating every absence as low utility biases taste coefficients, substitution patterns, value-of-time estimates, and welfare. Relaxing a constraint expands an opportunity set; it is not generally equivalent to changing preferences over a fixed set.
Two inverse problems should be distinguished. \emph{Constrained IRL} recovers reward when relevant constraints are known and enforced during inference; prior constraints can reduce the reward search space and rule out infeasible generated trajectories \citep{ding2022xmen}. \emph{Inverse constrained reinforcement learning} (ICRL), also called inverse constraint learning, starts with a known or separately specified reward and infers a constraint-cost function from demonstrations of constraint-abiding behavior \citep{malik2021icrl}. Offline variants learn constraints without repeated interaction, which is particularly relevant to passive mobility traces \citep{quan2024icsdice}. The harder joint problem treats both reward and constraints as unknown. It is generally weakly identified: a route that is never chosen can be explained by low utility, nonavailability, an individual constraint, omitted information, or lack of data support.
Identification therefore requires evidence beyond chosen trajectories. Network regulations, transit schedules, accessibility audits, vehicle characteristics, household commitments, stated availability, and sensor-quality variables can fix known constraints. Multiple tasks or contexts can reveal a constraint that remains stable while destinations or rewards change; diverse multi-task demonstrations also reduce the tendency to learn an overly conservative rule that forbids every unobserved action \citep{kim2023sharedconstraints}. Bad or infeasible demonstrations provide negative evidence, while the incremental imitation of good trajectories and avoidance of bad ones can learn safe behavior without replacing a trajectory-level rule by a statewise surrogate \citep{hoang2024goodbad}. A defensible analysis reports which constraints were supplied, inferred, or merely assumed; their units and budgets; heterogeneity across travelers; and sensitivity of reward estimates to alternative constraint specifications.
\begin{longtable}{@{}p{0.17\textwidth}p{0.22\textwidth}p{0.25\textwidth}p{0.25\textwidth}@{}}
\caption{Constraint roles in sequential mobility choice.}\label{tab:constraints}\\
\toprule
\textbf{Constraint role} & \textbf{Choice-model meaning} & \textbf{Mobility examples} & \textbf{Recommended treatment} \\
\midrule
\endfirsthead
\toprule
\textbf{Constraint role} & \textbf{Choice-model meaning} & \textbf{Mobility examples} & \textbf{Recommended treatment} \\
\midrule
\endhead
\endfoot
\bottomrule
\endlastfoot
Known physical or legal & Alternative availability and feasible transition & Connectivity, closures, access rules, operating hours & Exact action mask, state transition, or schedule network \\
\specialrule{0.25pt}{0.12em}{0.12em}
Known cumulative & Resource coupling across decisions & Time, fare, battery, duty-hour, or transfer budget & Augmented resource state or CMDP constraint \\
\specialrule{0.25pt}{0.12em}{0.12em}
Latent traveler or household & Unobserved availability, capability, or commitment & Mobility limitation, childcare, vehicle access, personal safety threshold & ICRL or hierarchical latent constraint, anchored by auxiliary evidence \\
\specialrule{0.25pt}{0.12em}{0.12em}
Operator or social & Limit on a recommended policy, not a traveler preference & Safety, fleet emissions, service equity, system budget & Constrained offline RL plus acceptance and equilibrium models \\
\end{longtable}
The last row is prescriptive and must not be folded into a descriptive choice model. An emissions cap on a routing recommender is a planner objective even if travelers did not behave as though they faced it. Conversely, a mobility limitation inferred from traveler behavior should not be described as a weak preference. Recommendation systems therefore need separate traveler utility, acceptance behavior, operator costs, and system constraints; constraint satisfaction alone does not establish behavioral validity.
\subsection{Risk and model uncertainty}
Risk sensitivity and robustness also differ. Conditional value at risk or distributional returns describe preferences over uncertain travel outcomes; robust MDPs guard against uncertain transition models \citep{iyengar2005robust,nilim2005robust,chow2015risk}. The first is behavioral, the second epistemic. Conflating them causes cautious predictions to be interpreted as risk aversion when they may reflect model uncertainty.
\subsection{Data fusion across sources, scales, and sampling densities}
Travel data sources trade behavioral detail against coverage. Household surveys and diaries record purpose, attitudes, constraints, and socioeconomic variables but contain few people and days. GPS and smartphone panels provide detailed trajectories but come from a selective population and often lack trip purpose. Smart cards are nearly continuous for participating transit users but omit other modes and may not identify destinations. Cellular and location-based-service records cover large populations at coarser and device-dependent resolutions. Counts, probes, and traffic sensors observe aggregate network states without individual choice sets, while census, land-use, and time-use data provide population margins rather than trajectories. Data fusion should exploit these complementary strengths without pretending that the records have the same unit, support, or error \citep{huang2018fusion,graellsgarrido2023fusion,vo2026fusion}.
The four-layer model supplies a principled formulation. Let $z^{(m)}$ indicate inclusion in source $m$, $y^{(m)}$ denote its observed record, $\mathcal{H}_m$ its spatial, temporal, semantic, and aggregation operator, and $q_m$ source-quality variables such as sampling interval, positional accuracy, penetration rate, or survey weight. Conditional on a latent trajectory $\tau$,
\begin{equation}
\begin{aligned}
p(\{z^{(m)},y^{(m)}\}_{m=1}^{M}\mid c)
&=\int p_\theta(\tau\mid c)
\prod_{m=1}^{M}p_{\eta_m}\!\left(z^{(m)}\mid\tau,c,q_m\right)\\[-0.1em]
&\quad\times\left[p_{\psi_m}\!\left(y^{(m)}\mid
\mathcal{H}_m(\tau,c),q_m,z^{(m)}=1\right)\right]^{z^{(m)}}d\tau .
\end{aligned}
\label{eq:datafusion}
\end{equation}
The operator $\mathcal{H}_m$ is essential. It may map a link-by-link route to sparse GPS pings, aggregate trips to zone-hour smart-card flows, censor modes unavailable to a source, or translate a full-day schedule into the questions asked by a diary. The inclusion model $p_{\eta_m}$ separates measurement error from sampling-frame selection: smartphone ownership, transit use, survey response, provider coverage, and device settings can determine who appears in each source. Equation~\eqref{eq:datafusion} assumes conditional independence across sources after the latent trajectory and context are specified. When recruitment, platform membership, geography, or devices create residual dependence, a shared selection variable or joint source model is required. Fusion then estimates a common latent behavioral object while preserving a separate observation and selection process for every source. Mobile-phone and transportation records have already been combined to recover real-time mobility, and survey-mode relationships have been used to update mode shares from large passive datasets \citep{huang2018fusion,graellsgarrido2023fusion}. Recent city-scale work illustrates the complementary-density case particularly clearly: sparse household surveys provide cross-modal relationships, whereas near-census smart-card data anchor absolute time-dependent flows \citep{vo2026fusion}.
\begin{table}[h!]
\centering
\caption{Complementary roles and failure modes in behavioral data fusion.}
\label{tab:datafusion}
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize\small{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
\begin{tabular}{@{}p{0.19\textwidth}p{0.33\textwidth}p{0.36\textwidth}@{}}
\toprule
\textbf{Source} & \textbf{Information contributed} & \textbf{Source-specific limitation to model} \\
\midrule
Survey / activity diary & Purpose, preferences, constraints, household roles, reported alternatives & Sparse days and people, recall error, nonresponse, expansion weights \\
\specialrule{0.25pt}{0.12em}{0.12em}
GPS / smartphone panel & Fine route geometry, timing, repeated individual behavior & Selective participation, battery-driven gaps, map-match ambiguity, weak semantics \\
\specialrule{0.25pt}{0.12em}{0.12em}
Smart card / ticketing & Dense and longitudinal transit transactions & Mode-specific population, missing alighting or access legs, card sharing \\
\specialrule{0.25pt}{0.12em}{0.12em}
Cellular / location service & Broad population coverage and long observation windows & Coarse or variable resolution, provider selection, uncertain activity stops \\
\specialrule{0.25pt}{0.12em}{0.12em}
Traffic counts / probes & Aggregate link flow, speed, congestion, and incident response & No person identity, purpose, feasible choice set, or complete trajectory \\
\specialrule{0.25pt}{0.12em}{0.12em}
Census / time use / land use & Population margins, demographic structure, opportunity and schedule context & Aggregated, infrequent, and not directly linked to individual choices \\
\bottomrule
\end{tabular}
\end{table}
Four fusion regimes should be distinguished. \emph{Record-level fusion} links sources for the same person or trip and must propagate probabilistic linkage and map-matching uncertainty. \emph{Representation-level fusion} uses source-specific encoders followed by attention, gating, or product-of-experts combination; missing-source masks and reliability inputs are preferable to zero filling. \emph{Population-level fusion} combines unlinked samples through shared latent classes, hierarchical parameters, calibration margins, or moment restrictions rather than concatenating rows. \emph{Prediction-level fusion} ensembles source-specific posteriors and is useful when legal or privacy constraints prevent record linkage. These strategies answer different questions and should not be presented as interchangeable feature engineering \citep{graellsgarrido2023fusion,bai2024multiresolution}.
Different data volumes require explicit balancing. If every passive record contributes equally, millions of low-detail observations can overwhelm a small survey that identifies purpose or preferences. Source-specific likelihood scaling, survey expansion weights, estimated noise variances, hierarchical partial pooling, or constraints on population moments can preserve the information role of each source. Conversely, up-weighting a rich survey does not repair nonresponse or outdated sampling frames. Multi-resolution fusion must retain within-bin variability when moving from seconds to minutes or links to zones; simple averaging can bias structural relationships \citep{bai2024multiresolution}. The target spatial and temporal grid should therefore be chosen by the behavioral decision interval, not by whichever file is largest.
For route choice, a useful joint model combines probabilistic map matching from GPS, purpose and attitudes from a survey subsample, link speeds and incidents from probes, counts as aggregate flow constraints, and network attributes from maps. For activity choice, diaries supply semantic episode labels, smart cards and phone traces supply repeated timing and location, time-use surveys constrain durations, and census margins expand the synthetic population. In both cases the model should learn from unmatched as well as matched data by marginalizing latent trajectories or classes. It should not require an exact person-level join when only population-level alignment is defensible.
Fusion is credible only if evaluated source by source. Recommended tests include leave-one-source-out prediction, training on dense sources and validating against an independently sampled survey, calibration within resolution and demographic strata, sensitivity to linkage thresholds and likelihood weights, and posterior predictive checks at both individual and aggregate levels. Source disagreement is diagnostic rather than a nuisance: it may reveal selection bias, reporting error, changing behavior, or a misspecified observation operator. An LLM can help harmonize schemas and extract semantics from diaries, but it cannot establish that two records describe the same trip or remove sampling bias without external evidence.
\subsection{Privacy in fused mobility data}
Mobility traces are highly identifying, and fusion can increase disclosure risk because quasi-identifiers from different sources intersect. Aggregation, secure data enclaves, federated learning, privacy-preserving record linkage, and differential privacy can reduce exposure \citep{dwork2014privacy}. Privacy protection changes statistical information, especially for rare routes and minority activity patterns. A model should report linkage authority, privacy parameters, performance loss, and subgroup effects rather than treating privacy as a separate engineering appendix. When person-level linkage is unnecessary, population-level constraints or federated sufficient statistics should be preferred.
\subsection{Multi-agent choice, strategic interaction, and equilibrium}
Observed choices shape the environment through congestion, crowding, prices, household coordination, platform recommendations, and the actions of automated agents. A single-agent MDP with fixed transitions can absorb this feedback into its reward or transition model. A genuine multi-agent formulation instead specifies a stochastic or Markov game. For decision maker $i$ of type $k_i$, a useful behavioral payoff is
\begin{equation}
r_i(s,\mathbf{a},k_i)=
\boldsymbol{\beta}_{k_i}^{\top}\mathbf{x}_i(s,a_i)
+h_{k_i}(a_i,m_{-i},s)+\epsilon_i(a_i),
\label{eq:markovgame}
\end{equation}
where $\mathbf{a}=(a_i,a_{-i})$ is the joint action and $m_{-i}$ summarizes relevant choices of others. The first term retains interpretable type-specific tastes; the interaction term represents congestion, competition, coordination, norms, or information spillovers. Each policy is a best response to beliefs about other policies, and the joint transition depends on the joint action. Dynamic discrete-game estimation already makes clear that multiple equilibria and repeated solution of best responses are central inferential difficulties \citep{aguirregabiria2007games}; multi-agent IRL adds the problem that rewards and the assumed solution concept must be recovered together \citep{yu2019maairl}.
\textbf{Not every heterogeneous population is a game.} If commuters, commercial drivers, tourists, cyclists, and travelers with mobility constraints have different $\boldsymbol{\beta}_{k}$ but do not materially affect one another after conditioning on network conditions, a latent-class or hierarchical choice model is sufficient. Strategic modeling is warranted when an agent anticipates that its action changes others' actions or future opportunities. Examples include drivers responding to congestion created by other drivers; households coordinating vehicles, escort trips, and activity times; ride-hailing drivers competing for demand; travelers reacting to personalized recommendations; electric vehicles competing for chargers; and autonomous fleets interacting with human drivers.
\textbf{The appropriate multi-agent architecture depends on interaction scale.} Atomic Markov games preserve named decision makers and are suitable for households, fleets, platforms, or a small number of strategically important actors. Centralized training with decentralized execution can condition a critic on global information while retaining local policies; value factorization such as QMIX makes cooperative joint action-values more scalable \citep{lowe2017maddpg,rashid2018qmix}. For thousands of individually negligible travelers, a mean-field model replaces the full joint action by a population or neighborhood distribution, reducing an exponential interaction space to a representative-agent problem coupled to aggregate dynamics \citep{yang2018meanfield}. Transportation applications already connect mean-field games to heterogeneous departure-time choice and connect Markov routing games to dynamic user equilibrium \citep{ameli2022departure,shou2022markovrouting}.
\begin{table}[t]
\centering
\caption{Multi-agent formulations for heterogeneous and strategic travel choice.}
\label{tab:multiagent}
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize
\@setfontsize\small{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
{9.8pt}{11.3pt}
\abovedisplayskip 7pt plus 2pt minus 4pt
\abovedisplayshortskip 0pt plus 2pt
\belowdisplayskip \abovedisplayskip
\belowdisplayshortskip 3pt plus 2pt minus 2pt
\begin{tabular}{@{}p{0.20\textwidth}p{0.30\textwidth}p{0.38\textwidth}@{}}
\toprule
\textbf{Interaction structure} & \textbf{Choice-model application} & \textbf{Learning formulation and behavioral requirement} \\
\midrule
Heterogeneous but conditionally independent & Traveler classes with distinct time, cost, safety, or accessibility preferences & Hierarchical or latent-class policy; no game claim unless choices affect beliefs or payoffs. \\
\specialrule{0.25pt}{0.12em}{0.12em}
Anonymous congestion / crowding & Route, departure-time, destination, platform, or transit choices in large populations & Mean-field RL/IRL or differentiable assignment; solve the policy--flow fixed point by type. \\
\specialrule{0.25pt}{0.12em}{0.12em}
Cooperative small group & Household schedules, shared vehicles, fleet coordination, joint ride matching & Dec-POMDP or cooperative Markov game; centralized training, decentralized choice, and explicit credit assignment. \\
\specialrule{0.25pt}{0.12em}{0.12em}
Mixed or competitive agents & Human drivers, autonomous fleets, ride-hailing platforms, carriers, or charging users & General-sum Markov game or multi-agent IRL; state the Nash, correlated, bounded-rational, or learning-dynamic solution concept. \\
\specialrule{0.25pt}{0.12em}{0.12em}
Leader--follower system & Tolling, information provision, recommendation, pricing, or capacity control & Stackelberg/bilevel model: operator commits, heterogeneous travelers adapt, and the network reaches a new equilibrium. \\
\bottomrule
\end{tabular}
\end{table}
\textbf{DICE provides a scalable but limited multi-agent bridge.} The joint occupancy $d(\mathbf{s},\mathbf{a})$ satisfies a conservation law, so multi-agent imitation can in principle match a demonstrated joint flow. Direct estimation is exponential in the number of decision makers. MisoDICE addresses cooperative mixed-quality demonstrations by decomposing joint value terms into local components and using a mixing network to maintain global--local consistency \citep{bui2025misodice}. In choice modeling, a corresponding design could learn type-specific local route or activity policies while a monotone mixer represents a household or fleet objective. For anonymous congestion, the more natural decomposition is by traveler type and aggregate flow rather than by named individual. For competitive or general-sum travel, however, a cooperative mixer is inappropriate: each type or actor requires its own reward and strategic response, and equilibrium selection cannot be replaced by joint occupancy matching.
\textbf{Estimation should separate preference, belief, and equilibrium.} A practical workflow is to estimate type membership and local choice shocks; infer reward or policy components conditional on public and private information; learn or calibrate the congestion and interaction transition; and solve for a specified equilibrium or adaptive learning process. Common incidents and information shocks must not be mistaken for strategic correlation. Panel data are especially valuable because repeated decisions reveal learning rules, belief updating, reciprocity, and adaptation. Counterfactuals should recompute all policies and network states after a toll, recommendation, capacity, or fleet-policy change rather than replaying policies fitted in the old regime.
The choice between descriptive and prescriptive MARL must also remain explicit. A learned best response may predict adaptive travelers, whereas a system-optimal controller may deliberately coordinate agents away from their observed behavior. Route-choice experiments with Markov routing games link learned policies to dynamic user equilibrium, while recent mixed human--autonomous routing simulators show why different objectives and adaptation speeds must be represented explicitly rather than treating every road user as one homogeneous optimizer \citep{shou2022markovrouting,akman2025routerl}. Evaluation should report individual likelihood and calibration by type, joint trajectory fidelity, congestion and welfare outcomes, equilibrium residuals, robustness to population composition, and distributional impacts across traveler groups.
This is essential for policy analysis. A road-capacity change alters travel time, which changes routes, which changes travel time again. An activity policy can shift both time and location demand. Learned preferences should be tested for stability across demand regimes, and counterfactuals should solve the behavioral-network fixed point rather than replay a policy in an unchanged environment.
\subsection{Large language models: semantic layer, behavioral agent, and scientific tool}
LLMs broaden the interface between unstructured urban information and structured choice models. Their value is clearest in three roles: transforming data into model-ready variables, supplying priors or representations when labeled choice data are scarce, and orchestrating tools that remain responsible for estimation and simulation. They should not be treated as drop-in replacements for a probability model merely because they can return a plausible option and explanation.
\textbf{Data processing and representation.} Travel surveys, incident reports, policy documents, map descriptions, images, and open-ended trip diaries contain context that is expensive to code manually. An LLM can extract activity purpose, constraints, perceived safety, disruption type, or household coordination cues into a schema; reconcile variable definitions across cities; propose map-matching candidates; and generate code for quality checks. The output should be typed, source-linked, and audited against a labeled sample. Personally identifying text should be minimized before model access, and generated labels should enter Equation~\eqref{eq:fourlayer} with measured error rather than as ground truth.
\textbf{Choice prediction and behavioral priors.} A language model can express a choice instance as natural language, use retrieved examples, and predict an action in zero- or few-shot settings. Budget-matched experiments on mode and trip-purpose tasks find that LLM approaches can be competitive when labels are scarce, while conventional supervised or tabular methods become stronger as labeled data grow \citep{mo2026llmtravel}. More broadly, a foundation model fine-tuned on millions of experimental choices has generalized across participants, cover stories, task modifications, and new cognitive domains \citep{binz2025centaur}. These results motivate pretraining across mobility datasets, but generated rationales are not identified utilities. A sound design uses the LLM as an encoder, prior, or residual, followed by a calibrated choice head with explicit alternatives, scale, and monotonicity tests.
\textbf{Synthetic populations and demonstrations.} LLMs can construct diverse traveler profiles, complete sparse diaries, write rare disruption scenarios, and propose high-level daily schedules. Synthetic data are useful for stress testing, privacy-preserving prototyping, and coverage of counterfactual contexts that are absent from historical traces. They are not evidence about population shares or elasticities unless calibrated to observed margins and validated on held-out human behavior. The lesson from AlphaGeometry is relevant: large-scale synthetic generation overcame scarce human proofs because a symbolic engine supplied valid problems and verified deductions \citep{trinh2024alphageometry}. Transportation synthesis likewise needs a generator constrained by census totals, time-use distributions, network feasibility, and behavioral moment matching.
\textbf{Microscopic and city-wide simulation.} An LLM agent can maintain memory, revise a plan after incidents, interact with information services, and produce natural-language reasons. LLMTraveler shows that memory-equipped agents can reproduce some day-to-day route-switching patterns in congestion games \citep{wang2025llmtraveler}; GATSim integrates memory, planning, reaction, and reflection in a mobility simulator and evaluates both individual and aggregate patterns on a stylized network \citep{liu2026gatsim}. Calling a large model for every agent at every time step, however, is expensive and can produce unstable aggregate behavior. A scalable architecture invokes the LLM sparsely to generate personas, activity templates, beliefs, or policy updates; distills those outputs into fast choice or IL policies; executes movement in a network microsimulator; and recalibrates aggregate flows. Recent city-scale work follows this division of labor by combining LLM-assisted semantic scheduling with cheaper learned alignment models rather than prompting at every decision \citep{santos2026citybehavex}.
\textbf{Model development and scientific discovery.} Recent mathematics and science systems show that the important advance is not free-form generation alone, but generation coupled to search, verification, and tools. AlphaGeometry and AlphaProof combine learned proposal mechanisms with symbolic deduction, formal proof checking, synthetic curricula, and reinforcement learning to solve olympiad-level problems \citep{trinh2024alphageometry,hubert2025alphaproof}. FunSearch pairs an LLM with an executable evaluator and discovered improved constructions and algorithms \citep{romeraparedes2024funsearch}. Coscientist connects LLM planning to literature search, code execution, laboratory interfaces, and experimental feedback \citep{boiko2023coscientist}. In choice modeling, an analogous research agent could propose utility features, correlation structures, state augmentations, missing-data models, or solver heuristics; compile and run estimators; search for counterexamples; and design informative interventions. Statistical tests, exact network constraints, held-out likelihood, formal proofs, and human review must serve as the evaluators. The credible promise is accelerated, auditable model discovery--not autonomous acceptance of an LLM's behavioral story.
Across these roles, four risks remain central: prompts and model versions can change results; pretrained corpora encode unknown selection and stereotypes; natural-language explanations can be persuasive but unfaithful; and API cost and latency can make replication difficult. Studies should archive prompts and model versions, report stochastic variation and token/energy cost, compare against non-LLM baselines under equal information, and evaluate subgroup calibration and intervention response.
\subsection{Risks of importing ML value and priorities for future research}
The same capabilities that create value can introduce new failure modes. Accelerator scale can encourage unnecessarily large models, concentrate reproducibility in well-resourced groups, and obscure energy or latency costs. Differentiable programming can make an entire pipeline optimizable while silently differentiating through an approximate solver, a misspecified observation model, or an equilibrium that is not unique. Minibatching, subgraph sampling, and mixed precision can bias a normalized likelihood or destabilize small utility differences if their corrections and tolerances are hidden. Pretraining, data fusion, and synthetic generation can improve apparent sample efficiency while importing platform selection, geographic stereotypes, privacy risks, or information leakage. Finally, benchmark optimization can reward a locally accurate black box whose utility scale, substitution, constraints, and response to intervention are behaviorally wrong.
These risks define a concrete research agenda rather than a reason to reject ML. Priority directions include accelerator-native sparse dynamic programming and equilibrium layers with certified residuals; differentiable choice estimators whose approximation error is propagated into inference; identification-aware representations that separate taste, scale, availability, observation, and congestion; transfer methods with explicit invariance tests and negative-transfer diagnostics; and sample-efficiency measures based on independent travelers, environments, and interventions. Benchmarks should pair predictive metrics with behavioral falsification and natural or designed policy shifts, while software reports seeds, hardware, precision, energy, sampling corrections, and solver tolerances. The long-run contribution of ML will be strongest when computational scale expands the set of estimable behavioral models without weakening the meaning of the quantities those models are meant to recover.
\section{A Behaviorally Disciplined Hybrid Architecture}
Sections~4--6 establish three premises for model design. First, route and activity decisions can share a sequential state--action representation, but formal relationships among recursive choice, dynamic discrete choice, and maximum-entropy control do not make their behavioral interpretations automatically equivalent \citep{mai2020relation}. Second, utility, reward, policy, occupancy, constraint, and trajectory distribution are different learned objects, so the estimand must follow the scientific question. Third, ML adds real value through scalable computation, flexible representations, incomplete-data learning, transfer, and simulation, but these gains do not by themselves establish preference recovery or counterfactual validity.
This section turns those premises into the paper's modeling prescription. The central message is to retain a behaviorally explicit backbone--decision state, feasible alternatives, transition and resource constraints, utility or reward normalization, observation and selection processes, and equilibrium feedback--and attach ML components only to declared bottlenecks. A graph encoder may enrich network context, a sequence model may supply memory, and an amortized solver may reduce computation, but none should silently redefine availability, preference scale, or the policy experiment. This division of labor follows the logic of hybrid choice models: learned representations can improve specification while the probabilistic and structural core preserves interpretable quantities and testable restrictions \citep{train2009discrete,sifringer2020representation,han2022tastenet}.
The result is not one universal algorithm. It is a use-case-driven architecture in which every component states what it learns, which data and constraints it uses, which approximation it introduces, and which behavioral claim it is permitted to support. The strength of the final conclusion is limited by the weakest relevant layer: a well-calibrated policy cannot repair a misspecified observation process, and an identified utility cannot support a congestion counterfactual without resolving system feedback. Applied to Section~\ref{sec:runningexample}, the architecture must preserve the school and network constraints, model each data source, select the learned object from Table~\ref{tab:runningexample}, and recompute congestion under the toll. The steps below operationalize this principle, and Table~\ref{tab:decision} maps scientific objectives to appropriate structural cores and ML augmentations.
\begin{enumerate}
\item \textbf{Construct a decision state, not just a feature vector.} For routes, include current link or node, destination, time, information, and any required memory. For activities, include location, time, remaining resources, commitments, and household context. Perform Markov sufficiency tests by checking whether longer history improves residual prediction.
\item \textbf{Separate feasibility, cumulative constraints, and preferences.} Use graph adjacency, transit schedules, time windows, access rules, and resource transitions to define $\mathcal{A}(s,c)$ and $P$. Represent known cumulative limits with resource states or CMDP costs. Infer a latent constraint only when it cannot be supplied and when the data contain evidence that distinguishes it from low utility. Do not ask a penalty term to learn a rule already known.
\item \textbf{Specify a structural reward backbone.} Include travel time, cost, reliability, activity duration, schedule delay, safety, comfort, and other policy-relevant terms. Fix one scale, state sign or monotonicity constraints, and declare which coefficients support welfare interpretation.
\item \textbf{Add a bounded learned residual or representation.} Use a graph encoder for spatial context, a transformer for history, or a multimodal encoder for imagery and text. Regularize the residual, test whether structural coefficients remain stable, and quantify how much predictive gain it contributes.
\item \textbf{Use LLMs at semantic boundaries.} Let an LLM parse unstructured inputs, retrieve analogous cases, generate high-level plans, or propose model specifications. Require schema validation, provenance, and an exact choice, planning, or simulation layer before outputs affect trajectories or policy conclusions.
\item \textbf{Choose the learning objective by use case.} Use likelihood-based recursive choice for explanation; IRL when reward transfer matters; occupancy matching for realistic simulation; quality-aware IL when demonstrations are heterogeneous; behavior cloning for supported prediction; and offline RL only for a separate prescriptive policy.
\item \textbf{Integrate observation and selection models.} Marginalize uncertain maps, missing segments, and unobserved activity episodes; model which travelers and trips enter each source; and retain multiple latent paths or schedules when the data do not identify one.
\item \textbf{Solve behavior and system feedback jointly when needed.} Feed predicted choices into congestion, crowding, or accessibility and iterate to a stable system outcome. Distinguish individual utility from the social objective.
\item \textbf{Audit with behavioral falsification.} Test signs, elasticities, substitution under link duplication, response to tolls and closures, destination reachability, schedule feasibility, calibration, and invariance across environments.
\end{enumerate}
\begin{longtable}{@{}p{0.20\textwidth}p{0.31\textwidth}p{0.37\textwidth}@{}}
\caption{Recommended model choice by scientific objective.}\label{tab:decision}\\
\toprule
\textbf{Objective} & \textbf{Preferred core} & \textbf{ML augmentation} \\
\midrule
\endfirsthead
\toprule
\textbf{Objective} & \textbf{Preferred core} & \textbf{ML augmentation} \\
\midrule
\endhead
\endfoot
\bottomrule
\endlastfoot
Behavioral explanation & Recursive or dynamic discrete choice & Structured residuals, hierarchical heterogeneity, latent observation model \\
\specialrule{0.25pt}{0.12em}{0.12em}
New-destination transfer & IRL plus exact planning & Goal-conditioned graph reward, amortized value function \\
\specialrule{0.25pt}{0.12em}{0.12em}
Population simulation & Occupancy matching or probabilistic choice & Graph/sequence policy with feasibility masks and calibration \\
\specialrule{0.25pt}{0.12em}{0.12em}
Missing-trajectory recovery & Latent path/schedule likelihood & Generative proposal or observation encoder with exact marginal checks \\
\specialrule{0.25pt}{0.12em}{0.12em}
Recommendation & Preference model plus constrained offline RL & Conservative value learning, acceptance model, uncertainty \\
\specialrule{0.25pt}{0.12em}{0.12em}
Policy counterfactual & Structural reward plus equilibrium & Differentiable solver, domain adaptation, sensitivity bounds \\
\end{longtable}
This architecture clarifies when simpler methods win. If the network is modest, behavior is well described by observed attributes, and policy interpretation is central, recursive logit may dominate a deep model. If only next-link prediction is required in a stable environment, masked behavior cloning may be sufficient. Complexity is justified by a demonstrated gain in scale, transfer, data integration, or specification robustness--not by model class alone.
\section{Evaluation and Benchmarking}
Evaluation must match the intended claim. A model proposed for supported next-step prediction needs held-out probabilistic accuracy and calibration; a model proposed for behavioral explanation also needs structural falsification; a transferable reward needs domain-shift tests; and a model proposed for policy analysis needs intervention and equilibrium evidence. Sections~5--7 distinguish these estimands and assign ML components different roles, so they cannot be ranked responsibly on one accuracy metric. In the running commute, next-link accuracy, recovery of time and cost trade-offs, complete-day feasibility, and post-toll flow response are four different tests. The purpose of this section is to translate that claim--evidence principle into a benchmark design and a minimum reporting suite.
The evidence forms a hierarchy rather than a menu of interchangeable scores. Predictive evaluation asks whether probabilities and trajectories match new observations. Sequential evaluation asks whether complete routes and schedules are coherent and feasible. Behavioral evaluation asks whether signs, elasticities, substitution, reward normalization, and inferred constraints remain plausible. Recovery tests use known synthetic truth; transfer tests change people, destinations, networks, or time; and counterfactual tests change the policy environment itself. System evaluation then measures congestion, accessibility, emissions, welfare, equity, privacy, and computational cost. A model may succeed at one layer and fail at the next, so every paper should report all layers required by its strongest substantive claim.
Random trip-level splits are inadequate because the same traveler, destination, and nearby links can appear in training and test data. A benchmark should instead contain four levels: (i) synthetic networks with known rewards, constraints, and observation noise; (ii) a public road or transit network with map-matched trajectories; (iii) an activity-diary or mobility dataset with time, purpose, and personal context; and (iv) an intervention or domain-shift setting involving closures, tolls, incidents, service changes, or a new city \citep{arora2021survey,levine2020offline,zhao2023deepirl}.
\begin{longtable}{@{}p{0.18\textwidth}p{0.34\textwidth}p{0.34\textwidth}@{}}
\caption{Minimum evaluation suite for learned travel behavior.}\label{tab:benchmark}\\
\toprule
\textbf{Dimension} & \textbf{Measures} & \textbf{Failure detected} \\
\midrule
\endfirsthead
\toprule
\textbf{Dimension} & \textbf{Measures} & \textbf{Failure detected} \\
\midrule
\endhead
\endfoot
\bottomrule
\endlastfoot
Density quality & Negative log likelihood, calibration, top-$k$ coverage & Good single path but poor distribution \\
\specialrule{0.25pt}{0.12em}{0.12em}
Trajectory fidelity & Edge overlap, edit distance, destination completion, schedule statistics & Locally accurate but incoherent sequence \\
\specialrule{0.25pt}{0.12em}{0.12em}
Behavioral validity & Signs, values of time, elasticities, substitution and monotonicity tests & Predictive shortcut or implausible utility \\
\specialrule{0.25pt}{0.12em}{0.12em}
Recovery & Reward ranking, parameters, constraints, shaping-invariant policy tests & Non-identification on synthetic truth \\
\specialrule{0.25pt}{0.12em}{0.12em}
Transfer & Held-out destination, traveler, time, region, network, and intervention & Memorization and environment dependence \\
\specialrule{0.25pt}{0.12em}{0.12em}
Sample efficiency & Learning curves over independent travelers, days, labels, queries, and planner calls & Apparent efficiency from repeated transitions, leakage, or unequal data budgets \\
\specialrule{0.25pt}{0.12em}{0.12em}
Counterfactual validity & Pre/post-intervention likelihood and flow change, elasticity and welfare stability, equilibrium residual & Good status-quo fit but incorrect response to policy change \\
\specialrule{0.25pt}{0.12em}{0.12em}
Online / continual validity & Prequential loss, adaptation delay, forgetting, query burden, intervention exposure & Updating that is unsafe, unstable, or confused with taste change \\
\specialrule{0.25pt}{0.12em}{0.12em}
Robustness & GPS noise, missingness, ambiguous matches, suboptimal demonstrations & Brittle preprocessing or expert assumption \\
\specialrule{0.25pt}{0.12em}{0.12em}
Fusion validity & Leave-one-source-out tests, source weights, linkage sensitivity, cross-resolution calibration & Dominance by a large weak source or incompatible sampling frames \\
\specialrule{0.25pt}{0.12em}{0.12em}
Feasibility / safety & Invalid moves, resource violations, CVaR, worst-case outcomes & Operationally unusable behavior \\
\specialrule{0.25pt}{0.12em}{0.12em}
System impact & Congestion, emissions, welfare, accessibility, equity & Individually accurate but harmful aggregate outcome \\
\specialrule{0.25pt}{0.12em}{0.12em}
Computation & Runtime, memory, planner calls, tolerance, precomputation, latency, energy & Unverifiable scalability claim \\
\specialrule{0.25pt}{0.12em}{0.12em}
Reproducibility & Repeated seeds, uncertainty intervals, tuning budget, software, hardware, and precision & Single-run improvement or unequal tuning and compute \\
\specialrule{0.25pt}{0.12em}{0.12em}
Privacy & Privacy budget, re-identification tests, utility loss by subgroup & Hidden disclosure or disparate degradation \\
\specialrule{0.25pt}{0.12em}{0.12em}
LLM reliability & Prompt/model version, repeated-run variance, schema validity, rationale faithfulness, token cost & Non-reproducible semantic or agentic behavior \\
\end{longtable}
Comparisons should control the information set, feasible action space, target-domain sample, and tuning budget. A neural model given destination, map imagery, and personal history should not be compared to a linear choice model lacking those inputs and described as intrinsically superior. Strong baselines include shortest path, path-based logit with sampling correction, recursive logit, a flexible supervised policy, and the proposed structured hybrid with ablations. Learning curves should use independent travelers or days rather than treating correlated link transitions as new samples. Stochastic pipelines should report repeated seeds and uncertainty intervals, while statistical uncertainty should include traveler or origin--destination clustering where appropriate \citep{henderson2018matters}.
Counterfactual evaluation deserves separate status. A model fitted before an intervention should predict route, mode, timing, or activity changes after the intervention without refitting. When no natural intervention is available, synthetic and semi-synthetic tests can vary network attributes while keeping known preferences fixed. Sensitivity analyses should show how results change with reward normalization, transition uncertainty, missing-data assumptions, and equilibrium specification.
\section{Research Agenda}
The central research gap is not a shortage of powerful predictors. It is the absence of reliable conditions under which a learned mobility sequence can be interpreted, transferred, and used for intervention. Sections~4--8 show that progress must occur jointly along three axes: \emph{identification}, so preferences, constraints, observations, and equilibrium effects are not conflated; \emph{generalization}, so learned objects remain valid across travelers, destinations, networks, and policies; and \emph{operational credibility}, so computation, data fusion, simulation, and evaluation preserve feasibility and expose approximation error. The running commute exposes all three gaps: toll avoidance is not identified without constraints and observation models, transfer requires stability beyond the observed destination and regime, and policy evaluation requires recomputing congestion. An accuracy gain on the current environment is useful, but it does not resolve any of these axes automatically.
The agenda below is therefore organized by the questions future work must answer. What behavioral object is recoverable, and under which assumptions? Where should it remain invariant, from routes to full activity schedules and from individual decisions to interacting populations? How should algorithms, LLM tools, fused data, and benchmarks make those claims scalable and falsifiable? Each direction names a concrete methodological deliverable and the failure it is intended to prevent.
\subsection{Establish equivalence and identification maps}
The deliverable should be a map, not another informal analogy: precise results stating when recursive logit, maximum-causal-entropy policies, dynamic logit, global path distributions, and regularized flows coincide, and which normalizations make their parameters comparable. Cycles, discounting, heterogeneous scale, stochastic travel time, non-additive attributes, and partial observation should be treated explicitly. A companion identification map should state what remains recoverable under reward shaping, omitted or learned constraints, learned states, and endogenous network conditions \citep{mai2020relation,ng1999shaping}. Benchmarks with known reward and constraint ground truth should then score preference recovery, feasibility recovery, and policy reproduction separately.
\subsection{Make activity choice a first-class sequential-learning problem}
Route choice is a useful test bed because its graph is explicit; activity choice is the harder test of whether the sequential framework is genuinely general. The needed deliverable is an open time--space activity environment with realistic durations, household coordination, accessibility, and resource constraints, accompanied by survey and passive-data observation models. Semi-Markov and multi-agent formulations should distinguish duration and coordination, while hierarchical rewards separate needs, preferences, and schedule pressure \citep{song2024stateirl,liu2025contextirl}. Success requires complete-day feasibility, population moments, and response to schedule or accessibility interventions, not only next-activity accuracy.
\subsection{Test transfer and adaptation layer by layer}
The deliverable should be a layer-specific transfer matrix rather than one aggregate accuracy gain. Studies should vary destinations, networks, populations, time periods, and interventions separately, then test whether rewards, policies, representations, observation models, or equilibrium parameters remain invariant \citep{zhao2023deepirl,zhang2024metairl}. Continual-learning studies should distinguish gradual drift from abrupt policy or network change, preserve historical model versions, and measure catastrophic forgetting. Opt-in active queries can target weakly identified trade-offs, but their value must be reported against respondent burden, safety, strategic response, and interference across travelers.
\subsection{Develop solver-aware representation learning}
The computational deliverable is a solver-aware learned component with a declared error contract. Graph and sequence encoders should be trained with network structure rather than attached after the fact. Promising designs include differentiable sparse Bellman solvers, residual value networks with certified error checks, destination-factorized rewards, and learned graph compression that preserves choice probabilities \citep{barnes2023maps,mai2023incomplete}. Large-scale claims should report numerical error, memory, planner calls, hardware, and energy together with speed; acceleration is not credible if it silently changes the choice probabilities being estimated.
\subsection{Separate preferences, constraints, beliefs, and observations}
The inferential deliverable is a model and data protocol that can distinguish dislike, nonavailability, limited information, observation error, and strategic response. Avoidance behavior alone does not reveal which layer caused it. Joint IRL--ICRL models should combine trajectories with road regulations, accessibility data, stated choices, incident information, and sensor-quality measures, and exploit repeated tasks in which rewards change while constraints remain stable \citep{malik2021icrl,quan2024icsdice}. Partial identification and sensitivity bounds may be more honest than one recovered reward--constraint surface. This distinction is also essential for equity: restricted access should not be reported as a weak preference for an inaccessible opportunity.
\subsection{Connect individual learning to equilibrium and policy}
The policy deliverable is an end-to-end counterfactual that recomputes behavior and network conditions after an intervention. Learned policies must therefore be embedded in congestion, crowding, land-use, platform, and recommendation feedback. Differentiable assignment and equilibrium layers can make system response part of estimation, but they also introduce endogeneity, equilibrium-selection, and stability questions \citep{ameli2022departure,shou2022markovrouting}. Evaluation should distinguish traveler utility, operator objectives, acceptance, and social welfare, and should show how heterogeneous groups adapt rather than replaying policies learned under the old regime.
\subsection{Develop verification-first LLM workflows}
The relevant deliverable is a verification-first workflow, not a collection of plausible demonstrations. Priorities include audited extraction of behavioral variables from unstructured data; retrieval-grounded choice prediction with calibrated probabilities; synthetic populations constrained to demographic and time-use margins; distillation of high-level LLM plans into fast microsimulation policies; and research agents whose proposed models are screened by identification analysis, exact solvers, falsification tests, and interventions \citep{trinh2024alphageometry,wang2025llmtraveler}. City-scale experiments should publish micro-level behavioral diagnostics, aggregate flow errors, subgroup impacts, LLM invocation rate, and total computational cost.
\subsection{Build shared benchmarks and reporting standards}
The infrastructure deliverable is a common experimental contract. A useful benchmark should expose the same network, contexts, demonstrations, observation uncertainty, and intervention splits to all model families. Software should provide interoperable likelihood, planning, occupancy, map-matching, constraint, and equilibrium interfaces. Each study should publish data splits where privacy permits, code, seeds, hyperparameter budgets, solver tolerances, compute, normalizations, and failed falsification tests \citep{henderson2018matters}. This infrastructure would make behavioral and operational value, rather than architectural novelty or unequal tuning, the unit of progress.
\subsection{Make source-aware fusion a behavioral research problem}
The data deliverable is a source-aware benchmark that pairs a small, semantically rich survey with larger GPS, smart-card, cellular, count, and network datasets that differ in resolution and sampling frame. The latent behavior and each source's observation, aggregation, missingness, and selection process should be scored separately. Research should establish when linked microdata are necessary, when population moments suffice, how fusion changes preference and reward identification, and how uncertainty propagates into route assignment, activity microsimulation, and policy counterfactuals \citep{graellsgarrido2023fusion,bai2024multiresolution,vo2026fusion}. The governing standard should be improvement on an independently sampled target population, not fit to the largest available source.
A practical sequencing follows from these deliverables. Near-term work can establish shared tasks, exact or certified baselines, source-aware observation models, and reporting standards. The next stage can test structured representations, transfer, and activity models against those controls. City-wide multi-agent, generative, and LLM systems should then be evaluated on intervention response and aggregate feedback rather than scaled before their behavioral components are validated. Across all stages, the decisive result is not that an ML model is larger or more accurate in-sample, but that it expands computational or empirical reach while preserving identifiable meaning and improving decisions in a changed mobility environment.
\section{Discussion}
The central connection developed in this review is simple but consequential. Sequential mobility choice is inverse decision making over purposeful, feasible trajectories. Route choice instantiates it on a spatial graph, whereas activity choice uses an augmented time-space-resource graph with durations and endogenous goals. They are members of one model class, not the same empirical problem. Recursive choice specifies a soft policy from utility; IRL asks which reward makes demonstrations probable; and IL asks which policy or occupancy reproduces them. The same observed sequence can be modeled at each of these levels, but the resulting models answer different questions.
This view changes how machine learning should enter transportation behavior. The highest-value contribution is not unconstrained function approximation. It is scalable representation and inference around strong structure: graphs define feasible movement; time and resource states define feasible schedules; structural reward terms retain policy meaning; source-aware observation operators connect sparse semantic and dense passive evidence without erasing their sampling differences; and network solvers translate learned values into coherent behavior. Deep components then represent context, interactions, heterogeneity, or long memory that classical specifications cannot easily capture. LLMs extend this stack at its semantic boundaries--turning text and multimodal context into structured variables, proposing high-level plans, and coordinating scientific tools--while calibrated estimators and simulators retain responsibility for behavioral and system claims.
The view also clarifies the limits of prediction. A model can predict the next link while failing to represent substitution; reproduce observed schedules while learning no transferable preference; match aggregate flow while hiding subgroup disparities; or recommend a system-optimal path that travelers reject. No single accuracy number resolves these failures. The target use--description, explanation, simulation, transfer, or prescription--must determine the learned object and evaluation design.
Finally, route and activity choice can contribute back to machine learning. Transportation networks offer explicit feasibility, abundant offline demonstrations of uneven quality, measurable domain shifts, and policy interventions. They are unusually rich laboratories for studying reward identification, constrained imitation, partial observation, LLM-agent validation, and multi-agent feedback. Progress here can inform a broader science of learning purposeful human behavior from observational sequences.
\section{Conclusion}
Sequential mobility choice provides a general language for route choice, activity choice, dynamic discrete choice, inverse reinforcement learning, and imitation learning as related views of constrained trajectory generation. Its route and activity instances share the soft Bellman bridge, episodic occupancy-flow conservation, and the inverse problem of recovering objectives from choices. They remain different in state, horizon, duration, coordination, observation, estimand, interpretation of randomness, identification assumptions, and intended counterfactual.
The resulting design principle is clear: keep feasibility, resource constraints, source-specific observation and sampling processes, and behavioral normalizations explicit; fuse evidence through a common latent trajectory or population model rather than naive record concatenation; use machine learning to enrich rewards, representations, heterogeneity, latent trajectories, and scalable value computation; use LLMs to connect unstructured context and high-level reasoning to verified model components; and evaluate models on transfer and intervention response as well as prediction. Such hybrid models can be more scalable than conventional route and activity models and more behaviorally valuable than black-box sequence predictors. Their goal is not merely to forecast where people go next, but to explain which trade-offs and constraints generate travel, how those mechanisms change across environments, and when learned behavior can support responsible transport policy.
\section*{Statement on the Use of Artificial Intelligence}
\addcontentsline{toc}{section}{Statement on the Use of Artificial Intelligence}
AI-assisted tools were used to identify potentially relevant literature and to improve the clarity, grammar, and consistency of the writing. The paper's central idea, analytical framework, review workflow, synthesis, methodological judgments, and conclusions were developed by the author, drawing on direct research experience in both choice modeling and inverse reinforcement and imitation learning. AI tools did not determine the paper's claims or replace scholarly judgment. All AI-assisted material, references, technical statements, and revisions were carefully reviewed and checked for accuracy, internal consistency, and alignment with the author's intended argument. The author takes full responsibility for the content of the paper.
\clearpage
\phantomsection
\addcontentsline{toc}{section}{References}
\bibliographystyle{plainnat}
\bibliography{references}