Lorenzo Muñoz, Stephane Hess, Thomas O. Hancock, Georges Sfeir
arXiv 17 Sep 2026 · Econometrics
arXiv:2609.20655 · PDF · Extracted main text
Machine learning (ML) techniques are increasingly drawing interest in the choice modelling (CM) field. The focus has primarily been on comparing the performance of these contrasting approaches or on improving behavioural insights for ML techniques, rather than translating ideas from one field into the other. In the present paper, we specifically focus on knowledge transfer from ML into CM in the context of model performance evaluation. In CM, model performance is typically evaluated using log-likelihood and related indicators, which are aggregate fit metrics that focus on overall fit. Conversely, in ML, the focus is on alternative-level misclassifications and correct classifications, which provide a more nuanced view of the results. To bridge these approaches, we explore the use of a probabilistic version of the confusion matrix, which reports the average probability of the model predicting each alternative, conditional on which alternative was observed to be chosen, across all choice tasks. This enables the computation of probabilistic ML metrics for both classic choice models and ML algorithms. We analyse model performance jointly in terms of overall fit and alternative-level predictions. Our findings demonstrate that models with similar log-likelihood can exhibit substantially different confusion matrices, revealing different probability patterns that aggregate metrics cannot capture. This framework identifies where models systematically `confuse' alternatives, highlighting trade-offs between alternatives, and potentially guiding model specification. Furthermore, evaluating these matrices and metrics out-of-sample reveals alternative-level prediction shifts that significantly impact forecasting performance.
appendix boundary found by appendix_command · 88% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Hillel, Tim and Bierlaire, Michel and Elshafie, Mohammed ZEB and Jin… (2021) A systematic review of machine learning classification methodologies for modelling passenger mode choice | 0.644 | 4 | 1 | 100% |
| 2 | Yacouby, Reda and Axman, Dustin (2020) Probabilistic Extension of Precision, Recall, and F1 Score for More Thorough Evaluation of Classification Models | 0.644 | 4 | 1 | 100% |
| 3 | Sander van Cranenburgh and Shenhao Wang and Akshay Vij and Francisco… (2022) Choice modelling in the age of machine learning - Discussion paper | 0.585 | 3 | 1 | 100% |
| 4 | Zhang, Jiawei and Yang, Yuhong and Ding, Jie (2023) Information criteria for model selection | 0.585 | 3 | 1 | 100% |
| 5 | Akaike, H (1974) A new look at the statistical model identification | 0.511 | 2 | 1 | 100% |
| 6 | Azam Ali and Arash Kalatian and Charisma F. Choudhury (2023) Comparing and contrasting choice model and machine learning techniques in the context of vehicle ownership decisions | 0.511 | 2 | 1 | 100% |
| 7 | Gideon Schwarz (1978) Estimating the Dimension of a Model | 0.511 | 2 | 1 | 100% |
| 8 | Hess, Stephane and Lancsar, Emily and Mariel, Petr and Meyerhoff, Jü… (2022) The path towards herd immunity: Predicting COVID-19 vaccination uptake through results from a stated choice study across six con… self | 0.511 | 2 | 1 | 100% |
| 9 | Ortelli, Nicola and Hillel, Tim and Pereira, Francisco C and de Lapp… (2021) Assisted specification of discrete choice models | 0.511 | 2 | 1 | 100% |
| 10 | Terven, Juan and Cordova-Esparza, Diana M and Ramirez-Pedraza, Alfon… (2023) Loss functions and metrics in deep learning | 0.511 | 2 | 1 | 100% |
Showing the top 10 of 65 scored citations.