EconBase
← All papers

The Partial Testimony of Logs: Evaluation of Language Model Generation under Confounded Model Choice

Jikai Jin, Vasilis Syrgkanis

arXiv 2 May 2026 · Machine Learning

arXiv:2605.01311 · PDF · DOI · OpenAlex · Extracted main text

Abstract

Offline evaluation of language models from usage logs is biased when model choice is confounded: the same user-side factors that influence which model is used can also influence how its output is judged, so raw comparisons of logged scores mix self-selected populations rather than estimating a common quantity of interest. A small randomized experiment can break this bias by overriding model choice, but in practice such experiments are scarce and costly. We study a three-source design that combines a large confounded observational log (OBS) for scale, a small randomized experiment (EXP) for unconfounded scoring, and an offline simulator (SIM) that replays candidate models on cached contexts. Our main result is an identification theorem showing that the randomized experiment and the simulator are together enough to recover causal model values; the observational log enters only afterward, to reduce estimation error rather than to make the causal comparison valid. Six estimator families are evaluated in a controlled semi-synthetic validation and in two real-task cached benchmarks for summarization and coding. No family dominates every regime; relative performance depends on the amount of unbiased EXP supervision and on how closely the target reward aligns with OBS-derived structure.

Citation extraction

39
references
87
in-text mentions
39
distinct cited
0
self-citations
10,000
main-text words

appendix boundary found by appendix_command · 49% of the source is main text. Read the extracted text to check this.

Most heavily cited references

The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.

ReferenceIntensityMentionsSectionsMain text
1Nathan Kallus, Aahlad Manas Puli, and Uri Shalit (2018) Removing hidden confounding by experimental grounding1.00063100%
2Evan T. R. Rosenman, Guillaume Basse, Art B. Owen, and Mike Baiocchi (2023) Combining observational and experimental datasets using shrinkage estimators1.00063100%
3Xuelin Yang, Licong Lin, Susan Athey, Michael I. Jordan, and Guido W… (2025) Cross-validated causal inference: a modern method to combine experimental and observational data, 20251.00063100%
4David Cheng and Tianxi Cai (2021) Adaptive combination of randomized and observational data, 20211.00053100%
5Xi Lin, Jens Magelund Tarp, and Robin J. Evans (2025) Combining experimental and observational data through a power likelihood0.92843100%
6Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelo… (2024) Chatbot arena: An open platform for evaluating LLMs by human preference0.84333100%
7Karl Moritz Hermann, Tomás Kociský, Edward Grefenstette, Lasse Espeh… Teaching machines to read and comprehend0.7373367%
8Abigail See, Peter J. Liu, and Christopher D. Manning Get to the point: Summarization with pointer-generator networks0.7373367%
9Heejung Bang and James M. Robins Doubly robust estimation in missing data and causal inference models0.64422100%
10Elias Bareinboim and Judea Pearl (2016) Causal inference and the data-fusion problem0.64422100%

Showing the top 10 of 39 scored citations.