Xinrui Ruan, Xinwei Ma, Yingfei Wang, Waverly Wei, Jingshen Wang
arXiv 7 Oct 2025 · Statistics — Methodology
arXiv:2510.05545 · PDF · DOI · OpenAlex · Extracted main text
Randomized experiments or randomized controlled trials (RCTs) are gold standards for causal inference, yet cost and sample-size constraints limit power. Meanwhile, modern RCTs routinely collect rich, unstructured data that are highly prognostic of outcomes but rarely used in causal analyses. We introduce CALM (Causal Analysis leveraging Language Models), a statistical framework that integrates large language models (LLMs) predictions with established causal estimators to increase precision while preserving statistical validity. CALM treats LLM outputs as auxiliary prognostic information and corrects their potential bias via a heterogeneous calibration step that residualizes and optimally reweights predictions. We prove that CALM remains consistent even when LLM predictions are biased and achieves efficiency gains over augmented inverse probability weighting estimators for various causal effects. In particular, CALM develops a few-shot variant that aggregates predictions across randomly sampled demonstration sets. The resulting U-statistic-like predictor restores i.i.d. structure and also mitigates prompt-selection variability. Empirically, in simulations calibrated to a mobile-app depression RCT, CALM delivers lower variance relative to other benchmarking methods, is effective in zero- and few-shot settings, and remains stable across prompt designs. By principled use of LLMs to harness unstructured data and external knowledge learned during pretraining, CALM provides a practical path to more precise causal analyses in RCTs.
appendix boundary found by none_found · 100% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Brown, Mann, Ryder, Subbiah, Kaplan, Dhariwal et al (2020) Language models are few-shot learners | 0.928 | 4 | 3 | 100% |
| 2 | Chernozhukov, Chetverikov, Demirer, Duflo, Hansen, Newey and Robins (2018) Double/debiased machine learning for treatment and structural parameters | 0.843 | 3 | 3 | 100% |
| 3 | Pratap, Homiar, Waninger, Herd, Suver, Volponi et al (2022) Real-world behavioral dataset from two fully remote smartphone-based randomized clinical trials for depression | 0.811 | 4 | 2 | 100% |
| 4 | van der Vaart (2000) Asymptotic Statistics, Cambridge, UK: Cambridge University Press | 0.644 | 2 | 2 | 100% |
| 5 | Bommasani, Hudson, Adeli, Altman, Arora, von Arx et al (2021) On the opportunities and risks of foundation models | 0.511 | 2 | 1 | 100% |
| 6 | Angelopoulos, Bates, Fannjiang, Jordan and Zrnic (2023) Prediction-powered inference | 0.511 | 2 | 1 | 100% |
| 7 | Guterman, Kiekhofer, Wood, Allen, Kahn, Dulaney et al (2023) Care ecosystem collaborative model and health care costs in Medicare beneficiaries with dementia: A secondary analysis of a rand… | 0.511 | 2 | 1 | 100% |
| 8 | OpenAI et al (2023) GPT-4 technical report | 0.511 | 2 | 1 | 100% |
| 9 | Touvron, Lavril, Izacard, Martinet, Lachaux, Lacroix et al (2023) Llama: Open and efficient foundation language models | 0.511 | 2 | 1 | 100% |
| 10 | De Bartolomeis, Abad, Wang, Donhauser, Duch, Yang and Dahabreh (2025) Efficient randomized experiments using foundation models | 0.405 | 1 | 1 | 100% |
Showing the top 10 of 58 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.
| Citing paper | Intensity | Mentions | Sections | |
|---|---|---|---|---|
| 1 | AI-Assisted Variance Reduction in Randomized Experiments | 1.000 | 5 | 3 |