Shanjukta Nath, Jiwon Hong, Jae Ho Chang, Keith Warren, Subhadeep Paul
arXiv 25 Sep 2025 · Econometrics
arXiv:2509.20634 · PDF · DOI · OpenAlex · Extracted main text
We find AI embeddings obtained using a pre-trained transformer-based Large Language Model (LLM) of 80,000-120,000 written affirmations and correction exchanges among residents in low-security correctional facilities to be highly predictive of recidivism. The prediction accuracy is 30% higher with embedding vectors than with only pre-entry covariates. However, since the text embedding vectors are high-dimensional, we perform Zero-Shot classification of these texts to a low-dimensional vector of user-defined classes to aid interpretation while retaining the predictive power. To shed light on the social dynamics inside the correctional facilities, we estimate peer effects in these LLM-generated numerical representations of language with a multivariate peer effect model, adjusting for network endogeneity. We develop new methodology and theory for peer effect estimation that accommodate sparse networks, multivariate latent variables, and correlated multivariate outcomes. With these new methods, we find significant peer effects in language usage for interaction and feedback.
appendix boundary found by appendix_command · 60% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Ma, Z., Ma, Z., and Yuan, H (2020) Universal latent space model fitting for large networks with edge covariates | 1.000 | 7 | 4 | 100% |
| 2 | Bramoullé, Y., Djebbari, H., and Fortin, B (2009) Identification of peer effects through social networks | 1.000 | 7 | 3 | 100% |
| 3 | Graham, B. S (2017) An econometric model of network formation with degree heterogeneity | 1.000 | 5 | 3 | 100% |
| 4 | Li, J., Xu, G., and Zhu, J (2023) Statistical inference on latent space models for network data | 0.935 | 11 | 3 | 82% |
| 5 | Johnsson, I. and Moon, H. R (2021) Estimation of peer effects in endogenous social networks: Control function approach | 0.928 | 20 | 4 | 80% |
| 6 | Athreya, A., Fishkind, D. E., Tang, M., Priebe, C. E., Park, Y., Vog… (2018) Statistical inference on random dot product graphs: a survey | 0.928 | 4 | 3 | 100% |
| 7 | Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K (2019) Bert: Pre-training of deep bidirectional transformers for language understanding | 0.928 | 4 | 3 | 100% |
| 8 | Dressel, J. and Farid, H (2018) The accuracy, fairness, and limits of predicting recidivism | 0.928 | 4 | 3 | 100% |
| 9 | Hoff, P. D., Raftery, A. E., and Handcock, M. S (2002) Latent space approaches to social network analysis | 0.928 | 4 | 3 | 100% |
| 10 | Zhu, X., Huang, D., Pan, R., and Wang, H (2020) Multivariate spatial autoregressive model for large scale social networks | 0.928 | 4 | 3 | 100% |
Showing the top 10 of 82 scored citations.