Susan Athey, Herman Brunborg, Tianyu Du, Ayush Kanodia, Keyon Vafa
arXiv 25 Jun 2024 · Machine Learning · 2 citations (OpenAlex)
arXiv:2406.17972 · PDF · DOI · OpenAlex · Extracted main text
Vafa et al. (2024) introduced a transformer-based econometric model, CAREER, that predicts a worker's next job as a function of career history (an "occupation model"). CAREER was initially estimated ("pre-trained") using a large, unrepresentative resume dataset, which served as a "foundation model," and parameter estimation was continued ("fine-tuned") using data from a representative survey. CAREER had better predictive performance than benchmarks. This paper considers an alternative where the resume-based foundation model is replaced by a large language model (LLM). We convert tabular data from the survey into text files that resemble resumes and fine-tune the LLMs using these text files with the objective to predict the next token (word). The resulting fine-tuned LLM is used as an input to an occupation model. Its predictive performance surpasses all prior models. We demonstrate the value of fine-tuning and further show that by adding more career data from a different population, fine-tuning smaller LLMs surpasses the performance of fine-tuning larger models.
appendix boundary found by appendix_titled_section at “Data Appendix” · 52% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Touvron, Hugo and Martin, Louis and Stone, Kevin and Albert, Peter a… (2023) Llama 2: Open Foundation and Fine-Tuned Chat Models | 0.928 | 4 | 3 | 100% |
| 2 | Vafa, Keyon and Palikot, Emil and Du, Tianyu and Kanodia, Ayush and… (2024) CAREER: A Foundation Model for Labor Sequence Data self | 0.855 | 16 | 6 | 62% |
| 3 | Autor, David H and Dorn, David (2013) The growth of low-skill service jobs and the polarization of the US labor market | 0.737 | 5 | 3 | 40% |
| 4 | Schmidt, Peter and Strauss, Robert P (1975) The Prediction of Occupation Using Multiple Logit Models | 0.737 | 3 | 2 | 100% |
| 5 | Boskin, Michael J (1974) A Conditional Logit Model of Occupational Choice | 0.644 | 2 | 2 | 100% |
| 6 | Vafa, Keyon and Athey, Susan and Blei, David M (2025) Estimating wage disparities using foundation models self | 0.585 | 3 | 1 | 100% |
| 7 | Survey Research Center, Institute for Social Research, University of… (2024) Panel Study of Income Dynamics, public use dataset | 0.511 | 2 | 2 | 50% |
| 8 | Blau, Francine D. and Kahn, Lawrence M (2017) The Gender Wage Gap: Extent, Trends, and Explanations | 0.511 | 2 | 1 | 100% |
| 9 | Brown, Randall S and Moon, Marilyn and Zoloth, Barbara S (1980) Incorporating occupational attainment in studies of male-female earnings differentials | 0.511 | 2 | 1 | 100% |
| 10 | Fairlie, Robert W. and Sundstrom, William A (1999) The Emergence, Persistence, and Recent Widening of the Racial Unemployment Gap | 0.511 | 2 | 1 | 100% |
Showing the top 10 of 73 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.
| Citing paper | Intensity | Mentions | Sections | |
|---|---|---|---|---|
| 1 | Mining Causality: AI-Assisted Search for Instrumental Variables | 0.644 | 2 | 2 |
| 2 | Causal Inference on Outcomes Learned from Text | 0.405 | 1 | 1 |