Keyon Vafa, Emil Palikot, Tianyu Du, Ayush Kanodia, Susan Athey, David M. Blei
arXiv 16 Feb 2022 · Machine Learning · 3 citations (OpenAlex)
arXiv:2202.08370 · PDF · DOI · OpenAlex · Extracted main text
Labor economists regularly analyze employment data by fitting predictive models to small, carefully constructed longitudinal survey datasets. Although machine learning methods offer promise for such problems, these survey datasets are too small to take advantage of them. In recent years large datasets of online resumes have also become available, providing data about the career trajectories of millions of individuals. However, standard econometric models cannot take advantage of their scale or incorporate them into the analysis of survey data. To this end we develop CAREER, a foundation model for job sequences. CAREER is first fit to large, passively-collected resume data and then fine-tuned to smaller, better-curated datasets for economic inferences. We fit CAREER to a dataset of 24 million job sequences from resumes, and adjust it on small longitudinal survey datasets. We find that CAREER forms accurate predictions of job sequences, outperforming econometric baselines on three widely-used economics datasets. We further find that CAREER can be used to form good predictions of other downstream variables. For example, incorporating CAREER into a wage model provides better predictions than the econometric models currently in use.
appendix boundary found by appendix_command · 50% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | J. Devlin, M. Chang, K. Lee, and K. Toutanova (2019) BERT: Pre-training of deep bidirectional transformers for language understanding | 1.000 | 5 | 3 | 100% |
| 2 | Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and… (2019) Language models are unsupervised multitask learners | 0.928 | 4 | 3 | 100% |
| 3 | Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever (2018) Improving language understanding by generative pre-training | 0.843 | 5 | 3 | 60% |
| 4 | Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Sy… (2023) Simple and controllable music generation | 0.843 | 3 | 3 | 100% |
| 5 | Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis… (2023) Starcoder: may the source be with you! | 0.843 | 3 | 3 | 100% |
| 6 | Robert E Hall (1972) Turnover in the labor force | 0.830 | 7 | 4 | 57% |
| 7 | Francine D Blau and Lawrence M Kahn (2017) The gender wage gap: Extent, trends, and explanations | 0.811 | 4 | 2 | 100% |
| 8 | Francisco J. R. Ruiz, Susan Athey, and David M. Blei (2020) SHOPPER: A probabilistic model of consumer choice with substitutes and complements self | 0.794 | 6 | 4 | 50% |
| 9 | Liangyue Li, How Jing, Hanghang Tong, Jaewon Yang, Qi He, and Bee-Ch… (2017) NEMO: Next career move prediction with contextual embedding | 0.737 | 5 | 4 | 40% |
| 10 | Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jo… (2017) Attention is all you need | 0.737 | 4 | 4 | 50% |
Showing the top 10 of 66 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.
| Citing paper | Intensity | Mentions | Sections | |
|---|---|---|---|---|
| 1 | Estimating Wage Disparities Using Foundation Models | 0.894 | 7 | 4 |
| 2 | LABOR-LLM: Language-Based Occupational Representations with Large Language Models | 0.855 | 16 | 6 |
| 3 | Model-Agnostic Covariate-Assisted Inference on Partially Identified Causal Effects | 0.405 | 1 | 1 |
| 4 | Causal Inference on Outcomes Learned from Text | 0.405 | 1 | 1 |