EconBase
← All papers

Impact of the Availability of ChatGPT on Software Development: A Synthetic Difference in Differences Estimation using GitHub Data

Alexander Quispe, Rodrigo Grijalba

arXiv 16 Jun 2024 · cs.SE

arXiv:2406.11046 · PDF · DOI · OpenAlex · Extracted main text

Abstract

Advancements in Artificial Intelligence, particularly with ChatGPT, have significantly impacted software development. Utilizing novel data from GitHub Innovation Graph, we hypothesize that ChatGPT enhances software production efficiency. Utilizing natural experiments where some governments banned ChatGPT, we employ Difference-in-Differences (DID), Synthetic Control (SC), and Synthetic Difference-in-Differences (SDID) methods to estimate its effects. Our findings indicate a significant positive impact on the number of git pushes, repositories, and unique developers per 100,000 people, particularly for high-level, general purpose, and shell scripting languages. These results suggest that AI tools like ChatGPT can substantially boost developer productivity, though further analysis is needed to address potential downsides such as low quality code and privacy concerns.

Citation extraction

20
references
20
in-text mentions
20
distinct cited
0
self-citations
6,023
main-text words

appendix boundary found by none_found · 100% of the source is main text. Read the extracted text to check this.

Most heavily cited references

The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.

ReferenceIntensityMentionsSectionsMain text
1Abadie, A., Diamond, A. and Hainmueller, J (2010) Synthetic control methods for comparative case studies: Estimating the effect of california’s tobacco control program0.40511100%
2Abu Jaber, M., Beganovic, A. and Abd Almisreb, A (2023) Methods and applications of chatgpt in software development: A literature review0.40511100%
3Ahmad, A., Waseem, M., Liang, P., Fahmideh, M., Aktar, M. S. and Mik… (2023) Towards human-bot collaborative software architecting with chatgpt0.40511100%
4Arkhangelsky, D., Athey, S., Hirshberg, D. A., Imbens, G. W. and Wag… (2021) Synthetic difference-in-differences0.40511100%
5Badini, S., Regondi, S., Frontoni, E. and Pugliese, R (2023) Assessing the capabilities of chatgpt to improve additive manufacturing troubleshooting0.40511100%
6Clarke, D., Pailañir, D., Athey, S. and Imbens, G (2023) Synthetic difference-in-differences estimation0.40511100%
7Del Rio-Chanona, M., Laurentsyeva, N. and Wachs, J (2023) Are large language models a threat to digital public goods? evidence from activity on stack overflow0.40511100%
8Demicri, O., Hannane, J. and Zhu, X (2023) Who is ai replacing? the impact of generative ai on online freelancing platforms0.40511100%
9Gallea, Q (2023) From mundane to meaningful: Ai's influence on work dynamics – evidence from chatgpt and stack overflow0.40511100%
10Jalil, S., Rafi, S., LaToza, T. D., Moran, K. and Lam, W (2023) Chatgpt and software testing education: Promises & perils0.40511100%

Showing the top 10 of 20 scored citations.