A recent study published in PNAS Nexus has revealed that the widespread adoption of large language models (LLMs), such as ChatGPT, is associated with a notable decrease in public knowledge sharing on platforms like Stack Overflow. The research points to a 25% reduction in user activity on the prominent programming Q&A site within six months following the release of ChatGPT, in contrast to similar platforms where access to ChatGPT is restricted.

Maria del Rio-Chanona, the study’s lead author and an associate faculty member at the Complexity Science Hub (CSH), commented on LLMs’ potency and transformative potential. “LLMs are incredibly powerful and have a profound impact on our global landscape. This leads us to ponder their long-term implications,” she stated.

The study proposed that rather than engaging in public question-and-answer exchanges on platforms like Stack Overflow, users are turning to private interactions with ChatGPT. This shift is significant, as LLMs like ChatGPT rely on the very type of open data they are beginning to supplant. “What will happen when the data from these public platforms dwindles?” Del Rio-Chanona, an assistant professor at University College London, an associate researcher at the Institute for New Economic Thinking at the Oxford Martin School, and the Bennett Institute for Public Policy at the University of Cambridge, further questioned.

The implications of these findings are profound. “We are observing a decline in the number of questions and answers on Stack Overflow since the introduction of ChatGPT. This reduction could potentially result in a scarcity of public data for training future models,” Del Rio-Chanona warned, indicating a severe challenge to the sustainability of AI development. In this research, she collaborated with Nadzeya Laurentsyeva from Ludwig Maximilian University of Munich and Johannes Wachs, a faculty member at CSH and professor at Corvinus University in Budapest, in a collective effort to understand and address this issue.

Johannes Wachs highlighted the value of Stack Overflow as a global knowledge repository. “Stack Overflow facilitates a worldwide educational exchange that even AI models like ChatGPT depend on,” he noted. Despite the current challenges, the potential for future AI models is promising. Ironically, AI’s displacement of human-driven content creation might hinder future AI models’ training, as training data derived from AI tends to yield inferior results, akin to the degradation seen when photocopying a document multiple times.

The study also draws attention to broader societal and economic shifts resulting from the transition from public to private data accumulation due to increased interaction with LLMs like ChatGPT. This migration of knowledge from open to private spheres could enhance the competitive edge of early adopters in the AI field, thereby centralising knowledge and economic power.

Furthermore, the decline in content creation on Stack Overflow has impacted users across all experience levels, from beginners to experts. However, as judged by user feedback, the quality of the contributions has remained relatively high, indicating that the displacement involves both high and low-quality posts alike.

Additionally, the research noted a particularly steep decline in posts related to Python and JavaScript, suggesting that users are possibly posing their questions about these popular programming languages directly to ChatGPT rather than on Stack Overflow. This phenomenon underscores the evolving nature of how and where we seek knowledge in the digital age.

More information: Maria del Rio-Chanona et al, Large language models reduce public knowledge sharing on online Q&A platforms, PNAS Nexus. DOI: 10.1093/pnasnexus/pgae400

Journal information: PNAS Nexus Provided by Complexity Science Hub