Key Points
- LLM use leads to a surge in manuscript submissions, especially among non‑native English speakers.
- AI‑generated papers exhibit higher linguistic complexity than traditional manuscripts.
- Despite greater complexity, LLM‑assisted papers are less likely to be published in peer‑reviewed journals.
- Reference lists of AI‑assisted work cite a wider variety of sources, including more books and recent studies.
- Many AI‑drafted manuscripts may undergo substantial human editing, obscuring true usage rates.
- Reliance on publication outcomes may underrepresent newer, AI‑heavy drafts.
Increased Submissions Driven by LLMs
Researchers observed a pronounced rise in manuscript submissions to pre‑print servers after scientists began using large language models to draft their work. The effect was especially strong for authors whose names indicated Asian origin and who were affiliated with institutions in Asia, suggesting that non‑native English speakers are leveraging LLMs to overcome language barriers.
Higher Linguistic Complexity but Lower Publication Rates
Analysis showed that papers written with LLM assistance tend to contain more complex language than those produced without AI help. Despite this increase in textual sophistication, LLM‑generated manuscripts were less likely to be accepted for publication in peer‑reviewed journals. In contrast, non‑LLM papers that used complex language were more likely to be published, indicating a reversal of the usual positive correlation between linguistic complexity and perceived scientific merit.
Diversified Citation Patterns
When the reference lists of AI‑assisted papers were examined, they revealed a broader range of sources compared with traditional manuscripts. LLM‑generated work was more prone to cite books and recent publications, suggesting that these tools may introduce a wider scholarly perspective into the literature.
Cautions and Limitations
The study authors note two important caveats. First, many manuscripts labeled as human‑written may actually contain AI‑generated text that was later heavily edited, meaning the true prevalence of LLM use could be higher than reported. Second, the reliance on publication outcomes as a proxy for quality may disadvantage newer drafts that are more likely to incorporate AI, because they have had less time to progress through the peer‑review process.
Implications for the Research Community
The findings highlight both opportunities and challenges associated with the growing use of LLMs in scientific writing. While the technology appears to empower researchers who face language obstacles and to broaden citation practices, it also raises questions about the relationship between textual complexity and publication success. Ongoing scrutiny of how AI tools are integrated into the research workflow will be essential to ensure that the benefits are realized without compromising the standards of scholarly communication.
Source: arstechnica.com