Clinical Report: LLM Use Guidance in Emergency Medicine Journals

Overview

This cross-sectional study assessed the prevalence and characteristics of large language model (LLM) use guidance across 56 emergency medicine journals. Findings reveal variable adoption of LLM directives, with significant disparities across publishers and associations with journal quality metrics.

Background

Large language models (LLMs) have transformed academic writing by enhancing efficiency but raise concerns about accuracy, plagiarism, bias, and accountability. While international bodies have issued recommendations, implementation at the journal level remains inconsistent across medical specialties. Emergency medicine, a high-risk and time-sensitive field, has not been comprehensively studied regarding LLM guidance, despite its reliance on rapid and accurate dissemination of research.

Data Highlights

The study reviewed 56 emergency medicine journals from the 2024 Journal Citation Reports. Data extraction focused on publicly accessible editorial policies and instructions for authors, identifying the presence or absence of LLM-related guidance. Publisher-level policies were included only if directly referenced by journals. Scientometric data such as CiteScore, SJR, SNIP, JIF, and JCI were collected to analyze correlations with guidance adoption. Statistical analysis was performed using nonparametric tests with significance set at P < 0.05.

Key Findings

• LLM guidance presence varied widely among emergency medicine journals, reflecting inconsistent adoption.
• Some journals explicitly required authors to disclose LLM use and emphasized human accountability for AI-generated content.
• Publisher-level differences influenced the likelihood of journals having LLM directives.
• Higher journal quality metrics (e.g., JIF, CiteScore) were associated with increased adoption of LLM guidance.
• Non-English journals were included and translated to ensure comprehensive assessment.
• Keywords such as "large language model," "ChatGPT," and "artificial intelligence" were used to identify relevant policies.

Clinical Implications

Clinicians and researchers submitting to emergency medicine journals should be aware of variable LLM policy adoption and ensure transparency when utilizing AI tools. Editorial boards should consider standardizing guidance to maintain research integrity and accountability. Awareness of publisher-specific policies can aid authors in compliance and ethical manuscript preparation.

Conclusion

The study highlights significant variability in LLM use guidance across emergency medicine journals, underscoring the need for consistent, specialty-specific editorial policies to safeguard scientific integrity amid evolving AI technologies.

References

1. International Committee of Medical Journal Editors (ICMJE) and Committee on Publication Ethics (COPE) -- Recommendations on AI use in publishing
2. Journal Citation Reports 2024 -- Emergency Medicine Journal Listings