Clinical Report: Identifying Pneumatic Retinopexy in Electronic Medical Records

Overview

In a retrospective diagnostic accuracy study of 542 patient records from two Australian hospitals, an on-premises open-weight large language model identified documented pneumatic retinopexy (PnR) with 0.98 sensitivity and 0.94 specificity. Its positive predictive value was 0.81, with false positives mainly involving pneumatic displacement or gas tamponade associated with pars plana vitrectomy. The authors support its use as a supervised case-finding tool, with positive classifications verified by a human reviewer.

Background

Accurate procedure identification in electronic medical records can support clinical registries, trial recruitment, quality assurance, and operative log validation. Administrative coding may not reliably represent ophthalmic procedures; the article cites an Australian study in which ophthalmology procedures were coded correctly in 53.9% of cases. PnR is a relevant test case because it may be performed outside the operating theatre, documented across multiple notes, and confused in text with other gas-based retinal procedures. Locally deployed open-weight language models may allow unstructured clinical text to be processed within institutional infrastructure.

Data Highlights

Key Findings

• The retrospective study included patients treated at the Royal Adelaide Hospital or The Queen Elizabeth Hospital between July 2023 and June 2024; candidate records were retrieved using “C3F8,” “SF6,” or “pneumatic.”
• Qwen3-235B-A22B was deployed on-premises and used a two-stage zero-shot prompt to classify whether PnR was documented.
• Against a manually established reference standard, the model produced 108 true positives, 407 true negatives, 25 false positives, and 2 false negatives among 542 records.
• Of the 25 false positives, 14 (56%) involved pneumatic displacement for submacular haemorrhage and 11 (44%) involved gas tamponade associated with PPV.
• The authors report that earlier supervised NLP approaches depended on labeled training data and performed less well for more complex ophthalmic procedures; the present zero-shot approach reduced task-specific development burden while keeping data within institutional infrastructure.
• The study assessed one model and prompt in a keyword-enriched cohort, had no rule-based comparator, and did not include external multicentre validation or prospective workflow assessment.

Clinical Implications

The authors describe the model as a potential case-finding instrument for research and administrative workflows, while noting that positive classifications require targeted human verification because the PPV was 0.81. They state that external multicentre validation and prospective workflow assessment are needed before implementation and favour human-supervised workflows over fully autonomous classification.

Conclusion

In this two-hospital cohort, an on-premises open-weight LLM showed high sensitivity and NPV for identifying documented PnR in longitudinal EMRs. Its observed errors, particularly confusion with other gas-based retinal procedures, support the authors’ emphasis on human verification and further validation.

Related Resources & Content

1. Study authors, Source not specified, Year not provided — Identifying Pneumatic Retinopexy in Electronic Medical Records with an On-Premises Large Language Model
2. EyeWiki, American Academy of Ophthalmology, Year not provided — Pneumatic Retinopexy
3. Ophthalmology, Year not provided — The Pneumatic Retinopexy versus Vitrectomy for the Management of Primary Rhegmatogenous Retinal Detachment Outcomes Randomized Trial (PIVOT)
4. BMJ Health & Care Informatics, Year not provided — Open-source large language model-based on-premises pipeline for automated data extraction from unstructured electronic health records: a pilot study
5. Eye — Performance of large language models for ophthalmic literature retrieval
6. JAMA Network Open — A Tool for Surgical Patient Triage Utilizing a Large Language Model Integrated with Electronic Health Records
7. Journal of Medical Internet Research (JMIR) — Automated Identification of Nursing Diagnoses and Interventions From Nursing Records Using a Retrieval-Augmented Large Language Model Approach: Quantitative Study
8. Pneumatic Retinopexy - EyeWiki
9. The Pneumatic Retinopexy versus Vitrectomy for the Management of Primary Rhegmatogenous Retinal Detachment Outcomes Randomized Trial (PIVOT)
10. Open-source large language model-based on-premises pipeline for automated data extraction from unstructured electronic health records: a pilot study
11. Pneumatic Retinopexy - EyeWiki
12. The Pneumatic Retinopexy versus Vitrectomy for the Management of Primary Rhegmatogenous Retinal Detachment Outcomes Randomized Trial (PIVOT) - Ophthalmology
13. Procedure Price Lookup for Outpatient Services | Medicare.gov 67110