Computer vision could turn surgical video into actionable clinical data

"One of the most tedious parts ... is at the end of surgery when we’re waiting for the team to count the instruments to make sure we didn’t lose one."

By
Meg Barbor
September 28, 2026
5 min

MDSpire News

Objective:

To explore how computer vision can extract actionable insights from surgical video, enhancing documentation and decision-making in surgery.

Approach:

Identifying Surgical Phases: Computer vision models were developed to identify individual phases of robotic prostatectomy with accuracy in the low- to mid-90% range.
Automated Operative Reports: AI-generated operative reports were created from surgical videos, achieving 87.3% accuracy compared to 72.8% for surgeon-written reports.
Evaluating Bladder Lesions: A computer vision model was trained to predict tumor histology from video during transurethral resection, achieving AUCs of 0.829 and 0.867 in different cohorts.
Automating Instrument Counting: A proof-of-concept project demonstrated a model's ability to detect and count surgical instruments in overlapping conditions.

Key Findings:

AI-generated operative reports showed higher accuracy than those written by surgeons.
The potential for video-linked operative reports could enhance surgical documentation.
Computer vision models can assist in evaluating ambiguous bladder lesions.
Automation of routine tasks like instrument counting could improve surgical efficiency.

Interpretation:

The findings suggest that computer vision has the potential to enhance surgical documentation, decision-making, and efficiency, although further validation is needed.

Limitations:

AI models may struggle with uncommon situations and subjective data can introduce noise.
Models can overfit to training data, necessitating validation across diverse settings.