Objective:

To develop a benchmark for anticancer drug reimbursement eligibility based on South Korea’s national guidelines and evaluate large language models (LLMs) for reliability in reimbursement eligibility adjudication under incomplete information.

Key Findings:

• Anticancer drug expenditures in South Korea surged by 168.2% from 2013 to 2022, complicating eligibility criteria.
• LLMs can support reimbursement-related tasks but require structured logical reasoning to assess multiple clinical attributes against complex rule sets.
• No standardized benchmark exists to evaluate LLM capabilities for reimbursement eligibility under real-world constraints.

Interpretation:

The study highlights the need for a structured benchmark to evaluate LLMs in the context of reimbursement eligibility, particularly under conditions of incomplete information.

Limitations:

• The benchmark focuses solely on three types of gynecologic cancers, which may limit its applicability.
• Validation was conducted by a limited number of experts, which may not encompass all clinical perspectives.

Conclusion:

The development of a benchmark for anticancer drug reimbursement eligibility is essential for assessing LLM reliability in real-world insurance review contexts.