Knowledge localization is associated with higher performance of domestic large language models in a Chinese radiation oncology examination

Category	Detail
Condition	Radiation Oncology
Key Mechanisms	Evaluation of large language models' performance in specialized clinical assessments.
Target Population	Chinese radiation oncologists and medical professionals.
Care Setting	Clinical assessments and examinations in radiation oncology.

Domestic models outperformed a single human reference participant in radiation oncology assessments.
International models showed significant performance decline in localized knowledge retrieval.
Translation of examination into English did not improve performance for international models.
Majority of errors in international models stemmed from discrepancies with Chinese clinical guidelines.
Regional clinical standards may significantly influence model performance.

Utilize models that align with regional clinical guidelines for improved accuracy.

Conduct error analysis to identify gaps in model performance related to localized knowledge.

Be aware of potential discrepancies between Western and Chinese clinical practices.

Patients undergoing radiation oncology assessments in China.

Models must interpret mixed-language inputs and adhere to localized clinical guidelines.

Incorporate regional clinical guidelines into model training and evaluation.
Ensure models are tested in the language and cultural context of the target population.
Regularly update models to reflect the latest clinical standards and practices.

Clinical Scorecard: Enhanced Knowledge Localization Improves Performance of Domestic Large Language Models in Chinese Radiation Oncology Assessments