Siyuan Zhang (Ph.D. student, Gu Lab) received the Best Student Paper Award at the 16th International Conference on Applied Human Factors and Ergonomics

Medical institutions often rely on manual analysis of adverse events, which requires substantial human resources, time, and specialized expertise. These demands can limit the efficiency of identifying potential risks. This study investigates whether large language models (LLMs) with advanced natural language processing capabilities can support the identification of risks associated with adverse events. By evaluating five widely used models on healthcare-specific tasks, the study found that the best-performing LLM was able to extract more than half of the risk factors with accuracy and generate reasonable explanations based on real cases, although its ability to address complex cases still needs further improvement.

The study also emphasizes the importance of developing human-centered evaluation strategies for LLMs in medical contexts. Such approaches complement technical assessments of LLM performance and are particularly relevant for enhancing human–AI interaction in healthcare.

As an initial attempt to apply generative AI to the analysis of complex medical records, the study drew considerable interest from scholars and reviewers, sparking active discussion and broad resonance at the conference.

Authors:

Siyuan Zhang, Xiuzhu Gu

Paper Title：

Is LLM a reliable risk detector? An evaluation of large language models in EMR-related medical incident detection

Supervisor：

Associate Professor Xiuzhu Gu

Related Sites

• Best Paper Awards | Conference Best Paper Awards
• Gu Lab