Artificial intelligence already surpasses the expertise of doctors in standardized medical knowledge tests on acute kidney injury (AKI). This is shown by a study by the University of Marburg and the University Hospital Giessen and Marburg (UKGM). 13 major language models achieved an average of 90 percent correct answers, while 123 participants in the DGIM Congress 2025 only managed 49 percent.

The team led by Dr. Philipp Russ and Prof. Dr. Ivica Grgic had both groups complete the same German-language test with two realistic patient cases and 15 multiple-choice questions. Several AI models answered all questions flawlessly and took only a fraction of the time. The subjects were medical students and specialists in internal medicine who took part in the 131st annual congress of the German Society of Internal Medicine (DGIM) in Wiesbaden.

The results show that current language models can now very reliably retrieve and apply guideline-compliant expertise on kidney diseases. At the same time, the authors emphasize the limits of the technology. “Human judgment and clinical experience remain crucial. The ultimate responsibility for patient care continues to lie with the treating physicians,” explained nephrologist and AI expert Prof. Dr. Ivica Grgic.

Study leader Dr. Philipp Russ sees the systems primarily as a supporting tool for everyday clinical practice: “Large language models can provide medical factual knowledge very quickly. This is an opportunity. At the same time, they have clear boundaries: they cannot feel empathy and do not grasp the human being in all his complexity.”

The study underlines the enormous potential of AI as a knowledge and decision-making support, but at the same time warns against overestimation. Independent clinical decision-making through AI is currently neither possible nor desirable. The results were published in the journal Scientific Reports.

Original Paper:

Potential of large language models for rapid clinical information support: evidence from acute kidney injury knowledge testing | Scientific Reports

Read Also:

VDI study: AI and robotics are driving a profound transformation of the German healthcare system – MedLabPortal

Editor: X-Press Journalistenbüro GbR

Gender Notice. The personal designations used in this text always refer equally to female, male and diverse persons. Double/triple naming and gendered designations are used for better readability. ected.