AI Reaches Physician‑Level Scores in Simulated Exams
Recent experiments with large language models—those behind chatbots like ChatGPT—show that they can reason through clinical case studies almost as well as seasoned physicians. Australian researchers from Flinders University highlighted these findings in a commentary published alongside a study in the journal Science. The key message is clear: excelling on paper‑based assessments does not automatically translate into safe, real‑world medical care.
Why Test Success Is Not the Same as Clinical Competence
In the controlled environment of an exam, AI can dissect a patient’s history, weigh differential diagnoses, and even arrive at a recommended treatment plan. In many of these artificial scenarios the algorithm matches or surpasses human performance. Yet a genuine patient is far more complex than a list of symptoms. Doctors use tactile feedback, visual cues, tone of voice, and subtle observations about a person’s environment—information that current language models simply cannot capture.
The Human Touch Remains Irreplaceable
When a physician palpates an abdomen, notes a fleeting tremor, or senses that a patient is hesitant to disclose certain details, they are gathering data that lies outside any textual dataset. Those nuances often tip the balance between a correct and a missed diagnosis. AI, which processes only the words it has been fed, lacks this embodied awareness.
Accountability and Legal Ambiguities
If an AI recommendation leads to harm, the question of liability becomes tangled. Is the software developer responsible? The hospital that purchased the system? Or the clinician who trusted the output? The commentary stresses that these legal frameworks are still in their infancy, leaving a dangerous gap in patient protection.
Bias, Fairness, and Data Gaps
Machine‑learning models inherit the biases present in their training data. Populations that are under‑represented in medical records may receive sub‑optimal advice from an AI trained predominantly on data from other groups. Without rigorous auditing, AI could unintentionally reinforce health disparities.
Treating Medical AI Like a Trainee
The authors advocate for viewing AI systems as medical interns who require close supervision. Just as no student would be allowed to treat patients unsupervised, an algorithm that existed only months ago should not be given unchecked autonomy. Proper oversight, continuous validation, and clear regulatory standards are essential before widespread deployment.
Potential Benefits Amid Caution
Despite the warnings, the researchers acknowledge that AI can alleviate clinicians’ workload, streamline documentation, and serve as a decision‑support tool when used responsibly. The goal is not to replace physicians but to augment their capacity, especially in overstretched health systems.
Source: https://scientias.nl/ai-denkt-al-bijna-net-zo-goed-als-een-dokter-en-dat-is-precies-het-probleem/