The AI text program ChatGPT was used to study 30 simple medical cases. The diagnostics were surprisingly accurate, but ChatGPT also showed some weaknesses.
Although AI text program ChatGPT doesn’t specialize in medicine, it diagnoses patients in the emergency room as accurately as doctors, according to a study published Wednesday. According to the authors of the Dutch study, the chatbot, which uses artificial intelligence (AI), even outperformed the work of doctors in some cases – but was still prone to errors.
For their study, researchers examined 30 cases of patients who had been treated in a Dutch emergency department in the past year. They fed ChatGPT with anonymized patient data, lab tests, and doctor observations and asked the chatbot to make five possible diagnoses. They then compared this with the doctors’ list of diagnoses and finally matched it with the correct diagnosis.
Doctors found the correct diagnosis in 87% of cases among the five suggestions, and with ChatGPT version 3.5 even in 97% of cases. “Simply put, this means that ChatGPT was able to suggest medical diagnoses, similar to what a human doctor would do,” said emergency doctor Hidde ten Berg from Jeroen Bosch Hospital in ‘s-Hertogenbosch.
Few patients, clear symptoms
As in other areas, the chatbot also presented some weaknesses. Sometimes the chatbot’s reasoning was “clinically implausible or contradictory,” the study said. This could lead to “misinformation or misdiagnosis” – with correspondingly serious consequences.
At the same time, scientists admitted that the number of cases examined was very low. Furthermore, only simple cases where the patient complained of only one main problem were examined.
ChatGPT can provide doctors with insights
The study does not mean that computers could one day take over emergency room management, co-author Steef Kurstjens told AFP news agency. But artificial intelligence can support doctors under pressure in diagnosing – for example, by “providing insights that the doctor hadn’t thought of”. AI has “the potential to save time and thus reduce emergency room wait times,” ten Berg said.
The study was published in the journal “Annals of Emergency Medicine”. Their results will be presented next week at the European Society of Emergency Medicine (Eusem) congress in Barcelona. (APA/dpa)