_JAMA Pediatrics and the NEJM were accessed for pediatric case challenges (N = 100). The text from each case was pasted into ChatGPT version 3.5 with the prompt List a differential diagnosis and a final diagnosis. _
A couple of key points:
These are case challenges, which are usually meant to be hard. I could find no comparison to actual physician results in the article, which would have been nice.
More importantly however: it was conducted in June 2023, and used GPT-3.5. GPT-4 improved substantially upon it, especially for complex scientific or scientific problems, and this shows in the newer studies that have used it.
I don’t think anyone’s advocating that an AI will replace doctors, much like it won’t replace white collar jobs either.
But if it helps achieve better outcomes for the patients, like the current research seems to indicate, aren’t you sworn to consider it in your practice?
Part of my significant suspicion regarding AI is that most of my medical experience and my intended specialty upon graduation is Emergency Medicine. The only thing AI might be useful for there is to function as a scribe. The AI is not going to tell me that the patient who denies any alcohol consumption smells like a liquor store, or that the patient that is completely unconscious has asterixis and flapping tremors. AI cannot tell me anything useful for my most critical patients, and for the less critical ones, I am perfectly capable of pulling up UpToDate or Dynamed and finding the thing I’m looking for myself. Maybe it can be useful for making suggestions for next steps, but for the initial evaluation? Nah. I don’t trust a glorified text predictor to catch the things that will kill my patients in the next 5 minutes.
From the article referenced in your news source:
A couple of key points:
I don’t think anyone’s advocating that an AI will replace doctors, much like it won’t replace white collar jobs either.
But if it helps achieve better outcomes for the patients, like the current research seems to indicate, aren’t you sworn to consider it in your practice?
Part of my significant suspicion regarding AI is that most of my medical experience and my intended specialty upon graduation is Emergency Medicine. The only thing AI might be useful for there is to function as a scribe. The AI is not going to tell me that the patient who denies any alcohol consumption smells like a liquor store, or that the patient that is completely unconscious has asterixis and flapping tremors. AI cannot tell me anything useful for my most critical patients, and for the less critical ones, I am perfectly capable of pulling up UpToDate or Dynamed and finding the thing I’m looking for myself. Maybe it can be useful for making suggestions for next steps, but for the initial evaluation? Nah. I don’t trust a glorified text predictor to catch the things that will kill my patients in the next 5 minutes.