AI vs. Human: Study Finds AI Chatbots Match Pharmacists in Real-World Medication Inquiries
Pharmacists are pivotal in ensuring the safe and effective administration of medications, often facing high workloads and a shortage of qualified professionals, especially in resource-limited settings. To address these challenges, the study examined the potential of AI chatbots to assist in managing drug-related inquiries.
Researchers from King Saud University, Saudi Arabia, analyzed 70 real-world drug inquiries from a Drug Information Inquiry Database, evaluating responses based on accuracy, detail, and risk of harm. They compared the outputs from GPT-based chatbots (GPT-3, GPT-3.5, GPT-4) with those from human pharmacists. The key findings were:
- GPT-4 demonstrated an accuracy rate of over 90%, which is comparable to that provided by human pharmacists.
- Both GPT-4 and human pharmacists delivered responses deemed 95% safe. However, GPT-4 also provided proactive risk mitigation information in 70% of cases, surpassing human counterparts.
- While GPT-4 showed strong performance in addressing dosage and administration queries, human pharmacists were more adept at managing inquiries related to drug interactions and pregnancy/lactation issues.
These findings underscore the potential of GPT-4 and similar AI chatbots as invaluable tools to support healthcare professionals, particularly in regions with limited access to medical care. The use of AI in drug-related inquiries not only promises to enhance healthcare accessibility but also provides substantial cost savings for countries aiming to establish and maintain high-quality drug information centers.
Given the ongoing global challenges in healthcare staffing and disparities in medical access, especially in rural and outbreak-prone areas, AI chatbots represent a significant opportunity. By leveraging technologies like the GPT-4 model, these areas could see improved access to essential drug-related information, thereby easing the burden on healthcare systems and enhancing patient outcomes.
The study emphasizes the importance of continuous monitoring and validation of AI models in healthcare and calls for further research into the integration of AI chatbots into clinical practice.
For more information on this study, please refer to the full paper published in SAGE Digital Health.
Dr. Nasser Mulhim
Self-employed
email us here
1 https://journals.sagepub.com/doi/10.1177/20552076241253523
