A pilot study assessing the accuracy of AI ChatGPT responses for AL amyloidosis Journal Article


Authors: Saba, L.; Comenzo, R.; Khouri, J.; Anwer, F.; Landau, H.; Chaulagain, C.
Article Title: A pilot study assessing the accuracy of AI ChatGPT responses for AL amyloidosis
Abstract: AL amyloidosis is a rare, complex and often challenging disease for both patients and healthcare providers. The availability of accurate medical information is crucial for effective diagnosis and management. In recent years, artificial intelligence (AI) has emerged as a potential tool for providing medical information. This study aims to assess the accuracy of AI ChatGPT responses for AL amyloidosis related common questions and compare them to expert opinions. A scoring system was developed to evaluate responses provided by five participating expert physicians. AI ChatGPT demonstrated an overall accuracy rate of 82% in answering AL amyloidosis-related questions. Responses on prognosis and patient support received the highest scores (100%), while questions related to treatment options showed lower accuracy (30%–60%). The results indicate that while the AI ChatGPT demonstrates overall accuracy, there are areas for improvement and potential discrepancies compared to expert opinions. These findings highlight the importance of ongoing refinement and validation of AI-powered medical tools and cannot yet replace the advice of experts in the disease. © 2024 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.
Keywords: controlled study; diagnostic accuracy; oncology; patient care; medical information; pilot study; artificial intelligence; scoring system; health care personnel; hematology; search engine; diagnostic test accuracy study; natural language processing; al amyloidosis; prognosis; human; article; chatgpt
Journal Title: European Journal of Haematology
Volume: 114
Issue: 3
ISSN: 0902-4441
Publisher: John Wiley & Sons  
Date Published: 2025-03-01
Start Page: 495
End Page: 499
Language: English
DOI: 10.1111/ejh.14347
PROVIDER: scopus
DOI/URL:
Notes: Article -- Source: Scopus
Altmetric
Citation Impact
BMJ Impact Analytics
MSK Authors
  1. Heather Jolie Landau
    419 Landau