Prospective validation of dermoscopy-based open-source artificial intelligence for melanoma diagnosis (PROVE-AI study) Journal Article

Authors: Marchetti, M. A.; Cowen, E. A.; Kurtansky, N. R.; Weber, J.; Dauscher, M.; DeFazio, J.; Deng, L.; Dusza, S. W.; Haliasos, H.; Halpern, A. C.; Hosein, S.; Nazir, Z. H.; Marghoob, A. A.; Quigley, E. A.; Salvador, T.; Rotemberg, V. M.
Article Title: Prospective validation of dermoscopy-based open-source artificial intelligence for melanoma diagnosis (PROVE-AI study)
Abstract: The use of artificial intelligence (AI) has the potential to improve the assessment of lesions suspicious of melanoma, but few clinical studies have been conducted. We validated the accuracy of an open-source, non-commercial AI algorithm for melanoma diagnosis and assessed its potential impact on dermatologist decision-making. We conducted a prospective, observational clinical study to assess the diagnostic accuracy of the AI algorithm (ADAE) in predicting melanoma from dermoscopy skin lesion images. The primary aim was to assess the reliability of ADAE’s sensitivity at a predefined threshold of 95%. Patients who had consented for a skin biopsy to exclude melanoma were eligible. Dermatologists also estimated the probability of melanoma and indicated management choices before and after real-time exposure to ADAE scores. All lesions underwent biopsy. Four hundred thirty-five participants were enrolled and contributed 603 lesions (95 melanomas). Participants had a mean age of 59 years, 54% were female, and 96% were White individuals. At the predetermined 95% sensitivity threshold, ADAE had a sensitivity of 96.8% (95% CI: 91.1–98.9%) and specificity of 37.4% (95% CI: 33.3–41.7%). The dermatologists’ ability to assess melanoma risk significantly improved after ADAE exposure (AUC 0.7798 vs. 0.8161, p = 0.042). Post-ADAE dermatologist decisions also had equivalent or higher net benefit compared to biopsying all lesions. We validated the accuracy of an open-source melanoma AI algorithm and showed its theoretical potential for improving dermatology experts’ ability to evaluate lesions suspicious of melanoma. Larger randomized trials are needed to fully evaluate the potential of adopting this AI algorithm into clinical workflows. © 2023, The Author(s).
Keywords: diagnostic accuracy; dermoscopy; oncology; biopsy; risk assessment; clinical study; artificial intelligence; decision making; dermatology; skin biopsies; potential impacts; prospectives; open-source; artificial intelligence algorithms; decisions makings; skin lesion images
Journal Title: npj Digital Medicine
Volume: 6
ISSN: 2398-6352
Publisher: Nature Publishing Group  
Date Published: 2023-07-12
Start Page: 127
Language: English
DOI: 10.1038/s41746-023-00872-1
PROVIDER: scopus
PMCID: PMC10338483
PUBMED: 37438476
Notes: Article -- MSK Cancer Center Support Grant (P30 CA008748) acknowledged in PubMed and PDF -- MSK corresponding author is Veronica Rotemberg -- Source: Scopus
Citation Impact
MSK Authors
  1. Elizabeth Ann Quigley
    19 Quigley
  2. Allan C Halpern
    385 Halpern
  3. Jennifer Defazio
    15 Defazio
  4. Stephen Dusza
    255 Dusza
  5. Liang Deng
    78 Deng
  6. Ashfaq A Marghoob
    507 Marghoob
  7. Michael Armando Marchetti
    149 Marchetti
  8. Jochen Weber
    12 Weber
  9. Emily Ann Cowen
    8 Cowen
  10. Zaeem Nazir
    8 Nazir
  11. Sharif Hosein
    2 Hosein