Machine learning natural language processing for identifying venous thromboembolism: Systematic review and meta-analysis Review
| Authors: | Lam, B. D.; Chrysafi, P.; Chiasakul, T.; Khosla, H.; Karagkouni, D.; McNichol, M.; Adamski, A.; Reyes, N.; Abe, K.; Mantha, S.; Vlachos, I. S.; Zwicker, J. I.; Patell, R. |
| Review Title: | Machine learning natural language processing for identifying venous thromboembolism: Systematic review and meta-analysis |
| Abstract: | Venous thromboembolism (VTE) is a leading cause of preventable in -hospital mortality. Monitoring VTE cases is limited by the challenges of manual medical record review and diagnosis code interpretation. Natural language processing (NLP) can automate the process. Rule -based NLP methods are effective but time consuming. Machine learning (ML)-NLP methods present a promising solution. We conducted a systematic review and metaanalysis of studies published before May 2023 that use ML-NLP to identify VTE diagnoses in the electronic health records. Four reviewers screened all manuscripts, excluding studies that only used a rule -based method. A meta -analysis evaluated the pooled performance of each study 's best performing model that evaluated for pulmonary embolism and/or deep vein thrombosis. Pooled sensitivity, speci ficity, positive predictive value (PPV), and negative predictive value (NPV) with con fidence interval (CI) were calculated by DerSimonian and Laird method using a random -effects model. Study quality was assessed using an adapted TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) tool. Thirteen studies were included in the systematic review and 8 had data available for meta -analysis. Pooled sensitivity was 0.931 (95% CI, 0.881-0.962), speci ficity 0.984 (95% CI, 0.967-0.992), PPV 0.910 (95% CI, 0.865-0.941) and NPV 0.985 (95% CI, 0.977-0.990). All studies met at least 13 of the 21 NLP-modi fied TRIPOD items, demonstrating fair quality. The highest performing models used vectorization rather than bag -of -words and deep -learning techniques such as convolutional neural networks. There was signi ficant heterogeneity in the studies, and only 4 validated their model on an external data set. Further standardization of ML studies can help progress this novel technology toward real -world implementation. |
| Keywords: | risks; benefits; limits; ai chatbot; gpt-4 |
| Journal Title: | Blood Advances |
| Volume: | 8 |
| Issue: | 12 |
| ISSN: | 2473-9529 |
| Publisher: | American Society of Hematology |
| Date Published: | 2024-06-25 |
| Start Page: | 2991 |
| End Page: | 3000 |
| Language: | English |
| ACCESSION: | WOS:001255170100001 |
| DOI: | 10.1182/bloodadvances.2023012200 |
| PROVIDER: | wos |
| PMCID: | PMC11215191 |
| PUBMED: | 38522096 |
| Notes: | Review -- MSK Cancer Center Support Grant (P30 CA008748) acknowledged in PDF -- Source: Wos |
Altmetric
Citation Impact
BMJ Impact Analytics
Related MSK Work