The ISMRM Open Science Initiative for Perfusion Imaging (OSIPI): Results from the OSIPI–dynamic contrast-enhanced challenge Journal Article
| Authors: | Shalom, E. S.; Kim, H.; van der Heijden, R. A.; Ahmed, Z.; Patel, R.; Hormuth, D. A. 2nd; DiCarlo, J. C.; Yankeelov, T. E.; Sisco, N. J.; Dortch, R. D.; Stokes, A. M.; Inglese, M.; Grech-Sollars, M.; Toschi, N.; Sahoo, P.; Singh, A.; Verma, S. K.; Rathore, D. K.; Kazerouni, A. S.; Partridge, S. C.; LoCastro, E.; Paudyal, R.; Wolansky, I. A.; Shukla-Dave, A.; Schouten, P.; Gurney-Champion, O. J.; Jiřík, R.; Macíček, O.; Bartoš, M.; Vitouš, J.; Das, A. B.; Kim, S. G.; Bokacheva, L.; Mikheev, A.; Rusinek, H.; Berks, M.; Hubbard Cristinacce, P. L.; Little, R. A.; Cheung, S.; O'Connor, J. P. B.; Parker, G. J. M.; Moloney, B.; LaViolette, P. S.; Bobholz, S.; Duenweg, S.; Virostko, J.; Laue, H. O.; Sung, K.; Nabavizadeh, A.; Saligheh Rad, H.; Hu, L. S.; Sourbron, S.; Bell, L. C.; Fathi Kazerooni, A. |
| Article Title: | The ISMRM Open Science Initiative for Perfusion Imaging (OSIPI): Results from the OSIPI–dynamic contrast-enhanced challenge |
| Abstract: | Purpose: (Formula presented.) has often been proposed as a quantitative imaging biomarker for diagnosis, prognosis, and treatment response assessment for various tumors. None of the many software tools for (Formula presented.) quantification are standardized. The ISMRM Open Science Initiative for Perfusion Imaging–Dynamic Contrast-Enhanced (OSIPI-DCE) challenge was designed to benchmark methods to better help the efforts to standardize (Formula presented.) measurement. Methods: A framework was created to evaluate (Formula presented.) values produced by DCE-MRI analysis pipelines to enable benchmarking. The perfusion MRI community was invited to apply their pipelines for (Formula presented.) quantification in glioblastoma from clinical and synthetic patients. Submissions were required to include the entrants' (Formula presented.) values, the applied software, and a standard operating procedure. These were evaluated using the proposed (Formula presented.) score defined with accuracy, repeatability, and reproducibility components. Results: Across the 10 received submissions, the (Formula presented.) score ranged from 28% to 78% with a 59% median. The accuracy, repeatability, and reproducibility scores ranged from 0.54 to 0.92, 0.64 to 0.86, and 0.65 to 1.00, respectively (0–1 = lowest–highest). Manual arterial input function selection markedly affected the reproducibility and showed greater variability in (Formula presented.) analysis than automated methods. Furthermore, provision of a detailed standard operating procedure was critical for higher reproducibility. Conclusions: This study reports results from the OSIPI-DCE challenge and highlights the high inter-software variability within (Formula presented.) estimation, providing a framework for ongoing benchmarking against the scores presented. Through this challenge, the participating teams were ranked based on the performance of their software tools in the particular setting of this challenge. In a real-world clinical setting, many of these tools may perform differently with different benchmarking methodology. © 2023 International Society for Magnetic Resonance in Medicine. |
| Keywords: | nuclear magnetic resonance imaging; magnetic resonance imaging; reproducibility; accuracy; reproducibility of results; algorithms; algorithm; sequence alignment; brain artery; glioblastoma; diagnosis; data analysis; benchmarking; contrast medium; contrast media; gadolinium pentetate; software; dynamic contrast-enhanced magnetic resonance imaging; perfusion weighted imaging; dce-mri; perfusion; anatomical variation; clinical outcome; information; contrast-enhanced ultrasound; glioblastomas; procedures; challenge; computer software; perfusion imaging; open access; dynamic contrast-enhanced; plasma volume; humans; human; article; standard operating procedures; measurement repeatability; reproducibilities; neuroradiologist; pipelines; open science; patient registry; dynamic contrast enhanced imaging; pythonidae; open-science; software-tools; denoising autoencoder; magnitude estimation method; open science initiative for perfusion imaging; rate constant |
| Journal Title: | Magnetic Resonance in Medicine |
| Volume: | 91 |
| Issue: | 5 |
| ISSN: | 0740-3194 |
| Publisher: | John Wiley & Sons |
| Date Published: | 2024-05-01 |
| Start Page: | 1803 |
| End Page: | 1821 |
| Language: | English |
| DOI: | 10.1002/mrm.29909 |
| PUBMED: | 38115695 |
| PROVIDER: | scopus |
| DOI/URL: | |
| Notes: | Article -- Source: Scopus |
Altmetric
Citation Impact
BMJ Impact Analytics
Related MSK Work