Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist
Machine learning (ML) has been increasingly used within cardiology, particularly in the domain of cardiovascular imaging. Due to the inherent complexity and flexibility of ML algorithms, inconsistencies in the model performance and interpretation may occur. Several review articles have been recently...
Gespeichert in:
Veröffentlicht in: | JACC. Cardiovascular imaging 2020-09, Vol.13 (9), p.2017-2035 |
---|---|
Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 2035 |
---|---|
container_issue | 9 |
container_start_page | 2017 |
container_title | JACC. Cardiovascular imaging |
container_volume | 13 |
creator | Sengupta, Partho P. Shrestha, Sirish Berthon, Béatrice Messas, Emmanuel Donal, Erwan Tison, Geoffrey H. Min, James K. D’hooge, Jan Voigt, Jens-Uwe Dudley, Joel Verjans, Johan W. Shameer, Khader Johnson, Kipp Lovstakken, Lasse Tabassian, Mahdi Piccirilli, Marco Pernot, Mathieu Yanamala, Naveena Duchateau, Nicolas Kagiyama, Nobuyuki Bernard, Olivier Slomka, Piotr Deo, Rahul Arnaout, Rima |
description | Machine learning (ML) has been increasingly used within cardiology, particularly in the domain of cardiovascular imaging. Due to the inherent complexity and flexibility of ML algorithms, inconsistencies in the model performance and interpretation may occur. Several review articles have been recently published that introduce the fundamental principles and clinical application of ML for cardiologists. This paper builds on these introductory principles and outlines a more comprehensive list of crucial responsibilities that need to be completed when developing ML models. This paper aims to serve as a scientific foundation to aid investigators, data scientists, authors, editors, and reviewers involved in machine learning research with the intent of uniform reporting of ML investigations. An independent multidisciplinary panel of ML experts, clinicians, and statisticians worked together to review the theoretical rationale underlying 7 sets of requirements that may reduce algorithmic errors and biases. Finally, the paper summarizes a list of reporting items as an itemized checklist that highlights steps for ensuring correct application of ML models and the consistent reporting of model specifications and results. It is expected that the rapid pace of research and development and the increased availability of real-world evidence may require periodic updates to the checklist.
[Display omitted]
•Algorithm complexity and flexibility of ML techniques can result in inconsistencies in model reporting and interpretations.•The PRIME checklist provides 7 items to be reported for reducing algorithmic errors and biases.•The checklist aims to standardize reporting on model design, data, selection, assessment, evaluation, replicability, and limitations.•As artificial intelligence and ML technologies continue to grow, the checklist will need periodic updates. |
doi_str_mv | 10.1016/j.jcmg.2020.07.015 |
format | Article |
fullrecord | <record><control><sourceid>hal_cross</sourceid><recordid>TN_cdi_hal_primary_oai_HAL_hal_03019705v1</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S1936878X20306367</els_id><sourcerecordid>oai_HAL_hal_03019705v1</sourcerecordid><originalsourceid>FETCH-LOGICAL-c2235-b9c6ff0f2fae4f713e6f1b3e0d7cb792851e4ede90627bd83a567dfc4c2ebb2b3</originalsourceid><addsrcrecordid>eNp9kMFKw0AQhoMoWKsv4GmP9pA4u0l2E_FSSrWFFktR8LZsNrPtxjTR3bTg25tS8ehphpn_G5gvCG4pRBQov6-iSu82EQMGEYgIaHoWDGgmeCjSnJ73fR7zMBPZ-2Vw5X0FwIEnYhDYlWs_W48lWePX3jrcYdN5YlpHJsqVtj0or_e1cmS-UxvbbMI11qrr80ult7ZBskDlmn5BpgdV71Vn24bcrdbz5XT0QMZkskX9UVvfXQcXRtUeb37rMHh7mr5OZuHi5Xk-GS9CzVichkWuuTFgmFGYGEFj5IYWMUIpdCFylqUUEywxB85EUWaxSrkojU40w6JgRTwMRqe7W1XLT2d3yn3LVlk5Gy_kcQYx0FxAeqB9lp2y2rXeOzR_AAV5FCsreRQrj2IlCNmL7aHHE4T9FweLTnptsdFY9vp0J8vW_of_AN5Lgmc</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist</title><source>Elsevier ScienceDirect Journals</source><source>Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals</source><creator>Sengupta, Partho P. ; Shrestha, Sirish ; Berthon, Béatrice ; Messas, Emmanuel ; Donal, Erwan ; Tison, Geoffrey H. ; Min, James K. ; D’hooge, Jan ; Voigt, Jens-Uwe ; Dudley, Joel ; Verjans, Johan W. ; Shameer, Khader ; Johnson, Kipp ; Lovstakken, Lasse ; Tabassian, Mahdi ; Piccirilli, Marco ; Pernot, Mathieu ; Yanamala, Naveena ; Duchateau, Nicolas ; Kagiyama, Nobuyuki ; Bernard, Olivier ; Slomka, Piotr ; Deo, Rahul ; Arnaout, Rima</creator><creatorcontrib>Sengupta, Partho P. ; Shrestha, Sirish ; Berthon, Béatrice ; Messas, Emmanuel ; Donal, Erwan ; Tison, Geoffrey H. ; Min, James K. ; D’hooge, Jan ; Voigt, Jens-Uwe ; Dudley, Joel ; Verjans, Johan W. ; Shameer, Khader ; Johnson, Kipp ; Lovstakken, Lasse ; Tabassian, Mahdi ; Piccirilli, Marco ; Pernot, Mathieu ; Yanamala, Naveena ; Duchateau, Nicolas ; Kagiyama, Nobuyuki ; Bernard, Olivier ; Slomka, Piotr ; Deo, Rahul ; Arnaout, Rima</creatorcontrib><description>Machine learning (ML) has been increasingly used within cardiology, particularly in the domain of cardiovascular imaging. Due to the inherent complexity and flexibility of ML algorithms, inconsistencies in the model performance and interpretation may occur. Several review articles have been recently published that introduce the fundamental principles and clinical application of ML for cardiologists. This paper builds on these introductory principles and outlines a more comprehensive list of crucial responsibilities that need to be completed when developing ML models. This paper aims to serve as a scientific foundation to aid investigators, data scientists, authors, editors, and reviewers involved in machine learning research with the intent of uniform reporting of ML investigations. An independent multidisciplinary panel of ML experts, clinicians, and statisticians worked together to review the theoretical rationale underlying 7 sets of requirements that may reduce algorithmic errors and biases. Finally, the paper summarizes a list of reporting items as an itemized checklist that highlights steps for ensuring correct application of ML models and the consistent reporting of model specifications and results. It is expected that the rapid pace of research and development and the increased availability of real-world evidence may require periodic updates to the checklist.
[Display omitted]
•Algorithm complexity and flexibility of ML techniques can result in inconsistencies in model reporting and interpretations.•The PRIME checklist provides 7 items to be reported for reducing algorithmic errors and biases.•The checklist aims to standardize reporting on model design, data, selection, assessment, evaluation, replicability, and limitations.•As artificial intelligence and ML technologies continue to grow, the checklist will need periodic updates.</description><identifier>ISSN: 1936-878X</identifier><identifier>EISSN: 1876-7591</identifier><identifier>DOI: 10.1016/j.jcmg.2020.07.015</identifier><language>eng</language><publisher>Elsevier Inc</publisher><subject>artificial intelligence ; cardiovascular imaging ; checklist ; Computer Science ; digital health ; machine learning ; Medical Imaging ; reporting guidelines ; reproducible research</subject><ispartof>JACC. Cardiovascular imaging, 2020-09, Vol.13 (9), p.2017-2035</ispartof><rights>2020 American College of Cardiology Foundation</rights><rights>Distributed under a Creative Commons Attribution 4.0 International License</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c2235-b9c6ff0f2fae4f713e6f1b3e0d7cb792851e4ede90627bd83a567dfc4c2ebb2b3</citedby><cites>FETCH-LOGICAL-c2235-b9c6ff0f2fae4f713e6f1b3e0d7cb792851e4ede90627bd83a567dfc4c2ebb2b3</cites><orcidid>0000-0001-8803-2004 ; 0000-0003-0752-9946 ; 0000-0001-5882-8925 ; 0000-0003-2677-3389 ; 0000-0002-9083-1582</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.sciencedirect.com/science/article/pii/S1936878X20306367$$EHTML$$P50$$Gelsevier$$H</linktohtml><link.rule.ids>230,314,776,780,881,3537,27901,27902,65306</link.rule.ids><backlink>$$Uhttps://hal.science/hal-03019705$$DView record in HAL$$Hfree_for_read</backlink></links><search><creatorcontrib>Sengupta, Partho P.</creatorcontrib><creatorcontrib>Shrestha, Sirish</creatorcontrib><creatorcontrib>Berthon, Béatrice</creatorcontrib><creatorcontrib>Messas, Emmanuel</creatorcontrib><creatorcontrib>Donal, Erwan</creatorcontrib><creatorcontrib>Tison, Geoffrey H.</creatorcontrib><creatorcontrib>Min, James K.</creatorcontrib><creatorcontrib>D’hooge, Jan</creatorcontrib><creatorcontrib>Voigt, Jens-Uwe</creatorcontrib><creatorcontrib>Dudley, Joel</creatorcontrib><creatorcontrib>Verjans, Johan W.</creatorcontrib><creatorcontrib>Shameer, Khader</creatorcontrib><creatorcontrib>Johnson, Kipp</creatorcontrib><creatorcontrib>Lovstakken, Lasse</creatorcontrib><creatorcontrib>Tabassian, Mahdi</creatorcontrib><creatorcontrib>Piccirilli, Marco</creatorcontrib><creatorcontrib>Pernot, Mathieu</creatorcontrib><creatorcontrib>Yanamala, Naveena</creatorcontrib><creatorcontrib>Duchateau, Nicolas</creatorcontrib><creatorcontrib>Kagiyama, Nobuyuki</creatorcontrib><creatorcontrib>Bernard, Olivier</creatorcontrib><creatorcontrib>Slomka, Piotr</creatorcontrib><creatorcontrib>Deo, Rahul</creatorcontrib><creatorcontrib>Arnaout, Rima</creatorcontrib><title>Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist</title><title>JACC. Cardiovascular imaging</title><description>Machine learning (ML) has been increasingly used within cardiology, particularly in the domain of cardiovascular imaging. Due to the inherent complexity and flexibility of ML algorithms, inconsistencies in the model performance and interpretation may occur. Several review articles have been recently published that introduce the fundamental principles and clinical application of ML for cardiologists. This paper builds on these introductory principles and outlines a more comprehensive list of crucial responsibilities that need to be completed when developing ML models. This paper aims to serve as a scientific foundation to aid investigators, data scientists, authors, editors, and reviewers involved in machine learning research with the intent of uniform reporting of ML investigations. An independent multidisciplinary panel of ML experts, clinicians, and statisticians worked together to review the theoretical rationale underlying 7 sets of requirements that may reduce algorithmic errors and biases. Finally, the paper summarizes a list of reporting items as an itemized checklist that highlights steps for ensuring correct application of ML models and the consistent reporting of model specifications and results. It is expected that the rapid pace of research and development and the increased availability of real-world evidence may require periodic updates to the checklist.
[Display omitted]
•Algorithm complexity and flexibility of ML techniques can result in inconsistencies in model reporting and interpretations.•The PRIME checklist provides 7 items to be reported for reducing algorithmic errors and biases.•The checklist aims to standardize reporting on model design, data, selection, assessment, evaluation, replicability, and limitations.•As artificial intelligence and ML technologies continue to grow, the checklist will need periodic updates.</description><subject>artificial intelligence</subject><subject>cardiovascular imaging</subject><subject>checklist</subject><subject>Computer Science</subject><subject>digital health</subject><subject>machine learning</subject><subject>Medical Imaging</subject><subject>reporting guidelines</subject><subject>reproducible research</subject><issn>1936-878X</issn><issn>1876-7591</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><recordid>eNp9kMFKw0AQhoMoWKsv4GmP9pA4u0l2E_FSSrWFFktR8LZsNrPtxjTR3bTg25tS8ehphpn_G5gvCG4pRBQov6-iSu82EQMGEYgIaHoWDGgmeCjSnJ73fR7zMBPZ-2Vw5X0FwIEnYhDYlWs_W48lWePX3jrcYdN5YlpHJsqVtj0or_e1cmS-UxvbbMI11qrr80ult7ZBskDlmn5BpgdV71Vn24bcrdbz5XT0QMZkskX9UVvfXQcXRtUeb37rMHh7mr5OZuHi5Xk-GS9CzVichkWuuTFgmFGYGEFj5IYWMUIpdCFylqUUEywxB85EUWaxSrkojU40w6JgRTwMRqe7W1XLT2d3yn3LVlk5Gy_kcQYx0FxAeqB9lp2y2rXeOzR_AAV5FCsreRQrj2IlCNmL7aHHE4T9FweLTnptsdFY9vp0J8vW_of_AN5Lgmc</recordid><startdate>202009</startdate><enddate>202009</enddate><creator>Sengupta, Partho P.</creator><creator>Shrestha, Sirish</creator><creator>Berthon, Béatrice</creator><creator>Messas, Emmanuel</creator><creator>Donal, Erwan</creator><creator>Tison, Geoffrey H.</creator><creator>Min, James K.</creator><creator>D’hooge, Jan</creator><creator>Voigt, Jens-Uwe</creator><creator>Dudley, Joel</creator><creator>Verjans, Johan W.</creator><creator>Shameer, Khader</creator><creator>Johnson, Kipp</creator><creator>Lovstakken, Lasse</creator><creator>Tabassian, Mahdi</creator><creator>Piccirilli, Marco</creator><creator>Pernot, Mathieu</creator><creator>Yanamala, Naveena</creator><creator>Duchateau, Nicolas</creator><creator>Kagiyama, Nobuyuki</creator><creator>Bernard, Olivier</creator><creator>Slomka, Piotr</creator><creator>Deo, Rahul</creator><creator>Arnaout, Rima</creator><general>Elsevier Inc</general><general>Elsevier/American College of Cardiology</general><scope>AAYXX</scope><scope>CITATION</scope><scope>1XC</scope><scope>VOOES</scope><orcidid>https://orcid.org/0000-0001-8803-2004</orcidid><orcidid>https://orcid.org/0000-0003-0752-9946</orcidid><orcidid>https://orcid.org/0000-0001-5882-8925</orcidid><orcidid>https://orcid.org/0000-0003-2677-3389</orcidid><orcidid>https://orcid.org/0000-0002-9083-1582</orcidid></search><sort><creationdate>202009</creationdate><title>Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist</title><author>Sengupta, Partho P. ; Shrestha, Sirish ; Berthon, Béatrice ; Messas, Emmanuel ; Donal, Erwan ; Tison, Geoffrey H. ; Min, James K. ; D’hooge, Jan ; Voigt, Jens-Uwe ; Dudley, Joel ; Verjans, Johan W. ; Shameer, Khader ; Johnson, Kipp ; Lovstakken, Lasse ; Tabassian, Mahdi ; Piccirilli, Marco ; Pernot, Mathieu ; Yanamala, Naveena ; Duchateau, Nicolas ; Kagiyama, Nobuyuki ; Bernard, Olivier ; Slomka, Piotr ; Deo, Rahul ; Arnaout, Rima</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c2235-b9c6ff0f2fae4f713e6f1b3e0d7cb792851e4ede90627bd83a567dfc4c2ebb2b3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>artificial intelligence</topic><topic>cardiovascular imaging</topic><topic>checklist</topic><topic>Computer Science</topic><topic>digital health</topic><topic>machine learning</topic><topic>Medical Imaging</topic><topic>reporting guidelines</topic><topic>reproducible research</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Sengupta, Partho P.</creatorcontrib><creatorcontrib>Shrestha, Sirish</creatorcontrib><creatorcontrib>Berthon, Béatrice</creatorcontrib><creatorcontrib>Messas, Emmanuel</creatorcontrib><creatorcontrib>Donal, Erwan</creatorcontrib><creatorcontrib>Tison, Geoffrey H.</creatorcontrib><creatorcontrib>Min, James K.</creatorcontrib><creatorcontrib>D’hooge, Jan</creatorcontrib><creatorcontrib>Voigt, Jens-Uwe</creatorcontrib><creatorcontrib>Dudley, Joel</creatorcontrib><creatorcontrib>Verjans, Johan W.</creatorcontrib><creatorcontrib>Shameer, Khader</creatorcontrib><creatorcontrib>Johnson, Kipp</creatorcontrib><creatorcontrib>Lovstakken, Lasse</creatorcontrib><creatorcontrib>Tabassian, Mahdi</creatorcontrib><creatorcontrib>Piccirilli, Marco</creatorcontrib><creatorcontrib>Pernot, Mathieu</creatorcontrib><creatorcontrib>Yanamala, Naveena</creatorcontrib><creatorcontrib>Duchateau, Nicolas</creatorcontrib><creatorcontrib>Kagiyama, Nobuyuki</creatorcontrib><creatorcontrib>Bernard, Olivier</creatorcontrib><creatorcontrib>Slomka, Piotr</creatorcontrib><creatorcontrib>Deo, Rahul</creatorcontrib><creatorcontrib>Arnaout, Rima</creatorcontrib><collection>CrossRef</collection><collection>Hyper Article en Ligne (HAL)</collection><collection>Hyper Article en Ligne (HAL) (Open Access)</collection><jtitle>JACC. Cardiovascular imaging</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Sengupta, Partho P.</au><au>Shrestha, Sirish</au><au>Berthon, Béatrice</au><au>Messas, Emmanuel</au><au>Donal, Erwan</au><au>Tison, Geoffrey H.</au><au>Min, James K.</au><au>D’hooge, Jan</au><au>Voigt, Jens-Uwe</au><au>Dudley, Joel</au><au>Verjans, Johan W.</au><au>Shameer, Khader</au><au>Johnson, Kipp</au><au>Lovstakken, Lasse</au><au>Tabassian, Mahdi</au><au>Piccirilli, Marco</au><au>Pernot, Mathieu</au><au>Yanamala, Naveena</au><au>Duchateau, Nicolas</au><au>Kagiyama, Nobuyuki</au><au>Bernard, Olivier</au><au>Slomka, Piotr</au><au>Deo, Rahul</au><au>Arnaout, Rima</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist</atitle><jtitle>JACC. Cardiovascular imaging</jtitle><date>2020-09</date><risdate>2020</risdate><volume>13</volume><issue>9</issue><spage>2017</spage><epage>2035</epage><pages>2017-2035</pages><issn>1936-878X</issn><eissn>1876-7591</eissn><abstract>Machine learning (ML) has been increasingly used within cardiology, particularly in the domain of cardiovascular imaging. Due to the inherent complexity and flexibility of ML algorithms, inconsistencies in the model performance and interpretation may occur. Several review articles have been recently published that introduce the fundamental principles and clinical application of ML for cardiologists. This paper builds on these introductory principles and outlines a more comprehensive list of crucial responsibilities that need to be completed when developing ML models. This paper aims to serve as a scientific foundation to aid investigators, data scientists, authors, editors, and reviewers involved in machine learning research with the intent of uniform reporting of ML investigations. An independent multidisciplinary panel of ML experts, clinicians, and statisticians worked together to review the theoretical rationale underlying 7 sets of requirements that may reduce algorithmic errors and biases. Finally, the paper summarizes a list of reporting items as an itemized checklist that highlights steps for ensuring correct application of ML models and the consistent reporting of model specifications and results. It is expected that the rapid pace of research and development and the increased availability of real-world evidence may require periodic updates to the checklist.
[Display omitted]
•Algorithm complexity and flexibility of ML techniques can result in inconsistencies in model reporting and interpretations.•The PRIME checklist provides 7 items to be reported for reducing algorithmic errors and biases.•The checklist aims to standardize reporting on model design, data, selection, assessment, evaluation, replicability, and limitations.•As artificial intelligence and ML technologies continue to grow, the checklist will need periodic updates.</abstract><pub>Elsevier Inc</pub><doi>10.1016/j.jcmg.2020.07.015</doi><tpages>19</tpages><orcidid>https://orcid.org/0000-0001-8803-2004</orcidid><orcidid>https://orcid.org/0000-0003-0752-9946</orcidid><orcidid>https://orcid.org/0000-0001-5882-8925</orcidid><orcidid>https://orcid.org/0000-0003-2677-3389</orcidid><orcidid>https://orcid.org/0000-0002-9083-1582</orcidid><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | ISSN: 1936-878X |
ispartof | JACC. Cardiovascular imaging, 2020-09, Vol.13 (9), p.2017-2035 |
issn | 1936-878X 1876-7591 |
language | eng |
recordid | cdi_hal_primary_oai_HAL_hal_03019705v1 |
source | Elsevier ScienceDirect Journals; Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals |
subjects | artificial intelligence cardiovascular imaging checklist Computer Science digital health machine learning Medical Imaging reporting guidelines reproducible research |
title | Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-13T02%3A32%3A05IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-hal_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Proposed%20Requirements%20for%20Cardiovascular%20Imaging-Related%20Machine%20Learning%20Evaluation%20(PRIME):%20A%20Checklist&rft.jtitle=JACC.%20Cardiovascular%20imaging&rft.au=Sengupta,%20Partho%20P.&rft.date=2020-09&rft.volume=13&rft.issue=9&rft.spage=2017&rft.epage=2035&rft.pages=2017-2035&rft.issn=1936-878X&rft.eissn=1876-7591&rft_id=info:doi/10.1016/j.jcmg.2020.07.015&rft_dat=%3Chal_cross%3Eoai_HAL_hal_03019705v1%3C/hal_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rft_els_id=S1936878X20306367&rfr_iscdi=true |