Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform
Objective To investigate the effects of Image Biomarker Standardisation Initiative (IBSI) compliance, harmonisation of calculation settings and platform version on the statistical reliability of radiomic features and their corresponding ability to predict clinical outcome. Methods The statistical re...
Gespeichert in:
| Veröffentlicht in: | European radiology 2020-11, Vol.30 (11), p.6241-6250 |
|---|---|
| Hauptverfasser: | , , , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 6250 |
|---|---|
| container_issue | 11 |
| container_start_page | 6241 |
| container_title | European radiology |
| container_volume | 30 |
| creator | Fornacon-Wood, Isabella Mistry, Hitesh Ackermann, Christoph J. Blackhall, Fiona McPartlin, Andrew Faivre-Finn, Corinne Price, Gareth J. O’Connor, James P. B. |
| description | Objective
To investigate the effects of Image Biomarker Standardisation Initiative (IBSI) compliance, harmonisation of calculation settings and platform version on the statistical reliability of radiomic features and their corresponding ability to predict clinical outcome.
Methods
The statistical reliability of radiomic features was assessed retrospectively in three clinical datasets (patient numbers: 108 head and neck cancer, 37 small-cell lung cancer, 47 non-small-cell lung cancer). Features were calculated using four platforms (PyRadiomics, LIFEx, CERR and IBEX). PyRadiomics, LIFEx and CERR are IBSI-compliant, whereas IBEX is not. The effects of IBSI compliance, user-defined calculation settings and platform version were assessed by calculating intraclass correlation coefficients and confidence intervals. The influence of platform choice on the relationship between radiomic biomarkers and survival was evaluated using univariable cox regression in the largest dataset.
Results
The reliability of radiomic features calculated by the different software platforms was only excellent (ICC > 0.9) for 4/17 radiomic features when comparing all four platforms. Reliability improved to ICC > 0.9 for 15/17 radiomic features when analysis was restricted to the three IBSI-compliant platforms. Failure to harmonise calculation settings resulted in poor reliability, even across the IBSI-compliant platforms. Software platform version also had a marked effect on feature reliability in CERR and LIFEx. Features identified as having significant relationship to survival varied between platforms, as did the direction of hazard ratios.
Conclusion
IBSI compliance, user-defined calculation settings and choice of platform version all influence the statistical reliability and corresponding performance of prognostic models in radiomics.
Key Points
• Reliability of radiomic features varies between feature calculation platforms and with choice of software version.
• Image Biomarker Standardisation Initiative (IBSI) compliance improves reliability of radiomic features across platforms, but only when calculation settings are harmonised.
• IBSI compliance, user-defined calculation settings and choice of platform version collectively affect the prognostic value of features. |
| doi_str_mv | 10.1007/s00330-020-06957-9 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_pubme</sourceid><recordid>TN_cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_7553896</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2408843644</sourcerecordid><originalsourceid>FETCH-LOGICAL-c560t-e91e73cd795cb2f810050117928dd0960fa9b63b62319a970853f8ea1a80f41b3</originalsourceid><addsrcrecordid>eNp9UUuLFDEQDqK44-of8CA5emmtdNKd5CLI4gsWBNFzSKerZ7KkkzbpXpx_b3ZnXPTioQjU96hKfYS8ZPCGAci3BYBzaKCt1etONvoR2THB24aBEo_JDjRXjdRaXJBnpdwAgGZCPiUXvBWK90LsSPmGwdvBB78eqY0jXXLax1RW7-itDRvSNNFsR5_m2pnQrlvGQm1GevD7QzjSEReMI8aVpkjdIXl3rzlTKf5as3Wrr-AS7DqlPD8nTyYbCr44v5fkx8cP368-N9dfP325en_duK6HtUHNUHI3St25oZ1U_XMHjEndqnEE3cNk9dDzoW8501ZLUB2fFFpmFUyCDfySvDv5Ltsw4-jqjtkGs2Q_23w0yXrzLxL9wezTrZFdx5Xuq8Hrs0FOPzcsq5l9cRiCjZi2YloBSom7Q1Zqe6K6nErJOD2MYWDu0jKntExNy9ynZXQVvfp7wQfJn3gqgZ8IpUJxj9ncpC3HerT_2f4GF6ii1w</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2408843644</pqid></control><display><type>article</type><title>Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform</title><source>MEDLINE</source><source>Springer Nature - Complete Springer Journals</source><creator>Fornacon-Wood, Isabella ; Mistry, Hitesh ; Ackermann, Christoph J. ; Blackhall, Fiona ; McPartlin, Andrew ; Faivre-Finn, Corinne ; Price, Gareth J. ; O’Connor, James P. B.</creator><creatorcontrib>Fornacon-Wood, Isabella ; Mistry, Hitesh ; Ackermann, Christoph J. ; Blackhall, Fiona ; McPartlin, Andrew ; Faivre-Finn, Corinne ; Price, Gareth J. ; O’Connor, James P. B.</creatorcontrib><description>Objective
To investigate the effects of Image Biomarker Standardisation Initiative (IBSI) compliance, harmonisation of calculation settings and platform version on the statistical reliability of radiomic features and their corresponding ability to predict clinical outcome.
Methods
The statistical reliability of radiomic features was assessed retrospectively in three clinical datasets (patient numbers: 108 head and neck cancer, 37 small-cell lung cancer, 47 non-small-cell lung cancer). Features were calculated using four platforms (PyRadiomics, LIFEx, CERR and IBEX). PyRadiomics, LIFEx and CERR are IBSI-compliant, whereas IBEX is not. The effects of IBSI compliance, user-defined calculation settings and platform version were assessed by calculating intraclass correlation coefficients and confidence intervals. The influence of platform choice on the relationship between radiomic biomarkers and survival was evaluated using univariable cox regression in the largest dataset.
Results
The reliability of radiomic features calculated by the different software platforms was only excellent (ICC > 0.9) for 4/17 radiomic features when comparing all four platforms. Reliability improved to ICC > 0.9 for 15/17 radiomic features when analysis was restricted to the three IBSI-compliant platforms. Failure to harmonise calculation settings resulted in poor reliability, even across the IBSI-compliant platforms. Software platform version also had a marked effect on feature reliability in CERR and LIFEx. Features identified as having significant relationship to survival varied between platforms, as did the direction of hazard ratios.
Conclusion
IBSI compliance, user-defined calculation settings and choice of platform version all influence the statistical reliability and corresponding performance of prognostic models in radiomics.
Key Points
• Reliability of radiomic features varies between feature calculation platforms and with choice of software version.
• Image Biomarker Standardisation Initiative (IBSI) compliance improves reliability of radiomic features across platforms, but only when calculation settings are harmonised.
• IBSI compliance, user-defined calculation settings and choice of platform version collectively affect the prognostic value of features.</description><identifier>ISSN: 0938-7994</identifier><identifier>EISSN: 1432-1084</identifier><identifier>DOI: 10.1007/s00330-020-06957-9</identifier><identifier>PMID: 32483644</identifier><language>eng</language><publisher>Berlin/Heidelberg: Springer Berlin Heidelberg</publisher><subject>Carcinoma, Non-Small-Cell Lung - diagnostic imaging ; Diagnostic Radiology ; Head and Neck Neoplasms - diagnostic imaging ; Humans ; Image Processing, Computer-Assisted - instrumentation ; Image Processing, Computer-Assisted - methods ; Imaging ; Imaging Informatics and Artificial Intelligence ; Internal Medicine ; Interventional Radiology ; Lung Neoplasms - diagnostic imaging ; Medicine ; Medicine & Public Health ; Neuroradiology ; Prognosis ; Proportional Hazards Models ; Radiology ; Reproducibility of Results ; Retrospective Studies ; Small Cell Lung Carcinoma - diagnostic imaging ; Software ; Tomography, X-Ray Computed ; Ultrasound</subject><ispartof>European radiology, 2020-11, Vol.30 (11), p.6241-6250</ispartof><rights>The Author(s) 2020</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c560t-e91e73cd795cb2f810050117928dd0960fa9b63b62319a970853f8ea1a80f41b3</citedby><cites>FETCH-LOGICAL-c560t-e91e73cd795cb2f810050117928dd0960fa9b63b62319a970853f8ea1a80f41b3</cites><orcidid>0000-0002-3736-2967</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://link.springer.com/content/pdf/10.1007/s00330-020-06957-9$$EPDF$$P50$$Gspringer$$Hfree_for_read</linktopdf><linktohtml>$$Uhttps://link.springer.com/10.1007/s00330-020-06957-9$$EHTML$$P50$$Gspringer$$Hfree_for_read</linktohtml><link.rule.ids>230,314,777,781,882,27905,27906,41469,42538,51300</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/32483644$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Fornacon-Wood, Isabella</creatorcontrib><creatorcontrib>Mistry, Hitesh</creatorcontrib><creatorcontrib>Ackermann, Christoph J.</creatorcontrib><creatorcontrib>Blackhall, Fiona</creatorcontrib><creatorcontrib>McPartlin, Andrew</creatorcontrib><creatorcontrib>Faivre-Finn, Corinne</creatorcontrib><creatorcontrib>Price, Gareth J.</creatorcontrib><creatorcontrib>O’Connor, James P. B.</creatorcontrib><title>Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform</title><title>European radiology</title><addtitle>Eur Radiol</addtitle><addtitle>Eur Radiol</addtitle><description>Objective
To investigate the effects of Image Biomarker Standardisation Initiative (IBSI) compliance, harmonisation of calculation settings and platform version on the statistical reliability of radiomic features and their corresponding ability to predict clinical outcome.
Methods
The statistical reliability of radiomic features was assessed retrospectively in three clinical datasets (patient numbers: 108 head and neck cancer, 37 small-cell lung cancer, 47 non-small-cell lung cancer). Features were calculated using four platforms (PyRadiomics, LIFEx, CERR and IBEX). PyRadiomics, LIFEx and CERR are IBSI-compliant, whereas IBEX is not. The effects of IBSI compliance, user-defined calculation settings and platform version were assessed by calculating intraclass correlation coefficients and confidence intervals. The influence of platform choice on the relationship between radiomic biomarkers and survival was evaluated using univariable cox regression in the largest dataset.
Results
The reliability of radiomic features calculated by the different software platforms was only excellent (ICC > 0.9) for 4/17 radiomic features when comparing all four platforms. Reliability improved to ICC > 0.9 for 15/17 radiomic features when analysis was restricted to the three IBSI-compliant platforms. Failure to harmonise calculation settings resulted in poor reliability, even across the IBSI-compliant platforms. Software platform version also had a marked effect on feature reliability in CERR and LIFEx. Features identified as having significant relationship to survival varied between platforms, as did the direction of hazard ratios.
Conclusion
IBSI compliance, user-defined calculation settings and choice of platform version all influence the statistical reliability and corresponding performance of prognostic models in radiomics.
Key Points
• Reliability of radiomic features varies between feature calculation platforms and with choice of software version.
• Image Biomarker Standardisation Initiative (IBSI) compliance improves reliability of radiomic features across platforms, but only when calculation settings are harmonised.
• IBSI compliance, user-defined calculation settings and choice of platform version collectively affect the prognostic value of features.</description><subject>Carcinoma, Non-Small-Cell Lung - diagnostic imaging</subject><subject>Diagnostic Radiology</subject><subject>Head and Neck Neoplasms - diagnostic imaging</subject><subject>Humans</subject><subject>Image Processing, Computer-Assisted - instrumentation</subject><subject>Image Processing, Computer-Assisted - methods</subject><subject>Imaging</subject><subject>Imaging Informatics and Artificial Intelligence</subject><subject>Internal Medicine</subject><subject>Interventional Radiology</subject><subject>Lung Neoplasms - diagnostic imaging</subject><subject>Medicine</subject><subject>Medicine & Public Health</subject><subject>Neuroradiology</subject><subject>Prognosis</subject><subject>Proportional Hazards Models</subject><subject>Radiology</subject><subject>Reproducibility of Results</subject><subject>Retrospective Studies</subject><subject>Small Cell Lung Carcinoma - diagnostic imaging</subject><subject>Software</subject><subject>Tomography, X-Ray Computed</subject><subject>Ultrasound</subject><issn>0938-7994</issn><issn>1432-1084</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><sourceid>C6C</sourceid><sourceid>EIF</sourceid><recordid>eNp9UUuLFDEQDqK44-of8CA5emmtdNKd5CLI4gsWBNFzSKerZ7KkkzbpXpx_b3ZnXPTioQjU96hKfYS8ZPCGAci3BYBzaKCt1etONvoR2THB24aBEo_JDjRXjdRaXJBnpdwAgGZCPiUXvBWK90LsSPmGwdvBB78eqY0jXXLax1RW7-itDRvSNNFsR5_m2pnQrlvGQm1GevD7QzjSEReMI8aVpkjdIXl3rzlTKf5as3Wrr-AS7DqlPD8nTyYbCr44v5fkx8cP368-N9dfP325en_duK6HtUHNUHI3St25oZ1U_XMHjEndqnEE3cNk9dDzoW8501ZLUB2fFFpmFUyCDfySvDv5Ltsw4-jqjtkGs2Q_23w0yXrzLxL9wezTrZFdx5Xuq8Hrs0FOPzcsq5l9cRiCjZi2YloBSom7Q1Zqe6K6nErJOD2MYWDu0jKntExNy9ynZXQVvfp7wQfJn3gqgZ8IpUJxj9ncpC3HerT_2f4GF6ii1w</recordid><startdate>20201101</startdate><enddate>20201101</enddate><creator>Fornacon-Wood, Isabella</creator><creator>Mistry, Hitesh</creator><creator>Ackermann, Christoph J.</creator><creator>Blackhall, Fiona</creator><creator>McPartlin, Andrew</creator><creator>Faivre-Finn, Corinne</creator><creator>Price, Gareth J.</creator><creator>O’Connor, James P. B.</creator><general>Springer Berlin Heidelberg</general><scope>C6C</scope><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7X8</scope><scope>5PM</scope><orcidid>https://orcid.org/0000-0002-3736-2967</orcidid></search><sort><creationdate>20201101</creationdate><title>Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform</title><author>Fornacon-Wood, Isabella ; Mistry, Hitesh ; Ackermann, Christoph J. ; Blackhall, Fiona ; McPartlin, Andrew ; Faivre-Finn, Corinne ; Price, Gareth J. ; O’Connor, James P. B.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c560t-e91e73cd795cb2f810050117928dd0960fa9b63b62319a970853f8ea1a80f41b3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>Carcinoma, Non-Small-Cell Lung - diagnostic imaging</topic><topic>Diagnostic Radiology</topic><topic>Head and Neck Neoplasms - diagnostic imaging</topic><topic>Humans</topic><topic>Image Processing, Computer-Assisted - instrumentation</topic><topic>Image Processing, Computer-Assisted - methods</topic><topic>Imaging</topic><topic>Imaging Informatics and Artificial Intelligence</topic><topic>Internal Medicine</topic><topic>Interventional Radiology</topic><topic>Lung Neoplasms - diagnostic imaging</topic><topic>Medicine</topic><topic>Medicine & Public Health</topic><topic>Neuroradiology</topic><topic>Prognosis</topic><topic>Proportional Hazards Models</topic><topic>Radiology</topic><topic>Reproducibility of Results</topic><topic>Retrospective Studies</topic><topic>Small Cell Lung Carcinoma - diagnostic imaging</topic><topic>Software</topic><topic>Tomography, X-Ray Computed</topic><topic>Ultrasound</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Fornacon-Wood, Isabella</creatorcontrib><creatorcontrib>Mistry, Hitesh</creatorcontrib><creatorcontrib>Ackermann, Christoph J.</creatorcontrib><creatorcontrib>Blackhall, Fiona</creatorcontrib><creatorcontrib>McPartlin, Andrew</creatorcontrib><creatorcontrib>Faivre-Finn, Corinne</creatorcontrib><creatorcontrib>Price, Gareth J.</creatorcontrib><creatorcontrib>O’Connor, James P. B.</creatorcontrib><collection>Springer Nature OA Free Journals</collection><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>MEDLINE - Academic</collection><collection>PubMed Central (Full Participant titles)</collection><jtitle>European radiology</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Fornacon-Wood, Isabella</au><au>Mistry, Hitesh</au><au>Ackermann, Christoph J.</au><au>Blackhall, Fiona</au><au>McPartlin, Andrew</au><au>Faivre-Finn, Corinne</au><au>Price, Gareth J.</au><au>O’Connor, James P. B.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform</atitle><jtitle>European radiology</jtitle><stitle>Eur Radiol</stitle><addtitle>Eur Radiol</addtitle><date>2020-11-01</date><risdate>2020</risdate><volume>30</volume><issue>11</issue><spage>6241</spage><epage>6250</epage><pages>6241-6250</pages><issn>0938-7994</issn><eissn>1432-1084</eissn><abstract>Objective
To investigate the effects of Image Biomarker Standardisation Initiative (IBSI) compliance, harmonisation of calculation settings and platform version on the statistical reliability of radiomic features and their corresponding ability to predict clinical outcome.
Methods
The statistical reliability of radiomic features was assessed retrospectively in three clinical datasets (patient numbers: 108 head and neck cancer, 37 small-cell lung cancer, 47 non-small-cell lung cancer). Features were calculated using four platforms (PyRadiomics, LIFEx, CERR and IBEX). PyRadiomics, LIFEx and CERR are IBSI-compliant, whereas IBEX is not. The effects of IBSI compliance, user-defined calculation settings and platform version were assessed by calculating intraclass correlation coefficients and confidence intervals. The influence of platform choice on the relationship between radiomic biomarkers and survival was evaluated using univariable cox regression in the largest dataset.
Results
The reliability of radiomic features calculated by the different software platforms was only excellent (ICC > 0.9) for 4/17 radiomic features when comparing all four platforms. Reliability improved to ICC > 0.9 for 15/17 radiomic features when analysis was restricted to the three IBSI-compliant platforms. Failure to harmonise calculation settings resulted in poor reliability, even across the IBSI-compliant platforms. Software platform version also had a marked effect on feature reliability in CERR and LIFEx. Features identified as having significant relationship to survival varied between platforms, as did the direction of hazard ratios.
Conclusion
IBSI compliance, user-defined calculation settings and choice of platform version all influence the statistical reliability and corresponding performance of prognostic models in radiomics.
Key Points
• Reliability of radiomic features varies between feature calculation platforms and with choice of software version.
• Image Biomarker Standardisation Initiative (IBSI) compliance improves reliability of radiomic features across platforms, but only when calculation settings are harmonised.
• IBSI compliance, user-defined calculation settings and choice of platform version collectively affect the prognostic value of features.</abstract><cop>Berlin/Heidelberg</cop><pub>Springer Berlin Heidelberg</pub><pmid>32483644</pmid><doi>10.1007/s00330-020-06957-9</doi><tpages>10</tpages><orcidid>https://orcid.org/0000-0002-3736-2967</orcidid><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0938-7994 |
| ispartof | European radiology, 2020-11, Vol.30 (11), p.6241-6250 |
| issn | 0938-7994 1432-1084 |
| language | eng |
| recordid | cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_7553896 |
| source | MEDLINE; Springer Nature - Complete Springer Journals |
| subjects | Carcinoma, Non-Small-Cell Lung - diagnostic imaging Diagnostic Radiology Head and Neck Neoplasms - diagnostic imaging Humans Image Processing, Computer-Assisted - instrumentation Image Processing, Computer-Assisted - methods Imaging Imaging Informatics and Artificial Intelligence Internal Medicine Interventional Radiology Lung Neoplasms - diagnostic imaging Medicine Medicine & Public Health Neuroradiology Prognosis Proportional Hazards Models Radiology Reproducibility of Results Retrospective Studies Small Cell Lung Carcinoma - diagnostic imaging Software Tomography, X-Ray Computed Ultrasound |
| title | Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-20T14%3A56%3A13IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_pubme&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Reliability%20and%20prognostic%20value%20of%20radiomic%20features%20are%20highly%20dependent%20on%20choice%20of%20feature%20extraction%20platform&rft.jtitle=European%20radiology&rft.au=Fornacon-Wood,%20Isabella&rft.date=2020-11-01&rft.volume=30&rft.issue=11&rft.spage=6241&rft.epage=6250&rft.pages=6241-6250&rft.issn=0938-7994&rft.eissn=1432-1084&rft_id=info:doi/10.1007/s00330-020-06957-9&rft_dat=%3Cproquest_pubme%3E2408843644%3C/proquest_pubme%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2408843644&rft_id=info:pmid/32483644&rfr_iscdi=true |