Development and validation of a supervised deep learning algorithm for automated whole‐slide programmed death‐ligand 1 tumour proportion score assessment in non‐small cell lung cancer
Aims Immunohistochemical programmed death‐ligand 1 (PD‐L1) staining to predict responsiveness to immunotherapy in patients with advanced non‐small cell lung cancer (NSCLC) has several drawbacks: a robust gold standard is lacking, and there is substantial interobserver and intraobserver variance, wit...
Gespeichert in:
| Veröffentlicht in: | Histopathology 2022-03, Vol.80 (4), p.635-647 |
|---|---|
| Hauptverfasser: | , , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 647 |
|---|---|
| container_issue | 4 |
| container_start_page | 635 |
| container_title | Histopathology |
| container_volume | 80 |
| creator | Hondelink, Liesbeth M Hüyük, Melek Postmus, Pieter E Smit, Vincent T H B M Blom, Sami Thüsen, Jan H Cohen, Danielle |
| description | Aims
Immunohistochemical programmed death‐ligand 1 (PD‐L1) staining to predict responsiveness to immunotherapy in patients with advanced non‐small cell lung cancer (NSCLC) has several drawbacks: a robust gold standard is lacking, and there is substantial interobserver and intraobserver variance, with up to 20% discordance around cutoff points. The aim of this study was to develop a new deep learning‐based PD‐L1 tumour proportion score (TPS) algorithm, trained and validated on a routine diagnostic dataset of digitised PD‐L1 (22C3, laboratory‐developed test)‐stained samples.
Methods and results
We designed a fully supervised deep learning algorithm for whole‐slide PD‐L1 assessment, consisting of four sequential convolutional neural networks (CNNs), using aiforia create software. We included 199 whole slide images (WSIs) of ‘routine diagnostic’ histology samples from stage IV NSCLC patients, and trained the algorithm by using a training set of 60 representative cases. We validated the algorithm by comparing the algorithm TPS with the reference score in a held‐out validation set. The algorithm had similar concordance with the reference score (79%) as the pathologists had with one another (75%). The intraclass coefficient was 0.96 and Cohen’s κ coefficient was 0.69 for the algorithm. Around the 1% and 50% cutoff points, concordance was also similar between pathologists and the algorithm.
Conclusions
We designed a new, deep learning‐based PD‐L1 TPS algorithm that is similarly able to assess PD‐L1 expression in daily routine diagnostic cases as pathologists. Successful validation on routine diagnostic WSIs and detailed visual feedback show that this algorithm meets the requirements for functioning as a ‘scoring assistant’. |
| doi_str_mv | 10.1111/his.14571 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_pubme</sourceid><recordid>TN_cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_9299490</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2598533037</sourcerecordid><originalsourceid>FETCH-LOGICAL-c4431-a0abd9bccf515e967d8d2f7b0ec7d0dbcb7558fb92634a7dcca8349bc86162c43</originalsourceid><addsrcrecordid>eNp1ks9u1DAQxiMEokvhwAsgS1zgkNaO4zi-IFXlTytV4gCcrYkz2XXl2MFOtuqNR-CFeBmeBO9uqQAJH-zD_Oabb6yvKJ4zesLyOd3YdMJqIdmDYsV4I8pKCPWwWFFOVUlZI4-KJyldU8okr6rHxRGvZdvIhq2KH29xiy5MI_qZgO_JFpztYbbBkzAQIGmZMG5twp70iBNxCNFbvybg1iHaeTOSIUQCyxxGmDN1swkOf377nrIOkimGdYRx3LfDvMkFZ9e7QYzMyxiWuEOmEPcTkwkRCaSEKe0dWU988Du1EZwjBvPlljzdgDcYnxaPBnAJn929x8WX9-8-n1-UVx8_XJ6fXZWmrjkrgULXq86YQTCBqpF921eD7Cga2dO-M50Uoh06VTW8BtkbAy2vc0PbsKYyNT8u3hx0p6XLq5hsLYLTU7QjxFsdwOq_K95u9DpstaqUqhXNAq_uBGL4umCa9WjTbhvwGJakK6FawTnlMqMv_0Gv8y_5vJ7O9qjkSgqVqdcHysSQUsTh3gyjehcKnUOh96HI7Is_3d-Tv1OQgdMDcGMd3v5fSV9cfjpI_gLQ-cro</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2630739759</pqid></control><display><type>article</type><title>Development and validation of a supervised deep learning algorithm for automated whole‐slide programmed death‐ligand 1 tumour proportion score assessment in non‐small cell lung cancer</title><source>MEDLINE</source><source>Wiley Online Library All Journals</source><creator>Hondelink, Liesbeth M ; Hüyük, Melek ; Postmus, Pieter E ; Smit, Vincent T H B M ; Blom, Sami ; Thüsen, Jan H ; Cohen, Danielle</creator><creatorcontrib>Hondelink, Liesbeth M ; Hüyük, Melek ; Postmus, Pieter E ; Smit, Vincent T H B M ; Blom, Sami ; Thüsen, Jan H ; Cohen, Danielle</creatorcontrib><description>Aims
Immunohistochemical programmed death‐ligand 1 (PD‐L1) staining to predict responsiveness to immunotherapy in patients with advanced non‐small cell lung cancer (NSCLC) has several drawbacks: a robust gold standard is lacking, and there is substantial interobserver and intraobserver variance, with up to 20% discordance around cutoff points. The aim of this study was to develop a new deep learning‐based PD‐L1 tumour proportion score (TPS) algorithm, trained and validated on a routine diagnostic dataset of digitised PD‐L1 (22C3, laboratory‐developed test)‐stained samples.
Methods and results
We designed a fully supervised deep learning algorithm for whole‐slide PD‐L1 assessment, consisting of four sequential convolutional neural networks (CNNs), using aiforia create software. We included 199 whole slide images (WSIs) of ‘routine diagnostic’ histology samples from stage IV NSCLC patients, and trained the algorithm by using a training set of 60 representative cases. We validated the algorithm by comparing the algorithm TPS with the reference score in a held‐out validation set. The algorithm had similar concordance with the reference score (79%) as the pathologists had with one another (75%). The intraclass coefficient was 0.96 and Cohen’s κ coefficient was 0.69 for the algorithm. Around the 1% and 50% cutoff points, concordance was also similar between pathologists and the algorithm.
Conclusions
We designed a new, deep learning‐based PD‐L1 TPS algorithm that is similarly able to assess PD‐L1 expression in daily routine diagnostic cases as pathologists. Successful validation on routine diagnostic WSIs and detailed visual feedback show that this algorithm meets the requirements for functioning as a ‘scoring assistant’.</description><identifier>ISSN: 0309-0167</identifier><identifier>EISSN: 1365-2559</identifier><identifier>DOI: 10.1111/his.14571</identifier><identifier>PMID: 34786761</identifier><language>eng</language><publisher>England: Wiley Subscription Services, Inc</publisher><subject>Aged ; Aged, 80 and over ; Algorithms ; Apoptosis ; artificial intelligence ; B7-H1 Antigen - analysis ; Carcinoma, Non-Small-Cell Lung - chemistry ; computational pathology ; Deep Learning ; Discordance ; Female ; Humans ; Immunotherapy ; Ligands ; Lung cancer ; Lung Neoplasms - chemistry ; Male ; Middle Aged ; Neural networks ; Non-small cell lung carcinoma ; non‐small cell lung cancer ; Original ; PD-L1 protein ; programmed death‐ligand 1 ; Small cell lung carcinoma ; Tumors ; Visual perception</subject><ispartof>Histopathology, 2022-03, Vol.80 (4), p.635-647</ispartof><rights>2021 The Authors. published by John Wiley & Sons Ltd</rights><rights>2021 The Authors. Histopathology published by John Wiley & Sons Ltd.</rights><rights>2021. This article is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c4431-a0abd9bccf515e967d8d2f7b0ec7d0dbcb7558fb92634a7dcca8349bc86162c43</citedby><cites>FETCH-LOGICAL-c4431-a0abd9bccf515e967d8d2f7b0ec7d0dbcb7558fb92634a7dcca8349bc86162c43</cites><orcidid>0000-0002-7173-7443</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://onlinelibrary.wiley.com/doi/pdf/10.1111%2Fhis.14571$$EPDF$$P50$$Gwiley$$Hfree_for_read</linktopdf><linktohtml>$$Uhttps://onlinelibrary.wiley.com/doi/full/10.1111%2Fhis.14571$$EHTML$$P50$$Gwiley$$Hfree_for_read</linktohtml><link.rule.ids>230,314,776,780,881,1411,27903,27904,45553,45554</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/34786761$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Hondelink, Liesbeth M</creatorcontrib><creatorcontrib>Hüyük, Melek</creatorcontrib><creatorcontrib>Postmus, Pieter E</creatorcontrib><creatorcontrib>Smit, Vincent T H B M</creatorcontrib><creatorcontrib>Blom, Sami</creatorcontrib><creatorcontrib>Thüsen, Jan H</creatorcontrib><creatorcontrib>Cohen, Danielle</creatorcontrib><title>Development and validation of a supervised deep learning algorithm for automated whole‐slide programmed death‐ligand 1 tumour proportion score assessment in non‐small cell lung cancer</title><title>Histopathology</title><addtitle>Histopathology</addtitle><description>Aims
Immunohistochemical programmed death‐ligand 1 (PD‐L1) staining to predict responsiveness to immunotherapy in patients with advanced non‐small cell lung cancer (NSCLC) has several drawbacks: a robust gold standard is lacking, and there is substantial interobserver and intraobserver variance, with up to 20% discordance around cutoff points. The aim of this study was to develop a new deep learning‐based PD‐L1 tumour proportion score (TPS) algorithm, trained and validated on a routine diagnostic dataset of digitised PD‐L1 (22C3, laboratory‐developed test)‐stained samples.
Methods and results
We designed a fully supervised deep learning algorithm for whole‐slide PD‐L1 assessment, consisting of four sequential convolutional neural networks (CNNs), using aiforia create software. We included 199 whole slide images (WSIs) of ‘routine diagnostic’ histology samples from stage IV NSCLC patients, and trained the algorithm by using a training set of 60 representative cases. We validated the algorithm by comparing the algorithm TPS with the reference score in a held‐out validation set. The algorithm had similar concordance with the reference score (79%) as the pathologists had with one another (75%). The intraclass coefficient was 0.96 and Cohen’s κ coefficient was 0.69 for the algorithm. Around the 1% and 50% cutoff points, concordance was also similar between pathologists and the algorithm.
Conclusions
We designed a new, deep learning‐based PD‐L1 TPS algorithm that is similarly able to assess PD‐L1 expression in daily routine diagnostic cases as pathologists. Successful validation on routine diagnostic WSIs and detailed visual feedback show that this algorithm meets the requirements for functioning as a ‘scoring assistant’.</description><subject>Aged</subject><subject>Aged, 80 and over</subject><subject>Algorithms</subject><subject>Apoptosis</subject><subject>artificial intelligence</subject><subject>B7-H1 Antigen - analysis</subject><subject>Carcinoma, Non-Small-Cell Lung - chemistry</subject><subject>computational pathology</subject><subject>Deep Learning</subject><subject>Discordance</subject><subject>Female</subject><subject>Humans</subject><subject>Immunotherapy</subject><subject>Ligands</subject><subject>Lung cancer</subject><subject>Lung Neoplasms - chemistry</subject><subject>Male</subject><subject>Middle Aged</subject><subject>Neural networks</subject><subject>Non-small cell lung carcinoma</subject><subject>non‐small cell lung cancer</subject><subject>Original</subject><subject>PD-L1 protein</subject><subject>programmed death‐ligand 1</subject><subject>Small cell lung carcinoma</subject><subject>Tumors</subject><subject>Visual perception</subject><issn>0309-0167</issn><issn>1365-2559</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><sourceid>24P</sourceid><sourceid>EIF</sourceid><recordid>eNp1ks9u1DAQxiMEokvhwAsgS1zgkNaO4zi-IFXlTytV4gCcrYkz2XXl2MFOtuqNR-CFeBmeBO9uqQAJH-zD_Oabb6yvKJ4zesLyOd3YdMJqIdmDYsV4I8pKCPWwWFFOVUlZI4-KJyldU8okr6rHxRGvZdvIhq2KH29xiy5MI_qZgO_JFpztYbbBkzAQIGmZMG5twp70iBNxCNFbvybg1iHaeTOSIUQCyxxGmDN1swkOf377nrIOkimGdYRx3LfDvMkFZ9e7QYzMyxiWuEOmEPcTkwkRCaSEKe0dWU988Du1EZwjBvPlljzdgDcYnxaPBnAJn929x8WX9-8-n1-UVx8_XJ6fXZWmrjkrgULXq86YQTCBqpF921eD7Cga2dO-M50Uoh06VTW8BtkbAy2vc0PbsKYyNT8u3hx0p6XLq5hsLYLTU7QjxFsdwOq_K95u9DpstaqUqhXNAq_uBGL4umCa9WjTbhvwGJakK6FawTnlMqMv_0Gv8y_5vJ7O9qjkSgqVqdcHysSQUsTh3gyjehcKnUOh96HI7Is_3d-Tv1OQgdMDcGMd3v5fSV9cfjpI_gLQ-cro</recordid><startdate>202203</startdate><enddate>202203</enddate><creator>Hondelink, Liesbeth M</creator><creator>Hüyük, Melek</creator><creator>Postmus, Pieter E</creator><creator>Smit, Vincent T H B M</creator><creator>Blom, Sami</creator><creator>Thüsen, Jan H</creator><creator>Cohen, Danielle</creator><general>Wiley Subscription Services, Inc</general><general>John Wiley and Sons Inc</general><scope>24P</scope><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7QP</scope><scope>7QR</scope><scope>7T5</scope><scope>7TK</scope><scope>7TM</scope><scope>8FD</scope><scope>FR3</scope><scope>H94</scope><scope>P64</scope><scope>RC3</scope><scope>7X8</scope><scope>5PM</scope><orcidid>https://orcid.org/0000-0002-7173-7443</orcidid></search><sort><creationdate>202203</creationdate><title>Development and validation of a supervised deep learning algorithm for automated whole‐slide programmed death‐ligand 1 tumour proportion score assessment in non‐small cell lung cancer</title><author>Hondelink, Liesbeth M ; Hüyük, Melek ; Postmus, Pieter E ; Smit, Vincent T H B M ; Blom, Sami ; Thüsen, Jan H ; Cohen, Danielle</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c4431-a0abd9bccf515e967d8d2f7b0ec7d0dbcb7558fb92634a7dcca8349bc86162c43</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Aged</topic><topic>Aged, 80 and over</topic><topic>Algorithms</topic><topic>Apoptosis</topic><topic>artificial intelligence</topic><topic>B7-H1 Antigen - analysis</topic><topic>Carcinoma, Non-Small-Cell Lung - chemistry</topic><topic>computational pathology</topic><topic>Deep Learning</topic><topic>Discordance</topic><topic>Female</topic><topic>Humans</topic><topic>Immunotherapy</topic><topic>Ligands</topic><topic>Lung cancer</topic><topic>Lung Neoplasms - chemistry</topic><topic>Male</topic><topic>Middle Aged</topic><topic>Neural networks</topic><topic>Non-small cell lung carcinoma</topic><topic>non‐small cell lung cancer</topic><topic>Original</topic><topic>PD-L1 protein</topic><topic>programmed death‐ligand 1</topic><topic>Small cell lung carcinoma</topic><topic>Tumors</topic><topic>Visual perception</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Hondelink, Liesbeth M</creatorcontrib><creatorcontrib>Hüyük, Melek</creatorcontrib><creatorcontrib>Postmus, Pieter E</creatorcontrib><creatorcontrib>Smit, Vincent T H B M</creatorcontrib><creatorcontrib>Blom, Sami</creatorcontrib><creatorcontrib>Thüsen, Jan H</creatorcontrib><creatorcontrib>Cohen, Danielle</creatorcontrib><collection>Wiley Online Library</collection><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>Calcium & Calcified Tissue Abstracts</collection><collection>Chemoreception Abstracts</collection><collection>Immunology Abstracts</collection><collection>Neurosciences Abstracts</collection><collection>Nucleic Acids Abstracts</collection><collection>Technology Research Database</collection><collection>Engineering Research Database</collection><collection>AIDS and Cancer Research Abstracts</collection><collection>Biotechnology and BioEngineering Abstracts</collection><collection>Genetics Abstracts</collection><collection>MEDLINE - Academic</collection><collection>PubMed Central (Full Participant titles)</collection><jtitle>Histopathology</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Hondelink, Liesbeth M</au><au>Hüyük, Melek</au><au>Postmus, Pieter E</au><au>Smit, Vincent T H B M</au><au>Blom, Sami</au><au>Thüsen, Jan H</au><au>Cohen, Danielle</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Development and validation of a supervised deep learning algorithm for automated whole‐slide programmed death‐ligand 1 tumour proportion score assessment in non‐small cell lung cancer</atitle><jtitle>Histopathology</jtitle><addtitle>Histopathology</addtitle><date>2022-03</date><risdate>2022</risdate><volume>80</volume><issue>4</issue><spage>635</spage><epage>647</epage><pages>635-647</pages><issn>0309-0167</issn><eissn>1365-2559</eissn><abstract>Aims
Immunohistochemical programmed death‐ligand 1 (PD‐L1) staining to predict responsiveness to immunotherapy in patients with advanced non‐small cell lung cancer (NSCLC) has several drawbacks: a robust gold standard is lacking, and there is substantial interobserver and intraobserver variance, with up to 20% discordance around cutoff points. The aim of this study was to develop a new deep learning‐based PD‐L1 tumour proportion score (TPS) algorithm, trained and validated on a routine diagnostic dataset of digitised PD‐L1 (22C3, laboratory‐developed test)‐stained samples.
Methods and results
We designed a fully supervised deep learning algorithm for whole‐slide PD‐L1 assessment, consisting of four sequential convolutional neural networks (CNNs), using aiforia create software. We included 199 whole slide images (WSIs) of ‘routine diagnostic’ histology samples from stage IV NSCLC patients, and trained the algorithm by using a training set of 60 representative cases. We validated the algorithm by comparing the algorithm TPS with the reference score in a held‐out validation set. The algorithm had similar concordance with the reference score (79%) as the pathologists had with one another (75%). The intraclass coefficient was 0.96 and Cohen’s κ coefficient was 0.69 for the algorithm. Around the 1% and 50% cutoff points, concordance was also similar between pathologists and the algorithm.
Conclusions
We designed a new, deep learning‐based PD‐L1 TPS algorithm that is similarly able to assess PD‐L1 expression in daily routine diagnostic cases as pathologists. Successful validation on routine diagnostic WSIs and detailed visual feedback show that this algorithm meets the requirements for functioning as a ‘scoring assistant’.</abstract><cop>England</cop><pub>Wiley Subscription Services, Inc</pub><pmid>34786761</pmid><doi>10.1111/his.14571</doi><tpages>13</tpages><orcidid>https://orcid.org/0000-0002-7173-7443</orcidid><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0309-0167 |
| ispartof | Histopathology, 2022-03, Vol.80 (4), p.635-647 |
| issn | 0309-0167 1365-2559 |
| language | eng |
| recordid | cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_9299490 |
| source | MEDLINE; Wiley Online Library All Journals |
| subjects | Aged Aged, 80 and over Algorithms Apoptosis artificial intelligence B7-H1 Antigen - analysis Carcinoma, Non-Small-Cell Lung - chemistry computational pathology Deep Learning Discordance Female Humans Immunotherapy Ligands Lung cancer Lung Neoplasms - chemistry Male Middle Aged Neural networks Non-small cell lung carcinoma non‐small cell lung cancer Original PD-L1 protein programmed death‐ligand 1 Small cell lung carcinoma Tumors Visual perception |
| title | Development and validation of a supervised deep learning algorithm for automated whole‐slide programmed death‐ligand 1 tumour proportion score assessment in non‐small cell lung cancer |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-27T21%3A25%3A06IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_pubme&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Development%20and%20validation%20of%20a%20supervised%20deep%20learning%20algorithm%20for%20automated%20whole%E2%80%90slide%20programmed%20death%E2%80%90ligand%201%20tumour%20proportion%20score%20assessment%20in%20non%E2%80%90small%20cell%20lung%20cancer&rft.jtitle=Histopathology&rft.au=Hondelink,%20Liesbeth%20M&rft.date=2022-03&rft.volume=80&rft.issue=4&rft.spage=635&rft.epage=647&rft.pages=635-647&rft.issn=0309-0167&rft.eissn=1365-2559&rft_id=info:doi/10.1111/his.14571&rft_dat=%3Cproquest_pubme%3E2598533037%3C/proquest_pubme%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2630739759&rft_id=info:pmid/34786761&rfr_iscdi=true |