RadBERT-CL: Factually-Aware Contrastive Learning For Radiology Report Classification
Radiology reports are unstructured and contain the imaging findings and corresponding diagnoses transcribed by radiologists which include clinical facts and negated and/or uncertain statements. Extracting pathologic findings and diagnoses from radiology reports is important for quality control, popu...
Gespeichert in:
Veröffentlicht in: | arXiv.org 2021-11 |
---|---|
Hauptverfasser: | , , , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | |
---|---|
container_issue | |
container_start_page | |
container_title | arXiv.org |
container_volume | |
creator | Jaiswal, Ajay Tang, Liyan Ghosh, Meheli Rousseau, Justin Peng, Yifan Ding, Ying |
description | Radiology reports are unstructured and contain the imaging findings and corresponding diagnoses transcribed by radiologists which include clinical facts and negated and/or uncertain statements. Extracting pathologic findings and diagnoses from radiology reports is important for quality control, population health, and monitoring of disease progress. Existing works, primarily rely either on rule-based systems or transformer-based pre-trained model fine-tuning, but could not take the factual and uncertain information into consideration, and therefore generate false-positive outputs. In this work, we introduce three sedulous augmentation techniques which retain factual and critical information while generating augmentations for contrastive learning. We introduce RadBERT-CL, which fuses these information into BlueBert via a self-supervised contrastive loss. Our experiments on MIMIC-CXR show superior performance of RadBERT-CL on fine-tuning for multi-class, multi-label report classification. We illustrate that when few labeled data are available, RadBERT-CL outperforms conventional SOTA transformers (BERT/BlueBert) by significantly larger margins (6-11%). We also show that the representations learned by RadBERT-CL can capture critical medical information in the latent space. |
format | Article |
fullrecord | <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_2590781684</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2590781684</sourcerecordid><originalsourceid>FETCH-proquest_journals_25907816843</originalsourceid><addsrcrecordid>eNqNi08LgjAcQEcQJOV3GHQezPm3bjWUDp7Eu_ywKZOx2TYLv30e-gCd3uG9t0MBi-OIFAljBxQ6N1FKWZazNI0D1DbwvJdNS3h9xRX0fgGlVnL7gBWYG-0tOC_fAtcCrJZ6xJWxeJukUWZccSNmYz3mCpyTg-zBS6NPaD-AciL88YjOVdnyB5mteS3C-W4yi9Wb6lh6oXkRZUUS_1d9AfH_P9Q</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2590781684</pqid></control><display><type>article</type><title>RadBERT-CL: Factually-Aware Contrastive Learning For Radiology Report Classification</title><source>Free E- Journals</source><creator>Jaiswal, Ajay ; Tang, Liyan ; Ghosh, Meheli ; Rousseau, Justin ; Peng, Yifan ; Ding, Ying</creator><creatorcontrib>Jaiswal, Ajay ; Tang, Liyan ; Ghosh, Meheli ; Rousseau, Justin ; Peng, Yifan ; Ding, Ying</creatorcontrib><description>Radiology reports are unstructured and contain the imaging findings and corresponding diagnoses transcribed by radiologists which include clinical facts and negated and/or uncertain statements. Extracting pathologic findings and diagnoses from radiology reports is important for quality control, population health, and monitoring of disease progress. Existing works, primarily rely either on rule-based systems or transformer-based pre-trained model fine-tuning, but could not take the factual and uncertain information into consideration, and therefore generate false-positive outputs. In this work, we introduce three sedulous augmentation techniques which retain factual and critical information while generating augmentations for contrastive learning. We introduce RadBERT-CL, which fuses these information into BlueBert via a self-supervised contrastive loss. Our experiments on MIMIC-CXR show superior performance of RadBERT-CL on fine-tuning for multi-class, multi-label report classification. We illustrate that when few labeled data are available, RadBERT-CL outperforms conventional SOTA transformers (BERT/BlueBert) by significantly larger margins (6-11%). We also show that the representations learned by RadBERT-CL can capture critical medical information in the latent space.</description><identifier>EISSN: 2331-8422</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Classification ; Learning ; Quality control ; Radiology ; Transformers</subject><ispartof>arXiv.org, 2021-11</ispartof><rights>2021. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>780,784</link.rule.ids></links><search><creatorcontrib>Jaiswal, Ajay</creatorcontrib><creatorcontrib>Tang, Liyan</creatorcontrib><creatorcontrib>Ghosh, Meheli</creatorcontrib><creatorcontrib>Rousseau, Justin</creatorcontrib><creatorcontrib>Peng, Yifan</creatorcontrib><creatorcontrib>Ding, Ying</creatorcontrib><title>RadBERT-CL: Factually-Aware Contrastive Learning For Radiology Report Classification</title><title>arXiv.org</title><description>Radiology reports are unstructured and contain the imaging findings and corresponding diagnoses transcribed by radiologists which include clinical facts and negated and/or uncertain statements. Extracting pathologic findings and diagnoses from radiology reports is important for quality control, population health, and monitoring of disease progress. Existing works, primarily rely either on rule-based systems or transformer-based pre-trained model fine-tuning, but could not take the factual and uncertain information into consideration, and therefore generate false-positive outputs. In this work, we introduce three sedulous augmentation techniques which retain factual and critical information while generating augmentations for contrastive learning. We introduce RadBERT-CL, which fuses these information into BlueBert via a self-supervised contrastive loss. Our experiments on MIMIC-CXR show superior performance of RadBERT-CL on fine-tuning for multi-class, multi-label report classification. We illustrate that when few labeled data are available, RadBERT-CL outperforms conventional SOTA transformers (BERT/BlueBert) by significantly larger margins (6-11%). We also show that the representations learned by RadBERT-CL can capture critical medical information in the latent space.</description><subject>Classification</subject><subject>Learning</subject><subject>Quality control</subject><subject>Radiology</subject><subject>Transformers</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><recordid>eNqNi08LgjAcQEcQJOV3GHQezPm3bjWUDp7Eu_ywKZOx2TYLv30e-gCd3uG9t0MBi-OIFAljBxQ6N1FKWZazNI0D1DbwvJdNS3h9xRX0fgGlVnL7gBWYG-0tOC_fAtcCrJZ6xJWxeJukUWZccSNmYz3mCpyTg-zBS6NPaD-AciL88YjOVdnyB5mteS3C-W4yi9Wb6lh6oXkRZUUS_1d9AfH_P9Q</recordid><startdate>20211119</startdate><enddate>20211119</enddate><creator>Jaiswal, Ajay</creator><creator>Tang, Liyan</creator><creator>Ghosh, Meheli</creator><creator>Rousseau, Justin</creator><creator>Peng, Yifan</creator><creator>Ding, Ying</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20211119</creationdate><title>RadBERT-CL: Factually-Aware Contrastive Learning For Radiology Report Classification</title><author>Jaiswal, Ajay ; Tang, Liyan ; Ghosh, Meheli ; Rousseau, Justin ; Peng, Yifan ; Ding, Ying</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-proquest_journals_25907816843</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Classification</topic><topic>Learning</topic><topic>Quality control</topic><topic>Radiology</topic><topic>Transformers</topic><toplevel>online_resources</toplevel><creatorcontrib>Jaiswal, Ajay</creatorcontrib><creatorcontrib>Tang, Liyan</creatorcontrib><creatorcontrib>Ghosh, Meheli</creatorcontrib><creatorcontrib>Rousseau, Justin</creatorcontrib><creatorcontrib>Peng, Yifan</creatorcontrib><creatorcontrib>Ding, Ying</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science & Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Access via ProQuest (Open Access)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Jaiswal, Ajay</au><au>Tang, Liyan</au><au>Ghosh, Meheli</au><au>Rousseau, Justin</au><au>Peng, Yifan</au><au>Ding, Ying</au><format>book</format><genre>document</genre><ristype>GEN</ristype><atitle>RadBERT-CL: Factually-Aware Contrastive Learning For Radiology Report Classification</atitle><jtitle>arXiv.org</jtitle><date>2021-11-19</date><risdate>2021</risdate><eissn>2331-8422</eissn><abstract>Radiology reports are unstructured and contain the imaging findings and corresponding diagnoses transcribed by radiologists which include clinical facts and negated and/or uncertain statements. Extracting pathologic findings and diagnoses from radiology reports is important for quality control, population health, and monitoring of disease progress. Existing works, primarily rely either on rule-based systems or transformer-based pre-trained model fine-tuning, but could not take the factual and uncertain information into consideration, and therefore generate false-positive outputs. In this work, we introduce three sedulous augmentation techniques which retain factual and critical information while generating augmentations for contrastive learning. We introduce RadBERT-CL, which fuses these information into BlueBert via a self-supervised contrastive loss. Our experiments on MIMIC-CXR show superior performance of RadBERT-CL on fine-tuning for multi-class, multi-label report classification. We illustrate that when few labeled data are available, RadBERT-CL outperforms conventional SOTA transformers (BERT/BlueBert) by significantly larger margins (6-11%). We also show that the representations learned by RadBERT-CL can capture critical medical information in the latent space.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | EISSN: 2331-8422 |
ispartof | arXiv.org, 2021-11 |
issn | 2331-8422 |
language | eng |
recordid | cdi_proquest_journals_2590781684 |
source | Free E- Journals |
subjects | Classification Learning Quality control Radiology Transformers |
title | RadBERT-CL: Factually-Aware Contrastive Learning For Radiology Report Classification |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-04T03%3A29%3A44IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=document&rft.atitle=RadBERT-CL:%20Factually-Aware%20Contrastive%20Learning%20For%20Radiology%20Report%20Classification&rft.jtitle=arXiv.org&rft.au=Jaiswal,%20Ajay&rft.date=2021-11-19&rft.eissn=2331-8422&rft_id=info:doi/&rft_dat=%3Cproquest%3E2590781684%3C/proquest%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2590781684&rft_id=info:pmid/&rfr_iscdi=true |