A dataset of clinically generated visual questions and answers about radiology images

Radiology images are an essential part of clinical decision making and population screening, e.g., for cancer. Automated systems could help clinicians cope with large amounts of images by answering questions about the image contents. An emerging area of artificial intelligence, Visual Question Answe...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Scientific data 2018-11, Vol.5 (1), p.180251-10, Article 180251
Hauptverfasser: Lau, Jason J., Gayen, Soumya, Ben Abacha, Asma, Demner-Fushman, Dina
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 10
container_issue 1
container_start_page 180251
container_title Scientific data
container_volume 5
creator Lau, Jason J.
Gayen, Soumya
Ben Abacha, Asma
Demner-Fushman, Dina
description Radiology images are an essential part of clinical decision making and population screening, e.g., for cancer. Automated systems could help clinicians cope with large amounts of images by answering questions about the image contents. An emerging area of artificial intelligence, Visual Question Answering (VQA) in the medical domain explores approaches to this form of clinical decision support. Success of such machine learning tools hinges on availability and design of collections composed of medical images augmented with question-answer pairs directed at the content of the image. We introduce VQA-RAD, the first manually constructed dataset where clinicians asked naturally occurring questions about radiology images and provided reference answers. Manual categorization of images and questions provides insight into clinically relevant tasks and the natural language to phrase them. Evaluating with well-known algorithms, we demonstrate the rich quality of this dataset over other automatically constructed ones. We propose VQA-RAD to encourage the community to design VQA tools with the goals of improving patient care. Design Type(s) image creation and editing objective • anatomical image analysis objective Measurement Type(s) image analysis Technology Type(s) visual observation method Factor Type(s) question type • answer type Sample Characteristic(s) Homo sapiens • head • chest • abdomen Machine-accessible metadata file describing the reported data (ISA-Tab format)
doi_str_mv 10.1038/sdata.2018.251
format Article
fullrecord <record><control><sourceid>proquest_pubme</sourceid><recordid>TN_cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_6244189</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2315956045</sourcerecordid><originalsourceid>FETCH-LOGICAL-c552t-de9f3b5303a0f30fc02381842ff92fac1a82a8715095c8aaccd9764473aefd0b3</originalsourceid><addsrcrecordid>eNp9kc1LXDEUxUNpqWLddlkCbrqZ8ebrfWwKIq0KQjd1He7kJa-RTGKT9yzz35uZUWuFdhFy4f5ycs89hHxksGQgutMy4IRLDqxbcsXekEMOii-kbMTbF_UBOS7lFgCYkKBaeE8OBEjVqkYdkpszuhUpdqLJURN89AZD2NDRRptxsgO992XGQH_Ntkw-xUIxDvWU3zbXepXmiWYcfApp3FC_xtGWD-Sdw1Ds8eN9RG6-ff1xfrm4_n5xdX52vTBK8Wkx2N6JlRIgEJwAZ4CLjnWSO9dzh4Zhx7FrmYJemQ7RmKFvGylbgdYNsBJH5Mte925ere1gbJwyBn2X6xh5oxN6_Xcn-p96TPe64VKyrq8Cnx8FctoZ1GtfjA0Bo01z0ZyJBpq64baiJ6_Q2zTnWO1pLpjqVVOX-l-KCdXwqscqtdxTJqdSsnXPIzPQ22j1Llq9jVbXaOuDTy-NPuNPQVbgdA-U2oqjzX_-_YfkA68Nr-A</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2135623601</pqid></control><display><type>article</type><title>A dataset of clinically generated visual questions and answers about radiology images</title><source>Nature Publishing Group website</source><source>MEDLINE</source><source>DOAJ Directory of Open Access Journals</source><source>PubMed Central Open Access</source><source>Springer Nature OA Free Journals</source><source>Free E-Journal (出版社公開部分のみ)</source><source>PubMed Central</source><creator>Lau, Jason J. ; Gayen, Soumya ; Ben Abacha, Asma ; Demner-Fushman, Dina</creator><creatorcontrib>Lau, Jason J. ; Gayen, Soumya ; Ben Abacha, Asma ; Demner-Fushman, Dina</creatorcontrib><description>Radiology images are an essential part of clinical decision making and population screening, e.g., for cancer. Automated systems could help clinicians cope with large amounts of images by answering questions about the image contents. An emerging area of artificial intelligence, Visual Question Answering (VQA) in the medical domain explores approaches to this form of clinical decision support. Success of such machine learning tools hinges on availability and design of collections composed of medical images augmented with question-answer pairs directed at the content of the image. We introduce VQA-RAD, the first manually constructed dataset where clinicians asked naturally occurring questions about radiology images and provided reference answers. Manual categorization of images and questions provides insight into clinically relevant tasks and the natural language to phrase them. Evaluating with well-known algorithms, we demonstrate the rich quality of this dataset over other automatically constructed ones. We propose VQA-RAD to encourage the community to design VQA tools with the goals of improving patient care. Design Type(s) image creation and editing objective • anatomical image analysis objective Measurement Type(s) image analysis Technology Type(s) visual observation method Factor Type(s) question type • answer type Sample Characteristic(s) Homo sapiens • head • chest • abdomen Machine-accessible metadata file describing the reported data (ISA-Tab format)</description><identifier>ISSN: 2052-4463</identifier><identifier>EISSN: 2052-4463</identifier><identifier>DOI: 10.1038/sdata.2018.251</identifier><identifier>PMID: 30457565</identifier><language>eng</language><publisher>London: Nature Publishing Group UK</publisher><subject>631/114/2164 ; 692/700/1421/1770 ; Algorithms ; Artificial intelligence ; Cancer ; Data Analysis ; Data collection ; Data Descriptor ; Data Mining ; Datasets ; Decision making ; Decision support systems ; Humanities and Social Sciences ; Humans ; Image processing ; Image Processing, Computer-Assisted - methods ; Learning algorithms ; Machine Learning ; multidisciplinary ; Radiography - methods ; Radiology Information Systems - classification ; Radiology Information Systems - standards ; Science</subject><ispartof>Scientific data, 2018-11, Vol.5 (1), p.180251-10, Article 180251</ispartof><rights>This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply 2018</rights><rights>Copyright Nature Publishing Group Nov 2018</rights><rights>2018. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><rights>Copyright © 2018, This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply 2018 This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c552t-de9f3b5303a0f30fc02381842ff92fac1a82a8715095c8aaccd9764473aefd0b3</citedby><cites>FETCH-LOGICAL-c552t-de9f3b5303a0f30fc02381842ff92fac1a82a8715095c8aaccd9764473aefd0b3</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC6244189/pdf/$$EPDF$$P50$$Gpubmedcentral$$Hfree_for_read</linktopdf><linktohtml>$$Uhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC6244189/$$EHTML$$P50$$Gpubmedcentral$$Hfree_for_read</linktohtml><link.rule.ids>230,314,727,780,784,864,885,27924,27925,41120,42189,51576,53791,53793</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/30457565$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Lau, Jason J.</creatorcontrib><creatorcontrib>Gayen, Soumya</creatorcontrib><creatorcontrib>Ben Abacha, Asma</creatorcontrib><creatorcontrib>Demner-Fushman, Dina</creatorcontrib><title>A dataset of clinically generated visual questions and answers about radiology images</title><title>Scientific data</title><addtitle>Sci Data</addtitle><addtitle>Sci Data</addtitle><description>Radiology images are an essential part of clinical decision making and population screening, e.g., for cancer. Automated systems could help clinicians cope with large amounts of images by answering questions about the image contents. An emerging area of artificial intelligence, Visual Question Answering (VQA) in the medical domain explores approaches to this form of clinical decision support. Success of such machine learning tools hinges on availability and design of collections composed of medical images augmented with question-answer pairs directed at the content of the image. We introduce VQA-RAD, the first manually constructed dataset where clinicians asked naturally occurring questions about radiology images and provided reference answers. Manual categorization of images and questions provides insight into clinically relevant tasks and the natural language to phrase them. Evaluating with well-known algorithms, we demonstrate the rich quality of this dataset over other automatically constructed ones. We propose VQA-RAD to encourage the community to design VQA tools with the goals of improving patient care. Design Type(s) image creation and editing objective • anatomical image analysis objective Measurement Type(s) image analysis Technology Type(s) visual observation method Factor Type(s) question type • answer type Sample Characteristic(s) Homo sapiens • head • chest • abdomen Machine-accessible metadata file describing the reported data (ISA-Tab format)</description><subject>631/114/2164</subject><subject>692/700/1421/1770</subject><subject>Algorithms</subject><subject>Artificial intelligence</subject><subject>Cancer</subject><subject>Data Analysis</subject><subject>Data collection</subject><subject>Data Descriptor</subject><subject>Data Mining</subject><subject>Datasets</subject><subject>Decision making</subject><subject>Decision support systems</subject><subject>Humanities and Social Sciences</subject><subject>Humans</subject><subject>Image processing</subject><subject>Image Processing, Computer-Assisted - methods</subject><subject>Learning algorithms</subject><subject>Machine Learning</subject><subject>multidisciplinary</subject><subject>Radiography - methods</subject><subject>Radiology Information Systems - classification</subject><subject>Radiology Information Systems - standards</subject><subject>Science</subject><issn>2052-4463</issn><issn>2052-4463</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2018</creationdate><recordtype>article</recordtype><sourceid>C6C</sourceid><sourceid>EIF</sourceid><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GNUQQ</sourceid><recordid>eNp9kc1LXDEUxUNpqWLddlkCbrqZ8ebrfWwKIq0KQjd1He7kJa-RTGKT9yzz35uZUWuFdhFy4f5ycs89hHxksGQgutMy4IRLDqxbcsXekEMOii-kbMTbF_UBOS7lFgCYkKBaeE8OBEjVqkYdkpszuhUpdqLJURN89AZD2NDRRptxsgO992XGQH_Ntkw-xUIxDvWU3zbXepXmiWYcfApp3FC_xtGWD-Sdw1Ds8eN9RG6-ff1xfrm4_n5xdX52vTBK8Wkx2N6JlRIgEJwAZ4CLjnWSO9dzh4Zhx7FrmYJemQ7RmKFvGylbgdYNsBJH5Mte925ere1gbJwyBn2X6xh5oxN6_Xcn-p96TPe64VKyrq8Cnx8FctoZ1GtfjA0Bo01z0ZyJBpq64baiJ6_Q2zTnWO1pLpjqVVOX-l-KCdXwqscqtdxTJqdSsnXPIzPQ22j1Llq9jVbXaOuDTy-NPuNPQVbgdA-U2oqjzX_-_YfkA68Nr-A</recordid><startdate>20181120</startdate><enddate>20181120</enddate><creator>Lau, Jason J.</creator><creator>Gayen, Soumya</creator><creator>Ben Abacha, Asma</creator><creator>Demner-Fushman, Dina</creator><general>Nature Publishing Group UK</general><general>Nature Publishing Group</general><scope>C6C</scope><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>3V.</scope><scope>7X7</scope><scope>7XB</scope><scope>88E</scope><scope>8FE</scope><scope>8FH</scope><scope>8FI</scope><scope>8FJ</scope><scope>8FK</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BBNVY</scope><scope>BENPR</scope><scope>BHPHI</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>FYUFA</scope><scope>GHDGH</scope><scope>GNUQQ</scope><scope>HCIFZ</scope><scope>K9.</scope><scope>LK8</scope><scope>M0S</scope><scope>M1P</scope><scope>M7P</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>7X8</scope><scope>5PM</scope></search><sort><creationdate>20181120</creationdate><title>A dataset of clinically generated visual questions and answers about radiology images</title><author>Lau, Jason J. ; Gayen, Soumya ; Ben Abacha, Asma ; Demner-Fushman, Dina</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c552t-de9f3b5303a0f30fc02381842ff92fac1a82a8715095c8aaccd9764473aefd0b3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2018</creationdate><topic>631/114/2164</topic><topic>692/700/1421/1770</topic><topic>Algorithms</topic><topic>Artificial intelligence</topic><topic>Cancer</topic><topic>Data Analysis</topic><topic>Data collection</topic><topic>Data Descriptor</topic><topic>Data Mining</topic><topic>Datasets</topic><topic>Decision making</topic><topic>Decision support systems</topic><topic>Humanities and Social Sciences</topic><topic>Humans</topic><topic>Image processing</topic><topic>Image Processing, Computer-Assisted - methods</topic><topic>Learning algorithms</topic><topic>Machine Learning</topic><topic>multidisciplinary</topic><topic>Radiography - methods</topic><topic>Radiology Information Systems - classification</topic><topic>Radiology Information Systems - standards</topic><topic>Science</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Lau, Jason J.</creatorcontrib><creatorcontrib>Gayen, Soumya</creatorcontrib><creatorcontrib>Ben Abacha, Asma</creatorcontrib><creatorcontrib>Demner-Fushman, Dina</creatorcontrib><collection>Springer Nature OA Free Journals</collection><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>ProQuest Central (Corporate)</collection><collection>ProQuest Health and Medical</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>Medical Database (Alumni Edition)</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Natural Science Collection</collection><collection>Hospital Premium Collection</collection><collection>Hospital Premium Collection (Alumni Edition)</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central</collection><collection>ProQuest Central</collection><collection>Biological Science Collection</collection><collection>ProQuest Central</collection><collection>Natural Science Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central</collection><collection>Health Research Premium Collection</collection><collection>Health Research Premium Collection (Alumni)</collection><collection>ProQuest Central Student</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Health &amp; Medical Complete (Alumni)</collection><collection>ProQuest Biological Science Collection</collection><collection>Health &amp; Medical Collection (Alumni Edition)</collection><collection>PML(ProQuest Medical Library)</collection><collection>Biological Science Database</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>MEDLINE - Academic</collection><collection>PubMed Central (Full Participant titles)</collection><jtitle>Scientific data</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Lau, Jason J.</au><au>Gayen, Soumya</au><au>Ben Abacha, Asma</au><au>Demner-Fushman, Dina</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>A dataset of clinically generated visual questions and answers about radiology images</atitle><jtitle>Scientific data</jtitle><stitle>Sci Data</stitle><addtitle>Sci Data</addtitle><date>2018-11-20</date><risdate>2018</risdate><volume>5</volume><issue>1</issue><spage>180251</spage><epage>10</epage><pages>180251-10</pages><artnum>180251</artnum><issn>2052-4463</issn><eissn>2052-4463</eissn><abstract>Radiology images are an essential part of clinical decision making and population screening, e.g., for cancer. Automated systems could help clinicians cope with large amounts of images by answering questions about the image contents. An emerging area of artificial intelligence, Visual Question Answering (VQA) in the medical domain explores approaches to this form of clinical decision support. Success of such machine learning tools hinges on availability and design of collections composed of medical images augmented with question-answer pairs directed at the content of the image. We introduce VQA-RAD, the first manually constructed dataset where clinicians asked naturally occurring questions about radiology images and provided reference answers. Manual categorization of images and questions provides insight into clinically relevant tasks and the natural language to phrase them. Evaluating with well-known algorithms, we demonstrate the rich quality of this dataset over other automatically constructed ones. We propose VQA-RAD to encourage the community to design VQA tools with the goals of improving patient care. Design Type(s) image creation and editing objective • anatomical image analysis objective Measurement Type(s) image analysis Technology Type(s) visual observation method Factor Type(s) question type • answer type Sample Characteristic(s) Homo sapiens • head • chest • abdomen Machine-accessible metadata file describing the reported data (ISA-Tab format)</abstract><cop>London</cop><pub>Nature Publishing Group UK</pub><pmid>30457565</pmid><doi>10.1038/sdata.2018.251</doi><tpages>10</tpages><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 2052-4463
ispartof Scientific data, 2018-11, Vol.5 (1), p.180251-10, Article 180251
issn 2052-4463
2052-4463
language eng
recordid cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_6244189
source Nature Publishing Group website; MEDLINE; DOAJ Directory of Open Access Journals; PubMed Central Open Access; Springer Nature OA Free Journals; Free E-Journal (出版社公開部分のみ); PubMed Central
subjects 631/114/2164
692/700/1421/1770
Algorithms
Artificial intelligence
Cancer
Data Analysis
Data collection
Data Descriptor
Data Mining
Datasets
Decision making
Decision support systems
Humanities and Social Sciences
Humans
Image processing
Image Processing, Computer-Assisted - methods
Learning algorithms
Machine Learning
multidisciplinary
Radiography - methods
Radiology Information Systems - classification
Radiology Information Systems - standards
Science
title A dataset of clinically generated visual questions and answers about radiology images
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-25T05%3A07%3A56IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_pubme&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=A%20dataset%20of%20clinically%20generated%20visual%20questions%20and%20answers%20about%20radiology%20images&rft.jtitle=Scientific%20data&rft.au=Lau,%20Jason%20J.&rft.date=2018-11-20&rft.volume=5&rft.issue=1&rft.spage=180251&rft.epage=10&rft.pages=180251-10&rft.artnum=180251&rft.issn=2052-4463&rft.eissn=2052-4463&rft_id=info:doi/10.1038/sdata.2018.251&rft_dat=%3Cproquest_pubme%3E2315956045%3C/proquest_pubme%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2135623601&rft_id=info:pmid/30457565&rfr_iscdi=true