Automatically assessing question answering system performance across possible confidence values
A mechanism is provided in a data processing system for assessing question answering system performance. The mechanism receives question answering system results. The question answering system results comprise questions posed to the question answering system, answers returned by the question answeri...
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Patent |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext bestellen |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | |
|---|---|
| container_issue | |
| container_start_page | |
| container_title | |
| container_volume | |
| creator | Block, Alexander M Van Der Stockt, Stefan A Chaney, Anna M Young, Kai G. H |
| description | A mechanism is provided in a data processing system for assessing question answering system performance. The mechanism receives question answering system results. The question answering system results comprise questions posed to the question answering system, answers returned by the question answering system for each question posed to the question answering system, and a confidence value for each answer. The question answering system is trained or tested using the ground truth questions and answers. The mechanism performs a matching operation comparing each question in the question answering system results to questions in the ground truth. A given question is determined to be on-topic or off-topic based on results of the matching operation. For a plurality of confidence threshold values, the mechanism determines a rightness or wrongness of each answer in the question answering system results. The mechanism generates performance statistics for the plurality of confidence threshold values based on whether each question is on-topic or off-topic and whether each answer is right or wrong. The mechanism presents the performance statistics to the user via a user interface. |
| format | Patent |
| fullrecord | <record><control><sourceid>epo_EVB</sourceid><recordid>TN_cdi_epo_espacenet_US10628749B2</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>US10628749B2</sourcerecordid><originalsourceid>FETCH-epo_espacenet_US10628749B23</originalsourceid><addsrcrecordid>eNqNjE0KwjAUhLNxIeodngcQtIo_SxXFvboOz3QqgTSJeanS29uCB3AzA998zFDpfZNDzdkadq4lFoGI9U96NZBsgyf28kHqkbSSUVNEqkKq2RsQmxREKHZhHw5kgq9siX56s-suxmpQsRNMfj1S0_PpdrzMEIOGRDbwyPp-XczXxXaz2h2K5T_OF3PQPtQ</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>patent</recordtype></control><display><type>patent</type><title>Automatically assessing question answering system performance across possible confidence values</title><source>esp@cenet</source><creator>Block, Alexander M ; Van Der Stockt, Stefan A ; Chaney, Anna M ; Young, Kai G. H</creator><creatorcontrib>Block, Alexander M ; Van Der Stockt, Stefan A ; Chaney, Anna M ; Young, Kai G. H</creatorcontrib><description>A mechanism is provided in a data processing system for assessing question answering system performance. The mechanism receives question answering system results. The question answering system results comprise questions posed to the question answering system, answers returned by the question answering system for each question posed to the question answering system, and a confidence value for each answer. The question answering system is trained or tested using the ground truth questions and answers. The mechanism performs a matching operation comparing each question in the question answering system results to questions in the ground truth. A given question is determined to be on-topic or off-topic based on results of the matching operation. For a plurality of confidence threshold values, the mechanism determines a rightness or wrongness of each answer in the question answering system results. The mechanism generates performance statistics for the plurality of confidence threshold values based on whether each question is on-topic or off-topic and whether each answer is right or wrong. The mechanism presents the performance statistics to the user via a user interface.</description><language>eng</language><subject>CALCULATING ; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS ; COMPUTING ; COUNTING ; PHYSICS</subject><creationdate>2020</creationdate><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://worldwide.espacenet.com/publicationDetails/biblio?FT=D&date=20200421&DB=EPODOC&CC=US&NR=10628749B2$$EHTML$$P50$$Gepo$$Hfree_for_read</linktohtml><link.rule.ids>230,308,777,882,25545,76296</link.rule.ids><linktorsrc>$$Uhttps://worldwide.espacenet.com/publicationDetails/biblio?FT=D&date=20200421&DB=EPODOC&CC=US&NR=10628749B2$$EView_record_in_European_Patent_Office$$FView_record_in_$$GEuropean_Patent_Office$$Hfree_for_read</linktorsrc></links><search><creatorcontrib>Block, Alexander M</creatorcontrib><creatorcontrib>Van Der Stockt, Stefan A</creatorcontrib><creatorcontrib>Chaney, Anna M</creatorcontrib><creatorcontrib>Young, Kai G. H</creatorcontrib><title>Automatically assessing question answering system performance across possible confidence values</title><description>A mechanism is provided in a data processing system for assessing question answering system performance. The mechanism receives question answering system results. The question answering system results comprise questions posed to the question answering system, answers returned by the question answering system for each question posed to the question answering system, and a confidence value for each answer. The question answering system is trained or tested using the ground truth questions and answers. The mechanism performs a matching operation comparing each question in the question answering system results to questions in the ground truth. A given question is determined to be on-topic or off-topic based on results of the matching operation. For a plurality of confidence threshold values, the mechanism determines a rightness or wrongness of each answer in the question answering system results. The mechanism generates performance statistics for the plurality of confidence threshold values based on whether each question is on-topic or off-topic and whether each answer is right or wrong. The mechanism presents the performance statistics to the user via a user interface.</description><subject>CALCULATING</subject><subject>COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS</subject><subject>COMPUTING</subject><subject>COUNTING</subject><subject>PHYSICS</subject><fulltext>true</fulltext><rsrctype>patent</rsrctype><creationdate>2020</creationdate><recordtype>patent</recordtype><sourceid>EVB</sourceid><recordid>eNqNjE0KwjAUhLNxIeodngcQtIo_SxXFvboOz3QqgTSJeanS29uCB3AzA998zFDpfZNDzdkadq4lFoGI9U96NZBsgyf28kHqkbSSUVNEqkKq2RsQmxREKHZhHw5kgq9siX56s-suxmpQsRNMfj1S0_PpdrzMEIOGRDbwyPp-XczXxXaz2h2K5T_OF3PQPtQ</recordid><startdate>20200421</startdate><enddate>20200421</enddate><creator>Block, Alexander M</creator><creator>Van Der Stockt, Stefan A</creator><creator>Chaney, Anna M</creator><creator>Young, Kai G. H</creator><scope>EVB</scope></search><sort><creationdate>20200421</creationdate><title>Automatically assessing question answering system performance across possible confidence values</title><author>Block, Alexander M ; Van Der Stockt, Stefan A ; Chaney, Anna M ; Young, Kai G. H</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-epo_espacenet_US10628749B23</frbrgroupid><rsrctype>patents</rsrctype><prefilter>patents</prefilter><language>eng</language><creationdate>2020</creationdate><topic>CALCULATING</topic><topic>COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS</topic><topic>COMPUTING</topic><topic>COUNTING</topic><topic>PHYSICS</topic><toplevel>online_resources</toplevel><creatorcontrib>Block, Alexander M</creatorcontrib><creatorcontrib>Van Der Stockt, Stefan A</creatorcontrib><creatorcontrib>Chaney, Anna M</creatorcontrib><creatorcontrib>Young, Kai G. H</creatorcontrib><collection>esp@cenet</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Block, Alexander M</au><au>Van Der Stockt, Stefan A</au><au>Chaney, Anna M</au><au>Young, Kai G. H</au><format>patent</format><genre>patent</genre><ristype>GEN</ristype><title>Automatically assessing question answering system performance across possible confidence values</title><date>2020-04-21</date><risdate>2020</risdate><abstract>A mechanism is provided in a data processing system for assessing question answering system performance. The mechanism receives question answering system results. The question answering system results comprise questions posed to the question answering system, answers returned by the question answering system for each question posed to the question answering system, and a confidence value for each answer. The question answering system is trained or tested using the ground truth questions and answers. The mechanism performs a matching operation comparing each question in the question answering system results to questions in the ground truth. A given question is determined to be on-topic or off-topic based on results of the matching operation. For a plurality of confidence threshold values, the mechanism determines a rightness or wrongness of each answer in the question answering system results. The mechanism generates performance statistics for the plurality of confidence threshold values based on whether each question is on-topic or off-topic and whether each answer is right or wrong. The mechanism presents the performance statistics to the user via a user interface.</abstract><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext_linktorsrc |
| identifier | |
| ispartof | |
| issn | |
| language | eng |
| recordid | cdi_epo_espacenet_US10628749B2 |
| source | esp@cenet |
| subjects | CALCULATING COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS COMPUTING COUNTING PHYSICS |
| title | Automatically assessing question answering system performance across possible confidence values |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-20T20%3A31%3A08IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-epo_EVB&rft_val_fmt=info:ofi/fmt:kev:mtx:patent&rft.genre=patent&rft.au=Block,%20Alexander%20M&rft.date=2020-04-21&rft_id=info:doi/&rft_dat=%3Cepo_EVB%3EUS10628749B2%3C/epo_EVB%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true |