A Systematic Review of Fairness, Accountability, Transparency and Ethics in Information Retrieval

We live in an information society that strongly relies on information retrieval systems, such as search engines and conversational assistants. Consequently, the trustworthiness of these systems is of critical importance, and has attracted a significant research attention in recent years. In this wor...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:ACM computing surveys 2023-12
Hauptverfasser: Bernard, Nolwenn, Balog, Krisztian
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title ACM computing surveys
container_volume
creator Bernard, Nolwenn
Balog, Krisztian
description We live in an information society that strongly relies on information retrieval systems, such as search engines and conversational assistants. Consequently, the trustworthiness of these systems is of critical importance, and has attracted a significant research attention in recent years. In this work, we perform a systematic literature review of the field of fairness, accountability, transparency, and ethics in information retrieval. In particular, we investigate the definitions, approaches, and evaluation methodologies proposed to build trustworthy information retrieval systems. This review reveals the lack of standard definitions, arguably due to the multi-dimensional nature of the different notions. In terms of approaches, most of the work focuses on building either a fair or a transparent information retrieval system. As for evaluation, fairness is often assessed by means of automatic evaluation, while accountability and transparency are most commonly evaluated using audits and user studies. Based on the surveyed literature, we develop taxonomies of requirements for the different notions, and further use these taxonomies to propose practical definitions to quantify the degree to which an information retrieval system satisfies a given notion. Finally, we discuss challenges that have yet to be solved for information retrieval systems to be trustworthy.
doi_str_mv 10.1145/3637211
format Article
fullrecord <record><control><sourceid>acm_cross</sourceid><recordid>TN_cdi_crossref_primary_10_1145_3637211</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>3637211</sourcerecordid><originalsourceid>FETCH-LOGICAL-a841-95a5016cf5a66c7a1d6e4f9ed78e5d6ee4619487f0b85762646e6ec6757eb2d63</originalsourceid><addsrcrecordid>eNo9kMFLwzAYxYMoWKd495Sbl1WTNvnSHcfYdDAQdPfyLf2CkTUdSZ30v3dj09N78H78Do-xeymepFT6uYTSFFJesExqbXJTKnnJMlGCyEUpxDW7SelLCFEoCRnDKf8YUk8t9t7yd9p7-uGd4wv0MVBKYz61tvsOPW781vfDmK8jhrTDSMEOHEPD5_2nt4n7wJfBdfFo6sJB1UdPe9zesiuH20R35xyx9WK-nr3mq7eX5Wy6yrFSMp9o1EKCdRoBrEHZACk3ocZUpA-dFMiJqowTm0obKEABAVkw2tCmaKAcsceT1sYupUiu3kXfYhxqKerjMfX5mAP5cCLRtv_Q3_gLlopdtA</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>A Systematic Review of Fairness, Accountability, Transparency and Ethics in Information Retrieval</title><source>ACM Digital Library Complete</source><creator>Bernard, Nolwenn ; Balog, Krisztian</creator><creatorcontrib>Bernard, Nolwenn ; Balog, Krisztian</creatorcontrib><description>We live in an information society that strongly relies on information retrieval systems, such as search engines and conversational assistants. Consequently, the trustworthiness of these systems is of critical importance, and has attracted a significant research attention in recent years. In this work, we perform a systematic literature review of the field of fairness, accountability, transparency, and ethics in information retrieval. In particular, we investigate the definitions, approaches, and evaluation methodologies proposed to build trustworthy information retrieval systems. This review reveals the lack of standard definitions, arguably due to the multi-dimensional nature of the different notions. In terms of approaches, most of the work focuses on building either a fair or a transparent information retrieval system. As for evaluation, fairness is often assessed by means of automatic evaluation, while accountability and transparency are most commonly evaluated using audits and user studies. Based on the surveyed literature, we develop taxonomies of requirements for the different notions, and further use these taxonomies to propose practical definitions to quantify the degree to which an information retrieval system satisfies a given notion. Finally, we discuss challenges that have yet to be solved for information retrieval systems to be trustworthy.</description><identifier>ISSN: 0360-0300</identifier><identifier>EISSN: 1557-7341</identifier><identifier>DOI: 10.1145/3637211</identifier><language>eng</language><publisher>New York, NY: ACM</publisher><subject>Information retrieval ; Information systems ; Social and professional topics</subject><ispartof>ACM computing surveys, 2023-12</ispartof><rights>Copyright held by the owner/author(s). Publication rights licensed to ACM.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-a841-95a5016cf5a66c7a1d6e4f9ed78e5d6ee4619487f0b85762646e6ec6757eb2d63</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,776,780,27903,27904</link.rule.ids></links><search><creatorcontrib>Bernard, Nolwenn</creatorcontrib><creatorcontrib>Balog, Krisztian</creatorcontrib><title>A Systematic Review of Fairness, Accountability, Transparency and Ethics in Information Retrieval</title><title>ACM computing surveys</title><addtitle>ACM CSUR</addtitle><description>We live in an information society that strongly relies on information retrieval systems, such as search engines and conversational assistants. Consequently, the trustworthiness of these systems is of critical importance, and has attracted a significant research attention in recent years. In this work, we perform a systematic literature review of the field of fairness, accountability, transparency, and ethics in information retrieval. In particular, we investigate the definitions, approaches, and evaluation methodologies proposed to build trustworthy information retrieval systems. This review reveals the lack of standard definitions, arguably due to the multi-dimensional nature of the different notions. In terms of approaches, most of the work focuses on building either a fair or a transparent information retrieval system. As for evaluation, fairness is often assessed by means of automatic evaluation, while accountability and transparency are most commonly evaluated using audits and user studies. Based on the surveyed literature, we develop taxonomies of requirements for the different notions, and further use these taxonomies to propose practical definitions to quantify the degree to which an information retrieval system satisfies a given notion. Finally, we discuss challenges that have yet to be solved for information retrieval systems to be trustworthy.</description><subject>Information retrieval</subject><subject>Information systems</subject><subject>Social and professional topics</subject><issn>0360-0300</issn><issn>1557-7341</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><recordid>eNo9kMFLwzAYxYMoWKd495Sbl1WTNvnSHcfYdDAQdPfyLf2CkTUdSZ30v3dj09N78H78Do-xeymepFT6uYTSFFJesExqbXJTKnnJMlGCyEUpxDW7SelLCFEoCRnDKf8YUk8t9t7yd9p7-uGd4wv0MVBKYz61tvsOPW781vfDmK8jhrTDSMEOHEPD5_2nt4n7wJfBdfFo6sJB1UdPe9zesiuH20R35xyx9WK-nr3mq7eX5Wy6yrFSMp9o1EKCdRoBrEHZACk3ocZUpA-dFMiJqowTm0obKEABAVkw2tCmaKAcsceT1sYupUiu3kXfYhxqKerjMfX5mAP5cCLRtv_Q3_gLlopdtA</recordid><startdate>20231215</startdate><enddate>20231215</enddate><creator>Bernard, Nolwenn</creator><creator>Balog, Krisztian</creator><general>ACM</general><scope>AAYXX</scope><scope>CITATION</scope></search><sort><creationdate>20231215</creationdate><title>A Systematic Review of Fairness, Accountability, Transparency and Ethics in Information Retrieval</title><author>Bernard, Nolwenn ; Balog, Krisztian</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a841-95a5016cf5a66c7a1d6e4f9ed78e5d6ee4619487f0b85762646e6ec6757eb2d63</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Information retrieval</topic><topic>Information systems</topic><topic>Social and professional topics</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Bernard, Nolwenn</creatorcontrib><creatorcontrib>Balog, Krisztian</creatorcontrib><collection>CrossRef</collection><jtitle>ACM computing surveys</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Bernard, Nolwenn</au><au>Balog, Krisztian</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>A Systematic Review of Fairness, Accountability, Transparency and Ethics in Information Retrieval</atitle><jtitle>ACM computing surveys</jtitle><stitle>ACM CSUR</stitle><date>2023-12-15</date><risdate>2023</risdate><issn>0360-0300</issn><eissn>1557-7341</eissn><abstract>We live in an information society that strongly relies on information retrieval systems, such as search engines and conversational assistants. Consequently, the trustworthiness of these systems is of critical importance, and has attracted a significant research attention in recent years. In this work, we perform a systematic literature review of the field of fairness, accountability, transparency, and ethics in information retrieval. In particular, we investigate the definitions, approaches, and evaluation methodologies proposed to build trustworthy information retrieval systems. This review reveals the lack of standard definitions, arguably due to the multi-dimensional nature of the different notions. In terms of approaches, most of the work focuses on building either a fair or a transparent information retrieval system. As for evaluation, fairness is often assessed by means of automatic evaluation, while accountability and transparency are most commonly evaluated using audits and user studies. Based on the surveyed literature, we develop taxonomies of requirements for the different notions, and further use these taxonomies to propose practical definitions to quantify the degree to which an information retrieval system satisfies a given notion. Finally, we discuss challenges that have yet to be solved for information retrieval systems to be trustworthy.</abstract><cop>New York, NY</cop><pub>ACM</pub><doi>10.1145/3637211</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 0360-0300
ispartof ACM computing surveys, 2023-12
issn 0360-0300
1557-7341
language eng
recordid cdi_crossref_primary_10_1145_3637211
source ACM Digital Library Complete
subjects Information retrieval
Information systems
Social and professional topics
title A Systematic Review of Fairness, Accountability, Transparency and Ethics in Information Retrieval
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-25T19%3A18%3A27IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-acm_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=A%20Systematic%20Review%20of%20Fairness,%20Accountability,%20Transparency%20and%20Ethics%20in%20Information%20Retrieval&rft.jtitle=ACM%20computing%20surveys&rft.au=Bernard,%20Nolwenn&rft.date=2023-12-15&rft.issn=0360-0300&rft.eissn=1557-7341&rft_id=info:doi/10.1145/3637211&rft_dat=%3Cacm_cross%3E3637211%3C/acm_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true