Designing and Evaluating User Experience of an AI-Based Defense System
In recent years, artificial intelligence (AI) has been applied in various fields, with rapid expansion of the scope of AI-human interactions. However, most AI technologies continue to exhibit black-box characteristics, i.e., their decisions and actions are not explainable, degrading user experience...
Gespeichert in:
| Veröffentlicht in: | IEEE access 2023, Vol.11, p.122045-122056 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 122056 |
|---|---|
| container_issue | |
| container_start_page | 122045 |
| container_title | IEEE access |
| container_volume | 11 |
| creator | Park, Sunyoung Kim, Hyun K. Park, Jaehyun Lee, Yuryeon |
| description | In recent years, artificial intelligence (AI) has been applied in various fields, with rapid expansion of the scope of AI-human interactions. However, most AI technologies continue to exhibit black-box characteristics, i.e., their decisions and actions are not explainable, degrading user experience (UX). Recently, research on explainable AI (XAI), focusing on both AI performance and UX, has garnered significant attention. However, development of generalizable UX evaluation tools for AI and improvement of AI UX have not been investigated adequately. In this study, a UX evaluation tool is developed for AI based on a systematic literature review and verified using exploratory and confirmatory factor analyses. Subsequently, based on identified AI UX factors, UX of an AI defense system is upgraded. Based on user evaluation, there were significant improvements in eight factors (satisfaction, safety, controllability, trust, causality, efficiency, accountability, and explainability) with the exception of fairness. The proposed evaluation tool is expected to serve as a cornerstone for future evaluation of AI UX advancement. |
| doi_str_mv | 10.1109/ACCESS.2023.3329257 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_ieee_</sourceid><recordid>TN_cdi_ieee_primary_10304133</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>10304133</ieee_id><doaj_id>oai_doaj_org_article_bcaed67b0d7649f98bba3ab3c5e42700</doaj_id><sourcerecordid>2887114393</sourcerecordid><originalsourceid>FETCH-LOGICAL-c359t-2ba8fa8dbe0e41cbbfcd6679f70eac678f6c4085908612e11989b637e618b34b3</originalsourceid><addsrcrecordid>eNpNUMtOwzAQjBBIoMIXwCES5xTbm_hxLG0KlSpxKJwt21lXqUpS7BTRvyclFepednc0M7uaJLmnZEwpUU-T6bRcrcaMMBgDMMUKcZHcMMpVBgXwy7P5OrmLcUP6kj1UiJtkPsNYr5u6WaemqdLy22z3pjuuHxFDWv7sMNTYOExb3zPSySJ7NhGrdIYem4jp6hA7_LxNrrzZRrw79VHyMS_fp6_Z8u1lMZ0sMweF6jJmjfRGVhYJ5tRZ613FuVBeEDSOC-m5y4ksFJGcMqRUSWU5CORUWsgtjJLF4Fu1ZqN3of404aBbU-s_oA1rbUJXuy1q6wxWXFhSCZ4rr6S1BowFV2DOBCG91-PgtQvt1x5jpzftPjT9-5pJKSjNQUHPgoHlQhtjQP9_lRJ9zF8P-etj_vqUf696GFQ1Ip4pgOQUAH4BNd-ABA</addsrcrecordid><sourcetype>Open Website</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2887114393</pqid></control><display><type>article</type><title>Designing and Evaluating User Experience of an AI-Based Defense System</title><source>IEEE Open Access Journals</source><source>DOAJ Directory of Open Access Journals</source><source>Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals</source><creator>Park, Sunyoung ; Kim, Hyun K. ; Park, Jaehyun ; Lee, Yuryeon</creator><creatorcontrib>Park, Sunyoung ; Kim, Hyun K. ; Park, Jaehyun ; Lee, Yuryeon</creatorcontrib><description>In recent years, artificial intelligence (AI) has been applied in various fields, with rapid expansion of the scope of AI-human interactions. However, most AI technologies continue to exhibit black-box characteristics, i.e., their decisions and actions are not explainable, degrading user experience (UX). Recently, research on explainable AI (XAI), focusing on both AI performance and UX, has garnered significant attention. However, development of generalizable UX evaluation tools for AI and improvement of AI UX have not been investigated adequately. In this study, a UX evaluation tool is developed for AI based on a systematic literature review and verified using exploratory and confirmatory factor analyses. Subsequently, based on identified AI UX factors, UX of an AI defense system is upgraded. Based on user evaluation, there were significant improvements in eight factors (satisfaction, safety, controllability, trust, causality, efficiency, accountability, and explainability) with the exception of fairness. The proposed evaluation tool is expected to serve as a cornerstone for future evaluation of AI UX advancement.</description><identifier>ISSN: 2169-3536</identifier><identifier>EISSN: 2169-3536</identifier><identifier>DOI: 10.1109/ACCESS.2023.3329257</identifier><identifier>CODEN: IAECCG</identifier><language>eng</language><publisher>Piscataway: IEEE</publisher><subject>AI system design ; AI UX ; AI UX measurement ; Artificial intelligence ; Artificial intelligence (AI) ; Explainable artificial intelligence ; Human factors ; Indexes ; Internet ; Literature reviews ; Surveys ; System analysis and design ; Usability ; User experience ; user experience (UX)</subject><ispartof>IEEE access, 2023, Vol.11, p.122045-122056</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c359t-2ba8fa8dbe0e41cbbfcd6679f70eac678f6c4085908612e11989b637e618b34b3</cites><orcidid>0000-0002-5264-6941 ; 0000-0002-5192-9405 ; 0000-0002-0615-5076</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/10304133$$EHTML$$P50$$Gieee$$Hfree_for_read</linktohtml><link.rule.ids>314,780,784,864,2101,4023,27632,27922,27923,27924,54932</link.rule.ids></links><search><creatorcontrib>Park, Sunyoung</creatorcontrib><creatorcontrib>Kim, Hyun K.</creatorcontrib><creatorcontrib>Park, Jaehyun</creatorcontrib><creatorcontrib>Lee, Yuryeon</creatorcontrib><title>Designing and Evaluating User Experience of an AI-Based Defense System</title><title>IEEE access</title><addtitle>Access</addtitle><description>In recent years, artificial intelligence (AI) has been applied in various fields, with rapid expansion of the scope of AI-human interactions. However, most AI technologies continue to exhibit black-box characteristics, i.e., their decisions and actions are not explainable, degrading user experience (UX). Recently, research on explainable AI (XAI), focusing on both AI performance and UX, has garnered significant attention. However, development of generalizable UX evaluation tools for AI and improvement of AI UX have not been investigated adequately. In this study, a UX evaluation tool is developed for AI based on a systematic literature review and verified using exploratory and confirmatory factor analyses. Subsequently, based on identified AI UX factors, UX of an AI defense system is upgraded. Based on user evaluation, there were significant improvements in eight factors (satisfaction, safety, controllability, trust, causality, efficiency, accountability, and explainability) with the exception of fairness. The proposed evaluation tool is expected to serve as a cornerstone for future evaluation of AI UX advancement.</description><subject>AI system design</subject><subject>AI UX</subject><subject>AI UX measurement</subject><subject>Artificial intelligence</subject><subject>Artificial intelligence (AI)</subject><subject>Explainable artificial intelligence</subject><subject>Human factors</subject><subject>Indexes</subject><subject>Internet</subject><subject>Literature reviews</subject><subject>Surveys</subject><subject>System analysis and design</subject><subject>Usability</subject><subject>User experience</subject><subject>user experience (UX)</subject><issn>2169-3536</issn><issn>2169-3536</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>ESBDL</sourceid><sourceid>RIE</sourceid><sourceid>DOA</sourceid><recordid>eNpNUMtOwzAQjBBIoMIXwCES5xTbm_hxLG0KlSpxKJwt21lXqUpS7BTRvyclFepednc0M7uaJLmnZEwpUU-T6bRcrcaMMBgDMMUKcZHcMMpVBgXwy7P5OrmLcUP6kj1UiJtkPsNYr5u6WaemqdLy22z3pjuuHxFDWv7sMNTYOExb3zPSySJ7NhGrdIYem4jp6hA7_LxNrrzZRrw79VHyMS_fp6_Z8u1lMZ0sMweF6jJmjfRGVhYJ5tRZ613FuVBeEDSOC-m5y4ksFJGcMqRUSWU5CORUWsgtjJLF4Fu1ZqN3of404aBbU-s_oA1rbUJXuy1q6wxWXFhSCZ4rr6S1BowFV2DOBCG91-PgtQvt1x5jpzftPjT9-5pJKSjNQUHPgoHlQhtjQP9_lRJ9zF8P-etj_vqUf696GFQ1Ip4pgOQUAH4BNd-ABA</recordid><startdate>2023</startdate><enddate>2023</enddate><creator>Park, Sunyoung</creator><creator>Kim, Hyun K.</creator><creator>Park, Jaehyun</creator><creator>Lee, Yuryeon</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>ESBDL</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>7SR</scope><scope>8BQ</scope><scope>8FD</scope><scope>JG9</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>DOA</scope><orcidid>https://orcid.org/0000-0002-5264-6941</orcidid><orcidid>https://orcid.org/0000-0002-5192-9405</orcidid><orcidid>https://orcid.org/0000-0002-0615-5076</orcidid></search><sort><creationdate>2023</creationdate><title>Designing and Evaluating User Experience of an AI-Based Defense System</title><author>Park, Sunyoung ; Kim, Hyun K. ; Park, Jaehyun ; Lee, Yuryeon</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c359t-2ba8fa8dbe0e41cbbfcd6679f70eac678f6c4085908612e11989b637e618b34b3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>AI system design</topic><topic>AI UX</topic><topic>AI UX measurement</topic><topic>Artificial intelligence</topic><topic>Artificial intelligence (AI)</topic><topic>Explainable artificial intelligence</topic><topic>Human factors</topic><topic>Indexes</topic><topic>Internet</topic><topic>Literature reviews</topic><topic>Surveys</topic><topic>System analysis and design</topic><topic>Usability</topic><topic>User experience</topic><topic>user experience (UX)</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Park, Sunyoung</creatorcontrib><creatorcontrib>Kim, Hyun K.</creatorcontrib><creatorcontrib>Park, Jaehyun</creatorcontrib><creatorcontrib>Lee, Yuryeon</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE Open Access Journals</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics & Communications Abstracts</collection><collection>Engineered Materials Abstracts</collection><collection>METADEX</collection><collection>Technology Research Database</collection><collection>Materials Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>DOAJ Directory of Open Access Journals</collection><jtitle>IEEE access</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Park, Sunyoung</au><au>Kim, Hyun K.</au><au>Park, Jaehyun</au><au>Lee, Yuryeon</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Designing and Evaluating User Experience of an AI-Based Defense System</atitle><jtitle>IEEE access</jtitle><stitle>Access</stitle><date>2023</date><risdate>2023</risdate><volume>11</volume><spage>122045</spage><epage>122056</epage><pages>122045-122056</pages><issn>2169-3536</issn><eissn>2169-3536</eissn><coden>IAECCG</coden><abstract>In recent years, artificial intelligence (AI) has been applied in various fields, with rapid expansion of the scope of AI-human interactions. However, most AI technologies continue to exhibit black-box characteristics, i.e., their decisions and actions are not explainable, degrading user experience (UX). Recently, research on explainable AI (XAI), focusing on both AI performance and UX, has garnered significant attention. However, development of generalizable UX evaluation tools for AI and improvement of AI UX have not been investigated adequately. In this study, a UX evaluation tool is developed for AI based on a systematic literature review and verified using exploratory and confirmatory factor analyses. Subsequently, based on identified AI UX factors, UX of an AI defense system is upgraded. Based on user evaluation, there were significant improvements in eight factors (satisfaction, safety, controllability, trust, causality, efficiency, accountability, and explainability) with the exception of fairness. The proposed evaluation tool is expected to serve as a cornerstone for future evaluation of AI UX advancement.</abstract><cop>Piscataway</cop><pub>IEEE</pub><doi>10.1109/ACCESS.2023.3329257</doi><tpages>12</tpages><orcidid>https://orcid.org/0000-0002-5264-6941</orcidid><orcidid>https://orcid.org/0000-0002-5192-9405</orcidid><orcidid>https://orcid.org/0000-0002-0615-5076</orcidid><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 2169-3536 |
| ispartof | IEEE access, 2023, Vol.11, p.122045-122056 |
| issn | 2169-3536 2169-3536 |
| language | eng |
| recordid | cdi_ieee_primary_10304133 |
| source | IEEE Open Access Journals; DOAJ Directory of Open Access Journals; Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals |
| subjects | AI system design AI UX AI UX measurement Artificial intelligence Artificial intelligence (AI) Explainable artificial intelligence Human factors Indexes Internet Literature reviews Surveys System analysis and design Usability User experience user experience (UX) |
| title | Designing and Evaluating User Experience of an AI-Based Defense System |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-08T18%3A13%3A12IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_ieee_&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Designing%20and%20Evaluating%20User%20Experience%20of%20an%20AI-Based%20Defense%20System&rft.jtitle=IEEE%20access&rft.au=Park,%20Sunyoung&rft.date=2023&rft.volume=11&rft.spage=122045&rft.epage=122056&rft.pages=122045-122056&rft.issn=2169-3536&rft.eissn=2169-3536&rft.coden=IAECCG&rft_id=info:doi/10.1109/ACCESS.2023.3329257&rft_dat=%3Cproquest_ieee_%3E2887114393%3C/proquest_ieee_%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2887114393&rft_id=info:pmid/&rft_ieee_id=10304133&rft_doaj_id=oai_doaj_org_article_bcaed67b0d7649f98bba3ab3c5e42700&rfr_iscdi=true |