Evaluating the agreement of users with usability problems identified by heuristic evaluation

Evaluating the agreement of users with usability problems identified by heuristic evaluation

•This paper Evaluated the agreement of users with usability problems.•The HE is the desired method for identifying recognizing, diagnosing problems.•The agreement of real user with problems identified by HE is at a medium level.•Younger users has a higher agreement with the problems identified in HE...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:International journal of medical informatics (Shannon, Ireland) Ireland), 2018-09, Vol.117, p.13-18
Hauptverfasser: Khajouei, Reza, Ameri, Arefeh, Jahani, Yunes
Format: Artikel
Sprache:eng
Schlagworte:
Heuristic Evaluation
Hospital information system
Usability
User-computer interface
Users
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 18
container_issue
container_start_page 13
container_title International journal of medical informatics (Shannon, Ireland)
container_volume 117
creator Khajouei, Reza
Ameri, Arefeh
Jahani, Yunes
description •This paper Evaluated the agreement of users with usability problems.•The HE is the desired method for identifying recognizing, diagnosing problems.•The agreement of real user with problems identified by HE is at a medium level.•Younger users has a higher agreement with the problems identified in HE.•The users with lower computer experience had a higher agreement with the results of HE.•The severity of some problems identified by experts are not acknowledge by users. Heuristic evaluation (HE) is an inspection method that can be used for identifying usability problems of health information systems. There is insufficient evidence to what extent these problems are experienced by users in their working environment. The objective of this study was to investigate the agreement of real end users of a health information system with the problems identified by HE. First, the emergency information system of educational hospitals in Kerman, Iran was evaluated by a heuristic evaluation. Then, a valid and reliable (Cronbach’s alpha = 0.95) electronic questionnaire was designed based on the results of the first phase. Among 127 identified problems, the highest number of problems was related to the "consistency and standards" heuristic (18.9%). The user agreement coefficient with heuristic results was 0.5. The highest agreement of the participants was on the problems related to the "help user recognize, diagnose and recover from errors" heuristic (78%). There was a significant relationship between participants age and work experience with a computer (p 
doi_str_mv 10.1016/j.ijmedinf.2018.05.012
format Article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_2074122642</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S1386505618301254</els_id><sourcerecordid>2074122642</sourcerecordid><originalsourceid>FETCH-LOGICAL-c368t-415be95706a878d6b044807f1c4bea2c1197e9a25163cfb2014c2c1886621e943</originalsourceid><addsrcrecordid>eNqFkE1PxCAQhonR-P0XNhy9tA6UUnrTGL8SEy96MyGUTnfZ9EOBavbfi9nVqycm8Mw7zEPIgkHOgMnLde7WA7Zu7HIOTOVQ5sD4HjlmquKZ4qLYT3WhZFZCKY_ISQhrAFZBKQ7JUQFQ8FrCMXm7_TT9bKIblzSukJqlRxxwjHTq6BzQB_rl4iqVpnG9ixv67qemxyFQ1ybMdQ5b2mzoCmfvQnSW4i5xGs_IQWf6gOe785S83t2-3DxkT8_3jzfXT5ktpIqZYGWDdVmBNKpSrWxACAVVx6xo0HDLWF1hbXjJZGG7Ju0rbLpVSkrOsBbFKbnY5qa_fcwYoh5csNj3ZsRpDppDJRjnUvCEyi1q_RSCx06_ezcYv9EM9I9Zvda_ZvWPWQ2lTmZT42I3Y27S81_br8oEXG0BTJt-OvQ6WIejTVEebdTt5P6b8Q3M8Y3u</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2074122642</pqid></control><display><type>article</type><title>Evaluating the agreement of users with usability problems identified by heuristic evaluation</title><source>Elsevier ScienceDirect Journals Complete</source><creator>Khajouei, Reza ; Ameri, Arefeh ; Jahani, Yunes</creator><creatorcontrib>Khajouei, Reza ; Ameri, Arefeh ; Jahani, Yunes</creatorcontrib><description>•This paper Evaluated the agreement of users with usability problems.•The HE is the desired method for identifying recognizing, diagnosing problems.•The agreement of real user with problems identified by HE is at a medium level.•Younger users has a higher agreement with the problems identified in HE.•The users with lower computer experience had a higher agreement with the results of HE.•The severity of some problems identified by experts are not acknowledge by users. Heuristic evaluation (HE) is an inspection method that can be used for identifying usability problems of health information systems. There is insufficient evidence to what extent these problems are experienced by users in their working environment. The objective of this study was to investigate the agreement of real end users of a health information system with the problems identified by HE. First, the emergency information system of educational hospitals in Kerman, Iran was evaluated by a heuristic evaluation. Then, a valid and reliable (Cronbach’s alpha = 0.95) electronic questionnaire was designed based on the results of the first phase. Among 127 identified problems, the highest number of problems was related to the "consistency and standards" heuristic (18.9%). The user agreement coefficient with heuristic results was 0.5. The highest agreement of the participants was on the problems related to the "help user recognize, diagnose and recover from errors" heuristic (78%). There was a significant relationship between participants age and work experience with a computer (p &lt; 0.05). The agreement rate of real end users with the results of HE was at a medium level. Therefore, in some areas, the perception of evaluators in using this method is not consistent with the users experience with a system. This can result in identifying minor issues and ignoring important system-critical usability issues. The HE is appropriate for identifying specific problems, such as problems related to identifying and correcting errors, but in some areas, such as consistency and standards in the design of a system, it identifies minor or ineffective problems.</description><identifier>ISSN: 1386-5056</identifier><identifier>EISSN: 1872-8243</identifier><identifier>DOI: 10.1016/j.ijmedinf.2018.05.012</identifier><identifier>PMID: 30032960</identifier><language>eng</language><publisher>Ireland: Elsevier B.V</publisher><subject>Heuristic Evaluation ; Hospital information system ; Usability ; User-computer interface ; Users</subject><ispartof>International journal of medical informatics (Shannon, Ireland), 2018-09, Vol.117, p.13-18</ispartof><rights>2018 Elsevier B.V.</rights><rights>Copyright © 2018 Elsevier B.V. All rights reserved.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c368t-415be95706a878d6b044807f1c4bea2c1197e9a25163cfb2014c2c1886621e943</citedby><cites>FETCH-LOGICAL-c368t-415be95706a878d6b044807f1c4bea2c1197e9a25163cfb2014c2c1886621e943</cites><orcidid>0000-0001-6094-5127</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://dx.doi.org/10.1016/j.ijmedinf.2018.05.012$$EHTML$$P50$$Gelsevier$$H</linktohtml><link.rule.ids>314,780,784,3550,27924,27925,45995</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/30032960$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Khajouei, Reza</creatorcontrib><creatorcontrib>Ameri, Arefeh</creatorcontrib><creatorcontrib>Jahani, Yunes</creatorcontrib><title>Evaluating the agreement of users with usability problems identified by heuristic evaluation</title><title>International journal of medical informatics (Shannon, Ireland)</title><addtitle>Int J Med Inform</addtitle><description>•This paper Evaluated the agreement of users with usability problems.•The HE is the desired method for identifying recognizing, diagnosing problems.•The agreement of real user with problems identified by HE is at a medium level.•Younger users has a higher agreement with the problems identified in HE.•The users with lower computer experience had a higher agreement with the results of HE.•The severity of some problems identified by experts are not acknowledge by users. Heuristic evaluation (HE) is an inspection method that can be used for identifying usability problems of health information systems. There is insufficient evidence to what extent these problems are experienced by users in their working environment. The objective of this study was to investigate the agreement of real end users of a health information system with the problems identified by HE. First, the emergency information system of educational hospitals in Kerman, Iran was evaluated by a heuristic evaluation. Then, a valid and reliable (Cronbach’s alpha = 0.95) electronic questionnaire was designed based on the results of the first phase. Among 127 identified problems, the highest number of problems was related to the "consistency and standards" heuristic (18.9%). The user agreement coefficient with heuristic results was 0.5. The highest agreement of the participants was on the problems related to the "help user recognize, diagnose and recover from errors" heuristic (78%). There was a significant relationship between participants age and work experience with a computer (p &lt; 0.05). The agreement rate of real end users with the results of HE was at a medium level. Therefore, in some areas, the perception of evaluators in using this method is not consistent with the users experience with a system. This can result in identifying minor issues and ignoring important system-critical usability issues. The HE is appropriate for identifying specific problems, such as problems related to identifying and correcting errors, but in some areas, such as consistency and standards in the design of a system, it identifies minor or ineffective problems.</description><subject>Heuristic Evaluation</subject><subject>Hospital information system</subject><subject>Usability</subject><subject>User-computer interface</subject><subject>Users</subject><issn>1386-5056</issn><issn>1872-8243</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2018</creationdate><recordtype>article</recordtype><recordid>eNqFkE1PxCAQhonR-P0XNhy9tA6UUnrTGL8SEy96MyGUTnfZ9EOBavbfi9nVqycm8Mw7zEPIgkHOgMnLde7WA7Zu7HIOTOVQ5sD4HjlmquKZ4qLYT3WhZFZCKY_ISQhrAFZBKQ7JUQFQ8FrCMXm7_TT9bKIblzSukJqlRxxwjHTq6BzQB_rl4iqVpnG9ixv67qemxyFQ1ybMdQ5b2mzoCmfvQnSW4i5xGs_IQWf6gOe785S83t2-3DxkT8_3jzfXT5ktpIqZYGWDdVmBNKpSrWxACAVVx6xo0HDLWF1hbXjJZGG7Ju0rbLpVSkrOsBbFKbnY5qa_fcwYoh5csNj3ZsRpDppDJRjnUvCEyi1q_RSCx06_ezcYv9EM9I9Zvda_ZvWPWQ2lTmZT42I3Y27S81_br8oEXG0BTJt-OvQ6WIejTVEebdTt5P6b8Q3M8Y3u</recordid><startdate>201809</startdate><enddate>201809</enddate><creator>Khajouei, Reza</creator><creator>Ameri, Arefeh</creator><creator>Jahani, Yunes</creator><general>Elsevier B.V</general><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0001-6094-5127</orcidid></search><sort><creationdate>201809</creationdate><title>Evaluating the agreement of users with usability problems identified by heuristic evaluation</title><author>Khajouei, Reza ; Ameri, Arefeh ; Jahani, Yunes</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c368t-415be95706a878d6b044807f1c4bea2c1197e9a25163cfb2014c2c1886621e943</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2018</creationdate><topic>Heuristic Evaluation</topic><topic>Hospital information system</topic><topic>Usability</topic><topic>User-computer interface</topic><topic>Users</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Khajouei, Reza</creatorcontrib><creatorcontrib>Ameri, Arefeh</creatorcontrib><creatorcontrib>Jahani, Yunes</creatorcontrib><collection>PubMed</collection><collection>CrossRef</collection><collection>MEDLINE - Academic</collection><jtitle>International journal of medical informatics (Shannon, Ireland)</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Khajouei, Reza</au><au>Ameri, Arefeh</au><au>Jahani, Yunes</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Evaluating the agreement of users with usability problems identified by heuristic evaluation</atitle><jtitle>International journal of medical informatics (Shannon, Ireland)</jtitle><addtitle>Int J Med Inform</addtitle><date>2018-09</date><risdate>2018</risdate><volume>117</volume><spage>13</spage><epage>18</epage><pages>13-18</pages><issn>1386-5056</issn><eissn>1872-8243</eissn><abstract>•This paper Evaluated the agreement of users with usability problems.•The HE is the desired method for identifying recognizing, diagnosing problems.•The agreement of real user with problems identified by HE is at a medium level.•Younger users has a higher agreement with the problems identified in HE.•The users with lower computer experience had a higher agreement with the results of HE.•The severity of some problems identified by experts are not acknowledge by users. Heuristic evaluation (HE) is an inspection method that can be used for identifying usability problems of health information systems. There is insufficient evidence to what extent these problems are experienced by users in their working environment. The objective of this study was to investigate the agreement of real end users of a health information system with the problems identified by HE. First, the emergency information system of educational hospitals in Kerman, Iran was evaluated by a heuristic evaluation. Then, a valid and reliable (Cronbach’s alpha = 0.95) electronic questionnaire was designed based on the results of the first phase. Among 127 identified problems, the highest number of problems was related to the "consistency and standards" heuristic (18.9%). The user agreement coefficient with heuristic results was 0.5. The highest agreement of the participants was on the problems related to the "help user recognize, diagnose and recover from errors" heuristic (78%). There was a significant relationship between participants age and work experience with a computer (p &lt; 0.05). The agreement rate of real end users with the results of HE was at a medium level. Therefore, in some areas, the perception of evaluators in using this method is not consistent with the users experience with a system. This can result in identifying minor issues and ignoring important system-critical usability issues. The HE is appropriate for identifying specific problems, such as problems related to identifying and correcting errors, but in some areas, such as consistency and standards in the design of a system, it identifies minor or ineffective problems.</abstract><cop>Ireland</cop><pub>Elsevier B.V</pub><pmid>30032960</pmid><doi>10.1016/j.ijmedinf.2018.05.012</doi><tpages>6</tpages><orcidid>https://orcid.org/0000-0001-6094-5127</orcidid></addata></record>
fulltext fulltext
identifier ISSN: 1386-5056
ispartof International journal of medical informatics (Shannon, Ireland), 2018-09, Vol.117, p.13-18
issn 1386-5056
1872-8243
language eng
recordid cdi_proquest_miscellaneous_2074122642
source Elsevier ScienceDirect Journals Complete
subjects Heuristic Evaluation
Hospital information system
Usability
User-computer interface
Users
title Evaluating the agreement of users with usability problems identified by heuristic evaluation
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-29T07%3A10%3A13IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Evaluating%20the%20agreement%20of%20users%20with%20usability%20problems%20identified%20by%20heuristic%20evaluation&rft.jtitle=International%20journal%20of%20medical%20informatics%20(Shannon,%20Ireland)&rft.au=Khajouei,%20Reza&rft.date=2018-09&rft.volume=117&rft.spage=13&rft.epage=18&rft.pages=13-18&rft.issn=1386-5056&rft.eissn=1872-8243&rft_id=info:doi/10.1016/j.ijmedinf.2018.05.012&rft_dat=%3Cproquest_cross%3E2074122642%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2074122642&rft_id=info:pmid/30032960&rft_els_id=S1386505618301254&rfr_iscdi=true