Senior-executive performance: Interrater reliability and rater effects in multi-source ratings
This article explores interrater reliability and rater effects in performance ratings at the senior‐executive level. Studies have shown that substantial rater effects affect the validity of multi‐source ratings, but it is unclear whether these effects hold true at the senior‐executive level. We pres...
Gespeichert in:
| Veröffentlicht in: | Asia Pacific journal of human resources 2011-12, Vol.49 (4), p.425-439 |
|---|---|
| Hauptverfasser: | , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 439 |
|---|---|
| container_issue | 4 |
| container_start_page | 425 |
| container_title | Asia Pacific journal of human resources |
| container_volume | 49 |
| creator | Hutchison, Ann Burch, Giles St J |
| description | This article explores interrater reliability and rater effects in performance ratings at the senior‐executive level. Studies have shown that substantial rater effects affect the validity of multi‐source ratings, but it is unclear whether these effects hold true at the senior‐executive level. We present a study of 189 senior executives in New Zealand and Australia, whose performance was rated by an average of 4.23 raters: superiors, peers, and subordinates. Intra‐class correlation coefficients revealed strong rater effects, and a multi‐trait multi‐method analysis showed that those effects came from individual raters, rather than rater source (i.e. superior, peer, or subordinate). The findings suggest that it may be unwise to aggregate performance ratings at the senior‐executive level, and to use such ratings, whether aggregated or single, to make critical decisions. |
| doi_str_mv | 10.1177/1038411111423187 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_istex</sourceid><recordid>TN_cdi_istex_primary_ark_67375_WNG_L0M8LT3K_4</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><informt_id>10.3316/ielapa.201200860</informt_id><sourcerecordid>2530839941</sourcerecordid><originalsourceid>FETCH-LOGICAL-c4796-8e7688b74dc1c8e0242a57ad1cf9a5062e321e0de79553a20f746c08a93d0aac3</originalsourceid><addsrcrecordid>eNqFkUtv1TAQhSMEEqWwZxmxYRUYP2I77KqKPsQtIChix8h1JsUlr9oO9P57HAV10Q1e2JbPd47H46J4yeANY1q_ZSCMZOuQXDCjHxUHTEtZ6Uayx3mf5WrVnxbPYrwB4CC0OCh-fKXRT6GiO3JL8r-pnCl0Uxjs6OhdeT4mCsHmqQzUe3vle5_2pR3bcjulriOXYunHclj65Ks4LcHRqvrxOj4vnnS2j_Ti33pYfDt5f3l8Vu0-nZ4fH-0qJ3WjKkNaGXOlZeuYMwRccltr2zLXNbYGxUlwRtCSbupaWA6dlsqBsY1owVonDovXW-4cptuFYsLBR0d9b0ealoiNklxKIyCTrx6QN7niMReHDTRSs0xlCDbIhSnGQB3OwQ827JEBru3Gh-3OFrlZ_vie9v_l8ejz2ZdaZdvJZguDT2hn2yX8mdIcsbXJoh_Xv8jKFK6xnfyaJwRT6KnPMHJgHMCo9V3VFuRjorv7-234hUoLXeP3j6e4gwuzuxQfUIq_QoypGA</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>909471483</pqid></control><display><type>article</type><title>Senior-executive performance: Interrater reliability and rater effects in multi-source ratings</title><source>Wiley-Blackwell Journals</source><source>PAIS Index</source><source>SAGE Journals Online</source><creator>Hutchison, Ann ; Burch, Giles St J</creator><creatorcontrib>Hutchison, Ann ; Burch, Giles St J</creatorcontrib><description>This article explores interrater reliability and rater effects in performance ratings at the senior‐executive level. Studies have shown that substantial rater effects affect the validity of multi‐source ratings, but it is unclear whether these effects hold true at the senior‐executive level. We present a study of 189 senior executives in New Zealand and Australia, whose performance was rated by an average of 4.23 raters: superiors, peers, and subordinates. Intra‐class correlation coefficients revealed strong rater effects, and a multi‐trait multi‐method analysis showed that those effects came from individual raters, rather than rater source (i.e. superior, peer, or subordinate). The findings suggest that it may be unwise to aggregate performance ratings at the senior‐executive level, and to use such ratings, whether aggregated or single, to make critical decisions.</description><identifier>ISSN: 1038-4111</identifier><identifier>EISSN: 1744-7941</identifier><identifier>DOI: 10.1177/1038411111423187</identifier><language>eng</language><publisher>Oxford, UK: Blackwell Publishing Ltd</publisher><subject>360-degree appraisal ; Australia ; Business ; Evaluation ; Executives ; Human resource management ; interrater reliability ; New Zealand ; Performance appraisal ; Performance management ; Performance measurement ; rater effects ; Rating ; Ratings & rankings ; Reliability ; senior executives ; Statistics ; Studies ; Surveys ; Workplaces</subject><ispartof>Asia Pacific journal of human resources, 2011-12, Vol.49 (4), p.425-439</ispartof><rights>2011 Australian Human Resources Institute (AHRI)</rights><rights>Copyright Sage Publications Ltd. Dec 2011</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c4796-8e7688b74dc1c8e0242a57ad1cf9a5062e321e0de79553a20f746c08a93d0aac3</citedby></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://onlinelibrary.wiley.com/doi/pdf/10.1177%2F1038411111423187$$EPDF$$P50$$Gwiley$$H</linktopdf><linktohtml>$$Uhttps://onlinelibrary.wiley.com/doi/full/10.1177%2F1038411111423187$$EHTML$$P50$$Gwiley$$H</linktohtml><link.rule.ids>314,780,784,1417,27865,27924,27925,45574,45575</link.rule.ids></links><search><creatorcontrib>Hutchison, Ann</creatorcontrib><creatorcontrib>Burch, Giles St J</creatorcontrib><title>Senior-executive performance: Interrater reliability and rater effects in multi-source ratings</title><title>Asia Pacific journal of human resources</title><description>This article explores interrater reliability and rater effects in performance ratings at the senior‐executive level. Studies have shown that substantial rater effects affect the validity of multi‐source ratings, but it is unclear whether these effects hold true at the senior‐executive level. We present a study of 189 senior executives in New Zealand and Australia, whose performance was rated by an average of 4.23 raters: superiors, peers, and subordinates. Intra‐class correlation coefficients revealed strong rater effects, and a multi‐trait multi‐method analysis showed that those effects came from individual raters, rather than rater source (i.e. superior, peer, or subordinate). The findings suggest that it may be unwise to aggregate performance ratings at the senior‐executive level, and to use such ratings, whether aggregated or single, to make critical decisions.</description><subject>360-degree appraisal</subject><subject>Australia</subject><subject>Business</subject><subject>Evaluation</subject><subject>Executives</subject><subject>Human resource management</subject><subject>interrater reliability</subject><subject>New Zealand</subject><subject>Performance appraisal</subject><subject>Performance management</subject><subject>Performance measurement</subject><subject>rater effects</subject><subject>Rating</subject><subject>Ratings & rankings</subject><subject>Reliability</subject><subject>senior executives</subject><subject>Statistics</subject><subject>Studies</subject><subject>Surveys</subject><subject>Workplaces</subject><issn>1038-4111</issn><issn>1744-7941</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2011</creationdate><recordtype>article</recordtype><sourceid>7TQ</sourceid><recordid>eNqFkUtv1TAQhSMEEqWwZxmxYRUYP2I77KqKPsQtIChix8h1JsUlr9oO9P57HAV10Q1e2JbPd47H46J4yeANY1q_ZSCMZOuQXDCjHxUHTEtZ6Uayx3mf5WrVnxbPYrwB4CC0OCh-fKXRT6GiO3JL8r-pnCl0Uxjs6OhdeT4mCsHmqQzUe3vle5_2pR3bcjulriOXYunHclj65Ks4LcHRqvrxOj4vnnS2j_Ti33pYfDt5f3l8Vu0-nZ4fH-0qJ3WjKkNaGXOlZeuYMwRccltr2zLXNbYGxUlwRtCSbupaWA6dlsqBsY1owVonDovXW-4cptuFYsLBR0d9b0ealoiNklxKIyCTrx6QN7niMReHDTRSs0xlCDbIhSnGQB3OwQ827JEBru3Gh-3OFrlZ_vie9v_l8ejz2ZdaZdvJZguDT2hn2yX8mdIcsbXJoh_Xv8jKFK6xnfyaJwRT6KnPMHJgHMCo9V3VFuRjorv7-234hUoLXeP3j6e4gwuzuxQfUIq_QoypGA</recordid><startdate>201112</startdate><enddate>201112</enddate><creator>Hutchison, Ann</creator><creator>Burch, Giles St J</creator><general>Blackwell Publishing Ltd</general><scope>BSCLL</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7TQ</scope><scope>DHY</scope><scope>DON</scope></search><sort><creationdate>201112</creationdate><title>Senior-executive performance: Interrater reliability and rater effects in multi-source ratings</title><author>Hutchison, Ann ; Burch, Giles St J</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c4796-8e7688b74dc1c8e0242a57ad1cf9a5062e321e0de79553a20f746c08a93d0aac3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2011</creationdate><topic>360-degree appraisal</topic><topic>Australia</topic><topic>Business</topic><topic>Evaluation</topic><topic>Executives</topic><topic>Human resource management</topic><topic>interrater reliability</topic><topic>New Zealand</topic><topic>Performance appraisal</topic><topic>Performance management</topic><topic>Performance measurement</topic><topic>rater effects</topic><topic>Rating</topic><topic>Ratings & rankings</topic><topic>Reliability</topic><topic>senior executives</topic><topic>Statistics</topic><topic>Studies</topic><topic>Surveys</topic><topic>Workplaces</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Hutchison, Ann</creatorcontrib><creatorcontrib>Burch, Giles St J</creatorcontrib><collection>Istex</collection><collection>CrossRef</collection><collection>PAIS Index</collection><collection>PAIS International</collection><collection>PAIS International (Ovid)</collection><jtitle>Asia Pacific journal of human resources</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Hutchison, Ann</au><au>Burch, Giles St J</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Senior-executive performance: Interrater reliability and rater effects in multi-source ratings</atitle><jtitle>Asia Pacific journal of human resources</jtitle><date>2011-12</date><risdate>2011</risdate><volume>49</volume><issue>4</issue><spage>425</spage><epage>439</epage><pages>425-439</pages><issn>1038-4111</issn><eissn>1744-7941</eissn><abstract>This article explores interrater reliability and rater effects in performance ratings at the senior‐executive level. Studies have shown that substantial rater effects affect the validity of multi‐source ratings, but it is unclear whether these effects hold true at the senior‐executive level. We present a study of 189 senior executives in New Zealand and Australia, whose performance was rated by an average of 4.23 raters: superiors, peers, and subordinates. Intra‐class correlation coefficients revealed strong rater effects, and a multi‐trait multi‐method analysis showed that those effects came from individual raters, rather than rater source (i.e. superior, peer, or subordinate). The findings suggest that it may be unwise to aggregate performance ratings at the senior‐executive level, and to use such ratings, whether aggregated or single, to make critical decisions.</abstract><cop>Oxford, UK</cop><pub>Blackwell Publishing Ltd</pub><doi>10.1177/1038411111423187</doi><tpages>15</tpages></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 1038-4111 |
| ispartof | Asia Pacific journal of human resources, 2011-12, Vol.49 (4), p.425-439 |
| issn | 1038-4111 1744-7941 |
| language | eng |
| recordid | cdi_istex_primary_ark_67375_WNG_L0M8LT3K_4 |
| source | Wiley-Blackwell Journals; PAIS Index; SAGE Journals Online |
| subjects | 360-degree appraisal Australia Business Evaluation Executives Human resource management interrater reliability New Zealand Performance appraisal Performance management Performance measurement rater effects Rating Ratings & rankings Reliability senior executives Statistics Studies Surveys Workplaces |
| title | Senior-executive performance: Interrater reliability and rater effects in multi-source ratings |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-20T21%3A30%3A51IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_istex&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Senior-executive%20performance:%20Interrater%20reliability%20and%20rater%20effects%20in%20multi-source%20ratings&rft.jtitle=Asia%20Pacific%20journal%20of%20human%20resources&rft.au=Hutchison,%20Ann&rft.date=2011-12&rft.volume=49&rft.issue=4&rft.spage=425&rft.epage=439&rft.pages=425-439&rft.issn=1038-4111&rft.eissn=1744-7941&rft_id=info:doi/10.1177/1038411111423187&rft_dat=%3Cproquest_istex%3E2530839941%3C/proquest_istex%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=909471483&rft_id=info:pmid/&rft_informt_id=10.3316/ielapa.201200860&rfr_iscdi=true |