Reliability From α to ω: A Tutorial

Reliability is a fundamental problem for measurement in all of science. Although defined in multiple ways, and estimated in even more ways, the basic concepts seem straightforward and need to be understood by practitioners as well as methodologists. Reliability theory is not just for the psychometri...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Psychological assessment 2019-12, Vol.31 (12), p.1395-1411
Hauptverfasser: Revelle, William, Condon, David M.
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 1411
container_issue 12
container_start_page 1395
container_title Psychological assessment
container_volume 31
creator Revelle, William
Condon, David M.
description Reliability is a fundamental problem for measurement in all of science. Although defined in multiple ways, and estimated in even more ways, the basic concepts seem straightforward and need to be understood by practitioners as well as methodologists. Reliability theory is not just for the psychometrician estimating latent variables, it is for everyone who wants to make inferences from measures of individuals or of groups. For the case of a single test administration, we consider multiple measures of reliability, ranging from the worst (β) to average (α, λ3) to best (λ4) split half reliabilities, and consider why model-based estimates (ωh, ωt) should be reported. We also address the utility of test-retest and alternate form reliabilities. The advantages of immediate versus delayed retests to decompose observed score variance into specific, state, and trait scores are discussed. But reliability is not just for test scores, it is also important when evaluating the use of ratings. Estimates that may be applied to continuous data include a set of intraclass correlations while discrete categorical data needs to take advantage of the family of κ statistics. Examples of these various reliability estimates are given using state and trait measures of anxiety given with different delays and under different conditions. An online supplemental materials is provided with more detail and elaboration. The online supplemental materials is also used to demonstrate applications of open source software to examples of real data, and comparisons are made between the many types of reliability. Public Significance Statement A tutorial on the estimation of the reliability of test scores considers classical and model based approaches. Examples using open source software applied to several real world data sets are provided.
doi_str_mv 10.1037/pas0000754
format Article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_2268573118</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2321827941</sourcerecordid><originalsourceid>FETCH-LOGICAL-a417t-fa7708257a85b85518aba70496b6702815831cb26bdc481ea0456ecb57baf47c3</originalsourceid><addsrcrecordid>eNp90EtKBDEQBuDgA2d8bDyANIggQmsqlXTS7kQcFQYEUXAXkpiBHnqm26R7MUfwOF5Er2SG8QEurE0t6uOn-AnZB3oKFOVZayJNIwVfI0MoscwB-dM62QaOqEpk_GkjHSinOYqSDsh2jFNK01WJLTJAQEWLshiSo3tfV8ZWddUtslFoZtn7W9Y12cfreXaRPfRdEypT75LNiamj3_vaO-RxdPVweZOP765vLy_GueEgu3xipKSKCWmUsEoIUMYaSXlZ2EJSpkAoBGdZYZ8dV-AN5aLwzgppzYRLhzvkeJXbhual97HTsyo6X9dm7ps-asYKJSQCqEQP_9Bp04d5-k4zZKCYLDn8q1IWSiYBkzpZKReaGIOf6DZUMxMWGqhe9q1_-0744CuytzP__EO_O00gXwHTGt3GhTOhq1zto-tD8PNuGZa0BqYBS4Gf-F-FQA</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2268372713</pqid></control><display><type>article</type><title>Reliability From α to ω: A Tutorial</title><source>APA PsycARTICLES</source><source>MEDLINE</source><creator>Revelle, William ; Condon, David M.</creator><contributor>Ben-Porath, Yossef S ; Sellbom, Martin ; Simms, Leonard J</contributor><creatorcontrib>Revelle, William ; Condon, David M. ; Ben-Porath, Yossef S ; Sellbom, Martin ; Simms, Leonard J</creatorcontrib><description>Reliability is a fundamental problem for measurement in all of science. Although defined in multiple ways, and estimated in even more ways, the basic concepts seem straightforward and need to be understood by practitioners as well as methodologists. Reliability theory is not just for the psychometrician estimating latent variables, it is for everyone who wants to make inferences from measures of individuals or of groups. For the case of a single test administration, we consider multiple measures of reliability, ranging from the worst (β) to average (α, λ3) to best (λ4) split half reliabilities, and consider why model-based estimates (ωh, ωt) should be reported. We also address the utility of test-retest and alternate form reliabilities. The advantages of immediate versus delayed retests to decompose observed score variance into specific, state, and trait scores are discussed. But reliability is not just for test scores, it is also important when evaluating the use of ratings. Estimates that may be applied to continuous data include a set of intraclass correlations while discrete categorical data needs to take advantage of the family of κ statistics. Examples of these various reliability estimates are given using state and trait measures of anxiety given with different delays and under different conditions. An online supplemental materials is provided with more detail and elaboration. The online supplemental materials is also used to demonstrate applications of open source software to examples of real data, and comparisons are made between the many types of reliability. Public Significance Statement A tutorial on the estimation of the reliability of test scores considers classical and model based approaches. Examples using open source software applied to several real world data sets are provided.</description><identifier>ISSN: 1040-3590</identifier><identifier>ISBN: 143389324X</identifier><identifier>ISBN: 9781433893247</identifier><identifier>EISSN: 1939-134X</identifier><identifier>DOI: 10.1037/pas0000754</identifier><identifier>PMID: 31380696</identifier><language>eng</language><publisher>United States: American Psychological Association</publisher><subject>Classical Test Theory ; Concepts ; Estimation bias ; Human ; Humans ; Latent Variables ; Measurement ; Mental Disorders - diagnosis ; Psychological research ; Psychometrics ; Psychometrics - methods ; Psychometrics - standards ; Quantitative psychology ; Reliability ; Reproducibility of Results ; Statistical Variables ; Test Reliability ; Test Scores</subject><ispartof>Psychological assessment, 2019-12, Vol.31 (12), p.1395-1411</ispartof><rights>2019 American Psychological Association</rights><rights>2019, American Psychological Association</rights><rights>Copyright American Psychological Association Dec 2019</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-a417t-fa7708257a85b85518aba70496b6702815831cb26bdc481ea0456ecb57baf47c3</citedby><orcidid>0000-0003-4880-9610 ; 0000-0002-8406-783X</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,776,780,27901,27902</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/31380696$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><contributor>Ben-Porath, Yossef S</contributor><contributor>Sellbom, Martin</contributor><contributor>Simms, Leonard J</contributor><creatorcontrib>Revelle, William</creatorcontrib><creatorcontrib>Condon, David M.</creatorcontrib><title>Reliability From α to ω: A Tutorial</title><title>Psychological assessment</title><addtitle>Psychol Assess</addtitle><description>Reliability is a fundamental problem for measurement in all of science. Although defined in multiple ways, and estimated in even more ways, the basic concepts seem straightforward and need to be understood by practitioners as well as methodologists. Reliability theory is not just for the psychometrician estimating latent variables, it is for everyone who wants to make inferences from measures of individuals or of groups. For the case of a single test administration, we consider multiple measures of reliability, ranging from the worst (β) to average (α, λ3) to best (λ4) split half reliabilities, and consider why model-based estimates (ωh, ωt) should be reported. We also address the utility of test-retest and alternate form reliabilities. The advantages of immediate versus delayed retests to decompose observed score variance into specific, state, and trait scores are discussed. But reliability is not just for test scores, it is also important when evaluating the use of ratings. Estimates that may be applied to continuous data include a set of intraclass correlations while discrete categorical data needs to take advantage of the family of κ statistics. Examples of these various reliability estimates are given using state and trait measures of anxiety given with different delays and under different conditions. An online supplemental materials is provided with more detail and elaboration. The online supplemental materials is also used to demonstrate applications of open source software to examples of real data, and comparisons are made between the many types of reliability. Public Significance Statement A tutorial on the estimation of the reliability of test scores considers classical and model based approaches. Examples using open source software applied to several real world data sets are provided.</description><subject>Classical Test Theory</subject><subject>Concepts</subject><subject>Estimation bias</subject><subject>Human</subject><subject>Humans</subject><subject>Latent Variables</subject><subject>Measurement</subject><subject>Mental Disorders - diagnosis</subject><subject>Psychological research</subject><subject>Psychometrics</subject><subject>Psychometrics - methods</subject><subject>Psychometrics - standards</subject><subject>Quantitative psychology</subject><subject>Reliability</subject><subject>Reproducibility of Results</subject><subject>Statistical Variables</subject><subject>Test Reliability</subject><subject>Test Scores</subject><issn>1040-3590</issn><issn>1939-134X</issn><isbn>143389324X</isbn><isbn>9781433893247</isbn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2019</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><recordid>eNp90EtKBDEQBuDgA2d8bDyANIggQmsqlXTS7kQcFQYEUXAXkpiBHnqm26R7MUfwOF5Er2SG8QEurE0t6uOn-AnZB3oKFOVZayJNIwVfI0MoscwB-dM62QaOqEpk_GkjHSinOYqSDsh2jFNK01WJLTJAQEWLshiSo3tfV8ZWddUtslFoZtn7W9Y12cfreXaRPfRdEypT75LNiamj3_vaO-RxdPVweZOP765vLy_GueEgu3xipKSKCWmUsEoIUMYaSXlZ2EJSpkAoBGdZYZ8dV-AN5aLwzgppzYRLhzvkeJXbhual97HTsyo6X9dm7ps-asYKJSQCqEQP_9Bp04d5-k4zZKCYLDn8q1IWSiYBkzpZKReaGIOf6DZUMxMWGqhe9q1_-0744CuytzP__EO_O00gXwHTGt3GhTOhq1zto-tD8PNuGZa0BqYBS4Gf-F-FQA</recordid><startdate>201912</startdate><enddate>201912</enddate><creator>Revelle, William</creator><creator>Condon, David M.</creator><general>American Psychological Association</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7RZ</scope><scope>PHGZM</scope><scope>PHGZT</scope><scope>PKEHL</scope><scope>PSYQQ</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0003-4880-9610</orcidid><orcidid>https://orcid.org/0000-0002-8406-783X</orcidid></search><sort><creationdate>201912</creationdate><title>Reliability From α to ω: A Tutorial</title><author>Revelle, William ; Condon, David M.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a417t-fa7708257a85b85518aba70496b6702815831cb26bdc481ea0456ecb57baf47c3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2019</creationdate><topic>Classical Test Theory</topic><topic>Concepts</topic><topic>Estimation bias</topic><topic>Human</topic><topic>Humans</topic><topic>Latent Variables</topic><topic>Measurement</topic><topic>Mental Disorders - diagnosis</topic><topic>Psychological research</topic><topic>Psychometrics</topic><topic>Psychometrics - methods</topic><topic>Psychometrics - standards</topic><topic>Quantitative psychology</topic><topic>Reliability</topic><topic>Reproducibility of Results</topic><topic>Statistical Variables</topic><topic>Test Reliability</topic><topic>Test Scores</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Revelle, William</creatorcontrib><creatorcontrib>Condon, David M.</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>APA PsycArticles®</collection><collection>ProQuest Central (New)</collection><collection>ProQuest One Academic (New)</collection><collection>ProQuest One Academic Middle East (New)</collection><collection>ProQuest One Psychology</collection><collection>MEDLINE - Academic</collection><jtitle>Psychological assessment</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Revelle, William</au><au>Condon, David M.</au><au>Ben-Porath, Yossef S</au><au>Sellbom, Martin</au><au>Simms, Leonard J</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Reliability From α to ω: A Tutorial</atitle><jtitle>Psychological assessment</jtitle><addtitle>Psychol Assess</addtitle><date>2019-12</date><risdate>2019</risdate><volume>31</volume><issue>12</issue><spage>1395</spage><epage>1411</epage><pages>1395-1411</pages><issn>1040-3590</issn><eissn>1939-134X</eissn><isbn>143389324X</isbn><isbn>9781433893247</isbn><abstract>Reliability is a fundamental problem for measurement in all of science. Although defined in multiple ways, and estimated in even more ways, the basic concepts seem straightforward and need to be understood by practitioners as well as methodologists. Reliability theory is not just for the psychometrician estimating latent variables, it is for everyone who wants to make inferences from measures of individuals or of groups. For the case of a single test administration, we consider multiple measures of reliability, ranging from the worst (β) to average (α, λ3) to best (λ4) split half reliabilities, and consider why model-based estimates (ωh, ωt) should be reported. We also address the utility of test-retest and alternate form reliabilities. The advantages of immediate versus delayed retests to decompose observed score variance into specific, state, and trait scores are discussed. But reliability is not just for test scores, it is also important when evaluating the use of ratings. Estimates that may be applied to continuous data include a set of intraclass correlations while discrete categorical data needs to take advantage of the family of κ statistics. Examples of these various reliability estimates are given using state and trait measures of anxiety given with different delays and under different conditions. An online supplemental materials is provided with more detail and elaboration. The online supplemental materials is also used to demonstrate applications of open source software to examples of real data, and comparisons are made between the many types of reliability. Public Significance Statement A tutorial on the estimation of the reliability of test scores considers classical and model based approaches. Examples using open source software applied to several real world data sets are provided.</abstract><cop>United States</cop><pub>American Psychological Association</pub><pmid>31380696</pmid><doi>10.1037/pas0000754</doi><tpages>17</tpages><orcidid>https://orcid.org/0000-0003-4880-9610</orcidid><orcidid>https://orcid.org/0000-0002-8406-783X</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 1040-3590
ispartof Psychological assessment, 2019-12, Vol.31 (12), p.1395-1411
issn 1040-3590
1939-134X
language eng
recordid cdi_proquest_miscellaneous_2268573118
source APA PsycARTICLES; MEDLINE
subjects Classical Test Theory
Concepts
Estimation bias
Human
Humans
Latent Variables
Measurement
Mental Disorders - diagnosis
Psychological research
Psychometrics
Psychometrics - methods
Psychometrics - standards
Quantitative psychology
Reliability
Reproducibility of Results
Statistical Variables
Test Reliability
Test Scores
title Reliability From α to ω: A Tutorial
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-18T23%3A44%3A38IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Reliability%20From%20%CE%B1%20to%20%CF%89:%20A%20Tutorial&rft.jtitle=Psychological%20assessment&rft.au=Revelle,%20William&rft.date=2019-12&rft.volume=31&rft.issue=12&rft.spage=1395&rft.epage=1411&rft.pages=1395-1411&rft.issn=1040-3590&rft.eissn=1939-134X&rft.isbn=143389324X&rft.isbn_list=9781433893247&rft_id=info:doi/10.1037/pas0000754&rft_dat=%3Cproquest_cross%3E2321827941%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2268372713&rft_id=info:pmid/31380696&rfr_iscdi=true