Reliability From α to ω: A Tutorial
Reliability is a fundamental problem for measurement in all of science. Although defined in multiple ways, and estimated in even more ways, the basic concepts seem straightforward and need to be understood by practitioners as well as methodologists. Reliability theory is not just for the psychometri...
Gespeichert in:
Veröffentlicht in: | Psychological assessment 2019-12, Vol.31 (12), p.1395-1411 |
---|---|
Hauptverfasser: | , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 1411 |
---|---|
container_issue | 12 |
container_start_page | 1395 |
container_title | Psychological assessment |
container_volume | 31 |
creator | Revelle, William Condon, David M. |
description | Reliability is a fundamental problem for measurement in all of science. Although defined in multiple ways, and estimated in even more ways, the basic concepts seem straightforward and need to be understood by practitioners as well as methodologists. Reliability theory is not just for the psychometrician estimating latent variables, it is for everyone who wants to make inferences from measures of individuals or of groups. For the case of a single test administration, we consider multiple measures of reliability, ranging from the worst (β) to average (α, λ3) to best (λ4) split half reliabilities, and consider why model-based estimates (ωh, ωt) should be reported. We also address the utility of test-retest and alternate form reliabilities. The advantages of immediate versus delayed retests to decompose observed score variance into specific, state, and trait scores are discussed. But reliability is not just for test scores, it is also important when evaluating the use of ratings. Estimates that may be applied to continuous data include a set of intraclass correlations while discrete categorical data needs to take advantage of the family of κ statistics. Examples of these various reliability estimates are given using state and trait measures of anxiety given with different delays and under different conditions. An online supplemental materials is provided with more detail and elaboration. The online supplemental materials is also used to demonstrate applications of open source software to examples of real data, and comparisons are made between the many types of reliability.
Public Significance Statement
A tutorial on the estimation of the reliability of test scores considers classical and model based approaches. Examples using open source software applied to several real world data sets are provided. |
doi_str_mv | 10.1037/pas0000754 |
format | Article |
fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_2268573118</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2321827941</sourcerecordid><originalsourceid>FETCH-LOGICAL-a417t-fa7708257a85b85518aba70496b6702815831cb26bdc481ea0456ecb57baf47c3</originalsourceid><addsrcrecordid>eNp90EtKBDEQBuDgA2d8bDyANIggQmsqlXTS7kQcFQYEUXAXkpiBHnqm26R7MUfwOF5Er2SG8QEurE0t6uOn-AnZB3oKFOVZayJNIwVfI0MoscwB-dM62QaOqEpk_GkjHSinOYqSDsh2jFNK01WJLTJAQEWLshiSo3tfV8ZWddUtslFoZtn7W9Y12cfreXaRPfRdEypT75LNiamj3_vaO-RxdPVweZOP765vLy_GueEgu3xipKSKCWmUsEoIUMYaSXlZ2EJSpkAoBGdZYZ8dV-AN5aLwzgppzYRLhzvkeJXbhual97HTsyo6X9dm7ps-asYKJSQCqEQP_9Bp04d5-k4zZKCYLDn8q1IWSiYBkzpZKReaGIOf6DZUMxMWGqhe9q1_-0744CuytzP__EO_O00gXwHTGt3GhTOhq1zto-tD8PNuGZa0BqYBS4Gf-F-FQA</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2268372713</pqid></control><display><type>article</type><title>Reliability From α to ω: A Tutorial</title><source>APA PsycARTICLES</source><source>MEDLINE</source><creator>Revelle, William ; Condon, David M.</creator><contributor>Ben-Porath, Yossef S ; Sellbom, Martin ; Simms, Leonard J</contributor><creatorcontrib>Revelle, William ; Condon, David M. ; Ben-Porath, Yossef S ; Sellbom, Martin ; Simms, Leonard J</creatorcontrib><description>Reliability is a fundamental problem for measurement in all of science. Although defined in multiple ways, and estimated in even more ways, the basic concepts seem straightforward and need to be understood by practitioners as well as methodologists. Reliability theory is not just for the psychometrician estimating latent variables, it is for everyone who wants to make inferences from measures of individuals or of groups. For the case of a single test administration, we consider multiple measures of reliability, ranging from the worst (β) to average (α, λ3) to best (λ4) split half reliabilities, and consider why model-based estimates (ωh, ωt) should be reported. We also address the utility of test-retest and alternate form reliabilities. The advantages of immediate versus delayed retests to decompose observed score variance into specific, state, and trait scores are discussed. But reliability is not just for test scores, it is also important when evaluating the use of ratings. Estimates that may be applied to continuous data include a set of intraclass correlations while discrete categorical data needs to take advantage of the family of κ statistics. Examples of these various reliability estimates are given using state and trait measures of anxiety given with different delays and under different conditions. An online supplemental materials is provided with more detail and elaboration. The online supplemental materials is also used to demonstrate applications of open source software to examples of real data, and comparisons are made between the many types of reliability.
Public Significance Statement
A tutorial on the estimation of the reliability of test scores considers classical and model based approaches. Examples using open source software applied to several real world data sets are provided.</description><identifier>ISSN: 1040-3590</identifier><identifier>ISBN: 143389324X</identifier><identifier>ISBN: 9781433893247</identifier><identifier>EISSN: 1939-134X</identifier><identifier>DOI: 10.1037/pas0000754</identifier><identifier>PMID: 31380696</identifier><language>eng</language><publisher>United States: American Psychological Association</publisher><subject>Classical Test Theory ; Concepts ; Estimation bias ; Human ; Humans ; Latent Variables ; Measurement ; Mental Disorders - diagnosis ; Psychological research ; Psychometrics ; Psychometrics - methods ; Psychometrics - standards ; Quantitative psychology ; Reliability ; Reproducibility of Results ; Statistical Variables ; Test Reliability ; Test Scores</subject><ispartof>Psychological assessment, 2019-12, Vol.31 (12), p.1395-1411</ispartof><rights>2019 American Psychological Association</rights><rights>2019, American Psychological Association</rights><rights>Copyright American Psychological Association Dec 2019</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-a417t-fa7708257a85b85518aba70496b6702815831cb26bdc481ea0456ecb57baf47c3</citedby><orcidid>0000-0003-4880-9610 ; 0000-0002-8406-783X</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,776,780,27901,27902</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/31380696$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><contributor>Ben-Porath, Yossef S</contributor><contributor>Sellbom, Martin</contributor><contributor>Simms, Leonard J</contributor><creatorcontrib>Revelle, William</creatorcontrib><creatorcontrib>Condon, David M.</creatorcontrib><title>Reliability From α to ω: A Tutorial</title><title>Psychological assessment</title><addtitle>Psychol Assess</addtitle><description>Reliability is a fundamental problem for measurement in all of science. Although defined in multiple ways, and estimated in even more ways, the basic concepts seem straightforward and need to be understood by practitioners as well as methodologists. Reliability theory is not just for the psychometrician estimating latent variables, it is for everyone who wants to make inferences from measures of individuals or of groups. For the case of a single test administration, we consider multiple measures of reliability, ranging from the worst (β) to average (α, λ3) to best (λ4) split half reliabilities, and consider why model-based estimates (ωh, ωt) should be reported. We also address the utility of test-retest and alternate form reliabilities. The advantages of immediate versus delayed retests to decompose observed score variance into specific, state, and trait scores are discussed. But reliability is not just for test scores, it is also important when evaluating the use of ratings. Estimates that may be applied to continuous data include a set of intraclass correlations while discrete categorical data needs to take advantage of the family of κ statistics. Examples of these various reliability estimates are given using state and trait measures of anxiety given with different delays and under different conditions. An online supplemental materials is provided with more detail and elaboration. The online supplemental materials is also used to demonstrate applications of open source software to examples of real data, and comparisons are made between the many types of reliability.
Public Significance Statement
A tutorial on the estimation of the reliability of test scores considers classical and model based approaches. Examples using open source software applied to several real world data sets are provided.</description><subject>Classical Test Theory</subject><subject>Concepts</subject><subject>Estimation bias</subject><subject>Human</subject><subject>Humans</subject><subject>Latent Variables</subject><subject>Measurement</subject><subject>Mental Disorders - diagnosis</subject><subject>Psychological research</subject><subject>Psychometrics</subject><subject>Psychometrics - methods</subject><subject>Psychometrics - standards</subject><subject>Quantitative psychology</subject><subject>Reliability</subject><subject>Reproducibility of Results</subject><subject>Statistical Variables</subject><subject>Test Reliability</subject><subject>Test Scores</subject><issn>1040-3590</issn><issn>1939-134X</issn><isbn>143389324X</isbn><isbn>9781433893247</isbn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2019</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><recordid>eNp90EtKBDEQBuDgA2d8bDyANIggQmsqlXTS7kQcFQYEUXAXkpiBHnqm26R7MUfwOF5Er2SG8QEurE0t6uOn-AnZB3oKFOVZayJNIwVfI0MoscwB-dM62QaOqEpk_GkjHSinOYqSDsh2jFNK01WJLTJAQEWLshiSo3tfV8ZWddUtslFoZtn7W9Y12cfreXaRPfRdEypT75LNiamj3_vaO-RxdPVweZOP765vLy_GueEgu3xipKSKCWmUsEoIUMYaSXlZ2EJSpkAoBGdZYZ8dV-AN5aLwzgppzYRLhzvkeJXbhual97HTsyo6X9dm7ps-asYKJSQCqEQP_9Bp04d5-k4zZKCYLDn8q1IWSiYBkzpZKReaGIOf6DZUMxMWGqhe9q1_-0744CuytzP__EO_O00gXwHTGt3GhTOhq1zto-tD8PNuGZa0BqYBS4Gf-F-FQA</recordid><startdate>201912</startdate><enddate>201912</enddate><creator>Revelle, William</creator><creator>Condon, David M.</creator><general>American Psychological Association</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7RZ</scope><scope>PHGZM</scope><scope>PHGZT</scope><scope>PKEHL</scope><scope>PSYQQ</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0003-4880-9610</orcidid><orcidid>https://orcid.org/0000-0002-8406-783X</orcidid></search><sort><creationdate>201912</creationdate><title>Reliability From α to ω: A Tutorial</title><author>Revelle, William ; Condon, David M.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a417t-fa7708257a85b85518aba70496b6702815831cb26bdc481ea0456ecb57baf47c3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2019</creationdate><topic>Classical Test Theory</topic><topic>Concepts</topic><topic>Estimation bias</topic><topic>Human</topic><topic>Humans</topic><topic>Latent Variables</topic><topic>Measurement</topic><topic>Mental Disorders - diagnosis</topic><topic>Psychological research</topic><topic>Psychometrics</topic><topic>Psychometrics - methods</topic><topic>Psychometrics - standards</topic><topic>Quantitative psychology</topic><topic>Reliability</topic><topic>Reproducibility of Results</topic><topic>Statistical Variables</topic><topic>Test Reliability</topic><topic>Test Scores</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Revelle, William</creatorcontrib><creatorcontrib>Condon, David M.</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>APA PsycArticles®</collection><collection>ProQuest Central (New)</collection><collection>ProQuest One Academic (New)</collection><collection>ProQuest One Academic Middle East (New)</collection><collection>ProQuest One Psychology</collection><collection>MEDLINE - Academic</collection><jtitle>Psychological assessment</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Revelle, William</au><au>Condon, David M.</au><au>Ben-Porath, Yossef S</au><au>Sellbom, Martin</au><au>Simms, Leonard J</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Reliability From α to ω: A Tutorial</atitle><jtitle>Psychological assessment</jtitle><addtitle>Psychol Assess</addtitle><date>2019-12</date><risdate>2019</risdate><volume>31</volume><issue>12</issue><spage>1395</spage><epage>1411</epage><pages>1395-1411</pages><issn>1040-3590</issn><eissn>1939-134X</eissn><isbn>143389324X</isbn><isbn>9781433893247</isbn><abstract>Reliability is a fundamental problem for measurement in all of science. Although defined in multiple ways, and estimated in even more ways, the basic concepts seem straightforward and need to be understood by practitioners as well as methodologists. Reliability theory is not just for the psychometrician estimating latent variables, it is for everyone who wants to make inferences from measures of individuals or of groups. For the case of a single test administration, we consider multiple measures of reliability, ranging from the worst (β) to average (α, λ3) to best (λ4) split half reliabilities, and consider why model-based estimates (ωh, ωt) should be reported. We also address the utility of test-retest and alternate form reliabilities. The advantages of immediate versus delayed retests to decompose observed score variance into specific, state, and trait scores are discussed. But reliability is not just for test scores, it is also important when evaluating the use of ratings. Estimates that may be applied to continuous data include a set of intraclass correlations while discrete categorical data needs to take advantage of the family of κ statistics. Examples of these various reliability estimates are given using state and trait measures of anxiety given with different delays and under different conditions. An online supplemental materials is provided with more detail and elaboration. The online supplemental materials is also used to demonstrate applications of open source software to examples of real data, and comparisons are made between the many types of reliability.
Public Significance Statement
A tutorial on the estimation of the reliability of test scores considers classical and model based approaches. Examples using open source software applied to several real world data sets are provided.</abstract><cop>United States</cop><pub>American Psychological Association</pub><pmid>31380696</pmid><doi>10.1037/pas0000754</doi><tpages>17</tpages><orcidid>https://orcid.org/0000-0003-4880-9610</orcidid><orcidid>https://orcid.org/0000-0002-8406-783X</orcidid><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | ISSN: 1040-3590 |
ispartof | Psychological assessment, 2019-12, Vol.31 (12), p.1395-1411 |
issn | 1040-3590 1939-134X |
language | eng |
recordid | cdi_proquest_miscellaneous_2268573118 |
source | APA PsycARTICLES; MEDLINE |
subjects | Classical Test Theory Concepts Estimation bias Human Humans Latent Variables Measurement Mental Disorders - diagnosis Psychological research Psychometrics Psychometrics - methods Psychometrics - standards Quantitative psychology Reliability Reproducibility of Results Statistical Variables Test Reliability Test Scores |
title | Reliability From α to ω: A Tutorial |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-18T23%3A44%3A38IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Reliability%20From%20%CE%B1%20to%20%CF%89:%20A%20Tutorial&rft.jtitle=Psychological%20assessment&rft.au=Revelle,%20William&rft.date=2019-12&rft.volume=31&rft.issue=12&rft.spage=1395&rft.epage=1411&rft.pages=1395-1411&rft.issn=1040-3590&rft.eissn=1939-134X&rft.isbn=143389324X&rft.isbn_list=9781433893247&rft_id=info:doi/10.1037/pas0000754&rft_dat=%3Cproquest_cross%3E2321827941%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2268372713&rft_id=info:pmid/31380696&rfr_iscdi=true |