The Misreporting Trade-Off Between List Experiments and Direct Questions in Practice: Partition Validation Evidence from Two Countries

To reduce strategic misreporting on sensitive topics, survey researchers increasingly use list experiments rather than direct questions. However, the complexity of list experiments may increase nonstrategic misreporting. We provide the first empirical assessment of this trade-off between strategic a...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Political analysis 2022-07, Vol.30 (3), p.381-402
Hauptverfasser: Kuhn, Patrick M., Vivyan, Nick
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 402
container_issue 3
container_start_page 381
container_title Political analysis
container_volume 30
creator Kuhn, Patrick M.
Vivyan, Nick
description To reduce strategic misreporting on sensitive topics, survey researchers increasingly use list experiments rather than direct questions. However, the complexity of list experiments may increase nonstrategic misreporting. We provide the first empirical assessment of this trade-off between strategic and nonstrategic misreporting. We field list experiments on election turnout in two different countries, collecting measures of respondents’ true turnout. We detail and apply a partition validation method which uses true scores to distinguish true and false positives and negatives for list experiments, thus allowing detection of nonstrategic reporting errors. For both list experiments, partition validation reveals nonstrategic misreporting that is: undetected by standard diagnostics or validation; greater than assumed in extant simulation studies; and severe enough that direct turnout questions subject to strategic misreporting exhibit lower overall reporting error. We discuss how our results can inform the choice between list experiment and direct question for other topics and survey contexts.
doi_str_mv 10.1017/pan.2021.10
format Article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2671915660</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><cupid>10_1017_pan_2021_10</cupid><sourcerecordid>2671915660</sourcerecordid><originalsourceid>FETCH-LOGICAL-c332t-65e0e37291c0987bb78b06fc052293fba243be2e454a5e35fe70a179a04fd6ad3</originalsourceid><addsrcrecordid>eNptkN1KAzEQhYMoWKtXvkDAS9maZH_SeKe1_kClFVZvl-zupKbYZE1Sqy_gc5vagjdezZnh48zMQeiUkgEllF900gwYYTR2e6hHM14kmRiK_ahJxhMqhvwQHXm_IJHmQvTQd_kK-FF7B511QZs5Lp1sIZkqha8hrAEMnmgf8PizA6eXYILH0rT4RjtoAn5agQ_aGo-1wTMnm6AbuMQzGc02c_wi33Qrf-X4Q7dgGsDK2SUu1xaP7MoEp8EfowMl3zyc7GofPd-Oy9F9MpnePYyuJkmTpiwkRQ4EUs4EbUj8pa75sCaFakjOmEhVLVmW1sAgyzOZQ5or4ERSLiTJVFvINu2js61v5-z75vRqYVfOxJUVKzgVNC8KEqnzLdU462M0quri69J9VZRUm6CrGHS1CTp2kU52tFzWTrdz-DP9j_8B-zaBYw</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2671915660</pqid></control><display><type>article</type><title>The Misreporting Trade-Off Between List Experiments and Direct Questions in Practice: Partition Validation Evidence from Two Countries</title><source>Worldwide Political Science Abstracts</source><source>EBSCOhost Political Science Complete</source><source>Cambridge University Press Journals Complete</source><creator>Kuhn, Patrick M. ; Vivyan, Nick</creator><creatorcontrib>Kuhn, Patrick M. ; Vivyan, Nick</creatorcontrib><description>To reduce strategic misreporting on sensitive topics, survey researchers increasingly use list experiments rather than direct questions. However, the complexity of list experiments may increase nonstrategic misreporting. We provide the first empirical assessment of this trade-off between strategic and nonstrategic misreporting. We field list experiments on election turnout in two different countries, collecting measures of respondents’ true turnout. We detail and apply a partition validation method which uses true scores to distinguish true and false positives and negatives for list experiments, thus allowing detection of nonstrategic reporting errors. For both list experiments, partition validation reveals nonstrategic misreporting that is: undetected by standard diagnostics or validation; greater than assumed in extant simulation studies; and severe enough that direct turnout questions subject to strategic misreporting exhibit lower overall reporting error. We discuss how our results can inform the choice between list experiment and direct question for other topics and survey contexts.</description><identifier>ISSN: 1047-1987</identifier><identifier>EISSN: 1476-4989</identifier><identifier>DOI: 10.1017/pan.2021.10</identifier><language>eng</language><publisher>New York, USA: Cambridge University Press</publisher><subject>Elections ; Errors ; Experiments ; Partition ; Respondents ; Simulation ; Validity ; Voter turnout</subject><ispartof>Political analysis, 2022-07, Vol.30 (3), p.381-402</ispartof><rights>The Author(s) 2021. Published by Cambridge University Press on behalf of the Society for Political Methodology</rights><rights>The Author(s) 2021. Published by Cambridge University Press on behalf of the Society for Political Methodology. This work is licensed under the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c332t-65e0e37291c0987bb78b06fc052293fba243be2e454a5e35fe70a179a04fd6ad3</citedby><cites>FETCH-LOGICAL-c332t-65e0e37291c0987bb78b06fc052293fba243be2e454a5e35fe70a179a04fd6ad3</cites><orcidid>0000-0002-8849-0248 ; 0000-0002-6793-0953</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.cambridge.org/core/product/identifier/S1047198721000103/type/journal_article$$EHTML$$P50$$Gcambridge$$Hfree_for_read</linktohtml><link.rule.ids>164,314,776,780,12825,27903,27904,55607</link.rule.ids></links><search><creatorcontrib>Kuhn, Patrick M.</creatorcontrib><creatorcontrib>Vivyan, Nick</creatorcontrib><title>The Misreporting Trade-Off Between List Experiments and Direct Questions in Practice: Partition Validation Evidence from Two Countries</title><title>Political analysis</title><addtitle>Polit. Anal</addtitle><description>To reduce strategic misreporting on sensitive topics, survey researchers increasingly use list experiments rather than direct questions. However, the complexity of list experiments may increase nonstrategic misreporting. We provide the first empirical assessment of this trade-off between strategic and nonstrategic misreporting. We field list experiments on election turnout in two different countries, collecting measures of respondents’ true turnout. We detail and apply a partition validation method which uses true scores to distinguish true and false positives and negatives for list experiments, thus allowing detection of nonstrategic reporting errors. For both list experiments, partition validation reveals nonstrategic misreporting that is: undetected by standard diagnostics or validation; greater than assumed in extant simulation studies; and severe enough that direct turnout questions subject to strategic misreporting exhibit lower overall reporting error. We discuss how our results can inform the choice between list experiment and direct question for other topics and survey contexts.</description><subject>Elections</subject><subject>Errors</subject><subject>Experiments</subject><subject>Partition</subject><subject>Respondents</subject><subject>Simulation</subject><subject>Validity</subject><subject>Voter turnout</subject><issn>1047-1987</issn><issn>1476-4989</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><sourceid>IKXGN</sourceid><sourceid>7UB</sourceid><sourceid>8G5</sourceid><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GNUQQ</sourceid><sourceid>GUQSH</sourceid><sourceid>M2O</sourceid><recordid>eNptkN1KAzEQhYMoWKtXvkDAS9maZH_SeKe1_kClFVZvl-zupKbYZE1Sqy_gc5vagjdezZnh48zMQeiUkgEllF900gwYYTR2e6hHM14kmRiK_ahJxhMqhvwQHXm_IJHmQvTQd_kK-FF7B511QZs5Lp1sIZkqha8hrAEMnmgf8PizA6eXYILH0rT4RjtoAn5agQ_aGo-1wTMnm6AbuMQzGc02c_wi33Qrf-X4Q7dgGsDK2SUu1xaP7MoEp8EfowMl3zyc7GofPd-Oy9F9MpnePYyuJkmTpiwkRQ4EUs4EbUj8pa75sCaFakjOmEhVLVmW1sAgyzOZQ5or4ERSLiTJVFvINu2js61v5-z75vRqYVfOxJUVKzgVNC8KEqnzLdU462M0quri69J9VZRUm6CrGHS1CTp2kU52tFzWTrdz-DP9j_8B-zaBYw</recordid><startdate>20220701</startdate><enddate>20220701</enddate><creator>Kuhn, Patrick M.</creator><creator>Vivyan, Nick</creator><general>Cambridge University Press</general><scope>IKXGN</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>0-V</scope><scope>3V.</scope><scope>7UB</scope><scope>7XB</scope><scope>88F</scope><scope>88J</scope><scope>8FK</scope><scope>8G5</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ALSLI</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>CCPQU</scope><scope>DPSOV</scope><scope>DWQXO</scope><scope>GNUQQ</scope><scope>GUQSH</scope><scope>KC-</scope><scope>M1Q</scope><scope>M2L</scope><scope>M2O</scope><scope>M2R</scope><scope>MBDVC</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>Q9U</scope><orcidid>https://orcid.org/0000-0002-8849-0248</orcidid><orcidid>https://orcid.org/0000-0002-6793-0953</orcidid></search><sort><creationdate>20220701</creationdate><title>The Misreporting Trade-Off Between List Experiments and Direct Questions in Practice: Partition Validation Evidence from Two Countries</title><author>Kuhn, Patrick M. ; Vivyan, Nick</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c332t-65e0e37291c0987bb78b06fc052293fba243be2e454a5e35fe70a179a04fd6ad3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Elections</topic><topic>Errors</topic><topic>Experiments</topic><topic>Partition</topic><topic>Respondents</topic><topic>Simulation</topic><topic>Validity</topic><topic>Voter turnout</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Kuhn, Patrick M.</creatorcontrib><creatorcontrib>Vivyan, Nick</creatorcontrib><collection>Cambridge University Press Wholly Gold Open Access Journals</collection><collection>CrossRef</collection><collection>ProQuest Social Sciences Premium Collection</collection><collection>ProQuest Central (Corporate)</collection><collection>Worldwide Political Science Abstracts</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>Military Database (Alumni Edition)</collection><collection>Social Science Database (Alumni Edition)</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>Research Library (Alumni Edition)</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>Social Science Premium Collection</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>ProQuest One Community College</collection><collection>Politics Collection</collection><collection>ProQuest Central Korea</collection><collection>ProQuest Central Student</collection><collection>Research Library Prep</collection><collection>ProQuest Politics Collection</collection><collection>Military Database</collection><collection>Political Science Database</collection><collection>Research Library</collection><collection>Social Science Database</collection><collection>Research Library (Corporate)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>ProQuest Central Basic</collection><jtitle>Political analysis</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Kuhn, Patrick M.</au><au>Vivyan, Nick</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>The Misreporting Trade-Off Between List Experiments and Direct Questions in Practice: Partition Validation Evidence from Two Countries</atitle><jtitle>Political analysis</jtitle><addtitle>Polit. Anal</addtitle><date>2022-07-01</date><risdate>2022</risdate><volume>30</volume><issue>3</issue><spage>381</spage><epage>402</epage><pages>381-402</pages><issn>1047-1987</issn><eissn>1476-4989</eissn><abstract>To reduce strategic misreporting on sensitive topics, survey researchers increasingly use list experiments rather than direct questions. However, the complexity of list experiments may increase nonstrategic misreporting. We provide the first empirical assessment of this trade-off between strategic and nonstrategic misreporting. We field list experiments on election turnout in two different countries, collecting measures of respondents’ true turnout. We detail and apply a partition validation method which uses true scores to distinguish true and false positives and negatives for list experiments, thus allowing detection of nonstrategic reporting errors. For both list experiments, partition validation reveals nonstrategic misreporting that is: undetected by standard diagnostics or validation; greater than assumed in extant simulation studies; and severe enough that direct turnout questions subject to strategic misreporting exhibit lower overall reporting error. We discuss how our results can inform the choice between list experiment and direct question for other topics and survey contexts.</abstract><cop>New York, USA</cop><pub>Cambridge University Press</pub><doi>10.1017/pan.2021.10</doi><tpages>22</tpages><orcidid>https://orcid.org/0000-0002-8849-0248</orcidid><orcidid>https://orcid.org/0000-0002-6793-0953</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 1047-1987
ispartof Political analysis, 2022-07, Vol.30 (3), p.381-402
issn 1047-1987
1476-4989
language eng
recordid cdi_proquest_journals_2671915660
source Worldwide Political Science Abstracts; EBSCOhost Political Science Complete; Cambridge University Press Journals Complete
subjects Elections
Errors
Experiments
Partition
Respondents
Simulation
Validity
Voter turnout
title The Misreporting Trade-Off Between List Experiments and Direct Questions in Practice: Partition Validation Evidence from Two Countries
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-22T13%3A09%3A57IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=The%20Misreporting%20Trade-Off%20Between%20List%20Experiments%20and%20Direct%20Questions%20in%20Practice:%20Partition%20Validation%20Evidence%20from%20Two%20Countries&rft.jtitle=Political%20analysis&rft.au=Kuhn,%20Patrick%20M.&rft.date=2022-07-01&rft.volume=30&rft.issue=3&rft.spage=381&rft.epage=402&rft.pages=381-402&rft.issn=1047-1987&rft.eissn=1476-4989&rft_id=info:doi/10.1017/pan.2021.10&rft_dat=%3Cproquest_cross%3E2671915660%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2671915660&rft_id=info:pmid/&rft_cupid=10_1017_pan_2021_10&rfr_iscdi=true