The Misreporting Trade-Off Between List Experiments and Direct Questions in Practice: Partition Validation Evidence from Two Countries
To reduce strategic misreporting on sensitive topics, survey researchers increasingly use list experiments rather than direct questions. However, the complexity of list experiments may increase nonstrategic misreporting. We provide the first empirical assessment of this trade-off between strategic a...
Gespeichert in:
Veröffentlicht in: | Political analysis 2022-07, Vol.30 (3), p.381-402 |
---|---|
Hauptverfasser: | , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 402 |
---|---|
container_issue | 3 |
container_start_page | 381 |
container_title | Political analysis |
container_volume | 30 |
creator | Kuhn, Patrick M. Vivyan, Nick |
description | To reduce strategic misreporting on sensitive topics, survey researchers increasingly use list experiments rather than direct questions. However, the complexity of list experiments may increase nonstrategic misreporting. We provide the first empirical assessment of this trade-off between strategic and nonstrategic misreporting. We field list experiments on election turnout in two different countries, collecting measures of respondents’ true turnout. We detail and apply a partition validation method which uses true scores to distinguish true and false positives and negatives for list experiments, thus allowing detection of nonstrategic reporting errors. For both list experiments, partition validation reveals nonstrategic misreporting that is: undetected by standard diagnostics or validation; greater than assumed in extant simulation studies; and severe enough that direct turnout questions subject to strategic misreporting exhibit lower overall reporting error. We discuss how our results can inform the choice between list experiment and direct question for other topics and survey contexts. |
doi_str_mv | 10.1017/pan.2021.10 |
format | Article |
fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2671915660</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><cupid>10_1017_pan_2021_10</cupid><sourcerecordid>2671915660</sourcerecordid><originalsourceid>FETCH-LOGICAL-c332t-65e0e37291c0987bb78b06fc052293fba243be2e454a5e35fe70a179a04fd6ad3</originalsourceid><addsrcrecordid>eNptkN1KAzEQhYMoWKtXvkDAS9maZH_SeKe1_kClFVZvl-zupKbYZE1Sqy_gc5vagjdezZnh48zMQeiUkgEllF900gwYYTR2e6hHM14kmRiK_ahJxhMqhvwQHXm_IJHmQvTQd_kK-FF7B511QZs5Lp1sIZkqha8hrAEMnmgf8PizA6eXYILH0rT4RjtoAn5agQ_aGo-1wTMnm6AbuMQzGc02c_wi33Qrf-X4Q7dgGsDK2SUu1xaP7MoEp8EfowMl3zyc7GofPd-Oy9F9MpnePYyuJkmTpiwkRQ4EUs4EbUj8pa75sCaFakjOmEhVLVmW1sAgyzOZQ5or4ERSLiTJVFvINu2js61v5-z75vRqYVfOxJUVKzgVNC8KEqnzLdU462M0quri69J9VZRUm6CrGHS1CTp2kU52tFzWTrdz-DP9j_8B-zaBYw</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2671915660</pqid></control><display><type>article</type><title>The Misreporting Trade-Off Between List Experiments and Direct Questions in Practice: Partition Validation Evidence from Two Countries</title><source>Worldwide Political Science Abstracts</source><source>EBSCOhost Political Science Complete</source><source>Cambridge University Press Journals Complete</source><creator>Kuhn, Patrick M. ; Vivyan, Nick</creator><creatorcontrib>Kuhn, Patrick M. ; Vivyan, Nick</creatorcontrib><description>To reduce strategic misreporting on sensitive topics, survey researchers increasingly use list experiments rather than direct questions. However, the complexity of list experiments may increase nonstrategic misreporting. We provide the first empirical assessment of this trade-off between strategic and nonstrategic misreporting. We field list experiments on election turnout in two different countries, collecting measures of respondents’ true turnout. We detail and apply a partition validation method which uses true scores to distinguish true and false positives and negatives for list experiments, thus allowing detection of nonstrategic reporting errors. For both list experiments, partition validation reveals nonstrategic misreporting that is: undetected by standard diagnostics or validation; greater than assumed in extant simulation studies; and severe enough that direct turnout questions subject to strategic misreporting exhibit lower overall reporting error. We discuss how our results can inform the choice between list experiment and direct question for other topics and survey contexts.</description><identifier>ISSN: 1047-1987</identifier><identifier>EISSN: 1476-4989</identifier><identifier>DOI: 10.1017/pan.2021.10</identifier><language>eng</language><publisher>New York, USA: Cambridge University Press</publisher><subject>Elections ; Errors ; Experiments ; Partition ; Respondents ; Simulation ; Validity ; Voter turnout</subject><ispartof>Political analysis, 2022-07, Vol.30 (3), p.381-402</ispartof><rights>The Author(s) 2021. Published by Cambridge University Press on behalf of the Society for Political Methodology</rights><rights>The Author(s) 2021. Published by Cambridge University Press on behalf of the Society for Political Methodology. This work is licensed under the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c332t-65e0e37291c0987bb78b06fc052293fba243be2e454a5e35fe70a179a04fd6ad3</citedby><cites>FETCH-LOGICAL-c332t-65e0e37291c0987bb78b06fc052293fba243be2e454a5e35fe70a179a04fd6ad3</cites><orcidid>0000-0002-8849-0248 ; 0000-0002-6793-0953</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.cambridge.org/core/product/identifier/S1047198721000103/type/journal_article$$EHTML$$P50$$Gcambridge$$Hfree_for_read</linktohtml><link.rule.ids>164,314,776,780,12825,27903,27904,55607</link.rule.ids></links><search><creatorcontrib>Kuhn, Patrick M.</creatorcontrib><creatorcontrib>Vivyan, Nick</creatorcontrib><title>The Misreporting Trade-Off Between List Experiments and Direct Questions in Practice: Partition Validation Evidence from Two Countries</title><title>Political analysis</title><addtitle>Polit. Anal</addtitle><description>To reduce strategic misreporting on sensitive topics, survey researchers increasingly use list experiments rather than direct questions. However, the complexity of list experiments may increase nonstrategic misreporting. We provide the first empirical assessment of this trade-off between strategic and nonstrategic misreporting. We field list experiments on election turnout in two different countries, collecting measures of respondents’ true turnout. We detail and apply a partition validation method which uses true scores to distinguish true and false positives and negatives for list experiments, thus allowing detection of nonstrategic reporting errors. For both list experiments, partition validation reveals nonstrategic misreporting that is: undetected by standard diagnostics or validation; greater than assumed in extant simulation studies; and severe enough that direct turnout questions subject to strategic misreporting exhibit lower overall reporting error. We discuss how our results can inform the choice between list experiment and direct question for other topics and survey contexts.</description><subject>Elections</subject><subject>Errors</subject><subject>Experiments</subject><subject>Partition</subject><subject>Respondents</subject><subject>Simulation</subject><subject>Validity</subject><subject>Voter turnout</subject><issn>1047-1987</issn><issn>1476-4989</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><sourceid>IKXGN</sourceid><sourceid>7UB</sourceid><sourceid>8G5</sourceid><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GNUQQ</sourceid><sourceid>GUQSH</sourceid><sourceid>M2O</sourceid><recordid>eNptkN1KAzEQhYMoWKtXvkDAS9maZH_SeKe1_kClFVZvl-zupKbYZE1Sqy_gc5vagjdezZnh48zMQeiUkgEllF900gwYYTR2e6hHM14kmRiK_ahJxhMqhvwQHXm_IJHmQvTQd_kK-FF7B511QZs5Lp1sIZkqha8hrAEMnmgf8PizA6eXYILH0rT4RjtoAn5agQ_aGo-1wTMnm6AbuMQzGc02c_wi33Qrf-X4Q7dgGsDK2SUu1xaP7MoEp8EfowMl3zyc7GofPd-Oy9F9MpnePYyuJkmTpiwkRQ4EUs4EbUj8pa75sCaFakjOmEhVLVmW1sAgyzOZQ5or4ERSLiTJVFvINu2js61v5-z75vRqYVfOxJUVKzgVNC8KEqnzLdU462M0quri69J9VZRUm6CrGHS1CTp2kU52tFzWTrdz-DP9j_8B-zaBYw</recordid><startdate>20220701</startdate><enddate>20220701</enddate><creator>Kuhn, Patrick M.</creator><creator>Vivyan, Nick</creator><general>Cambridge University Press</general><scope>IKXGN</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>0-V</scope><scope>3V.</scope><scope>7UB</scope><scope>7XB</scope><scope>88F</scope><scope>88J</scope><scope>8FK</scope><scope>8G5</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ALSLI</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>CCPQU</scope><scope>DPSOV</scope><scope>DWQXO</scope><scope>GNUQQ</scope><scope>GUQSH</scope><scope>KC-</scope><scope>M1Q</scope><scope>M2L</scope><scope>M2O</scope><scope>M2R</scope><scope>MBDVC</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>Q9U</scope><orcidid>https://orcid.org/0000-0002-8849-0248</orcidid><orcidid>https://orcid.org/0000-0002-6793-0953</orcidid></search><sort><creationdate>20220701</creationdate><title>The Misreporting Trade-Off Between List Experiments and Direct Questions in Practice: Partition Validation Evidence from Two Countries</title><author>Kuhn, Patrick M. ; Vivyan, Nick</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c332t-65e0e37291c0987bb78b06fc052293fba243be2e454a5e35fe70a179a04fd6ad3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Elections</topic><topic>Errors</topic><topic>Experiments</topic><topic>Partition</topic><topic>Respondents</topic><topic>Simulation</topic><topic>Validity</topic><topic>Voter turnout</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Kuhn, Patrick M.</creatorcontrib><creatorcontrib>Vivyan, Nick</creatorcontrib><collection>Cambridge University Press Wholly Gold Open Access Journals</collection><collection>CrossRef</collection><collection>ProQuest Social Sciences Premium Collection</collection><collection>ProQuest Central (Corporate)</collection><collection>Worldwide Political Science Abstracts</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>Military Database (Alumni Edition)</collection><collection>Social Science Database (Alumni Edition)</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>Research Library (Alumni Edition)</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>Social Science Premium Collection</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>ProQuest One Community College</collection><collection>Politics Collection</collection><collection>ProQuest Central Korea</collection><collection>ProQuest Central Student</collection><collection>Research Library Prep</collection><collection>ProQuest Politics Collection</collection><collection>Military Database</collection><collection>Political Science Database</collection><collection>Research Library</collection><collection>Social Science Database</collection><collection>Research Library (Corporate)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>ProQuest Central Basic</collection><jtitle>Political analysis</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Kuhn, Patrick M.</au><au>Vivyan, Nick</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>The Misreporting Trade-Off Between List Experiments and Direct Questions in Practice: Partition Validation Evidence from Two Countries</atitle><jtitle>Political analysis</jtitle><addtitle>Polit. Anal</addtitle><date>2022-07-01</date><risdate>2022</risdate><volume>30</volume><issue>3</issue><spage>381</spage><epage>402</epage><pages>381-402</pages><issn>1047-1987</issn><eissn>1476-4989</eissn><abstract>To reduce strategic misreporting on sensitive topics, survey researchers increasingly use list experiments rather than direct questions. However, the complexity of list experiments may increase nonstrategic misreporting. We provide the first empirical assessment of this trade-off between strategic and nonstrategic misreporting. We field list experiments on election turnout in two different countries, collecting measures of respondents’ true turnout. We detail and apply a partition validation method which uses true scores to distinguish true and false positives and negatives for list experiments, thus allowing detection of nonstrategic reporting errors. For both list experiments, partition validation reveals nonstrategic misreporting that is: undetected by standard diagnostics or validation; greater than assumed in extant simulation studies; and severe enough that direct turnout questions subject to strategic misreporting exhibit lower overall reporting error. We discuss how our results can inform the choice between list experiment and direct question for other topics and survey contexts.</abstract><cop>New York, USA</cop><pub>Cambridge University Press</pub><doi>10.1017/pan.2021.10</doi><tpages>22</tpages><orcidid>https://orcid.org/0000-0002-8849-0248</orcidid><orcidid>https://orcid.org/0000-0002-6793-0953</orcidid><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | ISSN: 1047-1987 |
ispartof | Political analysis, 2022-07, Vol.30 (3), p.381-402 |
issn | 1047-1987 1476-4989 |
language | eng |
recordid | cdi_proquest_journals_2671915660 |
source | Worldwide Political Science Abstracts; EBSCOhost Political Science Complete; Cambridge University Press Journals Complete |
subjects | Elections Errors Experiments Partition Respondents Simulation Validity Voter turnout |
title | The Misreporting Trade-Off Between List Experiments and Direct Questions in Practice: Partition Validation Evidence from Two Countries |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-22T13%3A09%3A57IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=The%20Misreporting%20Trade-Off%20Between%20List%20Experiments%20and%20Direct%20Questions%20in%20Practice:%20Partition%20Validation%20Evidence%20from%20Two%20Countries&rft.jtitle=Political%20analysis&rft.au=Kuhn,%20Patrick%20M.&rft.date=2022-07-01&rft.volume=30&rft.issue=3&rft.spage=381&rft.epage=402&rft.pages=381-402&rft.issn=1047-1987&rft.eissn=1476-4989&rft_id=info:doi/10.1017/pan.2021.10&rft_dat=%3Cproquest_cross%3E2671915660%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2671915660&rft_id=info:pmid/&rft_cupid=10_1017_pan_2021_10&rfr_iscdi=true |