Raw Data for: Unrepresentative Big Surveys Significantly Overestimate US Vaccine Uptake

This Dataverse stores the raw data analyzed in Bradley et al. The data is taken directly from public domains. Because links to these datasets may change or specifications may change, we store the versions used in Bradley et al. for preservation. See the paper and README in the Github https://github.com/vcbradley/ddc-vaccine-US for more details. Abstract of the paper below. Abstract : Accurate surveys are the primary tool for understanding public opinion towards and barriers preventing COVID-19 vaccine uptake. We compare three prominent surveys about vaccination in the US: Delphi-Facebook (n≈250,000 per week), Census Household Pulse (n≈75,000), and Axios-Ipsos (n≈1,000). We find that the two larger surveys are biased compared to the benchmark from the Centers for Disease Control and Prevention (CDC), and that their sample sizes lead to devastating overconfidence in those incorrect estimates. By April 26, 2021, Delphi-Facebook and Census Household Pulse estimated that at least 73% and 69% of US adults had received a first dose of COVID-19 vaccine, which was 16 and 12 percentage points higher, respectively, than the CDC's estimate (57%). Moreover, estimates of vaccine hesitancy disagree significantly between surveys -- we find that these differences cannot be explained entirely by Delphi-Facebook's under-representation of racial minorities and non-college educated adults. These are examples of the Big Data Paradox: when a confidence interval based on a large but biased sample exhibits both a seriously displaced center and a grossly underestimated width, thus leading us (confidently) away from the truth. With sufficient attention to quality control, small surveys like Axios-Ipsos can be far more reliable than large ones. We leverage a recently established data quality identity (Meng, Annals of Applied Statistics, 2018) to quantify sources of the estimation errors and to conduct a scenario analysis for implications on vaccine willingness and hesitancy. Our study quantifies how bias in large samples can lead to overconfidence in incorrect inferences, which is particularly problematic in studies, like those examined here, that inform high-stakes public policy decisions.