Assessing the validity of facilitated-volunteered geographic information: comparisons of expert and novice ratings

Facilitated-voluntary geographic information (f-VGI) is a promising method to enable systematic collection of data from residents about their physical and social environment. The method capitalizes on ubiquitous mobile smartphones to empower collection of geospatially-referenced data. It is important to evaluate the validity of usergenerated content for use in research or program planning. The purpose of this study was to test whether the aggregated environmental ("bikeability") ratings from novice community residents converges with ratings from experts using a robust researchbased, paper audit-tool (the established Pedestrian Environment Data Scan (PEDS) tool). Equivalence testing statistically showed overall agreement between the composite ratings of bikeability within the novice group. Agreement in categorical ratings between novices and experts were examined using the summary agreement index, which showed substantial agreement across the 10 locations rated by 11 novices using an f-VGI mobile application and four experts using PEDS; variability depended on the nature of the specific questions asked. Results reveal overall substantial agreement between novice and expert ratings for both composite scores and individual categorical ratings. However, additional research is needed to refine the methodology for use in formalized research applications.