Visual Route Recognition in Urban Spaces: A Scalable Approach Using Open Street View Data

This paper presents a novel pipeline for visual route recognition (VRR) in large-scale urban environments, leveraging open street view data. The proposed approach aims to identify the path of a video recorder by analyzing visual cues from continuous video frames and street landmarks, evaluated through datasets from New York and Taipei City. The pipeline begins with SemVG (Semantic Visual Geo-localization), a semantic fused feature extraction network that filters out non-landmark noise, generating robust visual representations. We construct a feature database from multi-perspective street view images to enable efficient feature retrieval for query video frames. Additionally, we introduce a spatio-temporal trajectory reconstruction method that corrects mismatches in the camera's motion path, ensuring consistency. Our contributions include the development of SemVG, a method for maintaining spatio-temporal consistency in trajectory reconstruction, and a large-scale Taipei dataset designed for visual route recognition. This work has implications for urban surveillance, law enforcement, and smart city applications, supporting urban planning, resource management, search and rescue, and augmented reality navigation by improving localization without specialized hardware.