Deep learning based virtual point tracking for real-time target-less dynamic displacement measurement in railway applications

•A novel approach of virtual point tracking for target-less displacement measurement.•Real-time point detection using lightweight convolutional neural network.•A rule engine based on railway domain knowledge is defined for point tracking.•Implementation of the proposed approach for real-time edge computing.•Our codes and data are available at the Github repository. In the application of computer-vision-based displacement measurement, an optical target is usually required to prove the reference. If the optical target cannot be attached to the measuring objective, edge detection and template matching are the most common approaches in target-less photogrammetry. However, their performance significantly relies on parameter settings. This becomes problematic in dynamic scenes where complicated background texture exists and varies over time. We propose virtual point tracking for real-time target-less dynamic displacement measurement, incorporating deep learning techniques and domain knowledge to tackle this issue. Our approach consists of three steps: 1) automatic calibration for detection of region of interest; 2) virtual point detection for each video frame using deep convolutional neural network; 3) domain-knowledge based rule engine for point tracking in adjacent frames. The proposed approach can be executed on an edge computer in a real-time manner (i.e. over 30 frames per second). We demonstrate our approach for a railway application, where the lateral displacement of the wheel on the rail is measured during operation. The numerical experiments have been performed to evaluate our approach’s performance and latency in a harsh railway environment with dynamic complex backgrounds. We make our code and data available at https://github.com/quickhdsdc/Point-Tracking-for-Displacement-Measurement-in-Railway-Applications.