Effective multiple pedestrian tracking system in video surveillance with monocular stationary camera

•Tracking system contains targets’ registration, model update and track termination.•Detected candidates are optimized with occlusion status and foreground masks.•Before target association, targets’ location are predicted with Kalman filter.•Targets are associated hierarchically, based on matching situations. Multiple pedestrian tracking in video surveillance is still a pressing challenge, especially under static and dynamic occlusions and target appearance variations. Considering these complex environments in video surveillance, a multiple pedestrian tracking system with special processing procedures is proposed in this paper. In the proposed tracking system, pedestrian candidates are detected on each frame and registered as the tracked targets or associated with existing targets when their situations are suitable. The registered pedestrian targets are tracked frame by frame and terminated when the termination criteria are satisfied. In order to distinguish these target individuals, multi-sample adaptive modeling (MSAM) is proposed, which is used to adapt to a new target’s unpredictable pose variation. Furthermore, static occlusions are annotated for each scene, which may occlude pedestrians in the annotated regions. These occluded targets are modified by the assigned rules and treated differently in the process of target association. Aiming to enhance the effect of target association, each target’s location on the current frame is predicted with information on the previous frames using a Kalman filter. The predicted location is regarded as the center of the search region of the corresponding target. The experimental results show that the proposed tracker achieves the best performance among the five state-of-the-art trackers on three publicly available databases.