Modeling interpretable social interactions for pedestrian trajectory

The abilities to understand pedestrian social interaction behaviors and to predict their future trajectories are critical for road safety, traffic management and more broadly autonomous vehicles and robots. Social interactions are intuitively heterogeneous and dynamic over time and circumstances, making them hard to explain. In this paper, we creatively investigate modeling interpretable social interactions for pedestrian trajectory, which is not considered by the existing trajectory prediction research. Moreover, we propose a two-stage methodology for interaction modeling - “mode extraction” and “mode aggregation”, and develop a long short-term memory (LSTM)-based model for long-term trajectory prediction, which naturally takes into account multi-types of social interactions. Different from previous models that do not explain how pedestrians interact socially, we extract latent modes that represent social interaction types which scales to an arbitrary number of neighbors. Extensive experiments over two public datasets have been conducted. The quantitative and qualitative results demonstrate that our method is able to capture the multi-modality of human motion and achieve better performance under specific conditions. Its performance is also verified by the interpretation of predicted modes, of which the results are in accordance with common sense. Besides, we have performed sensitivity analysis on the crucial hyperparameters in our model. Code is available at: https://github.com/xiaoluban/Modeling-Interpretable-Social-Interactions-for-Pedestrian-Trajectory. •People in crowds interacts with neighbors following different patterns.•Incorporating heterogeneous interaction modeling into trajectory prediction problem.•Extracting latent modes which scale to arbitrary number of neighbors.•Improving reliability of the prediction model with interpretation of extracted modes.