SPU-BERT: Faster human multi-trajectory prediction from socio-physical understanding of BERT

Accurately predicting pedestrian trajectories requires a human-like socio-physical understanding of movement, nearby pedestrians, and obstacles. However, traditional methods struggle to generate multiple trajectories in the same situation based on socio-physical understanding and are computationally intensive, making real-time application difficult. To overcome these limitations, we propose SPU-BERT, a fast multi-trajectory prediction model that incorporates two non-recursive BERTs for multi-goal prediction (MGP) and trajectory-to-goal prediction (TGP). First, MGP predicts multiple goals through generative models, followed by TGP generating trajectories that approach the predicted goals. SPU-BERT can simultaneously understand movement, social interaction, and scene context from trajectories and semantic maps using a single Transformer encoder, providing explainable results as evidence of socio-physical understanding. In experiments, SPU-BERT accurately predicted future trajectories (with 0.19 m and 7.54 pixels of ADE20 for the ETH/UCY datasets and SDD) with over 100 times faster computation (0.132 s) than the state-of-the-art method. The code is available at https://github.com/kina4147/SPUBERT. •SPU-BERT predicts socio-physically acceptable multiple trajectories for pedestrians.•SPU-BERT consists of sequentially connected two BERT models with CVAE in between.•SPU-BERT handles trajectories and semantic map for socio-physical understanding.•SPU-BERT provides fast computation and explainable results through attention weights.•SPU-BERT achieves accurate multi-trajectory prediction on two widely used datasets.