Let-It-Flow: Simultaneous Optimization of 3D Flow and Object Clustering

We study the problem of self-supervised 3D scene flow estimation from real large-scale raw point cloud sequences. The problem is crucial to various automotive tasks like trajectory prediction, object detection, and scene reconstruction. In the absence of ground truth scene flow labels, contemporary approaches concentrate on deducing and optimizing flow across sequential pairs of point clouds by incorporating structure-based regularization on flow and object rigidity. The rigid objects are estimated by a variety of 3D spatial clustering methods. While state-of-the-art methods successfully capture overall scene motion using the Neural Prior structure, they encounter challenges in discerning multi-object motions. We identified the structural constraints and the use of large and strict rigid clusters as the main pitfall of the current approaches, and we propose a novel clustering approach that allows for a combination of overlapping soft clusters and non-overlapping rigid clusters. Flow is then jointly estimated with progressively growing non-overlapping rigid clusters together with fixed-size overlapping soft clusters. We evaluate our method on multiple datasets with LiDAR point clouds, demonstrating superior performance over the self-supervised baselines and reaching new state-of-the-art results. Our method excels in resolving flow in complicated dynamic scenes with multiple independently moving objects close to each other, including pedestrians, cyclists, and other vulnerable road users. Our codes are publicly available on https://github.com/ctu-vras/let-it-flow .