Rare event simulation in autonomous vehicle motion planning

Methods of identifying corner case simulation scenarios that are used to train an autonomous vehicle motion planning model are disclosed. A system selects a scene that includes data captured by one or more vehicles over a time period. The data includes one or more actors that the vehicle's sensors perceived over the time period in a real-world environment. The system selects a scene that includes a safety threshold violation, and it identifies the trajectory of an actor that participated in the violation. The system generates simulated scenes that alter the trajectory of the actor in the selected scene, selects simulated scenes that are more likely to occur in the real world and that may include safety threshold violations that go beyond any that may be found in the original scene, and uses the selected simulated scenes to train an autonomous vehicle motion planning model.