Safety measurement of autonomous vehicle driving in simulation

The present technology is directed to training and the use of a machine learning model to measure the safety of an autonomous vehicle (AV) driving in simulation. An AV management system can run a simulation of an AV autonomously piloting itself and collect simulation driving data. Further, the AV management system can parse the simulation driving data into kinematic and semantic environmental features and output a simulation safety score of the simulation based on the kinematic and semantic environmental features. The simulation safety score indicates a probability of a safety critical event such as a collision or a near-miss between the AV and the at least one simulated object in the simulation.