SCENARIO BASED MONITORING AND CONTROL OF AUTONOMOUS VEHICLES

A system evaluates modifications to components of an autonomous vehicle (AV) stack. The system receives driving recommendations traffic scenarios based on user annotations of video frames showing each traffic scenario. For each traffic scenario, the system predicts driving recommendations based on the AV stack. The system determines a measure of quality of driving recommendation by comparing predicted driving recommendations based on the AV stack with the driving recommendations received for the traffic scenario. The measure of quality of driving recommendation is used for evaluating components of the AV stack. The system determines a driving recommendation for an AV corresponding to ranges of SOMAI (state of mind) score and sends signals to controls of the autonomous vehicle to navigate the autonomous vehicle according to the driving recommendation. The system identifies additional training data for training machine learning model based on the measure of driving quality.