PERCEPTION TESTING

A method of testing a candidate perception setup for an autonomous vehicle, comprising: receiving ground truth of a driving scenario run; providing a time-sequence of ground truth snapshots of the driving scenario run, wherein a decision making component independent of the ego agent determines a first time-sequence of decisions for the ground truth snapshots; providing a time-sequence of ablated snapshots of the driving scenario run, each ablated snapshot generated so as to cause a perception error that is representative of the candidate perception setup, wherein the decision making component determines a second time-sequence of decisions for the ablated snapshots; and computing a similarity measure between the first and second time-sequences of decisions denoting an extent to which the candidate perception setup caused a change in one or more decisions points of the second time-sequence relative to the first time-sequence, a decision point occurring when a decision changes between adjacent timesteps.