ADVERSARIAL ATTACKS ON PERCEPTION COMPONENTS

A computer-implemented method of generating black-box adversarial inputs to a perception component using a surrogate model of the perception component comprises receiving an initial input to the perception component and repeatedly perturbing the initial input until an adversarial input is found that satisfies an attack objective by: performing a primary attack process by perturbing the initial input based on a computed gradient of a surrogate attack loss function of the surrogate model that encodes the attack objective; wherein, if the primary attack process terminates without finding any perturbed input satisfying the promising attack condition, a backup attack process is performed to perform a randomized search of the input space of the perception component, guided by the surrogate model, until a perturbed input satisfying the promising attack condition is found; wherein the primary attack process is repeated based on the perturbed input found by the primary attack process or backup process.