System and method for determining defects using physics-based image perturbations

A system for characterizing a specimen is disclosed. In one embodiment, the system includes a characterization sub-system configured to acquire one or more images a specimen, and a controller communicatively coupled to the characterization sub-system. The controller may be configured to: receive from the characterization sub-system one or more training images of one or more defects of a training specimen; generate one or more augmented images of the one or more defects of the training specimen; generate a machine learning classifier based on the one or more augmented images of the one or more defects of the training specimen; receive from the characterization sub-system one or more target images of one or more target features of a target specimen; and determine one or more defects of the one or more target features with the machine learning classifier.