DeepMC: DNN test sample optimization method jointly guided by misclassification and coverage

Large-scale and high-quality test samples are extremely scarce in deep neural networks(DNN) testing. Existing test sample optimization methods exhibit the problem of low efficiency and low neuron coverage of optimized test samples, which consistently fail to expose erroneous behaviors of DNNs with corner-case inputs. In this paper, we propose DeepMC, an image classification DNN test sample optimization method jointly guided by misclassification and coverage. Specifically, we select the seed sample from the original test samples according to the misclassification probability. To maximize the misclassification probability and neuron coverage, we construct the joint optimization problem for the seed samples and use the gradient ascent to solve the joint optimization problem. We evaluate this method on two well-known datasets and prevalent image classification DNN models. Compare with DeepXplore, a DL white-box testing framework, DeepMC does not require multiple DNN models with similar functions for cross-referencing, saves 90% time consumption on MNIST, averagely covers 1.87% more neurons, and optimized test samples with more than 69% attack success rate. In addition, the test sample optimized by DeepMC can also be applied to optimize the robustness of the corresponding DNN with an average 3% improvement of the model’s accuracy.