AdaBias: An Optimization Method With Bias Correction for Differential Privacy Protection

A continuous increase in privacy attacks has caused the research and application of differential privacy (DP) to gradually increase. We can improve the efficiency of the DP model by Optimizing its parameters significantly. Inspired by the performance of various optimization methods for differential privacy, this paper proposes an improved RDP-AdaBound optimization method with bias correction, which is called "AdaBias", to increase the performance of Rényi differential privacy (RDP). The bias correction is used to realize the learning rate and speed up the convergence by upper and lower bound functions. We evaluate our method on the three datasets by training two different privacy model. We further compare three traditional optimization algorithms, namely, RDP-SGD, RDP-Adagrad, and RDP-Adam. And we use AdaBias to verify the performance of privacy protection on the COVID-19 dataset. Experimental results show that the new variant better implements learning rate adjustment to accommodate updates of noisy gradients. As a result, it can achieve higher accuracy and lower losses with a lower privacy budget, thereby better protecting data privacy.