Variational approach for privacy funnel optimization on continuous data

Here we consider a common data encryption problem encountered by users who want to disclose some data to gain utility but preserve their private information. Specifically, we consider the inference attack, in which an adversary conducts inference on the disclosed data to gain information about users’ private data. Following privacy funnel (Makhdoumi et al., 2014), assuming that the original data X is transformed into Z before disclosing and the log loss is used for both privacy and utility metrics, then the problem can be modeled as finding a mapping X→Z that maximizes mutual information between X and Z subject to a constraint that the mutual information between Z and private data S is smaller than a predefined threshold ϵ. In contrast to the original study (Makhdoumi et al., 2014), which only focused on discrete data, we consider the more general and practical setting of continuous and high-dimensional disclosed data (e.g., image data). Most previous work on privacy-preserving representation learning is based on adversarial learning or generative adversarial networks, which has been shown to suffer from the vanishing gradient problem, and it is experimentally difficult to eliminate the relationship with private data Y when Z is constrained to retain more information about X. Here we propose a simple but effective variational approach that does not rely on adversarial training. Our experimental results show that our approach is stable and outperforms previous methods in terms of both downstream task accuracy and mutual information estimation. •Propose a simple but effective algorithm for privacy funnel optimization.•Analyze influence to privacy cost caused by the variational approach.•The extent to trade off between privacy leaking and data distortion.•Evaluation by downstream task accuracy on a well-known multi-task dataset.•Evaluation by mutual information estimator and other proxies.