MACHINE LEARNING SECURITY

In various examples there is a method of empirically measuring a level of security of a training pipeline. The training pipeline is configured to train machine learning models using confidential training data. The method comprises storing a representation of a joint distribution of false positive rate and false negative rate of membership inference attacks on a plurality of machine learning models trained using the training pipeline. The method uses the representation to compute a posterior distribution of the level of security from observations of the membership inference attack on the plurality of machine learning models trained using the training pipelines. A confidence interval of the level of security is computed from the posterior distribution and the confidence interval is stored.