Statistical learning with group invariance: problem, method and consistency

Statistical learning theory (SLT) provides the theoretical basis for many machine learning algorithms (e.g. SVMs and kernel methods). Invariance, as one type of popular prior knowledge in pattern analysis, has been widely incorporated into various statistical learning algorithms to improve learning performance. Though successful in some applications, existing invariance learning algorithms are task-specific, and lack a solid theoretical basis including consistency . In this paper, we first propose the problem of statistical learning with group invariance (or group invariance learning in short) to provide a unifying framework for existing invariance learning algorithms in pattern analysis by exploiting group invariance. We then introduce the group invariance empirical risk minimization (GIERM) method to solve the group invariance learning problem, which incorporates the group action on the original data into empirical risk minimization (ERM). Finally, we investigate the consistency of the GIERM method in detail. Our theoretical results include three theorems, covering the necessary and sufficient conditions of consistency, uniform two-sided convergence and uniform one-sided convergence for the group invariance learning process based on the GIERM method.