On Agnostic Learning of Parities, Monomials, and Halfspaces

The authors study the learnability of several fundamental concept classes in the agnostic learning framework of [D. Haussler, Inform. and Comput., 100 (1992), pp. 78-150] and [M. Kearns, R. Schapire, and L. Sellie, Machine Learning, 17 (1994), pp. 115-141]. They show that under the uniform distribution, agnostically learning parities reduce to learning parities with random classification noise, commonly referred to as the noisy parity problem. Together with the parity learning algorithm of [A. Blum, A. Kalai, and H. Wasserman, J. ACM, 50 (2003), pp. 506-519], this gives the first nontrivial algorithm for agnostic learning of parities. They use similar techniques to reduce learning of two other fundamental concept classes under the uniform distribution to learning of noisy parities. Namely, they show that learning of disjunctive normal form expressions reduces to learning noisy parities of just logarithmic number of variables, and learning of ... -juntas reduces to learning noisy parities of ... variables. (ProQuest: ... denotes formulae/symbols omitted.)