Understanding kernel ridge regression: Common behaviors from simple functions to density functionals

Accurate approximations to density functionals have recently been obtained via machine learning (ML). By applying ML to a simple function of one variable without any random sampling, we extract the qualitative dependence of errors on hyperparameters. We find universal features of the behavior in extreme limits, including both very small and very large length scales, and the noise‐free limit. We show how such features arise in ML models of density functionals. © 2015 Wiley Periodicals, Inc. Machine learning with kernel ridge regression has recently been used to obtain approximations to density functionals. These approximations have been highly accurate for one‐dimensional systems including the spinless fermion in a box and orbital‐free bond breaking between diatomics. This article explores the behavior of machine learning models with respect to the properties of the kernel. Machine learning methods are tested on a simple function in order to elucidate this dependence.