Penalty processes for combining roughness and smoothness in spectral multivariate calibration

Tikhonov regularization (TR) has been successfully applied to form spectral multivariate calibration models by augmenting spectroscopic data with a regulation operator matrix. This matrix can be set to the identity matrix I (ridge regression), yielding what shall be considered rough regression vectors. It can also be set to the i‐th derivative operator matrix iL to form smoothed regression vectors. Two new penalty (regularization) methods are proposed that concurrently factor both roughness and smoothness in forming the model vector. This combination occurs by augmenting calibration spectra simultaneously with independently weighted I and iL matrices. The results of these two new methods are presented and compared with results using ridge regression forming rough model vectors and only using the smoothing TR processes. Partial least squares regression is also used to combine roughness and smoothness, and these results are compared with the TR variants. The sum of ranking differences algorithm and the two fusion rules sum and median are used for automatic model selection, that is, the appropriate tuning parameters for I and iL and partial least squares latent vectors. The approaches are evaluated using near‐infrared and ultraviolet‐visible spectral data sets. The near‐infrared set consists of corn samples for the analysis of protein and moisture content. The ultraviolet‐visible set consists of a three‐component system of inorganic elements. The general trends found are that when spectra are originally generally smooth, then using the smoothing methods provides no improvement in prediction errors. However, when spectra are considered noisy, then smoothing methods can assist in reducing prediction errors. This is especially true when the spectroscopic noise is more widespread across the wavelength regions. There was no difference in the results between the different smoothing methods. Copyright © 2016 John Wiley & Sons, Ltd. Two new penalty methods are proposed that concurrently factor both roughness and smoothness in forming the model vector. The results of these two new methods are presented and compared with results from using ridge regression and partial least squares. The sum‐of‐ranking‐differences algorithm and two other fusion rules are used for automatic model selection. The approaches are evaluated using near‐infrared and ultraviolet–visible spectral data sets. The general trends found are that when spectra are originally generally smooth, then using