Hyperspectral image synthesis from sparse RGB data: a comparative study combining linear regression, multilayer perceptron, and clustering

The problem of synthesizing hyperspectral images from RGB images is ill posed, with potentially infinite solutions, as it involves estimating data in a high-dimensional space, associated with hyperspectral bands, from limited information in a three-dimensional RGB space. However, under certain conditions related to lighting and physical properties of natural scenes, a feasible solution can be found. This study evaluates four methods for estimating hyperspectral data from RGB images: ridge linear regression, Minibatch K-means followed by linear regression, a multilayer perceptron (MLP) neural network, and Minibatch K-means combined with an MLP neural network. The results of each method are compared with each other and with the NTIRE 2020 Challenge. The comparison was performed using the mean absolute relative error (MARE) and execution time. The MLP method attained the lowest MARE (0.072) but with the longest execution time (220 s). Ridge regression attained the shortest execution time (0.47 s) at the cost of a higher MARE (0.089). The best trade-off was obtained by combining Minibatch K-means clustering with MLP, which reduced the execution time by 16 times (13.8 s) with a slightly higher MARE (0.075) compared to MLP alone. We have also confirmed that, for the case of natural scenes, points representing pixels that are close to each other in the RGB space are also close to each other in the hyperspectral space.