Quaternion tensor singular value decomposition using a flexible transform-based approach

•A flexible transform-based tensor product named *QT-product for Lth-order (L≥3) quaternion tensors is proposed.•Following the ?QT-product, a quaternion tensor singular value decomposition named TQt-SVD for Lth-order (L≥3) quaternion tensors is defined.•Based on the defined TQt-SVD, a rank named TQt-rank of Lth-order (L≥3) quaternion tensors is defined, which can well characterize the low rankness of color videos. A flexible transform-based tensor product named ★QT-product for Lth-order (L≥3) quaternion tensors is proposed. Based on the ★QT-product, we define the corresponding singular value decomposition named TQt-SVD and the rank named TQt-rank of the Lth-order (L≥3) quaternion tensor. Furthermore, with orthogonal quaternion transformations, the TQt-SVD can provide the best TQt-rank-s approximation of any Lth-order (L≥3) quaternion tensor. On the other side, since a color pixel with RGB channels can be well encoded as a pure quaternion, the proposed TQt-SVD, therefore, becomes a new mathematical tool for many color image processing tasks. In the experiments, we have verified the effectiveness of the proposed TQt-SVD in the application of the best TQt-rank-s approximation for color videos represented by third-order quaternion tensors.