A constraint optimization problem for model order estimation

•Automatic order estimation is needed for spectral estimation techniques.•Both statistical properties of the noise and the algebraic structure of of the signal model must be considered.•We propose a new model order estimation for the sum of complex exponentials when the signal to noise ratio is low and the signal is observed on a short time window.•Validation of the new technique by comparing its performance with other already known techniques in different scenarios are provided. In this paper, we present a new method for estimating the number of terms in a sum of exponentially damped sinusoids embedded in noise. In particular, we propose to combine the shift-invariance property of the Hankel matrix associated with the signal with a constraint over its singular values to penalize small-order estimations. With this new methodology, the algebraic structure of the Hankel matrix and the statistical properties of the noise are considered. The new order estimation technique shows significant improvements over subspace-based methods. In particular, when a good separation between the noise and the signal subspaces is not possible, the new methodology outperforms known techniques. We evaluate the performance of our method using numerical experiments and compare its performance with previous results found in the literature.