A Gridless DOA Estimation Method Based on Convolutional Neural Network With Toeplitz Prior

Most existing deep learning (DL) based direction-of-arrival (DOA) estimation methods treat direction finding problem as a multi-label classification task and the output of the neural network is a probability spectrum where the peaks indicate the true DOAs. These methods essentially belong to grid-based methods and may encounter grid mismatch effect. In this paper, we focus on gridless DL based DOA estimation under generalized linear array which can be regarded as a uniform linear array (ULA) with/without "holes". By using the Toeplitz structure, a deep convolutional neural network (CNN) is proposed to estimate the noiseless covariance matrix of the aforementioned ULA with "no holes," based on which the DOAs can be retrieved by using root-MUSIC. To increase the generalization, the parameters of the CNN with respect to different number of sources are pre-trained and stored in a database. We then propose another CNN for source enumeration in order to choose suitable parameters from the database. Our method can find more sources than sensors and do not suffer from the grid mismatch effect.