Deep Learning for Super-resolution Ultrasound Imaging with Spatiotemporal Data

Super-resolution ultrasound imaging (SRUS) is an active area of research as it brings up to a ten-fold improvement in the resolution of microvascular structures. The limitations to the clinical adoption of SRUS include long acquisition times and long image processing times. Both these limitations can be alleviated with deep learning approaches to the processing of SRUS images. In this study we propose an optimized architecture based on modern improvements to convolutional neural networks from the ConvNeXt architecture and further customize the choice of features to improve performance on the specific tasks of both MB detection and localization within a single network. We employ a spatiotemporal input of up to five successive image frames to increase the number of MBs detected. The output structure produces three classifications: a MB detection Boolean for each pixel in the central image frame, as well as x and z offsets at 4-fold subpixel resolution for each MB detected. Ultrasound simulations generated images based on the L22-14v transducer (Verasonics) for training and testing of the proposed SRUS-ConvNeXt network. In vivo image data of a mouse brain was used as further validation of the architecture. The proposed network had the highest performance as measured by F1 score when configured for a 3-frame spatiotemporal input. The smallest localization error of {\lambda}/22 was achieved when the network was configured for a single input frame. The flexibility of the proposed architecture allows extension to 10-fold upscaling for SRUS images with a much lower impact to number of parameters and subsequent increase in inference time than typical U-Net style approaches. This network is promising in the quest to develop a SRUS deep network architecture for real time image formation.