HUDNet: A dynamic calibration method for automotive augmented reality head-up-displays

The automotive augmented reality head-up-display (AR-HUD) system relies on multiple free-form surfaces to project the virtual image into human eyes. A wide range of views and the extensive display area result in extraordinarily complex distortions. Providing a way to correct such distortions is a major leap forward. Methods widely used for calibration, such as lookup tables and interpolation, require considerable memory capacity, complicated calibration procedures, etc. Additionally, when attempting to meet the high-accuracy requirements for such systems, computational memory grow rapidly. In this context, we propose a fully connected neural network (HUDNet) for an automotive AR-HUD to correct dynamic distortion. Here, we propose a parallel prediction framework for the center and the edge of the image, and separate the distortion information from the disparity information in order to balance the unstable central field of view error. Ultimately, we introduce the transfer learning method to improve accuracy and its effect on accuracy is assessed. The present study provides a reference for us to apply deep learning methods to predict dynamic image distortion of various complex multi-freeform surface reflection systems in the future. [Display omitted] •It is effective to decouple disparity and distortion in calibrating AR-HUD.•Positional coding and proper data processing improve the accuracy of the network.•Deep learning improves the mobility of calibration data among different products.