The application of nonlinear least-squares estimation algorithms in atmospheric density model calibration

Purpose The purpose of this paper is to focus on the performance of three typical nonlinear least-squares estimation algorithms in atmospheric density model calibration. Design/methodology/approach The error of Jacchia-Roberts atmospheric density model is expressed as an objective function about temperature parameters. The estimation of parameter corrections is a typical nonlinear least-squares problem. Three algorithms for nonlinear least-squares problems, Gauss–Newton (G-N), damped Gauss–Newton (damped G-N) and Levenberg–Marquardt (L-M) algorithms, are adopted to estimate temperature parameter corrections of Jacchia-Roberts for model calibration. Findings The results show that G-N algorithm is not convergent at some sampling points. The main reason is the nonlinear relationship between Jacchia-Roberts and its temperature parameters. Damped G-N and L-M algorithms are both convergent at all sampling points. G-N, damped G-N and L-M algorithms reduce the root mean square error of Jacchia-Roberts from 20.4% to 9.3%, 9.4% and 9.4%, respectively. The average iterations of G-N, damped G-N and L-M algorithms are 3.0, 2.8 and 2.9, respectively. Practical implications This study is expected to provide a guidance for the selection of nonlinear least-squares estimation methods in atmospheric density model calibration. Originality/value The study analyses the performance of three typical nonlinear least-squares estimation methods in the calibration of atmospheric density model. The non-convergent phenomenon of G-N algorithm is discovered and explained. Damped G-N and L-M algorithms are more suitable for the nonlinear least-squares problems in model calibration than G-N algorithm and the first two algorithms have slightly fewer iterations.