A Big Data‐Driven Nonlinear Least Squares Four‐Dimensional Variational Data Assimilation Method: Theoretical Formulation and Conceptual Evaluation

A new nonlinear least squares four‐dimensional variational data assimilation method (NLS‐4DVar) is proposed incorporating the use of “big data.” This distinctive four‐dimensional ensemble‐variational data assimilation method (4DEnVar) is made up of two ensembles, a preprepared historical big data ensemble and a small “online” ensemble. The historical ensemble portrays both the ensemble‐constructed background error covariance and tangent models more accurately, as compared with the standard NLS‐4DVar method, with no heavy increase in computational cost in terms of real‐time operations. The online ensemble maintains the flow dependence of the ensemble‐estimated background error covariance. The ensemble analysis scheme proposed by merging the local ensemble transform Kalman filter scheme with a sophisticated sampling approach is able to adjust the ensemble spreads suitably and maintain them steadily. The updating scheme also largely guarantees the partial flow dependence of the historical ensemble. Experimental results using the shallow‐water equations demonstrate that the new big data method provides substantial performance improvement over the standard NLS‐4DVar method. Key Points This big data‐driven NLS‐4DVar is made up of two ensembles, a preprepared historical “big data” ensemble and a small “online” ensemble The ensemble update scheme merges the LETKF scheme with a sophisticated sampling approach The new BD‐NLS4DVar provides substantial performance improvement over the standard NLS‐4DVar method