Observation data compression for variational assimilation of dynamical systems

Accurate estimation of error covariances (both background and observation) is crucial for efficient observation compression approaches in data assimilation of large-scale dynamical problems. We propose a new combination of a covariance tuning algorithm with existing PCA-type data compression approaches, either observation- or information-based, with the aim of reducing the computational cost of real-time updating at each assimilation step. Relying on a local assumption of flow-independent error covariances, dynamical assimilation residuals are used to adjust the covariance in each assimilation window. The estimated covariances then contribute to better specify the principal components of either the observation dynamics or the state-observation sensitivity. The proposed approaches are first validated on a shallow water twin experiment with correlated and non-homogeneous observation error. Proper selection of flow-independent assimilation windows, together with sampling density for background error estimation, and sensitivity of the approaches to the observations error covariance knowledge, are also discussed and illustrated with various numerical tests and results. The method is then applied to a more challenging industrial hydrological model with real-world data and non-linear transformation operator provided by an operational precipitation-flow simulation software. •We address the problem of observation data compression where the error covariance matrices are not known a priori.•An algorithm combining error covariance tuning and data compression is developed.•The novel approach is tested on a 2D shallow water twin experiment and an operating hydrological model.•The optimal sampling density and truncation parameters are discussed with numerical results.