Model performance analysis for Bayesian biomass dynamics models using bias, precision and reliability metrics

► Performance comparison of Bayesian biomass production models implemented in OpenBUGS. ► The production models are formulated as observation error, process error and state-space models. ► Bias, precision and reliability metrics are keys to evaluate model performance. ► The state-space model captures model uncertainty adequately while the others do not. Bayesian observation error (OEM), process error (PEM) and state-space (SSM) implementations of a Fox biomass dynamics model are compared using a simulation–estimation approach and by applying them to data for the octopus fishery off Mauritania. Estimation performance is evaluated in terms of bias, precision, and reliability measured by the extreme tail-area probability and the mean highest posterior density interval. The PEM generally performs poorest of the three methods in terms of the these performance metrics. In contrast, the OEM is precise, but under-represents uncertainty. The OEM is outperformed by the SSM in terms of its ability to provide posterior distributions which adequately capture parameter uncertainty. It is key to consider the above four metrics when comparing estimation performance in a Bayesian context. Finally, although model performance measures are useful, there is still a need to examine goodness of fit statistics in actual applications.