Simulation testing performance of ensemble models when catch data are underreported

Abstract Ensemble model use in stock assessment is increasing, yet guidance on construction and an evaluation of performance relative to single models is lacking. Ensemble models can characterize structural uncertainty and avoid the conundrum of selecting a “best” assessment model when alternative m...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:ICES journal of marine science 2024-08, Vol.81 (6), p.1053-1072
Hauptverfasser: Brooks, Elizabeth N, Brodziak, Jon K T
Format: Artikel
Sprache:eng
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Abstract Ensemble model use in stock assessment is increasing, yet guidance on construction and an evaluation of performance relative to single models is lacking. Ensemble models can characterize structural uncertainty and avoid the conundrum of selecting a “best” assessment model when alternative models explain observed data equally well. Through simulation, we explore the importance of identifying candidate models for both assessment and short-term forecasts and the consequences of different ensemble weighting methods on estimated quantities. Ensemble performance exceeded a single best model only when the set of candidate models spanned the true model configuration. Accuracy and precision depended on the model weighting scheme, and varied between two case studies investigating the impact of catch accuracy. Information theoretic weighting methods performed well in the case study with accurate catch, while equal weighting performed best when catch was underreported. In both cases, equal weighting produced multimodality. Ensuring that an ensemble spans the true state of nature will be challenging, but we observed that a change in sign of Mohn’s rho across candidate models coincided with the true OM being bounded. Further development of protocols to select an objective and balanced set of candidate models, and diagnostics to assess adequacy of candidates are recommended.
ISSN:1054-3139
1095-9289
DOI:10.1093/icesjms/fsae067