Climatology, persistence, and their linear combination as standards of reference in skill scores

Skill scores measure the accuracy of the forecasts of interest relative to the accuracy of forecasts based on naive forecasting methods, with either climatology or persistence usually playing the role of the naive method. In formulating skill scores, it is generally agreed that the naive method that produces the most accurate forecasts should be chosen as the standard of reference. The conditions under which climatological forecasts are more accurate than persistence forecasts--and vice versa--were first described in the meteorological literature more than 30 years ago. At about the same time, it was also shown that a linear combination of climatology and persistence produces more accurate forecasts than either of these standards of reference alone. Surprisingly, these results have had relatively little if any impact on the practice of forecast verification in general and the choice of a standard of reference in formulating skill scores in particular. The purposes of this paper are to describe these results and discuss their implications for the practice of forecast verification. Expressions for the mean-square errors of forecasts based on climatology, persistence, and an optimal linear combination of climatology and persistence--as well as expressions for the respective skill scores--are presented and compared. These pairwise comparisons identify the conditions under which each naive method is superior as a standard of reference. Since the optimal linear combination produces more accurate forecasts than either climatology or persistence alone, it leads to lower skill scores than the other two naive forecasting methods. Decreases in the values of the skill scores associated with many types of operational weather forecasts can be anticipated if the optimal linear combination of climatology and persistence is used as a standard of reference. The conditions under which this practice might lead to substantial decreases in such skill scores are identified.