Transformer for EI Niño-Southern Oscillation Prediction

Accurate prediction of EI Niño-southern oscillation (ENSO) is of great significance to seasonal climate forecast. Recently, a convolutional neural network (CNN) has shown an optimal skill for ENSO prediction. However, it is difficult for the convolutional kernel to capture long-range precursors of ENSO due to its build-in local property. The transformer model has long been used in natural language processing (NLP) for its ability to focus on global features. Here, we introduce it to the ENSO research community and propose the ENSO transformer (ENSOTR). We show that using the ENSOTR model, the monthly average Niño3.4 index can be skillfully predicted up to one and a half years ahead. The model can also predict strong EI Niño cases more than a year ahead, such as 1997-1998. Experimental results show that our model achieves better skill than CNN for ENSO prediction.