Contrasting the Skills and Biases of Deterministic Predictions for the Two Types of El Niño

As the most striking interannual variability in the tropical Pacific, El Niño-Southern Oscillation (ENSO) has been intensively studied for several decades. Based on the profound effects of ENSO on environmental and socioeconomic activities worldwide, understanding the changes in ENSO's characteristics remains important and challenging (McPhaden et al., 2006; Ashok and Yamagata, 2009). Currently, due to improved observations (e.g., McPhaden et al., 1998) and modeling techniques (Delecluse et al., 1998), the ability to predict ENSO has become markedly better over the past few decades (Latif et al., 1998; Jin et al., 2008). Indeed, ENSO forecasts have reached the stage where skillful predictions can be made 6-12 months in advance. Several operational centers have used climate models to routinely make ENSO predictions in real time (Latif et al., 1998; Kirtman et al., 2001). However, the skill with respect to sea surface temperature (SST) forecasts in the equatorial Pacific is strongly model-dependent and widely divergent across various prediction systems (Jin et al., 2008; Barnston et al., 2012). Broadly, there is still room for improvement in ENSO prediction (Barnston et al., 2012).

It has been demonstrated that the real-time ENSO prediction skill over the past decade is obviously lower than that in the 1980s and 1990s (Barnston et al., 2012). For example, the correlation between observations and ENSO hindcasts over a nine-year sliding window has an average value of 0.65 during 1981-2010 at a six-month lead time, but decreases to 0.42 for 2002-11 period (Barnston et al., 2012). One possible reason for the shift in the ENSO prediction skill is because a different type of El Niño——as compared to the canonical eastern Pacific (EP) El Niño (McPhaden et al., 2011; Yu et al., 2012)——emerged in the 2000s. For this type of El Niño, the maximum anomalous SST is mostly confined to the central Pacific, and is thus referred to as the central Pacific (CP) El Niño (Yu and Kao, 2007; Kao and Yu, 2009; Yu et al., 2010; Zheng et al., 2014b).

The limited predictability may be attributable to factors such as errors in oceanic initial conditions, state-dependent stochastic forcing, or model errors (e.g., Moore and Kleeman, 1996; Karspeck et al., 2006; Duan and Zhao, 2015). However, a systematic examination of climate models' performances in predicting the two types of El Niño has been less well explored, and it remains controversial as to whether their predictabilities are distinct in different state-of-the-art climate models (e.g., Jeong et al., 2012; Yang and Jiang, 2014; Imada et al., 2015; Luo et al., 2016). Based on version 2 of the National Centers for Environmental Prediction Climate Forecast System, (Yang and Jiang, 2014) compared the model skill in using the El Niño Modoki index (EMI) and ño3 index to predict the two types of El Niño, and showed that the EMI was more persistent and predictable than the ño3 index during boreal summer and autumn. On the contrary, (Jeong et al., 2012) and (Luo et al., 2016) found EP events to be more predictable than CP events when adopting the coupled climate prediction multi-model ensemble suite of the Asia-Pacific Economic Cooperation Climate Center. Using version 5 of the Model for Interdisciplinary Research on Climate, (Imada et al., 2015) also found that CP El Niño has limited predictability and a shorter lead time for prediction compared to EP El Niño.

In the present study, we use the ensemble prediction system (EPS) developed at the Institute of Atmospheric Physics (IAP), Chinese Academy of Sciences (Zheng et al., 2006, 2007, 2009a; Zheng and Zhu, 2010a, 2016), to investigate the predictability of the two types of El Niño. Hindcast experiments are carried out with the EPS for 21 major El Niño events observed since 1950. The predictability of the ensemble-mean forecasts is validated and compared with respect to EP and CP El Niño. The common forecast biases for the two types of El Niño are identified and contrasted throughout the different phases of the El Niño lifecycle.

The IAP ENSO EPS has three main components: an intermediate coupled model (ICM), an air-sea coupled data assimilation system, and a stochastic model-error model. The ICM was developed by (Keenlyside and Kleeman, 2002) and (Zhang et al., 2005), and consists of a dynamical ocean model, an SST anomaly model that empirically parameterizes the temperature of subsurface water entrained into the mixed layer based on sea level anomalies, and a statistical wind stress model. The dynamical component of the ICM is described in detail by (Keenlyside and Kleeman, 2002). All coupled model components exchange simulated anomaly fields, such as the wind stress in the atmosphere and the SST in the ocean, once a day. The air-sea coupled data assimilation system (Zheng and Zhu, 2008, 2010a, 2015) uses an ensemble Kalman filter (EnKF) approach to minimize the errors in both the atmospheric and oceanic initial conditions by assimilating available atmosphere and ocean observations simultaneously into the ICM (please refer to the electronic supplementary material for details). A stochastic error model (Zheng et al., 2009a; Zheng and Zhu, 2016) is embedded within the ICM to perturb the modeled SST anomaly field randomly by adding error terms to the right-hand-sides of the model equations. This stochastic error model is designed to account for the temporal evolution of the forecasted uncertainties in the SST anomaly field (Zheng et al., 2006, 2009a, b; Feng et al., 2015; Zheng and Zhu, 2016). The performance of the prediction system is documented in (Zheng and Zhu, 2016), in which a 20-year retrospective forecast comparison shows that a good forecast skill of the EPS with a prediction lead time of up to one year is possible. Therefore, this EPS system is suitable for examining the predictability of the two types of El Niño.

Because of the need to initialize the model for a long period (1950-2012) to predict the CP and EP El Niño events, the observational data available to us in this study only include version 3b of the Extended Reconstructed SST (Smith et al., 2008) dataset (horizontal resolution: 2^°), and wind stress data obtained as the ensemble mean of a 24-ensemble member ECHAM4.5 simulation (Röckner et al., 1996; please refer to the online supplementary file for details). The SST data from 1950 to 2012 are used to select the EP and CP El Niño events for the hindcast experiments (for details, see section 3.1). For the hindcast experiments, the model's spin-up and data assimilation cycle are described in detail in the online supplementary file.

The El Niño events are selected in this study based on the National Oceanic and Atmospheric Administration (NOAA) criterion that the Ocean Niño Index (ONI) [i.e., the three-month running mean of SST anomalies in the Niño3.4 region (5^°N-5^°S, 120^°-170^°W)] must be greater than or equal to 0.5^°C for a period of at least five consecutive overlapping seasons. A total of 21 events are identified based on this criterion and are listed in Table 1. Similar to previous studies (e.g., Yu et al., 2012; Yu and Kim, 2013), we then determine the type of these 21 El Niño events based on a consensus of three identification methods, which are: the EMI method of (Ashok et al., 2007); the cold tongue (CT) and warm pool (WP) index method of (Ren and Jin, 2011); and the pattern correlation (PTN) method of (Yu and Kim, 2013). With the EMI method, an El Niño event is considered to be CP-type when the value of the December-January-February (DJF)-averaged EMI is equal to or greater than 0.7 standard deviations. With the CT/WP method, an El Niño event is classified as CP-type (EP-type) when the DJF-averaged value of the WP index is greater (less) than the averaged value of the CT index. Using the PTN method, the El Niño type is determined based on whether the spatial pattern of El Niño SST anomalies resembles more closely the typical SST anomaly pattern of EP- or CP-type El Niño. According to the consensus listed in Table 1, nine of the twenty-one major El Niño events are classified to be EP-type, and the other twelve are CP-type.

Next, the EPS is used to perform retrospective ensemble forecasts for the 21 major El Niño events. A unique index (i.e., ño3.4 index), rather than the ño3 and EMI indices used in previous studies (e.g., Yang and Jiang, 2014, Luo et al., 2016), is adopted to evaluate the skill of the EPS in forecasting the two types of El Niño. A 12-month hindcast is initialized each month during the period 1950-2010 with 100 ensemble members. Figure 1 shows the deterministic retrospective forecast results for the strongest EP El Niño (i.e., the 1997/98 event) and the strongest CP El Niño (i.e., the 2009/10 event) since 1950. Here, the strength of the El Niño event is measured by the peak value of the ño3.4 SST anomalies. The 1997/98 El Niño is also the strongest EP type El Niño event during the past century (McPhaden and Yu, 1999; Picaut et al., 2002), and the 2009/10 El Niño event is considered the largest CP-type El Niño event in the historical record (e.g., Yu et al., 2012). For both events, the EPS system can successfully predict their onset and development as early as 12 months in advance, albeit with errors still existing in the forecasts of their magnitudes.

The anomaly correlations and root-mean-square errors (RMSEs) between the observed and ensemble-mean-predicted SST anomalies in the ño3.4 region are shown in Fig. 2 as a function of lead time. The hindcasts for the both types of El Niño have generally high skill at all lead times. The correlations are greater than 0.93 for the one-month lead time and remain above 0.6 even for the twelve-month lead time. The skill scores of the hindcast are significantly higher for the EP events than for the CP events at all lead times. There is a distinct difference between the EP and CP events in skill score beyond the two-month lead time. At the nine-month lead time, the correlation coefficient for the EP events is still above 0.8, and is approximately 0.2 higher than that for the CP events. The RMSEs of the hindcast for both the EP and CP events remain smaller than 0.75^°C up to the nine-month lead time. Beyond the nine-month lead time, the RMSEs of the hindcast are about 0.1^°C-0.15^°C smaller for the CP events than for the EP events. This is because the CP events have a relatively weak signal compared to the EP events (Zheng et al., 2014b; Fang et al., 2015), and thus they are more affected by atmospheric noise; this prevents the development of oceanic signals that can be used for prediction (e.g., Imada et al., 2015).

Figure 1.  Deterministic predictions for the largest (a) EP and (b) CP El Niño since 1950. The thick black curves are the observed ño3.4 SST anomalies, and the thin curves of red, green, blue and orange are the ensemble-mean predictions starting 12, 9, 6 and 3 months, respectively, before the peak of each El Niño.

Figure 2.  Anomaly correlations (upper panel) and RMSEs (lower panel) between the observed and predicted SST anomalies in the ño3.4 region as a function of lead time. Values are shown separately for all El Niño events (dot-dashed lines), EP El Niño events only (lines with solid circles), and CP El Niño events only (lines with open circles). The results are obtained as the means of the ensemble hindcasts made for the El Niño events during the period 1950-2012.

Figure 3.  (a-d) Horizontal distribution of the anomaly correlations between observed and ensemble-mean-forecasted SST anomalies for the EP El Niño events at a lead time of (a) 3 months, (b) 6 months, (c) 9 months and (d) 12 months. The contour interval is 0.2, and the shaded areas represent correlation coefficients above 0.5 with 0.1 interval. (e-h) As in (a-d) but for the prediction of CP El Niño events.

The horizontal distributions of the anomaly correlations and RMSEs between the observed and predicted SST anomalies at lead times of three, six, nine and twelve months are displayed in Figs. 3 and 4. Geographically, the performance of this system is particularly good in the central-eastern equatorial Pacific (Zhang et al., 2005; Zheng and Zhu, 2015). Also, for predicting the EP events, the correlation is above 0.8 in the central basin and above 0.7 in the eastern equatorial Pacific at the three-month lead time (Fig. 3a). As the lead time increases, the correlation drops first and fastest in the eastern basin. However, even at the six-month lead time (Fig. 3b), the skill does not drop much in the central basin (the correlation remains greater than 0.7) and there is only a slight decrease in the eastern basin. At a twelve-month lead time (Fig. 3d), correlations larger than 0.7 can still be found over a sizeable region of the central Pacific, but the correlation drops below 0.6 east of 100^°W. Consistent with Fig. 2, there is an obvious decrease in the correlations for the prediction of the CP events over the entire region. The correlations for the CP events are approximately 0.1-0.2 lower than those for the EP events at all lead times. The difference is particularly significant in the central and eastern Pacific (Fig. 3). At the same time, the forecasted errors in the EP El Niño predictions are approximately 0.2^°C larger than those in the CP El Niño predictions over the eastern Pacific at all lead times (Fig. 4).

Figure 4.  As in Fig. 3, but for the RMSEs. Contour interval is 0.2^°C, and the shaded areas represent RMSEs larger than 0.5^°C with 0.1^°C interval.

In coupled prediction systems, climate drift is still a significant problem. In some cases, systematic model biases can be much larger than the anomalies to be predicted (e.g., Schneider et al., 2003; Zhang et al., 2005). Typically, the systematic errors can be identified by averaging the differences between the predicted and observed physical fields over all ensemble members. In Fig. 5, we show the systematic errors of the IAP ENSO EPS in the ensemble-mean prediction of equatorial Pacific SST anomalies as a function of initial calendar month. When the systematic errors are calculated from the predictions for all 21 El Niño events (the top row), the errors are characterized by a warm bias in the eastern basin and a cold bias in the central part of the basin. The largest warm bias (close to 0.5^°C) occurs in May with the predictions starting from January and April. When stratifying the systematic errors by the predictions of the EP El Niño events (middle row) and the CP El Niño events (bottom row), it is noticeable that the systematic errors are significantly smaller for the EP El Niño predictions than for the CP El Niño predictions. The systematic errors in the El Niño predictions are mostly related to predicting CP El Niño events. The warm bias might be caused by the model's deficiency in simulating the thermocline feedback over the equatorial eastern Pacific during the CP El Niño events. Moreover, the obvious systematic errors in forecasting the CP El Niño events also contribute to the forecast errors in the CP El Niño predictions (Fig. 4). The weak systematic errors in the EP El Niño predictions also indicate the errors in forecasting the EP El Niño events are mainly stochastic (Zheng et al., 2016).

Figure 5.  Systematic errors of the predicted SST anomalies along the equator from the IAP ENSO EPS at different lead times. Results are shown for predictions starting in January (first column), April (second column), July (third column), and October (fourth column); and for the predictions for all the El Niño events (top row), the EP El Niño events only (middle row), and the CP El Niño events only (bottom row). Contour interval is 0.1^°C, and the shaded areas represent biases larger (smaller) than 0.3^°C (-0.3^°C).

The robust center of high systematic error in May over the eastern Pacific might indicate that CP El Niño events are more difficult to predict through the spring season. Errors in observations can easily lead to a spring prediction barrier (SPB; Webster and Yang, 1992) in the prediction of CP events due to the larger forecast bias (Zheng and Zhu, 2010b). To examine the seasonality of the prediction skill in a deterministic sense, Fig. 6 displays the anomaly correlations for the ensemble-mean forecast calculated as a function of the initial month and lead time. As shown in many previous studies, the skill in predicting SST anomalies depends sensitively on the initial month (e.g., Latif et al., 1998; Jin et al., 2008; Zheng and Zhu, 2010b). For example, as shown in Fig. 6, the correlation is relatively low for the predictions initialized before and even during the spring season, and is significantly higher for the predictions initialized thereafter. Moreover, the SPB can be characterized as a decay in the anomaly correlation skill of ENSO forecasts made before and during the spring being much more obvious and rapid than those made after spring (e.g., Webster and Yang, 1992; Zhang et al., 2005). Our results further indicate that a stronger SPB exists for CP El Niño predictions than for EP El Niño predictions. The ensemble-mean forecasts for the EP El Niño events have higher anomaly correlation coefficients than those for the CP El Niño events at all lead times and initial months. In particular, the decline in correlation skill for the CP El Niño predictions initialized before and during spring is much more rapid than that for the EP El Niño predictions.

The different speeds of decline in the correlations of the EP and CP El Niño events are likely related to the different seasonal evolution of the EP and CP events (Kao and Yu, 2009; Yeh et al., 2014). While both types of El Niño reach their maximum amplitude during boreal winter, the onset of the positive SST anomalies for EP events typically happens in the eastern Pacific during spring, but for CP events this usually takes place from the eastern subtropics into the tropical central Pacific during summer. Due to the difference in evolution, SST anomalies tend to be more persistent from spring to summer for EP events than for CP events. Consequently, EP El Niño tends to have a high persistence prediction skill across the spring season and a weak SPB, whereas CP El Niño tends to have a more rapid decline in persistence prediction skill and a strong SPB.

Figure 6.  Anomaly correlations in the ño3.4 region as a function of lead time and start month for the ensemble-mean forecasts performed using the IAP ENSO EPS: (a) for the EP El Niño events; (b) for the CP El Niño events.

It has been noticed that CP-type El Niño events have occurred more frequently in recent decades, and that this type of El Niño may be generated by a mechanism distinct from that of the traditional EP-type El Niño (Yu et al., 2010, 2017). In this study, the deterministic prediction skill of the IAP ENSO EPS is examined separately for EP- and CP-type El Niño. The prediction skill is found to be lower for CP-type El Niño. Beyond a three-month lead time, the prediction skill is consistently higher for predicting EP events (0.1-0.2 higher in terms of the correlation coefficient) than CP events. Also, the system produces overly warm SSTs in the eastern basin during the spring season when predicting CP El Niño events. This bias indicates that the SST anomalies in the prediction model may be too sensitive to wind forcing. This oversensitivity may be because most El Niño prediction models have been designed, tuned and tested to capture the thermocline dynamics of the traditional EP-type El Niño. These thermocline dynamics are suggested to be less important for CP El Niño, and thus may manifest as a systematic error when predicting such events. Our results indicate that further improvements in El Niño prediction can be realized if we can improve the performance of coupled atmosphere-ocean models in predicting CP El Niño, which should be recognized as a key and high-priority task for the climate prediction community.

Efforts to improve the ability of the IAP ENSO EPS in predicting CP El Niño are currently in progress. For example, the current version of the system does not consider the physical processes involving freshwater flux and salinity variability over the tropical Pacific (Zheng and Zhang, 2012; Zheng et al., 2014a), and the effects of salinity are not included in the current model. Because El Niño events, especially CP events, can be modulated by the interannually varying salinity (Zhu et al., 2014; Zheng and Zhang, 2015), such effects need to be considered. Using data assimilation methods in the initialization system to include more accurate salinity information may also help improve CP El Niño forecasts (Zheng and Zhu, 2010a, 2015). Moreover, as demonstrated in previous studies (Chen et al., 2004; Zheng et al., 2009a, 2016; Wang et al., 2010; Barnston et al., 2012), strong decadal variations exist in the predictability of ENSO, with the most recent decade having the lowest predictability among the past six. Because of the relatively low reliability of the data in the 1950s-70s compared to recent decades, it is better to assess the predictability of the two types of El Niño by comparing the difference in prediction skill between the recent and previous decades by including as many El Niño events as possible.