Performance of the Seasonal Forecasting of the Asian Summer Monsoon by BCC_CSM1.1(m)

Dynamical monsoon prediction is an important issue in scientific research and operational forecasting due to prominent socioeconomic demands. Given its remarkable technical difficulties, substantial effort has been devoted to improving the accuracy of monsoon forecasts during the past several decades.

Atmosphere-alone models forced by prescribed lower-boundary conditions have been used in seasonal monsoon forecasting for a number of decades (Charney and Shukla, 1981; Zeng et al., 1990, 1997; Shukla, 1998). Although atmospheric models have shown reasonable skill (e.g., Kang et al., 2002; Wang et al., 2004; Zhou et al., 2009), they nevertheless exhibit noticeable shortcomings due to the lack of air-sea interaction, which is especially important for the Asian summer monsoon (e.g., Wang et al., 2005; Wu and Kirtman, 2005). Instead, coupled atmosphere-land-ocean models offer overall better performance, and thus have gradually become the major tool of dynamical climate prediction (e.g., Palmer et al., 2004; Li et al., 2005; Saha et al., 2006; Weisheimer et al., 2009; Ma and Wang, 2014). Previous studies have also proven that one-tier monsoon prediction by coupled models is more skillful than two-tier prediction by atmospheric models that use previously forecasted SST (e.g., Kug et al., 2008; Kumar et al., 2008; Zhu and Shukla, 2013).

While monsoon forecasting with coupled climate system models shows promising results, it is also highly limited or affected by many factors such as model resolution, initial conditions, model physics and dynamics, and ensemble forecasting methods (e.g., Pope and Stratton, 2002; Krishnamurti et al., 2006; Fu et al., 2009; Yang et al., 2011; Kumar and Krishnamurti, 2012; Wen et al., 2012). State-of-the-art climate models remain incapable of accurately reproducing the realistic climatology and spatiotemporal variation of monsoon (e.g., Wang et al., 2008; Lee et al., 2010; Li et al., 2012; Rajeevan et al., 2012; Jiang et al., 2013; Liu et al., 2013, 2014a). Also, compared with large-scale monsoon features related to strong oceanic-atmospheric events such as El Ni\ no-Southern Oscillation (ENSO), it is harder for coupled climate models to predict the regional characteristics of monsoon (e.g., Yang et al., 2008; Drbohlav and Krishnamurthy, 2010). In addition, they possess limited skill in predicting the relationships among different scales of monsoon variability (Achuthavarier and Krishnamurthy, 2010; Joseph et al., 2010; Liu et al., 2014b). In brief, the improvement of dynamical monsoon forecasting still faces various scientific and technical problems.

In China, the especially urgent concern of accurately forecasting the Asian monsoon has propelled many meteorologists to develop numerical climate models and apply these models in short-term climate prediction (Ding et al., 2004; Wang et al., 2015). Although the forecasting of tropical SST and ENSO has exhibited continuous success (Zhou and Zeng, 2001; Zheng et al., 2006, 2009; Zhu et al., 2013) and the forecast method has been developed from the two-tier approach (Zeng et al., 1990, 1997; Lang et al., 2004) to the one-tier approach (Li et al., 2005; Liu et al., 2014b; Ma and Wang, 2014), dynamical prediction of the Asian monsoon remains an arduous task because of its large uncertainty (Wang, 1997; Wang et al., 2015). In spite of the challenges, the development of dynamical monsoon prediction by improving numerical models and forecast methods remains a major research topic. At present, whether or not a model can reasonably simulate or predict monsoon features is one of the most important criteria to assess the model's performance. How to fully diagnose the performance of a model in reproducing the variability of monsoon and its interaction with other climate phenomena is a vital issue for guiding further work on model development. In this study, we analyze the performance of a coupled climate system model used at the Beijing Climate Center (BCC) in terms of its seasonal forecasting of the Asian summer monsoon. Several questions are addressed: (1) To what degree can the model reproduce the observed climatology and interannual variability of the Asian summer monsoon? (2) What skill does the model possess regarding its forecasting of various major dynamical monsoon indices? (3) And to what extent can the model's deficiencies in monsoon forecasting be attributed to its inability to capture the relationship between the monsoon and underlying SST?

In section 2, a brief overview of the model and the observational data used in the study is provided. In sections 3-5, we analyze the model's predictions of monsoon climatology and interannual variability, several major dynamical monsoon indices and their relationships with large-scale circulation and precipitation patterns, and SST variability and its connection with monsoon, respectively. A summary of the results and a further discussion are presented in section 6.

The model adopted in this work is version 1.1 of the BCC Climate System Model (Wu et al., 2013) with a moderate atmospheric resolution [BCC_CSM1.1 (m)]. The atmospheric component of the model is the BCC Atmospheric General Model with a T106 horizontal resolution and 26 hybrid sigma/pressure layers in the vertical direction (Wu et al., 2010). The land component is version 1.0 of the BCC Atmosphere and Vegetation Interaction Model. The ocean and sea ice components are version 4 of the Geophysical Fluid Dynamics Laboratory Modular Ocean Model and the Sea Ice Simulator, respectively. The different components are coupled without any flux adjustment. BCC_CSM1.1 (m) is one of the climate system models joining phase 5 of the Coupled Model Intercomparison Project (CMIP).

Several seasonal hindcast experiments aimed at summer monsoon prediction are implemented using the BCC_CSM1.1 (m). The hindcasts are initiated from the first day of each calendar month from 1991 to 2013 and ended with a 9-month forecast integration. The atmospheric initial conditions are obtained from the four-times daily air temperature, winds, and surface pressure fields of the National Centers for Environmental Prediction (NCEP) Reanalysis, and the oceanic initial conditions are from the sea temperature of the NCEP Global Oceanic Data Assimilation System. The reanalysis data are used to initialize the model by a nudging method, which operates from late 1980s to the end of 2013. Each hindcast experiment includes 15 members, produced by a lagged average forecasting with a combination of different atmospheric and oceanic initial conditions at the end of the month preceding the beginning of the hindcast.

In addition, CMIP-type and Atmospheric Model Intercomparison Project (AMIP)-type simulations from 1990 to 2013 are also implemented with the same greenhouse gas forcing as in the hindcasts. The former is a coupled free run by BCC_CSM1.1(m), while the latter is an ensemble mean of four integrations by the atmospheric component of BCC_CSM1.1(m), which are initialized with different atmospheric initial conditions and forced by observed monthly SST and sea ice from the Hadley Centre Sea Ice and Sea Surface Temperature dataset (Rayner et al., 2003). The data with the same period of time as the hindcasts are used in this study.

The observational data used for model verification include multi-level zonal wind and meridional wind from the NCEP/Department of Energy (DOE) Reanalysis 2 (Kanamitsu et al., 2002), the Optimum Interpolation Sea Surface Temperature (Reynolds et al., 2002), and the monthly precipitation from the Global Precipitation Climatology Project (Adler et al., 2003).

In this study, summer monsoon refers to the June-July-August mean monsoon, and monsoon climatologies are computed based on the data from 1991 to 2013. The forecasts of the summer mean initialized on 1 June are defined as 0-month lead forecasts, and the runs initialized on 1 May are used as the 1-month lead forecasts, and so forth until the 6-month lead forecasts initialized on 1 December. For an objective assessment and better description of the performance of the forecast system, in addition to focusing on the ensemble mean features, the average results of individual members are also partially addressed to properly present the ability of the model itself.

Figure 1.  Climatologies of summer precipitation (shading; units: mm d^-1) and 850-hPa winds (vectors; units: m s^-1) for (a) observations, (b) CMIP simulation, and (c) hindcasts initialized on 1 June. Also shown are biases of (d) CMIP, (e) 0-month lead and (f) 6-month lead forecasts.

Observed, simulated, and predicted climatologies of precipitation and 850-hPa winds are shown in Fig. 1. The hindcasts basically capture the general distribution of maximum precipitation centers. However, overestimated rainfall appears over the south of the Indian subcontinent, the western Indo-China Peninsula, and the tropical western North Pacific. Underestimated rainfall is seen over the northern Bay of Bengal, the South China Sea, the eastern equatorial Indian Ocean, and the Maritime Continent. The biases of wind match those of precipitation, and they both show little increase from 0 to 6 months of lead time over most regions except some sparse areas. Compared to the long-term simulation, although small differences appear over the tropical Pacific and Indian oceans due partially to different initializations, an overall reflection of the distributions of biases is found, implying a quick formation and stable maintenance of prediction biases from the beginning of the forecast (Liu et al., 2013). Some of the biases, especially the dry biases over the Bay of Bengal and South China Sea and the cyclonic wind bias over the northwestern Pacific, are also fairly common in other state-of-the-art climate forecast models with varying degrees of magnitude (e.g., Lee et al., 2010; Drbohlav and Krishnamurthy, 2010; Liu et al., 2013).

We further examine the standard deviations of the interannual variation of summer mean 850-hPa zonal wind and precipitation (Fig. 2). The observed low-level zonal wind shows strong interannual variability over the western tropical Pacific, with a maximum center near the east of the Philippine Sea (Fig. 2a). The forecast with the shortest lead time reproduces the observed features reasonably, but with a more intense center that shifts southeastward toward the dateline (Fig. 2c). As the lead time increases, the stronger-than-observed center near the dateline shows a gradual decrease in error (Figs. 2c and d). For precipitation, the 0-month lead prediction shows a remarkable overestimation of the interannual variance over the southeastern Arabian Sea, western Indo-China Peninsula, and most of the western tropical Pacific (Fig. 2g), which weakens apparently in the long-lead forecast (Fig. 2h). The CMIP-type simulations (Figs. 2b and f) show a similar spatial distribution of variance, although weaker, compared to the short-lead forecast, suggesting the possible existence of obvious initial error over the tropical western Pacific and northern Indian Ocean for summer monsoon forecasts by BCC_CSM1.1(m).

Figure 2.  Standard deviations of interannual variability of the summer mean for 850-hPa zonal wind (units: m s^-1; left column) and precipitation (units: mm d^-1; right column) from 1991 to 2013. Panels (a) and (e) are for observations, (b) and (f) for CMIP simulations, (c) and (g) for 0-month lead predictions, and (d) and (h) for 6-month lead predictions.

Figure 3.  Spatial distributions of temporal correlations between ensemble mean predictions and observations for summer mean (a-d) 850-hPa zonal wind, (e-h) precipitation, and (i-l) SST from 1991 to 2013. Results for forecasts of 0-, 2-, 4-, and 6-month lead times are shown in each panel in sequence. The shading level above 0.41 represents the statistical significance of correlation above the 95% confidence level.

Figure 4.  Spatial distributions of forecast error (left column) and predictability error (right column) of interannual variation of 850-hPa zonal wind (units: m s^-1) from 1991 to 2013. Panels for ensemble mean predictions from short to long lead times are shown from top to bottom.

Figure 3 shows the model's forecast skill for wind, precipitation, and SST measured by the magnitude of the temporal correlation coefficient (TCC). For the 0-month lead forecast, significant TCCs between predicted and observed 850-hPa zonal wind are situated over most of the tropical Indian and Pacific oceans, except the Arabian Sea (Fig. 3a). As the lead time increases, the area with significant skill becomes gradually smaller and its central magnitude decreases obviously (Figs. 3b-d). The forecast becomes unskillful over most regions at the 6-month lead time. The prediction of precipitation shows considerable skill over the tropical central Pacific at the shortest lead time, but quickly becomes unskillful at other lead times, except near the equator (Figs. 3e-h). Precipitation is controlled by complicated physics other than large-scale circulation, which partially results in the low prediction skill and poor predictability for precipitation in climate models. In contrast, the prediction of SST is often more skillful over most of the tropical region, especially the central eastern Pacific, and the TCCs at short lead times are almost universally significant (Figs. 3i-l). This feature shows the long persistence of the predictable signal from slowly varying components of the climate system and the possible existence of the predictability source for monsoon variations (e.g., Charney and Shukla, 1981; Shukla, 1998).

We also assess the forecast error and predictability error, defined as the difference between the ensemble mean forecast and observation and the difference between two forecasts with different initializations, respectively (e.g., Lorenz, 1982; Drbohlav and Krishnamurthy, 2010). Here, the root-mean-square error (RMSE) between two ensemble mean forecasts initiated one month apart is used as the predictability error. For instance, at the 1-month lead time, the forecast error is the RMSE between the 1-month lead forecast and observation, and the predictability error is the RMSE between the 1-month lead forecast and 0-month lead forecast. Forecast error originates from errors in both the initial conditions and model imperfection, whereas predictability error is largely dependent on error in the initial conditions only, assuming the model to be perfect. In spite of some differences among forecasts of different lead times, the forecast error of 850-hPa zonal wind is mainly distributed over the eastern equatorial Indian Ocean, the western tropical Pacific, and the northern subtropical Pacific (Figs. 4a-d). The predictability error at the 1-month lead time shows a maximum over the tropical western North Pacific, denoting that forecasts of zonal wind suffer more from initial error over this region than other regions (Fig. 4e). In contrast, model imperfection should contribute more to the forecast error of wind over the eastern tropical Indian Ocean. The error distribution of precipitation basically matches that of zonal wind, but maximum centers of forecast error are also found over the south of the Indian subcontinent and the western Indo-China Peninsula, where predictability error is likewise evident. These disparities between forecast errors and predictability errors are generally consistent with the differences between the hindcast and CMIP-type simulations, as shown in Figs. 1 and 2. As the lead time increases, the predictability errors of zonal wind and precipitation gradually decrease, suggesting that long-lead predictions become more similar to each other and less dependent on the initial conditions because of the domination of slowly varying components.

Figure 5.  Left panels: interannual variations of (a) SEASM index, (b) WY index, and (c) SASM index for observations and predictions of different lead times. The decimals shown in brackets are the temporal correlation coefficients between observations and predictions. Right panels: interannual variations of pattern correlations between observations and predictions of different lead times for (d) 850-hPa zonal wind over the SEASM region, (e) vertical shear of zonal wind over the WY index region, and (f) vertical shear of meridional wind over the SASM region. The decimals shown beside the figure legends are averaged correlation coefficients from 1991 to 2013.

In this section, we explore the model's forecast skill in terms of the variability of monsoon and its relationship with large-scale circulation and precipitation patterns, with a focus on several dynamical monsoon indices. Possibly due to the existence of the "spring predictability barrier" phenomenon (Webster and Yang, 1992), long-lead predictions of the summer monsoon are often poor, as shown in Fig. 3. Thus, the model's predictions with lead times ranging from 0 to 3 months are the main concern in this section.

Figures 5a-c show the interannual variations of monsoon indices including the Southeast Asian summer monsoon (SEASM), the Webster-Yang (WY), and the South Asian summer monsoon (SASM) indices. The SEASM index is defined as the horizontal shear of 850-hPa zonal wind between (5^°-15^°N, 90^°-130^°E) and (22.5^°-32.5^°N, 110^°-140^°E) (Wang and Fan, 1999). The WY index is defined as the vertical shear of zonal winds between the 850- and 200-hPa levels averaged over (0^°-20^°N, 40^°-110^°E) (Webster and Yang, 1992), and the SASM index is defined as the vertical shear of meridional winds between the 850- and 200-hPa levels averaged over (10^°-30^°N, 70^°-110^°E) (Goswami et al., 1999). The ensemble predictions capture the observed interannual variability of the SEASM index well, with a TCC of 0.75 at the minimum lead time and 0.6 at the 3-month lead time (Fig. 5a). In contrast, the forecasts of the WY index are overall unskillful, which can be largely attributed to the model's obvious incapability in reproducing the variability of this index after 2007 (Fig. 5b). With a small spatial scale and small amplitude of variation, the SASM index is also unsuccessfully forecasted (Fig. 5c). Low SASM forecast skill is also found in other climate forecast systems (e.g., Kim et al., 2012; Jiang et al., 2013), possibly because the interannual variation of the SASM is affected not only by ENSO, but also by regional factors that are harder to predict, such as tropical eastern Indian Ocean SST. Nevertheless, the uncertainty in the observational data may also partially account for the above feature since the sudden drop of the WY index near 2009 does not appear in the European Centre for Medium-Range Weather Forecasts Interim Reanalysis (ERA-Interim) or the NCEP Climate Forecast System Reanalysis (e.g., Kim et al., 2012; Jiang et al., 2013). When compared to ERA-Interim, the TCCs between hindcasts and observations are 0.71 and 0.69 for the WY index, and 0.27 and 0.32 for the SASM index, at 0- and 3-month lead times, respectively (Fig. 6).

To examine the prediction skill with respect to the spatial variability of monsoon, we display the pattern correlation coefficients (PCCs) between observations and ensemble mean predictions at different lead times for 850-hPa zonal wind or vertical wind shear over the regions applied to define the monsoon indices. As shown in Figs. 5d and e, reasonable skill is found over both the SEASM region and WY index region. For the latter in particular, the PCC is about 0.53 at the minimum lead time and drops to 0.35 at the 3-month lead time. In contrast, the spatial variability of vertical wind shear over the SASM region is poorly captured, as is its temporal variability (Fig. 5f).

Figure 6.  As in Figs. 5a-c, but the observational data is replaced by European Centre for Medium-Range Weather Forecasts Interim Reanalysis.

The relationships between the above dynamical monsoon indices and the associated large-scale circulation and precipitation patterns are further evaluated and shown in Figs. 7 and 8. In observations, a stronger-than-normal SEASM often corresponds to westerly wind anomalies across the Maritime Continent and the tropical western Pacific surrounded by anomalies of cyclonic wind convergence over both of its sides in the two hemispheres, in association with more precipitation over the tropical western Pacific and less precipitation over an extensive belt from Australia across Indonesia to the Indian subcontinent (Fig. 7a). Compared to the observations, the member-average predictions indicate that the model captures a less significant connection between the SEASM and winds and precipitation, which is nearly confined to the tropical western North Pacific (Figs. 7b-e). However, the ensemble mean predictions present an overestimated basin-wide atmospheric response, marked in particular by the strong cyclonic wind anomaly over the western North Pacific and easterly wind anomaly over the equatorial eastern Pacific, as well as the horseshoe-pattern-like wet anomaly over the western Pacific and dry anomaly over the western Indian Ocean (Figs. 7f-i). The 0-month forecast shows the best skill, but it still exhibits a farther northward cyclonic wind response and cannot reproduce the observed belt of insufficient precipitation (Fig. 7f). As the lead time increases, the link between the SEASM and the wind and precipitation over the western Indian Ocean shows a gradual intensification.

Figure 7.  Patterns of regressions (vectors) of 850-hPa winds on the SEASM index and correlations (shading) between precipitation and the monsoon index for observations and predictions of different lead times. The results are examined by the (b-e) member average of individual predictions and (f-i) ensemble mean of predictions. The shading level above 0.41 represents the statistical significance of correlation above the 95% confidence level. Decimals shown in the upper right corners are the pattern correlation coefficients between observations and predictions.

Figure 8.  As in Fig. 7, but for the results of ensemble mean predictions for the WY index (left column) and SASM index (right column).

The observed WY index shows a close connection with the low-level winds over the southern Asian monsoon region and some sparse areas over the tropical Pacific (Fig. 8a). In contrast, the member-average predictions clearly overestimate the relationship between the WY index and the zonal wind over the equatorial central-eastern Pacific. Furthermore, the ensemble mean results show that a stronger-than-normal WY index corresponds to a remarkable westerly wind anomaly over the southern Asian monsoon region and easterly wind anomaly over the tropical central-eastern Pacific, and is coupled with significant cyclonic wind anomalies over both the western North and South Pacific. Associated with this feature, obvious dry-wet differences between the tropical western Indian Ocean and the Indian subcontinent, and between the tropical central-eastern Pacific and the tropical western Pacific, are found (Figs. 8b-e).

Figure 9.  Temporal correlations of (a) Ni\ no3.4 index and (b) IOD index between observations and predictions as a function of lead time and target season. Also shown are the interannual variations of (c) Ni\ no3.4 index and (e) IOD index, and the pattern correlations between observations and predictions of different lead times for SST over the (d) Ni\ no3 and Ni\ no4 region and (f) IOD region. The decimals shown in brackets in (c) and (e) are the temporal correlations between observations and predictions, and those in (d) and (f) are the multi-year averaged pattern correlations.

Figure 10.  First principal components (first and third columns) and spatial modes (second and fourth columns) for the EOF analysis of SSTs over the tropical Pacific (left two columns) and Indian Ocean (right two columns). Observations (first row) and ensemble mean predictions of different leads (second to fifth rows) are shown. The percentages in the annotation are the variances of the modes, and the decimals are the temporal or pattern correlation coefficients between observations and predictions.

Figure 11.  Patterns of regressions (vectors) of 850-hPa winds on first principal components of SSTs over the tropical Pacific (left column) and Indian Ocean (right column), and of correlations (shading) between precipitation and the first principal components for (a, f) observations and (b-e and g-j) ensemble mean predictions of different lead times. The shading level above 0.41 represents the statistical significance of correlation above the 95% confidence level. Decimals shown in the upper right corners are the pattern correlation coefficients between observations and predictions.

As shown by Fig. 8f, a strong SASM index is significantly related to the southerly wind anomaly over the region from the Arabian Sea to the Bay of Bengal and the wet anomaly over the Indian subcontinent. Also, it is closely teleconnected to the circulation anomaly over parts of the subtropical Pacific. Compared with observations, the individual members are unable to reasonably capture both the local and remote connection of the SASM index with circulation and precipitation patterns. For ensemble mean predictions, not only the local strong anomalies of meridional wind and precipitation, but also the significant north wind anomaly over the western Indian Ocean and easterly wind anomaly over the tropical central Pacific, appear to match an anomalously high SASM index. For the 2- and 3-month lead forecasts in particular, zonal wind anomalies extend from the equatorial central Pacific to the Maritime Continent (Figs. 8g-j).

The predictability of monsoon is partially attributed to the memory of the underlying surface state, and thus it is particularly important to assess the model's ability in predicting SST over the tropical Pacific and Indian oceans. Predictions of two commonly-used SST indices, the Ni\ no3.4 index and Indian Ocean dipole (IOD) index, are examined in Fig. 9. The former is defined as the SST averaged over (5^°S-5^°N, 170^°-120^°W) and the latter is defined as the difference in SST between (10^°S-10^°N, 50^°-70^°E) and (10^°S-0^°N, 90^°-110^°E) (Saji et al., 1999). The Ni\ no3.4 index is predicted well in all target seasons except summer, when ENSO's amplitude is often small and long-lead forecasts suffer from the influence of the spring predictability barrier (Fig. 9a). The forecasts of the IOD index, however, are generally poor at most lead times except for the target autumn and winter (Fig. 9b). For the June-July-August mean, the TCCs between observations and 0-, 1-, 2- and 3-month lead predictions are 0.92, 0.91, 0.83, and 0.66 for the Ni\ no3.4 index (Fig. 9c), and 0.65, 0.52, 0.33, and 0.09 for the IOD index (Fig. 9e), respectively. Besides, the PCCs of SST over the El Ni\ no region (5^°S-5^°N, 160^°E-90^°W) and the IOD region attain average maxima of 0.71 and 0.48 at the 0-month lead time, respectively (Figs. 9d and f). Compared to the results shown in Figs. 5d-f, the interannual variations of PCCs for regional SST forecasts often show insignificant correlations with those of PCCs for monsoon forecasts, except for the PCC variations for the SEASM and Ni\ no SST (R=0.41) and for the SASM and IOD SST (R=0.53) at the 0-month lead time. The prediction skill in terms of the tropical large-scale monsoon feature is highly related to ENSO amplitude (Kim et al., 2012), and in this study some particular years with extremely anomalous Ni\ no3.4 index or IOD index values also have high PCCs over certain monsoon regions (Figs. 9c, e and 5d-f). However, overall, there is no significant interannual relationship between the amplitude of ENSO/IOD and the forecast skill of regional monsoon.

We further conduct an empirical orthogonal function analysis of the tropical Pacific and Indian Ocean SST, for both observation and prediction. The first mode of the Pacific SST is characterized by out-of-phase anomalies over the eastern and western tropical Pacific, with a maximum center over the central-eastern equatorial Pacific. The extensive significant anomaly over the eastern Pacific is captured well by the 0- and 1-month lead forecasts, but its spatial range quickly reduces to a narrow band near the equator at the lead times of 2 and 3 months (Fig. 10b). In the tropical Indian Ocean, the observed first spatial mode shows a generally consistent anomaly over much of the ocean, with maximum centers near the eastern and western sides. The predictions, however, show a less extensive range of uniform anomaly and unrealistic maximums over the central areas (Fig. 10d). As a result, the PCCs between observed and predicted spatial modes over the Indian Ocean are obviously lower than those over the Pacific, although the TCCs of the principal components between observations and predictions are often significant (Figs. 10a and c).

To some extent, the model's good performance in predicting the principal SST mode over the tropical Pacific also means a skillful forecast of ENSO variability, since the correlation between the corresponding principal component and Ni\ no3.4 SST index is as high as 0.9 for observations. For the principal SST mode over the tropical Indian Ocean, although it is highly related to ENSO, as proven by previous studies (e.g., Klein et al., 1999; Saji et al., 2006; Du et al., 2013), its linear TCCs with both the principal SST mode over the tropical Pacific and the Ni\ no3.4 SST index are insignificant for both observations and predictions, indicating a weak connection of SST variability between the two oceans in summer during the past two decades. Also, this mode is rarely related to the IOD index because the latter is often represented by the second SST mode, rather than the first mode (Saji et al., 1999).

As an important underlying forcing, SST contributes substantially to the interannual variability of monsoon. The regressions of low-level atmospheric circulation on the principal components of SST over the tropical Pacific and Indian oceans are presented in Fig. 11. The correlations of the SST mode over the tropical Pacific with circulation and precipitation patterns are often significant over the Maritime Continent, the equatorial Pacific, and the South Pacific in observations (Fig. 11a). The predictions basically capture the observed relationships, but their magnitudes are obviously overestimated, especially over the subtropical Pacific in the two hemispheres and part of the Indian Ocean, implying that a stronger-than-observed response of the subtropical circulation to the tropical SST is always predicted (Figs. 11b-e). For the SST mode over the tropical Indian Ocean, strong responses of circulation and precipitation in observations are limited to some local areas only in the South Asian summer monsoon region and the subtropical Pacific (Fig. 11f). The forecasts, however, tend to show an exaggerated connection between SST mode and circulation over the tropical Indian Ocean and central-western Pacific (Figs. 11g-j).

The relationships between the first principal components of SST and various dynamical monsoon indices are further exhibited in Table 1. The correlations with the Ni\ no3.4 SST index and IOD index are also shown in the table. The observed SEASM index is significantly correlated with Ni\ no3.4 SST index, but insignificantly with the first principal component of SST over the tropical Pacific, although the latter two factors are highly connected with each other. This feature is because the interrelation between the SEASM and SST is mainly limited to the near-equatorial region, compared with the subtropical region. The predictions partially capture the close relationship between the SEASM and the SST over the Pacific, but only at the lead time of 3 months. Both the WY index and SASM index show a weak connection with the principal component of tropical Pacific SST in observations, but in most predictions these relationships are seriously overestimated. In contrast, to some extent, the correlations of these two indices with the principal component of the Indian Ocean SST are underestimated in predictions compared to observations. In addition, overestimated relationships between the IOD index and SEASM and WY indices, as well as underestimated relationships between the IOD index and SASM index, are found in most predictions.

The above results reveal the possible influence of unrealistic SST forcing on the failure of monsoon forecasts. We further compare hindcasts and an AMIP-type simulation to better understand this possible influence. In the AMIP simulation, the TCCs between simulated and observed WY index, SEASM index, and SASM index are 0.34, 0.27, and -0.03, respectively, which are slightly higher than those in the 0-month lead forecasts for the former but obviously lower than most predictions for the latter two. This feature implies that inclusion of ocean-atmosphere coupling is of critical importance to SEASM and SASM forecasts in the model, although a SST drift may also be introduced. Figure 12 shows the patterns of regression of winds against the monsoon indices and principal components of SST, and correlations between these indices and precipitation, in the AMIP simulation. Compared to the forecasts shown in Figs. 7, 8 and 11, overall similar features are found in the AMIP simulation, albeit with relatively weaker magnitudes over many regions, especially the subtropical Pacific. This feature indicates that the connections among SST, monsoons, and large-scale circulation patterns in the AMIP simulation are reproduced and further exaggerated by the coupled ensemble forecasts. In particular, the atmospheric response to tropical Pacific SST in the AMIP simulation shows a higher PCC with the observation than those in all predictions, suggesting that the unrealistic SST forcing of monsoon is less significantly highlighted in the AMIP run compared to the coupled forecasts. This feature can also be seen in Table 1, in which the correlations between the first principal component of Pacific SST and the WY and SASM indices is often less significant in both the AMIP and CMIP simulations than in most predictions. Of course, the important role of SST could also be related to the configuration of the AMIP simulation itself or the enhancement of signals by ensemble average forecasting methods. Nevertheless, the strong responses of circulation and precipitation over the tropical central-eastern Indian Ocean to the principal component of Indian Ocean SST, as shown in Figs. 11g-j for ensemble predictions, do not appear in the AMIP simulation (Fig. 12e), partially showing the negative impacts of ocean-atmosphere coupling and initial conditions on the monsoon forecasts by the coupled BCC_CSM1.1(m).

Overall, the predictions overestimate the relationships between the basin-wide SST mode and regional circulation and precipitation over the tropical Pacific and Indian oceans, which should be closely associated with the evident forecast error and predictability error, as shown in Fig. 4. Monsoon forecast skill is dependent on the model's ability to accurately capture the links of monsoon with SST and large-scale circulation. Therefore, the unskillful forecasts of the WY index and SASM index (Fig. 5) are partially attributed to the obviously overestimated links of monsoon with SST and circulation over the tropical Pacific, as well as the unreasonable contribution from the variability of SST and circulation over the tropical Indian Ocean (Table 1, Figs. 8 and 11). In contrast, although with an unrealistic signature of SST mode, the more extensive response of the SEASM to large-scale circulation and its closer connection to ENSO compared to the SASM and WY indices should be partially responsible for the reasonable forecast of the SEASM index (Figs. 5, 7, and 11).

In this study, we analyze the performance of BCC_ CSM1.1(m) in terms of its seasonal forecasting of the Asian summer monsoon using various hindcast experiments initialized with different initial conditions. Despite reasonable skill regarding monsoon climatology and interannual variability, predictions show apparent biases for winds and precipitation over the monsoon region, as well as an apparent overestimation of the interannual variance over the tropical western Pacific. Compared to the insignificant TCCs of precipitation between observations and predictions over most of the monsoon region, significant skill with respect to 850-hPa zonal wind is found over the tropical western Pacific, in association with the skillful prediction of SST over most of the tropical ocean, especially the central-eastern Pacific. Nevertheless, forecasts often show gradually decreasing skill with increasing lead time and become unskillful over many regions when the lead time is longer than 4 months. Comparisons between forecast error and predictability error further indicate that the prediction of wind often suffers apparent initial errors over the tropical western North Pacific, and the rainfall forecast is affected by obvious initial errors over the south of the Indian subcontinent and the western Indo-China Peninsula, as well as the tropical western Pacific.

Figure 12.  As in Fig. 11, but for regressions on the (a) SEASM index, (b) WY index, (c) SASM index, and the first principal components of SSTs over the (d) tropical Pacific and (f) Indian Ocean in the AMIP-type simulation.

The ensemble mean predictions show varying skill for different dynamical monsoon indices. The temporal variability of the SEASM index is more skillfully predicted than that of the WY index and SASM index. Nevertheless, the spatial pattern correlation over the region in which the WY index is defined is the highest, followed by the SEASM region and the SASM region. Although high skill in some particular years is associated with extremely anomalous Ni\ no3.4 or IOD SST index, overall there is no significant linear correlation between the amplitude of ENSO/IOD and the forecast skill of regional monsoon. Nevertheless, significant correlations between the interannual variations of PCCs for the SEASM and Ni\ no SST, and between those for the SASM and IOD SST, are found at the shortest lead time. Furthermore, the relationships between these monsoon indices and the circulation and precipitation patterns are examined. Compared to the less extensive response of circulation to the dynamical monsoon indices shown by averaging the results of individual members, the ensemble mean predictions significantly overestimate the connection of the SEASM with the low-level winds over the western North Pacific and the equatorial eastern Pacific, as well as with the precipitation over the subtropical Pacific. Meanwhile, the relationships between the WY index and the circulation and precipitation over the tropical oceans, and between the SASM index and the winds over the monsoon region and equatorial Pacific, are also exaggerated obviously.

The temporal and spatial variations of the SST in both observations and predictions are examined by an empirical orthogonal function analysis. Reasonable skill is found for the primary spatial mode and the principal component of tropical Pacific SST, in spite of a clear decline with an increase in lead time. In contrast, predictions show limited skill in terms of the tropical Indian Ocean SST, especially its spatial mode. The relationships between the first principal component of Pacific SST and the winds over the subtropics, and between the first principal component of Indian Ocean SST and the winds over the tropical Indian Ocean and central-western Pacific, are always exaggerated by most ensemble mean predictions. The observed connections of the WY index and SASM index with the principal component of the Pacific SST are clearly overestimated in most forecasts, while their links with the tropical Indian Ocean SST are often underestimated. Comparisons with the AMIP-type simulation show that unrealistic connections among monsoons, atmospheric circulation, and tropical Pacific SST in the atmosphere-only model are similarly reproduced and further exaggerated by the coupled ensemble forecasts, suggesting the existence of an unreasonable atmospheric response to tropical SST in both the simulation and predictions.

Our analyses indicate some shortcomings regarding monsoon forecasts, providing information for further development of BCC_CSM1.1(m). The predictions show reasonable skill with respect to the interannual variability of tropical SST, but the model is incapable of reproducing the appropriate interaction between monsoon and SST. The various degrees of overestimation or underestimation of the relationships partially account for the unskillful forecasts of the Asian monsoon. Therefore, in addition to further improvements in the forecasting of the underlying boundary conditions, a realistic description of the connection between monsoon and surface conditions, especially between the Asian monsoon and the tropical SST over the Pacific and Indian oceans, is of critical importance.

It should also be noted that a considerable proportion of the above-described overestimation originates from the ensemble averaging process, in which internal noise is greatly filtered out and signals from external forcings are excessively highlighted. This feature indicates that ensemble forecasting methods sometimes obviously improve the prediction skill for many climate factors at the cost of losing the reasonable relationships among them. In this case, although we have to use various ensemble methods to improve the dynamical forecast skill, the number of members to compose an ensemble forecast is critical, being neither too large nor too small. In this context, the optimal use of ensemble forecasting methods deserves further assessment and exploration.

In addition, the uncertainty of observational data and its influence on the evaluation of forecast results deserves more attention. As mentioned in section 4, the forecast of the WY index is poor compared to NCEP/DOE Reanalysis 2 data but good compared to the ERA-Interim data. This is because of the different variations of the index in different datasets, as shown in this and other studies (e.g., Kim et al., 2012; Jiang et al., 2013; Ma and Wang, 2014). Thus, a comprehensive validation based on more observational data is necessary.