Simulation data from 20 CMIP6 models (Table 1) are used in this study. For each model, the near-surface air temperature and precipitation results from the historical simulation and the SSP245 and SSP585 experiments are employed. The SSP245 and SSP585 reflect a set of alternative futures of social development and greenhouse gas emission. The SSP245 represents the combined scenario of a moderate socio-economic development path (i.e., SSP2) with the medium-low radiation forcing which peaks at 4.5 W m−2 by 2100. The SSP585 represents the combined scenario of a high energy-intensive, socio-economic developmental path (i.e., SSP5) with strong radiative forcing which peaks at 8.5 W m−2 by 2100 (O'Neill et al., 2016; Riahi et al., 2017).
ID Model name Institution and country Atmospheric resolution (lon×lat: number of grids, L: vertical levels) 1 ACCESS-CM2 Commonwealth Scientific and Industrial Research Organization, Australian Research Council Centre of Excellence for Climate System Science, Australia 192×144, L85 2 ACCESS-ESM1-5 Commonwealth Scientific and Industrial Research Organization, Australia 192×145, L38 3 BCC-CSM2-MR Beijing Climate Center, China 320×160, L46 4 CanESM5 Canadian Centre for Climate Modelling and Analysis, Canada 128 × 64, L49 5 CESM2 National Center for Climate Research, USA 288 × 192, L32 6 CESM2-WACCM National Center for Climate Research, USA 288 × 192, L70 7 EC-Earth3 EC-Earth Consortium, Europe 512 × 256, L91 8 EC-Earth3-Veg EC-Earth Consortium, Europe 512 × 256, L91 9 FGOALS-g3 Chinese Academy of Sciences, China 180 × 80, L26 10 GFDL-CM4 National Oceanic and Atmospheric Administration, Geophysical Fluid Dynamics Laboratory, USA 288 × 180, L33 11 GFDL-ESM4 National Oceanic and Atmospheric Administration, Geophysical Fluid Dynamics Laboratory, USA 288 × 180, L49 12 INM-CM4-8 Institute for Numerical Mathematics, Russia 180 × 120, L21 13 INM-CM5-0 Institute for Numerical Mathematics, Russia 180 × 120, L73 14 IPSL-CM6A-LR Institute Pierre Simon Laplace, France 144 × 143, L79 15 MIROC6 Atmosphere and Ocean Research Institute, The University of Tokyo, Japan 256 × 128, L81 16 MPI-ESM1-2-HR Max Planck Institute for Meteorology, Germany 384 × 192, L95 17 MPI-ESM1-2-LR Max Planck Institute for Meteorology, Alfred Wegener Institute, Germany 192× 96, L47 18 MRI-ESM2-0 Meteorological Research Institute, Japan 320 ×160, L80 19 NorESM2-LM NorESM Climate modeling Consortium, Norway 144 × 96, L32 20 NorESM2-MM NorESM Climate modeling Consortium, Norway 288 × 192, L32
Table 1. Basic information for the CMIP6 models used in this study.
The observed temperature and precipitation data of CN05.1 with a resolution of 0.25°×0.25° (Wu and Gao, 2013) are used to validate the performance of the CMIP6 models. For convenience, all data are converted to the same 1°×1° grid using a bilinear interpolation scheme before analysis. As recommended by the CMIP6, the period 1995–2014 is used as the reference period for the evaluation and projection. The ensemble in this study is calculated with the same weight. The statistical significance is examined by the Student’s t-test.
A Taylor diagram (Taylor, 2001) is used to evaluate spatial distributions of temperature and precipitation over China. This diagram provides a concise statistical summary of how well a simulated pattern matches an observed pattern in terms of the spatial correlation coefficient (SCC), the root-mean-square error (RMSE), and the ratio of variances. The interannual variability of the simulations relative to the observations is assessed by the interannual variability skill score (IVS) (Gleckler et al., 2008; Scherrer, 2011), which is calculated as
where STDm and STDo are the standard deviations of the simulation and observation, respectively. IVS is a symmetric variability statistic that is used to measure the similarity of interannual variation between the simulation and observation. A smaller IVS value indicates a better simulation of interannual variability.
To quantitatively examine regional differences, following Zhou et al. (2014), we divide China into eight subregions: Northeast China (NEC; 39°–54°N, 119°–134°E), North China (NC; 36°–46°N, 111°–119°E), East China (EC; 27°–36°N, 116°–122°E), Central China (CC; 27°–36°N, 106°–116°E), Northwest China (NWC; 36°–46°N, 75°–111°E), Tibetan Plateau (SWC1; 27°–36°N, 77°–106°E), Southwest China (SWC2; 22°–27°N, 98°–106°E), and South China (SC; 20°–27°N, 106°–120°E) (see Fig. 1), all of which are based on administrative boundaries and societal and geographical conditions (National Report Committee, 2007).
Figures 2a–f show the climatological spatial distributions of annual, winter (December to February, DJF), and summer (June to August, JJA) temperatures from observations and the ensemble simulation of all models (AMME), respectively. In general, the AMME simulated spatial patterns approximate those of the observations. However, relative to the observations, a general underestimation of annual temperature is noted over most of China in the AMME simulation. The most pronounced cold bias is located in the Tibetan Plateau (Fig. 2g). This phenomenon was also present in the CMIP3 and CMIP5 simulations as revealed by previous studies (Jiang et al., 2005; Xu and Xu, 2012a; Jiang et al., 2016). For winter (Fig. 2h) and summer (Fig. 2i) temperatures, there are notable warm biases in parts of northern China, in addition to the cold bias in the Tibetan Plateau.
Figure 2. Spatial distributions of (a−c) observed temperature (units: °C), (d−f) AMME simulated temperature (units: °C), and (g−i) AMME simulation biases from the observation (simulation minus observation, units: °C) for the period 1995–2014. The panels from the left to right side are for annual (ANN), winter (DJF), and summer (JJA), respectively. The black lines in (g)−(i) show the boundary of subregions. Note that the scales of color bars are different.
For observed precipitation (Figs. 3a–c), the annual, winter, and summer precipitation amounts decrease from the southeast coast to the northwest areas. These spatial patterns are captured by the AMME simulation (Figs. 3d–f) but with overall wet biases (Figs. 3g–i). The wet bias for annual precipitation appears in most parts of northern and western China, particularly on the northern and southern flanks of the Tibetan Plateau (Fig. 3g), which was also reported for the CMIP3 and CMIP5 simulations (Jiang et al., 2005; Xu and Xu, 2012a; Jiang et al., 2016). Compared with the CMIP5, the wet bias in the CMIP6 models was observed to be smaller (Jiang et al., 2020; Zhu et al., 2020). The spatial distributions of wet biases for winter precipitation resemble that for annual precipitation, but with larger bias magnitudes (Fig. 3h). Besides the wet bias, dry biases are also notable for summer precipitation in parts of Northwest China and East China (Fig. 3i).
Figure 3. Spatial distributions of (a−c) observed precipitation (units: mm), (d−f) AMME simulated precipitation (units: mm), and (g−i) AMME simulation biases from the observation ((simulation minus observation)/observation, units: %) for the period 1995–2014. The panels from the left to right side are for annual (ANN), winter (DJF), and summer (JJA), respectively. The black lines in (g)−(i) show the boundary of subregions. Note that the scales of color bars are different.
Figure 4 shows the Taylor diagrams for annual, winter, and summer temperature and precipitation over China as simulated by the 20 CMIP6 models and AMME against the observations. The azimuthal position of the model point indicates the SCC between the simulated and observed patterns. The distance from the reference point (REF) to the model point indicates the normalized RMSE of the simulation relative to the observation. The radial distance from the origin to the model point indicates the ratio of standard deviations between the simulation and observation. The overall model biases are excluded in this diagram. Clearly, the CMIP6 models show better performance for temperature than for precipitation. For temperature, regardless of whether winter, summer, or annual mean values are used, the SCCs between the simulations and observations are all greater than 0.9, the RMSEs of the simulations relative to the observations are generally below 0.5, and the ratios of variances to the observations are close to 1 for most models. These results indicate that the CMIP6 models effectively capture the climatological distributions in terms of annual, summer, and winter temperatures.
Figure 4. Taylor diagrams of (a) annual (ANN), (b) winter (DJF), and (c) summer (JJA) temperature (red dots; units: °C) and precipitation (blue dots; units: mm) over China for the period 1995–2014. The black dot in each panel represents AMME.
Compared with temperature, the SCCs for precipitation over China are relatively lower and the RMSEs are relatively higher. Specifically, the SCCs and RMSEs are mainly in the range of 0.6–0.9 (still statistically significant) and 0.5–1, respectively. In addition, the ratios of variances mostly lie between 1 and 1.5. Overall, the simulations of most models are reliable for the spatial patterns of annual, summer, and winter precipitation, although the variances are overestimated.
Figure 5 presents the IVS values of the simulations for the interannual variability of annual, winter, and summer temperature and precipitation over China. In this study, the IVS values were first calculated in each grid of China and then averaged. For temperature (Fig. 5a), the IVS values are below 1.5 for all models except for CanESM5 which shows a value of 4.0 in summer. This suggests that the CMIP6 models can reasonably reproduce the observed interannual variability of annual, winter, and summer temperature. In comparison, the model performances for the interannual variability of annual and winter temperatures are better than their performances in summer. For precipitation (Fig. 5b), though the IVS values are larger than those for temperature, the relatively low IVS values in annual mean and summer imply a reasonable reproduction of the observed interannual variability by the CMIP6 models. It also reflects the dominant contribution of summer precipitation to annual precipitation (Sui et al., 2013). There is a large range for the winter IVS values, which vary from 7.1 to 62.9 and are much larger than those of annual mean and summer. This result indicates large inconsistencies among the models and poor simulations for the interannual variability of winter precipitation.
Figure 5. Interannual variability skill score (IVS) of the CMIP6 models for annual (ANN), winter (DJF), and summer (JJA) (a) temperature and (b) precipitation over China. Note that the IVS for winter precipitation is divided by 10.
According to Gleckler et al. (2008), the rankings for all models that considered the three factors of the Taylor diagram and the interannual variability skill score are summarized in Fig. 6. This figure depicts the overall performance of individual models. A smaller ranking value indicates a better performing model. The rankings for the Taylor diagram are the average of the rankings of SCC, RMSE, and ratio of variance. On the whole, the AMME outperforms its ensemble members in a comprehensive manner. For a given individual model, the performance ranks are somewhat different for different metrics. Considering the comprehensive performance for both spatial patterns and IVS, the relatively “highest-ranked” and “lowest-ranked” models are selected based on Fig. 6 and listed in Table 2. For these “highest-ranked” and “lowest-ranked” models, their comprehensive performances (arithmetic average of the rankings for Taylor Diagram and IVS) rank in the top three and bottom three among all models, respectively. Note that ACCESS-ESM1-5 and CESM2-WACCM (ACCESS-ESM1-5 and CESM2) show the same ranking for annual (summer) precipitation.
Figure 6. Portrait diagram of the rankings of model performance for annual (ANN), winter (DJF), and summer (JJA) (a) temperature (units: °C) and (b) precipitation (units: mm). The colors in the label bar indicate the rankings. A smaller ranking number indicates a better model performance. Columns from the left to the right side in each group show the rankings of the SCC, ratio of variances, and RMSE, mean rankings of the three factors in the Taylor diagram, and IVS rankings, respectively.
ANN DJF JJA Highest ranked models Lowest ranked models Highest ranked models Lowest ranked models Highest ranked models Lowest ranked models Tas CESM2-WACCM CanESM5 ACCESS-CM2 CanESM5 CESM2 CanESM5 GFDL-ESM4 IPSL-CM6A-LR CESM2-WACCM IPSL-CM6A-LR CESM2-WACCM FGOALS-g3 MPI-ESM1-2-HR MIROC6 NorESM2-MM MIROC6 NorESM2-MM INM-CM5-0 Pre EC-Earth3 ACCESS-ESM1-5 EC-Earth3 MPI-ESM1-2-LR ACCESS-CM2 ACCESS-ESM1-5 EC-Earth3-Veg CESM2 EC-Earth3-Veg INM-CM4-8 BCC-CSM2-MR CESM2 MRI-ESM2-0 CESM2-WACCM GFDL-CM4 INM-CM5-0 INM-CM4-8 FGOALS-g3 FGOALS-g3 MPI-ESM1-2-LR
Table 2. Highest and lowest ranking models selected for the ensembles for annual (ANN), winter (DJF), and summer (JJA) temperature and precipitation.
Some studies have shown that increasing the model resolution is an effective way to improve the performance of model simulations (Yao et al., 2017; Zhou et al., 2018b; Bador et al., 2020), thus we examine the relationships between the model performances and resolutions. The analyses show that the comprehensive performances of the models and their resolutions are significantly correlated. The correlation coefficients are 0.50 and 0.81 for annual and summer temperatures, respectively. The comprehensive performance of the models for winter precipitation also show a significant correlation of 0.65 with their resolutions, which is consistent with the previous finding that model resolution influences the simulation of winter precipitation in China (Gao et al., 2006; Jiang et al., 2016, 2020).
Figure 7 shows the spatial distributions of the biases from the “highest-ranked” model ensemble (hereafter BMME) and the “lowest-ranked” model ensemble (hereafter WMME) for annual temperature and precipitation. Compared with the AMME simulation (Fig. 2g), the cold bias over the Tibetan Plateau is reduced in the BMME simulation (Fig. 7a) and augmented in the WMME simulation (Fig. 7b). The regionally averaged BMME, AMME, and WMME biases in SWC1 are −1.3°C, −2.0°C, and −4.3°C, respectively (Fig. 8a). From a seasonal perspective, the performance of the BMME for winter temperature is better than that of the AMME and WMME simulations over SWC1, CC, EC, SC, and SWC2 (Fig. 8c). However, due to an overall warm bias, the BMME does not perform better than the AMME in simulating summer temperature but does indicate a smaller spread (Fig. 8e).
Figure 7. Spatial distributions of (a, c) BMME and (b, d) WMME simulation biases for annual (a, b) temperature (simulation minus observation, units: °C) and (c, d) precipitation [(simulation minus observation)/observation, units: %]. The black lines show the boundary of subregions.
Figure 8. Biases of the BMME, AMME, and WMME simulations for annual (ANN), winter (DJF), and summer (JJA) temperature (left panel, units: °C) and precipitation (right panel, units: %) in eight subregions of China. Boxes indicate the range of biases from the ensemble models and the black lines show the ensemble mean values. Note that the vertical scales are different.
For annual precipitation, the wet biases in the AMME simulation (Fig. 3g) decrease in the BMME simulation (Fig. 7c) and increase in the WMME simulation (Fig. 7d). When regionally averaged, the percentage-based wet biases over NWC, SWC1, NC, and NEC are 199%, 191%, 45%, and 28% respectively for the WMME simulation. These decrease to 136%, 147%, 40%, and 32% for the AMME simulation; the wet biases further reduce to 39%, 96%, 4%, and 23% in the BMME simulation, respectively (Fig. 8b). Similar results are obtained for the simulation of winter precipitation (Fig. 8d). Nevertheless, there is no improvement in the BMME simulation for summer precipitation over subregions except for EC, NWC, and NEC when compared to the AMME and WMME simulations, although the model spread is reduced.
In short, the BMME generally shows better performance than the AMME and WMME in reproducing the spatial patterns of annual and winter temperature and precipitation, particularly in subregions with complex terrain. Similarly, regardless of whether for annual, winter, or summer temperature (precipitation), the BMME presents the smallest IVS values, followed by the AMME and then by the WMME. The IVS values for annual, winter, and summer temperature (precipitation) over China are 0.1 (0.9), 0.2 (8.3), and 0.3 (1.0) from the BMME simulation, 0.2 (1.4), 0.3 (22.2), and 0.5 (1.0) from the AMME simulation, and 0.6 (2.4), 0.7 (43.4), and 1.1 (1.4) from the WMME simulation, respectively.
|ID||Model name||Institution and country||Atmospheric resolution (lon×lat: number of grids, L: vertical levels)|
|1||ACCESS-CM2||Commonwealth Scientific and Industrial Research Organization, Australian Research Council Centre of Excellence for Climate System Science, Australia||192×144, L85|
|2||ACCESS-ESM1-5||Commonwealth Scientific and Industrial Research Organization, Australia||192×145, L38|
|3||BCC-CSM2-MR||Beijing Climate Center, China||320×160, L46|
|4||CanESM5||Canadian Centre for Climate Modelling and Analysis, Canada||128 × 64, L49|
|5||CESM2||National Center for Climate Research, USA||288 × 192, L32|
|6||CESM2-WACCM||National Center for Climate Research, USA||288 × 192, L70|
|7||EC-Earth3||EC-Earth Consortium, Europe||512 × 256, L91|
|8||EC-Earth3-Veg||EC-Earth Consortium, Europe||512 × 256, L91|
|9||FGOALS-g3||Chinese Academy of Sciences, China||180 × 80, L26|
|10||GFDL-CM4||National Oceanic and Atmospheric Administration, Geophysical Fluid Dynamics Laboratory, USA||288 × 180, L33|
|11||GFDL-ESM4||National Oceanic and Atmospheric Administration, Geophysical Fluid Dynamics Laboratory, USA||288 × 180, L49|
|12||INM-CM4-8||Institute for Numerical Mathematics, Russia||180 × 120, L21|
|13||INM-CM5-0||Institute for Numerical Mathematics, Russia||180 × 120, L73|
|14||IPSL-CM6A-LR||Institute Pierre Simon Laplace, France||144 × 143, L79|
|15||MIROC6||Atmosphere and Ocean Research Institute, The University of Tokyo, Japan||256 × 128, L81|
|16||MPI-ESM1-2-HR||Max Planck Institute for Meteorology, Germany||384 × 192, L95|
|17||MPI-ESM1-2-LR||Max Planck Institute for Meteorology, Alfred Wegener Institute, Germany||192× 96, L47|
|18||MRI-ESM2-0||Meteorological Research Institute, Japan||320 ×160, L80|
|19||NorESM2-LM||NorESM Climate modeling Consortium, Norway||144 × 96, L32|
|20||NorESM2-MM||NorESM Climate modeling Consortium, Norway||288 × 192, L32|