How Well Do CMIP6 and CMIP5 Models Simulate the Climatological Seasonal Variations in Ocean Salinity?

Ocean salinity is the key variable used to track variations in the hydrological cycle in the Earth’s climate system. This is because >80% of evaporation and >75% of precipitation at the Earth’s surface occurs over the ocean (Schmitt, 2008; Durack et al., 2012, 2018; Cheng et al., 2020). Ocean salinity also reveals changes in river runoff, the formation and melting of sea ice, and the ocean circulation (Delcroix et al., 1996). Changes in salinity are therefore an important indicator of global climate change, and ocean salinity has been identified as an essential climate variable in the Global Climate Observation System.

Because the ocean circulation is partly driven by changes in the density of seawater (determined by the salinity and temperature), changes in salinity are also a driver for changes in ocean circulation (Roemmich et al., 1994; Lagerloef et al., 2010)—for example, the surface waters in the subpolar regions of the Atlantic become denser (colder and saltier) and sink to great depths, which maintains the Atlantic Meridional Overturning Circulation (AMOC). As a potential tipping element in the Earth’s climate system, monitoring and predicting changes in the AMOC has received much attention, and one of the key tasks is to correctly simulate the temperature and salinity states in climate or earth system models (Delworth et al., 1993, 2012; Danabasoglu et al., 2012; Kwon and Frankignoul, 2012; Liu et al., 2020).

Simulating salinity is challenging, although substantial improvements have been made in global models over recent decades. Since a standard experimental protocol and infrastructure were provided in the Coupled Model Intercomparison Project (CMIP), scientists have been able to systematically analyze model simulations to promote model developments and study climate change (Taylor et al., 2012; Eyring et al., 2016). However, despite the importance of ocean salinity in the Earth’s climate system, few studies have comprehensively evaluated the performance of ocean salinity simulations (especially below the sea surface) in the CMIP phase 5 (CMIP5) and phase 6 (CMIP6) models (Durack et al., 2014; Bindoff et al., 2019; Eyring et al., 2021). Few studies have compared the CMIP5 and CMIP6 models, particularly the climatological state and seasonal variations.

The Intergovernmental Panel on Climate Change Assessment Report 5 (IPCC-AR5) (IPCC, 2013), the IPCC Special Report on the Ocean and Cryosphere in a Changing Climate (Bindoff et al., 2019), and the IPCC-AR6 (Eyring et al., 2021) provided several analyses of model performance in the salinity field—for example, the model bias of the zonal mean salinity of the multimodel mean (MMM). Previous IPCC assessments have shown a reasonable agreement of the basin-scale patterns of salinity between models and observations, but have also shown some biases. The IPCC-AR5 identified a negative salinity bias near the surface and a positive bias in the subsurface within 15°–90°N. The IPCC-AR5 speculated that most of the salinity errors result from surface flux errors in the upper ocean and that the errors at intermediate depths are caused by the formation of water masses (Flato et al., 2013). The IPCC-AR6 provided a brief assessment of the model bias of the climatological zonal mean 0–5000-m salinity and the spatial distribution of the sea surface salinity (SSS) in the CMIP6 models (Eyring et al., 2021, Fig. 3.25), with a strong upper ocean (<300 m) negative salinity (fresh) bias and a tendency toward a positive salinity (salty) bias below 300 m in the Northern Hemisphere for the CMIP6-MMM.

Some regional evaluations are available. For example, in the Indian Ocean, the CMIP5-MMM displays an annual and seasonal salinity bias in three regions: the western Indian Ocean (negative bias); the Bay of Bengal (positive bias); and the southeastern Indian Ocean (negative bias) (Fathrio et al., 2017). These biases are attributed to the biases in both precipitation and ocean dynamics. In the Arctic Ocean, Khosravi et al. (2021) showed that the simulated Atlantic water is too deep and thick (i.e., spanning greater water volume) in the CMIP5/6-MMM and concluded that there was no clear improvement for the CMIP6-MMM compared with the CMIP5-MMM. An analysis of the salinity budget in the tropical Pacific suggests that model biases in surface forcing are the main factors contributing to the uncertainty in salinity (Lin, 2007; Zhang and Busalacchi, 2009). All these regional studies provide useful insights.

This study was designed to fill the gap in previous research by comprehensively evaluating the performance of the CMIP5 and CMIP6 models in simulating the climatological states and seasonal variation in ocean salinity from global to regional scales, which will support further studies of model diagnostics, our understanding of mechanisms, and model development. The comparison is also useful to test whether the most recent generation of climate models has improved the simulation of ocean salinity and to gain insights into the evolution of model performance. To assess the performance of the CMIP5 and CMIP6 models in simulating the climatological state and seasonal variation of ocean salinity, we used new salinity data from the surface to 2000-m depth, to provide a more complete view of the simulation of ocean salinity. The data and methods are introduced in section 2. Our results are provided in section 3, which shows the annual mean and seasonal variation of the SSS and the vertical integrated salinity for both latitude–longitude and latitude–depth sections. Section 4 presents our summary and conclusions.

Our evaluations are based on 40 models from CMIP5 (Taylor et al., 2012; https://www.wcrp-climate.org/wgcm-cmip/wgcm-cmip5) and 33 models from CMIP6 (Eyring et al., 2016; https://www.wcrp-climate.org/wgcm-cmip/wgcm-cmip6). Tables 1 and 2 list the model names and basic information for CMIP5 and CMIP6, respectively. Our study focused on the time period 2005–17 for consistency with the best available observations; the Argo network started to achieve near-global coverage around 2005, but much of the salinity data suffers from data bias after about 2017 (Roemmich et al., 2019). Liu et al. (2021) showed that using other time periods did not significantly change the climatological seasonal variation. We used Representative Concentration Pathway 4.5 (RCP 4.5) data for the CMIP5 models because most of the historical simulations ended at 2005 (Meehl et al., 2007; Van Vuuren et al., 2011). Using different RCPs did not change the results of this study because the projections under different scenarios are very similar within the time period 2005–17. We used the Shared Socioeconomic Pathway 2-4.5 (SSP2-4.5) projections after 2014 because the historical simulations in most of the CMIP6 models ended in 2014. Again, the difference between scenarios was negligible for the time period 2014–17. Furthermore, because different forcings being used for different time periods could potentially impact the analyses, we have replicated our results using another time period (1980–2000) to test the impact of time window choice. The results [shown in the electronic supplementary material (ESM)] reveal a very minor impact, because the long-term trends of the salinity changes are much smaller than its climatological state and seasonal variation (see Cheng et al., 2020 for long-term changes).

The MMMs of the models were calculated from the altimetric mean of all available model results, consistent with IPCC-AR6 (Eyring et al., 2021). The standard deviation was used to define the model spread, corresponding to the inter-model 33rd and 66th percentiles defined in the IPCC reports (IPCC, 2021). As the total number of models is different in CMIP5 (40 models) and CMIP6 (33 models), which could affect the calculation, we included several tests using models (a total of 20) from the same organizations (Table S1 in the ESM) for both CMIP5 and CMIP6. The results indicated that the difference in the number of models did not affect the conclusions of this study (See the ESM).

We used five widely available observational datasets of salinity as the reference for the model evaluations. The primary dataset used was an ocean objective analysis from the Institute of Atmospheric Physics (IAP). These data are available with a horizontal resolution of (1° × 1°) and 41 vertical levels for the upper 0–2000 m. The key advantage of these data is the application of an advanced and carefully evaluated gap-filling algorithm that reduces the bias due to imperfect sampling (Cheng et al., 2020).

Four other products were also used. Two of the products used Argo data combined with all other available instruments (e.g., data from gliders, conductivity temperature depth measurements, bottles, moorings, and marine mammal observations): the EN4 ocean objective analysis product (EN4.2.1) from the UK Met Office Hadley Centre (Good et al., 2013) and data from the Japan Meteorological Administration (Ishii et al., 2017; referred to as ISHII hereafter). These data were collectively grouped to give the All-data product.

Two Argo-only salinity gridded products were also used: one from the Scripps Institution of Oceanography (SCRIPPS) (Roemmich and Gilson, 2009) and the Barnes objective analysis (BOA) Argo product (Li et al., 2017). Two Argo-only products were used because the Argo products are very similar to each other as a result of being based on the same data source and having similar gap-filling methodologies. The comparison between the All-data products and the Argo-only products gives an insight into the capability of the observational system (Liu et al., 2021). A comprehensive comparison of different salinity products for SSS can be found in Yu et al. (2021).

To facilitate intercomparison and validation against the observations, all the monthly fields of salinity were re-gridded to (1.0° × 1.0°) grids using the nearest neighbor method, consistent with the horizonal resolution of the observational datasets. All the data were also interpolated to 41 standard levels from the surface to 2000 m, consistent with the observational salinity products from the IAP. For all the data, gridded monthly salinity fields from 2005 to 2017 were averaged to construct the 12-month climatological means, which were then used to analyze the seasonal variation. A three-month running smoother was applied to reduce the month-to-month data noise; a nine-point smoother was used to smooth the spatial fields to better illustrate the large-scale patterns.

Two observational datasets of evaporations and precipitation are used for surface freshwater flux (E–P) analyses, which helps to interpret the salinity bias. The first dataset of precipitation and evaporation data is from the European Centre for Medium-Range Weather Forecasts Interim Reanalysis (ERA-Interim) product (Berrisford et al., 2011; Dee et al., 2011). The second E–P dataset is a combination of the precipitation from Global Precipitation Climatology Project (GPCP) Version 2.3 (Adler et al., 2018) and evaporation from Objectively Analyzed air–sea Flux (OAFlux) Version 3 (Yu and Weller, 2007). The resolution is a 1° horizontal grid, and the time span is the same as the salinity datasets (2005–17).

Figure 1 shows the geographical distribution of the annual mean SSS for the observations and the CMIP5 and CMIP6 MMMs. The basin-scale patterns of salinity distributions for both CMIP5 and CMIP6 are consistent with the observations (Fig. 1), including several “salty pools” (salinity >35 g kg⁻¹) located in the subtropics of all three basins and “fresh pools” in the tropics, coastal regions, and high-latitude regions. This pattern of ocean salinity is consistent with the distribution of the sea surface freshwater flux (evaporation minus precipitation or E−P) (Fig. 2), and therefore, the salinity field shows the oceanic footprint of the global water cycle (Schmitt, 2008; Durack et al., 2012, 2018; Yu et al., 2020). The pattern correlations between individual CMIP6 models and observation ensemble mean are mostly >0.9, larger than CMIP5 models (mostly between 0.85–0.9), indicating an improvement in CMIP6 models for SSS simulation.

Figure 1.  (a) Global climatology of the SSS and (b) spread across the five observational products. (c) MMM and (d) spread (defined as the multimodel standard deviation) for CMIP5. (e) MMM and (f) spread of CMIP6 models. The model bias between the MMM and the observational mean for the (g) CMIP5 and (h) CMIP6 models, respectively. The unit of salinity is g kg⁻¹. Data for the time period 2005–17 are used.

Figure 2.  (a) Global climatology of the E–P and (b) spread across the two observational products. (c) MMM and (d) spread (defined as the multimodel standard deviation) for CMIP5. (e) MMM and (f) spread of CMIP6 models. The model bias between the MMM and the observational mean for the (g) CMIP5 and (h) CMIP6 models, respectively. The unit is mm d⁻¹. Data for the time period 2005–2017 are used.

Figure 3.  (a) Global climatology of the S1000 (0–1000-m averaged salinity) and (b) spread across five observational products. (c) MMM and (d) spread (defined as the multimodel standard deviation) for CMIP5. (e) MMM and (f) spread of CMIP6 models. The model bias between the MMM and the observational mean for the (g) CMIP5 and (h) CMIP6 models, respectively. The unit of salinity is g kg⁻¹. Data for the time period 2005–17 are used.

Although the individual models differ in the pattern of climatological SSS (Figs. S1 and S2 in the ESM), we can identify some common model biases in the MMM. In the Pacific Ocean, the MMM shows fresh biases in the subtropical regions and positive errors along the equator for both the CMIP5-MMM and CMIP6-MMM (Figs. 1g, 1h), coincident with the well-established rainfall bias patterns (Eyring et al., 2021, Fig. 3.13) and E–P bias patterns (Fig. 2). In particular, climate models always show a double intertropical convergence zone (ITCZ) in the tropical Pacific, leading to excessive rainfall on both sides of the equatorial Pacific (Tian and Dong, 2020; Zhou et al., 2020). The CMIP5-MMM bias is less than −0.6 g kg⁻¹ at about 5°–20°S, and the bias is stronger and more widely distributed in the CMIP6-MMM (spreading at 5°–20°S and 5°–20°N).

The Indian Ocean shows a dipole pattern of salinity from west (saltier) to east (fresher). The higher salinity in the Arabian Sea is linked to both stronger evaporation in this region and the transport of saltier waters from the Red Sea. The higher precipitation and strong river runoff in the Bay of Bengal are responsible for its lower salinity. Both the CMIP5 and CMIP6 models show a fresher bias in the west and a saltier bias in the east (i.e., the climatological west–east salinity contrast is weaker in the models), generally consistent with E–P bias distributions. Near the equator regions, the SSS/S pattern (Figs. 1g, 1h; 3g, 3h) is different from the E–P bias (Figs. 2g, 2h), suggesting that the ocean dynamics plays a more important role. Fathrio et al. (2017) indicated that the model biases in the Indian Ocean are primarily linked to biases in the E−P (especially P) field, with ocean circulation biases having a secondary role. The spread of models is very large in the Bay of Bengal in both the CMIP5 and CMIP6 models, probably linked to the poor representation of river runoff in the models because the Bay of Bengal receives large amounts of freshwater through the Ganges, Brahmaputra, and Irrawaddy rivers.

A large fresh bias of SSS occurs in most of the Atlantic Ocean (except the northern Gulf of Mexico and the gulf extension regions) for the CMIP5-MMM and CMIP6-MMM and most of the models. This bias has been identified in previous studies (e.g., Eyring et al., 2021) and has important implications for the simulation of the AMOC because the models might be too stable with a smaller near-surface salinity (Liu et al., 2020).

The bias shows a relatively large spread across both the observational datasets (Fig. 1b) and models (Figs. 1d, 1f) in coastal regions, mainly because the Argo network is in the open ocean and the Argo-only products do not represent coastal conditions. The large spread of salinity among the observational datasets in the Arctic Ocean is probably due to the lack of in situ observations, and the changes in Arctic salinity are subject to large uncertainties (Khosravi et al., 2021).

For the global average, both the CMIP5-MMM and CMIP6-MMM show a fresher ocean at the sea surface than the observations. This negative salinity bias is more severe in the CMIP6-MMM than in the CMIP5-MMM (Figs. 1g, 1h); the global mean bias is −0.44 g kg⁻¹ for the CMIP6-MMM and −0.25 g kg⁻¹ for the CMIP5-MMM. This indicates a notable increase in the model bias for CMIP6, implying that the simulation of hydrological processes is still challenging in climate simulations. The increased negative model salinity bias may be attributed to stronger negative E–P bias in CMIP6 models (Figs. 2g, 2h). Previous studies have speculated that the absence of direct feedback between ocean salinity and the atmosphere is one reason why model simulations are not well-constrained (Durack et al., 2012). Another feature of the CMIP6 models is a slightly reduced model spread compared with the CMIP5 models (Fig. 1f versus Fig. 1d; Table 3) in the Southern Hemisphere and North Atlantic and South Pacific oceans. This is consistent with reduced E–P spread for the CMIP6 models (Figs. 2d, 2f). The global average model spread (the standard deviation across models) is 0.56 g kg⁻¹ for CMIP5 and 0.53 g kg⁻¹ for CMIP6.

Because the different number of models in CMIP5 (40 models) and CMIP6 (33 models) could affect the results, we selected 20 models from the same institutes from CMIP5 and CMIP6 and re-calculated the MMM and spread. The results were generally consistent with those using all the models—for example, the global mean SSS bias is −0.40 g kg⁻¹ for 20 common CMIP6 models (with a spread of 0.48 g kg⁻¹) and −0.21 g kg⁻¹ for 20 common CMIP5 models (with a spread of 0.54 g kg⁻¹). This test indicates that our findings are robust (Fig. S3 in the ESM). Using a climatology from another time period (i.e., 1980−2000 in Fig. S4 in the ESM) does not impact the conclusions shown above. But the observation spread is slightly increased for the 1980–2000 climatology compared with the 2005–17 climatology (especially in the open ocean), indicating the impact of improved data quality in the recent decades.

Figure 3 compares the spatial pattern of the 0–1000-m average salinity (S1000) for CMIP5 and CMIP6 with the observations. We selected the 0–1000-m layer because it includes most of the upper ocean water masses and the uplift of deep ocean water masses. The apparent salinity contrast between the salty Atlantic Ocean and the fresh Pacific/Indian oceans is robust in all models (Figs. 3c, 3e; individual models presented in Figs. S5 and S6 in ESM). The spatial pattern of S1000 is generally well-simulated by the models. The pattern correlations between individual CMIP6 models and the observation ensemble mean are mostly >0.9, larger than CMIP5 models (mostly between 0.85–0.9), indicating an improvement in CMIP6 models for S1000.

Figure 4.  Zonal mean and spread (defined as the multimodel standard deviation) of the vertical salinity distributions for the world ocean in the (a, b) observations, (c, d) CMIP5 models, and (e, f) CMIP6 models. The model bias between the MMM and the observational mean for the (g) CMIP5 models (h) CMIP6 models. The unit of salinity is g kg⁻¹. Data for the time period 2005–17 are used. Note that the depth scale of the zonally averaged plot is not linear.

Figure 5.  Zonal mean salinity (SSS) for the world ocean in (a) the observations, (b) the CMIP5 models, and (c) the CMIP6 models.

Figure 6.  Zonal mean salinity (S1000 m) for the world ocean in (a) the observations, (b) the CMIP5 models, and (c) the CMIP6 models.

However, the fresh bias of the CMIP6-MMM and CMIP5-MMM seen in the SSS is more spatially consistent and more severe for S1000 (Figs. 3g, 3h versus Figs. 1g, 1h) in the Pacific, Indian, and South Atlantic oceans, indicating the persistence of the negative bias in the ocean subsurface. The global mean S1000 bias is −0.26 g kg⁻¹ for the CMIP6-MMM and −0.07 g kg⁻¹ for the CMIP5-MMM (Table 3). Using the 20 models from the same organization gives similar results (−0.23 g kg⁻¹ for the CMIP6-MMM and −0.03 g kg⁻¹ for the CMIP5-MMM). The CMIP6-MMM bias for subsurface salinity (S1000) is therefore much stronger than the CMIP5-MMM (Figs. 3g, 3h).

Both the CMIP6-MMM and CMIP5-MMM show a pronounced positive bias for S1000 in the North Atlantic Ocean. The area with a positive bias is smaller in the CMIP6-MMM than in the CMIP5-MMM (0°–80°N for the CMIP5-MMM and 0°–40°N for the CMIP6-MMM), but the area with a bias >0.3 g kg⁻¹ extends more widely from the Caribbean Sea into the Strait of Gibraltar for the CMIP6-MMM than for the CMIP5-MMM. The Atlantic salinity bias patterns are less consistent with E–P biases (Figs. 3g, 3h versus Figs. 2g, 2h), indicating ocean dynamics (and ice changes in the polar regions) plays a more important role. Further studies are needed to disclose the culprits with a rigorous salinity budget analysis.

The model spread of S1000 is decreased at many locations in the CMIP6 models compared with the CMIP5 models (Figs. 3d, 3f), although the global average model spread is only marginally improved from 0.25 g kg⁻¹ for CMIP5 to 0.22 g kg⁻¹ for CMIP6. The reduction is also shown in the subset of 20 models from the same organizations (Fig. S7 in the ESM), with a global averaged spread of 0.24 g kg⁻¹ for CMIP5 and 0.20 g kg⁻¹ for CMIP6. Using a different climatology (1980–2000) does not significantly change the results (Fig. S8 in the ESM). Although the model spread is decreased for CMIP6 models in general, the improvements in many places are marginal (i.e., in the North Pacific Ocean and Southern Ocean), suggesting that the physical processes represented by the models still need to be improved. Note that the observational spread for S1000 (Fig. 3b) is smaller than SSS (Fig. 1b), mainly because the SSS is strongly impacted by river runoff, ice, and surface freshwater changes, which has more noise than a vertical integrated quantity (Cheng et al., 2020).

Figure 4 shows the vertical structure of the zonal mean salinity from the surface to 2000 m in an attempt to better link the surface salinity (section 3.1) with the vertical average salinity, which mainly reveals subsurface changes (section 3.2), and to understand the vertical distribution of the model error. The “salty pools” at midlatitudes in both hemispheres extend from the surface down to about 400 m in the observations, following the pathway of the subtropical gyre (Fig. 4a). In both the CMIP5-MMM and the CMIP6-MMM, the “salty pools” are much fresher and shallower (if indexed by the 35 g kg⁻¹ contour). This model bias for freshwater extends to least 350 m in both hemispheres (Figs. 4g, 4h).

The tropical fresh pool is broader and deeper in the models than in the observations (Figs. 4c, 4e versus Figs. 4a), consistent with strong negative E–P biases outside of the equator (Fig. 2). Notably, there is only one center of salinity minimum in the tropical fresh pool in the observations, located within the region 5°–10°N in the upper 60 m, corresponding to the location of the ITCZ. By contrast, there are two minima in the models located on both sides of the equator and extending to about 100 m, corresponding to a double ITCZ bias in the model simulation of the rainfall belts (Zhou et al., 2020). The surface freshwater flux bias is probably responsible for this salinity bias because the vertical structure of salinity is largely set by the surface freshwater flux. A further analysis of the impact of ocean dynamics should be carried out, similar to Yu (2011) for mixed-layer salinity, but is beyond the scope of this study.

The ventilation of high-latitude (<40°S, >40°N) freshwater can be seen in both the CMIP5 and CMIP6 models (Figs. 4c, 4e versus Fig. 4a), suggesting that the models can describe the process of formation of cold water and freshwater (i.e., the North Pacific Intermediate Water and the Antarctic Intermediate Water) at high latitudes, followed by subduction into the subsurface (Curry et al., 2003; Talley, 2008). However, the ventilation of freshwater seems to be too strong in the models and becomes even stronger in the CMIP6-MMM, indicating the bias in the formation of water masses in the models (Sallée et al., 2013). Because surface freshwater fluxes are key to the water mass formation, these biases can be largely traced back to the negative biases in E–P in both CMIP5 and CMIP6 models within 40°–50°N and 50°–40°S (Fig. 2). There is a small positive bias in the CMIP5-MMM in the deep layers at 400–2000 m, but this changes to a negative bias in the CMIP6-MMM (except in the Mediterranean Sea, where it shows a positive bias). The origin of such bias has not been discovered.

The CMIP6 model spread decreases from that of CMIP5 in almost every zonal band (Figs. 4d, 4f). One exception is located in the upper 400 m within 35°–65°N, where the model spread is larger than at other locations. This is probably influenced by changes in the ice cover in polar regions and river runoff in coastal regions. The observational products also show a large spread, mainly due to the difference between the All-data and the Argo-only data. This is caused by the insufficiency of the Argo observing network, which does not sample the coastal and polar oceans (Liu et al., 2021).

The zonal mean SSS (S1000) for each individual model and the observational products are presented in Fig. 5 (Fig. 6), which shows the performance of individual models compared with the observations. The observational products have a large spread north of 40°N, mainly as a result of the difference between the Argo-only (SCRIPPS and BOA) and the All-data (IAP, ISHII, and EN4) products, as shown in Liu et al. (2021). Because the Argo-only data do not represent the salinities in the coastal and polar regions, the other three products (IAP, ISHII, and EN4) are averaged and compared with the models in Figs. 5 and 6.

Both the CMIP5 and CMIP6 models show similar zonal fluctuations in salinity compared with the observational products (Figs. 5 and 6). The zonal mean SSS shows a salinity <35 g kg⁻¹ in the tropics (20°S–20°N) and at high latitudes (<40°S, >40°N), and a higher salinity in the subtropics (20°–40°N, 20°–40°S) (Fig. 5), consistent with the zonal–depth section in Fig. 4. By contrast, the S1000 salinity is lower in the Southern Hemisphere (34.5–35 g kg⁻¹) than in the Northern Hemisphere (34.5–35.6 g kg⁻¹), indicating the impact of the northward extension of fresh, intermediate waters. Almost all the CMIP6 models have a common fresh bias for the SSS and S1000 within the region 30°S–30°N, whereas several CMIP5 models show a positive bias (e.g., HadCM3, GISS-E2-H, and CESM1-WACCM) (Figs. 5b and 6b). North of 30°N, FIO-ESM has the largest negative SSS bias (>2 g kg⁻¹) in the CMIP5 models (Fig. 5b), whereas E3SM-1-0 and HadGEM3 have the strongest SSS bias (>2 g kg⁻¹) in the CMIP6 group (Fig. 5c). INM-CM4-8 and INM-CM5-0 show large spikes (>1 g kg⁻¹) at some latitudes (about 65°N) for S1000, implying dramatic regional biases.

We assessed the seasonal variation of salinity by examining the zonally averaged SSS and S1000 relative to the annual mean (Figs. 7 and 8). Within the region 5°–20°N (20°S–5°N), the sea surface is saltier (fresher) in the first half of the year and becomes fresher (saltier) in the second half of the year (Fig. 7a). Liu et al. (2021) indicated that this variation in SSS is mainly driven by the seasonal changes in the surface freshwater fluxes (E−P) associated with the seasonal shift in the ITCZ. The minimum (less than −0.1 g kg⁻¹) of the SSS anomalies is located at about 8°N in October in the Northern Hemisphere and at about 3°S in April in the Southern Hemisphere, corresponding to the location of the strongest ITCZ (Bingham et al., 2010, 2012; Yu et al., 2021). Both the CMIP5-MMM and the CMIP6-MMM show a similar seasonal evolution of the SSS within 20°S–20°N compared with the observations (Figs. 7c, 7e versus Fig. 7a). However, the magnitude of the seasonal variation in the models (from −0.2 g kg⁻¹ to 0.2 g kg⁻¹ for the CMIP5-MMM; Figs. 7c, 7e) tends to be stronger than that in the observations (about 0.1 g kg⁻¹; Fig. 7a). The CMIP6-MMM is closer to the observations than the CMIP5-MMM within the region 20°S–20°N, but the spread becomes larger (Figs. 7d, f), indicating increasing model divergence in simulating the seasonal variation of the SSS in the recent generation of climate models.

Figure 7.  Zonal mean seasonal variation and spread of the SSS in the (a, b) observations, (c, d) CMIP5 models, and (e, f) CMIP6 models for the world ocean. The unit of salinity is g kg⁻¹. The bias is calculated as the zonal mean salinity difference between CMIP ensemble mean and observation ensemble mean at each month. The anomalies are relative to the annual mean.

Figure 8.  Zonal mean seasonal variation and spread of the S1000 m in the (a, b) observations, (c, d) CMIP5 models, and (e, f) CMIP6 models for the world ocean. The unit of salinity is g kg⁻¹. The anomalies are relative to the annual mean.

Outside the tropics (60°–20°S and 20°–60°N), the models show a similar seasonal variation (from positive anomalies before June to negative anomalies after July) to the observations (Figs. 7c, 7e versus Fig. 7a). The magnitude of the variation tends to be smaller in the northern polar regions and larger in the Southern Hemisphere. In almost all zonal bands (except the midlatitudes at about 40°S and 30°N), the CMIP6 model spreads are larger than those of the CMIP5 models. In the subpolar regions of the Northern Hemisphere (>50°N), the spread of the observations is compatible with the model spread, mainly as a result of the differences in the Argo-only and All-data products and the lack of observations.

The observational variation does not show a clear large-scale pattern for S1000 (Fig. 8a), but shows many zonal bands and fine structures. However, it is difficult to determine these structures as a result of the relatively large spread in the observations (Fig. 8b). The S1000 variation in the northern polar regions (>60°N) is broadly consistent with that of the SSS (Fig. 8a versus Fig. 7a), which is probably associated with the deep-reaching ocean circulation that conveys surface anomalies downward. This is supported by a previous study of Bingham et al. (2012), who suggests that ocean dynamics are more important in high-latitude regions than E–P for SSS seasonal variation. In contrast with the observations, both the CMIP5-MMM and CMIP6-MMM show a well-organized, large-scale pattern for S1000—that is, a transfer from positive (February–July) to negative (August–January) anomalies within 40°S–5°N and a change from negative (March–August) to positive (September–February) anomalies within 10°–30°N (Figs. 8c, 8e versus Fig. 8a). However, the spread in the observational datasets is as large as that in the models (Fig. 8b versus Figs. 8d, 8f), and there is low confidence for both the observational and modeled seasonal variation in S1000. This is surprising and suggests a limited modeling and observing capability for the variation in the subsurface salinity at the seasonal scale.

Figures 9 and 10 show the geographical distributions of the magnitude of the seasonal variation in salinity (defined as the difference between the minimum and maximum of the seasonal change in each grid cell) for SSS and S1000, respectively. The magnitude of the variation indicates the strength of the changes in salinity at the regional scale and has important implications for the role of salinity in the regional climate (Boyer and Levitus, 2002; Liu et al., 2021) because the local change in salinity could regulate the ocean circulation. The observations show a higher magnitude of changes in SSS (>0.6 g kg⁻¹, the 95th percentile of the regional changes in SSS) near the ITCZ regions, mainly in the northern non-equatorial oceans (Bingham et al., 2010, 2012; Liu et al., 2021). However, both the CMIP5-MMM and CMIP6-MMM models show a more symmetrical distribution of the SSS maximum bands on both sides of the equator, again indicating the typical double ITCZ bias in climate models.

Figure 9.  Magnitude of the seasonal variation of the SSS in (a) the observations, (c) the CMIP5 models, and (e) the CMIP6 models for the world ocean. The spread across observational products (b) and models (d, f) is also presented. The unit of salinity is g kg⁻¹.

Figure 10.  Magnitude of the seasonal variation of the S1000 m in (a) the observations, (c) the CMIP5 models, and (e) the CMIP6 models for the world ocean. The spread across observational products (b) and models (d, f) is also presented. The unit of salinity is g kg⁻¹.

The magnitude of the variations is also stronger in the models than in the observations within 20°S–20°N, probably due to the stronger rainfall modeled in the tropics compared to what is found in the observations (Eyring et al., 2021). The stronger surface fluxes penetrate downward and are probably responsible for the larger magnitude of variation in S1000 in Fig. 10 in the tropical Pacific (for both the CMIP5-MMM and CMIP6-MMM). Notably, the CMIP6 model spread of S1000 has increased dramatically from the CMIP5 models in the Pacific basin, leading to poorer results for CMIP6 compared to those for CMIP5 when compared with the observations in the tropics; but CMIP6 results are closer to the observations in the northwest Pacific. By contrast, the models show a weaker magnitude of variation than the observations in the Atlantic and Indian oceans, although the CMIP6-MMM is improved compared with the CMIP5-MMM, especially in the northwest Indian Ocean and the western Atlantic Ocean.

In summary, we found that the new generation climate models (CMIP6) only show marginal improvements in simulating the seasonal variation of SSS and S1000 in some specific places. The persistent model bias in the magnitude of the seasonal variation in both the CMIP5 and CMIP6 models deserves careful analysis in the future.

This section investigates the seasonal variation of the global-averaged SSS and S1000, which are associated with the global-scale ocean freshwater budget. Based on the global average, the SSS is higher from November to May and lower from June to October, showing a robust seasonal variation (Fig. 11a) (Liu et al., 2021). This seasonal variation is caused by both E−P and river runoff. The two Argo-only products do not show a complete global average and were therefore not considered in the following analyses. Most of the models simulate this seasonal variation well. The CMIP6-MMM shows much better consistency with the observational average than the CMIP5-MMM (Figs. 11b, 11c), indicating an improvement in this global metric. However, because there are viable and persistent regional biases, this slightly better agreement between the CMIP6-MMM and the observations should not be over-emphasized. Despite the general consistency between the models and observations, some individual models show almost the opposite variation (e.g., the BNU-ESM, CESM1-WACCM, and GFDL-CM3 models in the CMIP5 family). Some models in the CMIP6 group show different phase changes (e.g., the INM-CM4-8 and INM-CM5-0 models).

Figure 11.  Global mean SSS in (a) the observations, (b) the CMIP5 models, and (c) the CMIP6 models. The anomalies are relative to the annual average. The MMM is shown as the dashed black line. The red dashed line is the mean of IAP, ISHII, and EN4.

Furthermore, we provided global SSS changes within 60°S–60°N to reduce the impact of polar regions (Fig. 12), which are less accurately represented in both observational datasets and models (Figs. 1–3). Within 60°S–60°N, the modeled global mean SSS seasonal variation becomes more consistent with observations. And the five observational products also show consistent variations, similar to the findings of Liu et al. (2021). This suggests that the polar regions play a key role in setting the seasonal variation of global SSS.

Figure 12.  Same as Fig. 11 but for SSS change within 60ºS–60ºN.

It is surprising that the observational changes are not consistent among the five data products for the global mean S1000 (Fig. 13a). The EN4 and IAP models show a positive peak in June and a negative peak in January, but the ISHII model shows a positive peak in August. The SCRIPPS and BOA models are similar to each other, showing a weaker magnitude of variation than the other three All-data products and a change from positive anomalies in the first half of the year to negative anomalies in the second half of the year. These notable differences indicate that the current ocean subsurface observational system is still insufficient to robustly represent the global seasonal change in salinity. In contrast with the observations, most of the CMIP5 and CMIP6 models show a change from positive anomalies in the first half of the year (January–July) to negative anomalies in the second half of the year (August–December), consistent with the SCRIPPS and BOA data. However, the magnitude of the variation in the models is much stronger than in the SCRIPPS and BOA data. Finally, averaged S1000 within 60^oS–60^oN is calculated, and the results (Fig. S9 in ESM) are generally consistent with the global average (Fig. 13), implying that the subsurface changes in the open ocean dominate the global averages for the 0–1000-m average.

Figure 13.  Global mean S1000 m in (a) the observations, (b) the CMIP5 models, and (c) the CMIP6 models. The anomalies are relative to the annual average. The MMM is shown as the dashed black line. The red dashed line is the mean of IAP, ISHII, and EN4.

In summary, our assessment of the global mean salinity indicates a consistent representation of the SSS for the models and observations, but suggests poor constraint of the variation in subsurface salinity.

We evaluated the CMIP5 and CMIP6 models for the climatological seasonal variation in salinity and compared them with five observational products. Based on this investigation, our conclusions are as follows.

(1) Although the basic features of ocean salinity can be well-simulated by the CMIP5 and CMIP6 models, there are notable systematic errors. The common model biases include: (i) the fresh biases in the subtropical and polar Pacific and Atlantic oceans extending from the surface to at least 400 m, which are probably associated with the biases in surface flux; (ii) the global mean fresh biases for SSS and S1000; (iii) the basin-scale fresh bias for the subsurface salinity (S1000) in the Pacific, Indian, and North Atlantic oceans, but the positive bias in the North Atlantic Ocean; (iv) the magnitude of seasonal variations in salinity is stronger in the models than in the observations in the tropics for SSS in all three basins—by contrast, this error appears only in the Pacific Ocean for S1000. The Atlantic and Indian oceans show a weaker magnitude of variation for S1000.

(2) For the climatological mean salinity state, the model spread is decreased across the CMIP6 models compared with the CMIP5 models in many locations. However, for the seasonal variation, the model spread of the CMIP6 models is larger than that of the CMIP5 models in many parts of the global ocean (except in some locations in the southern oceans).

(3) Our study is consistent with the IPCC-AR6 and IPCC-AR5 for the zonal mean salinity at different depths. In particular, we confirm the IPCC-AR6 conclusion that “the structure of the salinity biases in the multi-model mean has not changed substantially between CMIP5 and CMIP6 (medium confidence), though there is limited evidence that the magnitude of subsurface biases has been reduced”.

(4) As a result of the substantial uncertainty in the observational datasets for the global mean 0–1000-m seasonal change in salinity, the models cannot be fully evaluated. This study therefore stresses that the global ocean observation system for salinity should be extended and improved in the future.

Most of the climate models cannot represent the melting/formation of ice sheets and have large errors in simulations of the changes in sea ice (e.g., in the Southern Ocean). These processes might contribute to the difference in the comparison between the observations and models. A detailed diagnostic study of the model biases is required in the future to fully understand the models.

Analyzing the model performance of salinity at all time scales, including inter-annual and decadal scales, helps in both fundamental research and for the creation of accurate predictions. For example, SSS in the Indo-Pacific sectors is known to experience large variations associated with ENSO and other climate variability that alter density stratification in the upper ocean (Zhu et al., 2014; Zhang et al., 2016; Kido et al., 2019), but such SSS variations may not be adequately represented in CMIP5 and CMIP6 models due to biases in climatological salinity.

Our analysis of salinity provides a useful base to understand the simulation of the global energy cycle and the water cycle, because these cycles are closely coupled (Trenberth et al., 2007; Dai et al., 2009; Bronselaer and Zanna, 2020). For instance, the air–sea freshwater and heat fluxes are coupled via evaporation and precipitation, and the air–sea freshwater flux bias is associated with both the salinity bias and the ocean temperature (heat content) bias. It would be an intriguing follow-on study to investigate the coupling of the water and heat cycles and to understand the formation of model bias in ocean temperatures and salinity.

Acknowledgements. This study was supported by the National Natural Science Foundation of China (Grant No. 42076202) and the Strategic Priority Research Program of the Chinese Academy of Sciences (XDB42040402).

Data availability. The salinity data produced in this study are available at http://www.ocean.iap.ac.cn/. The data are available at: ISHII (https://climate.mri-jma.go.jp/pub/ocean/ts/v7.2/); EN4 (http:// www.metoffice.gov.uk/hadobs/en4/index.html); SCRIPPS (http://sio-argo.ucsd.edu/RG_Climatology.html); and BOA (www.argo.ucsd.edu/Gridded_fields.html).

Electronic supplementary material: Supplementary material is available in the online version of this article at https://doi.org/10.1007/s00376-022-1381-2.