Cloud Top Pressure Retrieval Using Polarized and Oxygen A-band Measurements from GF5 and PARASOL Satellites

Clouds cover 60% to 70% of the Earth's surface and significantly impact both the atmospheric radiation budget and climate change (Wang et al., 2022). The radiative properties of clouds and their height affect the overall radiation balance and can lead to Earth’s warming or cooling (Stephens and Webster, 1981; Letu et al., 2019, 2022, 2023). Cloud top pressure (CTP, hPa) is a cloud macroscopic physical parameter that is essential for describing the physical properties of clouds and retrieving other cloud properties (Nakajima et al., 2019; Ri et al., 2022; Li et al., 2022). Under certain conditions, CTP can be converted to cloud top height (CTH, km) and cloud top temperature (CTT, K) based on atmospheric profiles, and these parameters are collectively referred to as cloud top parameters. Usually, cloud-top properties play a significant role in the observation or investigation of extreme weather events, and accurately measuring CTP is especially important for monitoring and early warning of strong convective weather as well as gathering forecast data in the aviation and navigation industries (Van et al 2004).

Passive satellite remote sensing enables sophisticated CTP retrieval, categorized by available datasets and attributes. GOES-12 uses the infrared window method, deriving CTH from satellite-measured radiance using Planck's equation for cloud top temperature, and then comparing the result to atmospheric temperature profiles (Rossow et al., 1989). Assuming clouds are blackbodies and ignoring transmittance and extinction, this method only resolves opaque high clouds (Nieman et al., 1993). The polarized reflectance method used by POLDER-3 retrieves CTH from atmospheric molecular optical thickness and pressure relationships. Molecular optical thickness above clouds is derived from the 490 nm polarized reflectance sensitivity to atmospheric molecules (Buriez et al., 1997, 2005). However, such a retrieval method depends on limited scattering angles. MODIS employs the CO₂ slicing method, retrieving CTP from the 15 μm CO₂ absorption band radiance ratios (Wielicki and Coakley, 1981). As CO₂ absorption is insensitive in the lower atmosphere, it generates mid- and high-level CTH accurately but poorly resolves low clouds (Menzel and Strabala, 1997; Eyre et al., 1989). POLDER-3 uses the O₂ A-band method, exploiting the O₂ absorption by solar radiation (Buriez et al., 1997). Depending solely on O₂ absorption and less influenced by atmospheric constituents, this method is highly variable for thin low COT clouds and is surface albedo-sensitive (Zhang et al., 2011). The first two methods derive CTH from the top-of-atmosphere radiance, physically close to actual cloud top (Simpson et al., 2000; Vanbauce et al., 2003). However, atmospheric gas and molecule distribution and concentration during retrieval also influence the latter methods, representing an intermediate level rather than cloud top layer (Vanbauce, 2003; Platnick et al., 2003).

The O₂ A-band and polarized reflectance method are the two primary algorithms that can be utilized in CTH retrieval for the passive polarized sensors. Buriez et al. (1997) suggested that apparent pressure could be obtained from two spectral channels (763 nm and 765 nm) centered on the O₂ A-band. Thus, the apparent pressure assumes that the atmosphere is a purely absorbing medium, and the clouds are perfect reflecting surfaces (Desmons et al., 2013). The theory is based upon the fact that the absorption of O₂ expresses the penetration depth of radiation within the atmosphere. The absorption ratio of the wide (763 nm) and narrow (765 nm) channels measured by the POLDER characterized the depth of photons and oxygen transmittance from the top of the atmosphere to a level of pressure and back to space (Ferlay et al., 2008). Ferlay et al. (2010) examined the relationship among POLDER oxygen parameters, cloud pressure, and vertical extent, showing that for clouds with a high vertical extent (deep convective, cirrus, or low-level clouds), the cloud top oxygen pressure obtained by the parameterized formula is closer to the actual CTP than the POLDER O₂ pressure. However, for low-level liquid clouds (CTP > 680 hPa), POLDER retrievals tend to slightly underestimate the actual cloud top and intermediate pressures. However, the POLDER CTP retrieval results still have good accuracy in the high cloud fraction region, where the uncertainty in the retrieval of the CTH of upper-level clouds (mainly ice clouds) is higher due to the much faster vertical variation of atmospheric pressure at low altitudes than at high altitudes.

Using the polarized reflectance at either 443 nm or 490 nm to retrieve the CTP is also widely achieved by passive polarized sensors (Buriez et al., 1997). When the illumination coming from the observed surface is insignificant, the polarized radiance at 443 nm is mostly related to the optical thickness of the air molecules above the surface. This method is therefore restricted to thick clouds (clouds with spherical albedo>0.3) and the 80° to 120° region of highest molecular scattering polarization, which is outside the range of sun glint. The polarized irradiance of the 443 nm (or 490 nm) channel comes from atmospheric molecular and cloud contributions, Because of the link between molecular scattering efficiency and λ^–4,the polarized irradiance of the 865 nm channel comes from cloud contributions only. The difference in polarized information between the 490 nm and 865 nm cloud layers is simulated using the vector radiative transfer equation (RTM), and the equation for retrieving the Rayleigh pressure at the cloud top is fitted based on this information (Vanbauce et al., 2003). Li et al. (2018) proposed a 490 nm and 865 nm polarization reflectance difference retrieval for CTH, increasing the range of scattering angles used for CTH retrieval, and found that the standard deviation of the values is lower than that of the POLDER-3 CTH (~1.5 km).

Recent studies have demonstrated that machine learning (ML) techniques (Hopfield, 1982), such as k-nearest neighbors, random forest (RF) (Breiman, 2001), support vector machines, and (Liu et al.,2022) artificial neural networks, provide solutions to certain nonlinear problems in remote sensing, Earth system science, and geosciences. The four aforementioned ML techniques were employed by (Kühnlein et al. (2014) to develop a cloud top height (CTH) prediction model. The ML-based retrieval algorithm can estimate CTH directly from satellite observations and numerical weather prediction data. Retrieval results indicate that compared to conventional physically-based algorithms, the optimized ML method yields more accurate CTH estimates, especially for high and optically thin clouds, which are underrepresented in model simulations. Because mismatches remain between the simulated conditions in radiative transfer models and actual atmospheric conditions, uncertainties may arise under cloudy skies (Min et al., 2020).

This study proposes a synergistic retrieval algorithm for CTP using polarized sensors covered with oxygen bands. This algorithm addresses the issues of underestimation in retrieved CTP from optically thin low clouds by the O₂ A-band method and the dependence of polarization-based retrievals on narrow scattering angle ranges. Due to the constraints of polarized reflectance sensitivity within specific scattering angles of polarized channels, the synergistic method incorporates sensitivities from three polarized bands for CTP and the O₂ A-band through a random forest regression model. Compared to single-channel retrievals at 490 nm, this method provides a wider range of scattering angle sensitivities and compensates for errors associated with optically thin clouds in O₂ A-band retrievals.

The remainder of this paper is organized as follows. Section 2 describes in detail the data sources and sensitivity analysis used in this study. This includes introduction to the four satellite datasets from POLDER, DPC, MODIS and CALIPSO, as well as the research and validation work applied to the cloud top height retrieval algorithms. The variation characteristics of polarized reflectance and reflectance in different bands under different cloud properties simulated by vector RTM are also presented. Section 3 then explains the establishment and application of CTP retrieval methods. The CTP results from POLDER and DPC are presented respectively. Comparisons are made between CTP derived from different algorithms and sensors. The sources and magnitudes of differences are quantified and possible reasons explored. Finally, section 4 summarizes the major conclusions from this study regarding the retrieval of cloud properties from satellite data. The key findings are highlighted and the significance and implications of these results are discussed. Limitations of the current work and directions for future research to improve cloud property retrievals are suggested.

In this study, contributions to CTH retrieval from both scalar measurements from O₂ A-band and multi-angle polarized measurements are investigated using L1 products from POLDER and DPC. Since the three satellites AQUA, Cloud-Aerosol LiDAR and Infrared Pathfinder Satellite Observation (CALIPSO), and PARASOL are in A-train orbits, their observation times are relatively consistent. The sample data utilized in this work include CAL data from the CALIPSO L2 cloud product, which has high precision to provide high-quality training samples for ML methods. Finally, the findings are analyzed and compared using the MODIS L2 cloud product MYD06 and CAL data.

The POLDER-3 sensor, developed at CNES, was launched with the PARASOL satellite in 2004 to conduct atmospheric polarized detection analyses. POLDER-3 L1 products include TOA radiation (443, 490, 565, 670, 763, 765, 865, 910, and 1020 nm) at 16 angles measured by 8 channels with a resolution of 6 km × 7 km. Among them, retrievals at 490, 670, and 865 nm can obtain additional polarized observation information by converting the satellite readings into Stokes parameters I, Q, and U, which represent the total radiation intensity, linearly polarized intensity parallel to or perpendicular to the reference plane, and linearly polarized intensity at an angle of 45° to the reference plane, respectively. The L2 products include multiple datasets on clouds and aerosols, from which Rayleigh pressure and cloud phase products used in the present study were used.

The DPC (directional polarized camera) (Li et al., 2018b) was launched on the GF-5 satellite (a high-resolution earth observation program) in 2018. The DPC overpasses at 1330 local time with the descending node, and the observation orbit is close to that of the A-train. It has a swath of 1850 km and a spatial resolution of 3.3 km. Comparatively, DPC maintains capabilities comparable to those of the POLDER-3 in terms of band settings and imaging mechanisms, although it has a notably higher spatial resolution and field width, which can provide more detailed atmospheric and ground information.

The MYD06 products cover most cloud top parameters (pressure, temperature, and height) and optical parameters. The MYD06 CTP is retrieved by the CO₂ slicing method, the CTP values are retrieved using several different infrared bands between 12 and 15 μm that are inside a CO₂ absorption band whose sensitivity changes with altitude throughout the atmosphere. Clouds with variable heights are displayed at different positions in the CO₂ band map, where high clouds appear in all bands and low clouds do not appear in the high absorption band (Menzel and Strabala, 1997). The MYD06 CTP product is obtained by converting the results to a resolution of 5 km by averaging 5 × 5 cloud pixel arrays of 1 km (per pixel) to reduce noise (Platnick et al., 2017). Naud et al. (2007) compared CTH retrieved by the infrared brightness temperature method, 11 μm water vapor channel brightness temperature method, and the CO₂ slicing method and concluded that the CO₂ slicing method presented retrieval errors at 60 and 110 hPa and had the highest retrieval accuracy among the three methods. Therefore, the results in this paper are compared with the MYD06 product.

CALIPSO is a sun-synchronous orbiting satellite that was launched by NASA in April 2006. The local time when CALIPSO satellite passes the descending node is 13:30. Globally, the orbital pitch is ~1.55°. CALIPSO is equipped with three sensors: a cloud-aerosol LiDAR with orthogonal polarization (CALIOP), an imaging infrared radiometer (IIR), and a wide-field camera (WFC). Among them, CALIOP is a dual-wavelength polarized-sensitive lidar capable of providing high-resolution vertical profiles of aerosols and clouds across three observation bands: one measuring the backscatter intensity at 1064 nm, and the two others measuring the orthogonally polarized components of the 532 nm backscattered signal. IIR can measure the size of high-altitude cloud ice crystals, as well as cloud absorption and scattering of thermal energy (Winker et al., 2009). The CALIOP and IIR measurements are combined with the hybrid extinction retrieval algorithm (HERA) (Omar et al., 2009) to obtain L2 products. The CALIOP L2 product provides a global vertical structure of abundant tropospheric and lower-stratospheric aerosols and clouds. This study primarily used the L2 product CTP, and only the pressure value of the topmost cloud was selected for data extraction. Because the spatial resolution of the CAL_L2 product is 1 km, the dataset of stored values was matched only after upscaling the CALIOP CTP (DPC, 3 km; POLDER, 5.5 km).

The University of Lille’s ARTDECO RTM was applied in this experiment (Shang et al., 2020). Using the spectral response functions of the POLDER and DPC sensors, three polarized bands (490, 670, and 865 nm) and the O₂ A-band ratios (763 and 765 nm) were simulated and examined. The topmost layer in the simulation for this research is the atmospheric molecular layer, followed by the middle layer (the cloud layer), and the bottom layer (the atmospheric molecular and aerosol layer). The satellite sensors primarily pick up polarized radiation signals from the ground, clouds, atmospheric molecules, and aerosols. After passing through the atmosphere, solar energy is split into direct and dispersed rays. A portion of the scattered radiation takes the form of polarized radiation due to the scattering characteristics of air molecules, clouds, aerosols, and other particles. Thus, the reflectance at the TOA (R_measure) can be expressed according to Eq. (1):

where R_m, R_c, R_a, and R_s represent the contribution of atmospheric molecules, clouds, aerosols, and surface to the top of the atmosphere reflectance, respectively. The impact of the surface type on polarized radiation can be ignored without considering aerosols (Cheng et al., 2008). The reflectance R adopted in this study is defined as follows (Gu et al., 2011):

where I is the radiance (W cm^–2 μm^–1 sr^–1), E₀ is the extraterrestrial solar irradiance (W cm^–2 μm^–1), and μ_s is the cosine of the solar zenith angle. The polarized reflectance R_p adopted in this study is defined as follows (Labonnote et al., 2000):

where $ \sqrt{{Q}^{2}+{U}^{2}} $ is the polarized radiance, E₀ is the extraterrestrial solar irradiance, and μ_s is the cosine of the solar zenith angle.

As the observed central wavelengths of the DPC and POLDER sensors are practically analogous, we only selected the spectral response function of DPC for the input of the simulation. To determine the best combination for the development of ML algorithms, this section presents the sensitivity of the simulated polarized and non-polarized reflectance at the TOA to cloud parameters (cloud phase (CP), CTH, COT, and CER). To evaluate those specific parameters that affected the satellite observations, we changed one parameter in the simulation while leaving the remaining initial circumstances unchanged. To represent the optical scattering properties of ice and water clouds, sensitivity studies were performed for specific liquids and ice crystals. Lorenz-Mie theory (Wiscombe, 1980) was used to calculate the single scattering of liquid clouds, and the distribution of single-layer water clouds in the atmosphere was simulated using a log-normal distribution. The inhomogeneous hexagonal monocrystal (IHM) model (Labonnote et al., 2001) was used to calculate the single scattering of ice clouds, where ice crystals are assumed to be randomly oriented and contain spherical impurities of air or soot bubbles. In addition, other input settings were added to the RTM and the k-distribution coefficient was used as the absorption line to simulate the gas content in the entire atmosphere at different heights.

In practice, the polarized radiance in the 490 nm band is primarily related to the optical thickness of atmospheric molecules above the measuring surface, and the method is restricted to the maximum region of molecular scattering polarized (80°–120°), which is outside the sun-glint area, and optically thick clouds (cloud sphere albedo>0.3) (Vanbauce et al., 2003). To investigate whether the information provided by polarized channels other than 490 nm can be used to support CTH retrievals, we simulated the effect of different COTs on polarized radiation from bands at 490, 670, and 865 nm to determine the variation of TOA-normalized polarized reflectance at high and low COTs.

The conditions for this simulation are set to have a constant aerosol optical thickness, surface albedo, CER, and CTH, and only COT variations are considered when calculating the normalized polarized reflectance at 490, 670, and 865 nm. The cloud settings are as follows: the CTH of water and ice clouds are 2 and 12 km, respectively, the CER of water and ice clouds are 10 and 40 μm, respectively, and the effective variance varied of water clouds is 0.01. The effective radius of the spherical bubble inclusions within the ice crystal is 1.5 μm, the effective variance is 0.05, and the mean free path length is 15. The number of simulated photons is 10 000 000, and the surface albedo is 0.1. The solar zenith angle is 20°, the zenith angle range is 0°−90° (average 30°), the relative azimuth is 0°, and the COT values are 0.5, 1, 5, and 10, as shown in Fig. 1. The comparison of Figs. 1a–c indicates that the polarized radiation is affected by the wavelength and COT of the liquid clouds. The normalized polarized reflectance with COT has the same pattern for different wavelength cases, and signal saturation is observed for COT > 5. The simulation results also show that there is a peak in the normalized polarized reflectance at a scattering angle of approximately 140°, and the height of these peaks increases as the COT value increases. Figures 1d–f indicate that the change of the normalized polarized reflectance with the COT of the ice clouds is less pronounced than that of the water clouds and considerably less than the peak at approximately 140°, thus indicating a more stable state compared with that of the water clouds. In addition, the intensity of polarized radiation decreases as the wavelength increases in terms of the magnitude of the value domain of normalized polarized reflectance. This phenomenon also shows that the smaller the wavelength, the more information it likely provides for CTH retrieval.

Figure 1.  Simulations of normalized polarized reflectance of (a–c) liquid clouds, and (d–f) ice clouds at 490, 670, and 865 nm, with four COTs of 0.5, 1, 5, and 10.

According to the simulation shown above, the polarized reflectance of actual clouds decreases as the optical thickness increases, although saturation is observed at COT > 5. Therefore, it is necessary to determine whether the altered CTH of the COT < 5 case has an impact on the normalized polarized reflectance. The intensity of polarized light is higher for the 490 nm channel; thus, this channel is used for the experiment. The simulation parameters are COT values of 2−6 and CTH values of 1−5 km, and the results are displayed in Fig. 2. A comparison of panels (a) and (b) shows that the variation of both COT and CTH affect the normalized polarized reflectance. The variation of CTH in panel (a) shows that the normalized polarized reflectance decreases uniformly as CTH increases, which also confirms that the higher the cloud layer, the lower the optical thickness of the atmospheric molecules above the cloud layer, which leads to a decrease in the molecular contribution to the polarized radiation. A comparison of panels (a) and (b) further shows that the variation of either COT or CTH affects the normalized polarized reflectance. Panel (b) shows that the normalized polarized reflectance degrades as the COT increases, and saturation is observed at COT > 8. To treat the saturation of the polarized radiation in the low COT scenario for this decay phenomenon, a proposed relationship can be added that will lessen the impact on the COT at the retrieval CTH. Its polynomial formula is given as follows:

Figure 2.  Simulation of the variation of normalized polarized reflectance of liquid clouds at 490 nm with (a) CTH and (b) COT, with CTH varying from 1–5 km and COT varying from 1–8.

In this formula, R_p represents the polarized reflectance, COT represents the cloud optical thickness, SAC represents the scattering angle, and b₀–b₄ are the coefficients, as illustrated in the Table1.

The theory of atmospheric radiative transfer states that the main contributor to the brightness of polarized irradiance is single scattering, while polarized multiple scattering has a more limited impact (Hansen, 1974). The next step in the simulation is CER, which plays a vital role in the single scattering properties of cloud particles. Except for COT being fixed and CER being transformed, the simulation conditions are the same as for COT. The simulation's findings are displayed in Fig. 3, further noting that the findings in Figs. 3a–c and Figs. 1a–c are similar to the simulation of the liquid cloud COT. The difference is that the cloud bow at a scattering angle of approximately 140° shifts to a larger scattering angle as the CER increases, and the cloud bow covers a larger range of scattering angles. The findings in panels (d–f) support the hypothesis that CER has little impact on the normalized polarized reflectance of ice clouds because the geometrical optics approximation predicts that the IHM is scale-invariant, which reduces the effect of CER variations on polarized reflectance.

Figure 3.  Simulations of the normalized polarized reflectance of (a–c) liquid clouds and (d–f) ice clouds at 490, 670, and 865 nm with CERs of 5, 10, 20, and 30 μm for liquid clouds and 10, 20, 40, and 60 μm for ice clouds.

Figure 4 illustrates the variation of the normalized polarized reflectance with the scattering angle when only the CTH is modified, and all other conditions are left the same. Different CTH values of water clouds still exhibit distinct variances in their enhanced scattering angles, with 490 nm being the most notable (Figs. 4a–c). The polarized reflectance consistently decreased with an increase in CTH in the scattering angle range of 100°–120°. For instance, as the associated scattering angle range and polarized reflectance intensity decreased, as the simulated wavelength increased, the sensitivity of CTH retrieval at 670 nm was significantly weaker than that at 490 nm. Figures 4d–f display the variation of ice clouds with different scattering angle CTPs, thus revealing a similar pattern to water clouds; however, the range of scattering angles available for retrieval is smaller than that of ice clouds due to the cloud bow phenomenon of water clouds. Similarly, the sensitivity of the normalized polarized reflectance decreases with increasing simulated wavelength.

Figure 4.  Simulations of normalized polarized reflectance of (a–c) liquid clouds and (d–f) ice clouds at 490, 670, and 865 nm with CTHs of 1, 2, 3, 4, and 5 km for liquid clouds and 5, 7, 9, 11, and 13 km for ice clouds.

The simulation results demonstrate that in either water clouds or ice clouds, the normalized polarized reflectance at 490 nm fluctuates with the CTH significantly stronger than that at the other two wavelengths. The combination of the three can be a powerful source of data for ML models for CTH retrieval because the polarized radiation at 490 nm is primarily provided by both molecular scattering and cloud scattering, while the radiation at 865 nm is only related to the cloud itself and the radiation at 670 nm also depends on the COT of the cloud.

The polarized radiation reflected by the water and ice clouds was found to be sensitive to CTH according to the results of the cloud characteristics based on the vector radiation simulation. It is important to distinguish the cloud phase state when retrieving CTHs because the polarized radiation of ice clouds was less affected by COT and CER than that of water clouds, whose polarized reflectance was simultaneously affected by COT and CER and resulted in an increase or decrease in the radiation within the cloud bow.

The cloud parameters are configured identically to those in section 3.1.2 for scalar radiation simulations used in the O₂ A-band. The simulation channels are substituted with the reflectance of 763 nm and 765 nm, which are common to POLDER and DPC sensors. The O₂ A-band ratio is defined as the reflectance ratio between the absorption channel at 763 nm (thin), centered on the A-band, and the broad channel at 765 nm (broad), spanning 40 nm on either side of the A-band. The air mass is also utilized as a variable to visually represent the relationship between the A-band ratio and CTH. The O₂ A-band ratio and air mass factor are formulated as (Merlin et al., 2016):

In Eq. (5), R_{763 nm} and R_{765 nm} represent the reflectances as measured by the 763 and 765 nm channels, respectively. In Eq. (6), θ_s represents the solar zenith angle, and θ_v is the satellite viewing angle.

Simulation results clearly show that the liquid cloud Fig. 5a and ice cloud Fig. 5b A-band ratio exhibits the same trend with airmass under identical COT. Both decrease with increasing airmass factor, indicating greater absorption; thus, CTH variation directly alters cloud reflection path length. Increased CTH elongates the above-cloud photon path, enhancing in-band absorption without an out-band change. Therefore, the ratio increases with CTH, but decreases with CTH, as Fig. 5 shows. We note that the change in CTH is not limited by the observed angle, because the change in ratio caused by the change in CTH is equivalent to almost any angle.

Figure 5.  Simulations of the O₂ A-band ratio of (a) liquid clouds and (b) ice clouds, with CTHs of 1, 2, 3, 4, and 5 km for liquid clouds and 5, 7, 9, 11, and 13 km for ice clouds.

The above analysis shows that the sensitivities have covered more scattering angle range, but the useful angle range is 80°–120°. The polarized radiation of water clouds was greatly affected by COT and CER. We can neglect this effect to some extent after repairing the polarized reflectance of those water clouds with a COT < 10 by introducing a polynomial fit. The amount of information available in the polarized band decreases as the wavelength increases, and the range of scattering angles covered decreases as well. The sensitivity analysis shows that both the polarized reflectance and O₂ A-band ratio have a good sensitivity response to CTH. In the process of ML model training, multi-bands contain richer information than a single polarized or non-polarized band, and combining both can retrieve CTP more accurately.

The cloud detection algorithm is based on a series of single-pixel detections, and we set thresholds based on the wavelength properties of GF-5 and the reflectance properties of the features. We also introduce apparent pressure to increase the stability of high-layer cloud identification and use the multi-angle polarized properties of the 865 nm band to address the inability of the POLDER algorithm to detect in the solar scintillation region (Lesi et al., 2021). By using the RTM simulation in conjunction with Mie theory and the IHM model (Labonnote et al., 2001), the cloud phase detection technique determines the polarized reflectance of liquid clouds with various size distributions and ice clouds with various crystal forms. Then, the typical features of the polarized reflectance curves of liquid and ice clouds are summarized to develop the P-CP algorithm (Shang et al., 2020). The cloud parameter preprocessing section of this work uses the aforementioned cloud identification and cloud phase detection algorithms, which may be applied to both the POLDER and DPC sensors.

This study uses high-accuracy LiDAR CALIOP CTP as the sample data, with the most sensitive band to CTP extracted as the training value via the ML approach (Fig. 6, sensitivity analysis with ML). Following the extraction of the appropriate band values for cloud and cloud phase detection from the L1 product of the DPC, COT retrieval is carried out. The contemporaneous MYD06 COT is utilized because the retrieval technique of the DPC COT is still being tested (Fig. 6, cloud parameter reprocessing). Finally, the RF model for CTP retrieval is updated to include the acquired polarized and O₂ A-band reflectance of water and ice phase clouds. Simultaneously, incorrect values of wide-angle observations or those filtered for large retrieval errors of CTP were eliminated throughout the process of averaging retrieval results across numerous angle measurement directions. The complete CTP search procedure is described above (Fig. 6, CTP retrieval).

Figure 6.  CTP retrieval flow chart.

A random forest model (RF) is a classifier containing multiple decision trees, the final class of which is decided by the patterns of each tree's output. This model was first proposed by Ho (1995) and derived from the random decision trees that make up a forest, further noting that the method was established by Breiman et al. in 1996. To create a collection of decision trees, this method combines Ho's random subspace method with Breiman's theory of bootstrap aggregation. When compared to other ML techniques, RF can handle categorical and numerical information, solve classification and regression problems, and is extremely resistant to overfitting via decision tree averaging. The hyperparameters of a flexible and simple ML method such as RF often do not need to be adjusted to provide decent results.

In this study, LIDAR CALIOP cloud layer data were used as the sample dataset, and datasets of CALIOP and DPC (POLDER) from overlapped observation regions were extracted for January, April, July, and October 2019 (2012) to balance the seasonal pattern variations. These datasets included non-polarized data (763 and 765 nm) at the effective observation angles for the L1 product, polarized data (polarized reflectance 490, 670, and 865 nm), solar zenith angle, satellite observation angle, and relative azimuth. When selecting the sample data, the original 1-km resolution of cloud layer data was upscaled to 3 km and 7 km for space matching, corresponding to DPC and POLDER, respectively, noting that the matching time did not exceed 1 hour.

To find the optimal combination among these parameters, we constructed different parameter combinations for the sensitivity analysis, and the comparison results are shown in Table 2. Important metrics used to evaluate the accuracy of the RF models in this study include the square of the coefficient of determination (R²) and the root-mean-square error (RMSE). It was also found that the polarized and non-polarized data contained decisive information mainly for estimation, thus confirming that the combination of polarized and non-polarized information could help the model to better estimate CTP and should be considered in the retrieval algorithm. The model with the lowest error was selected as the optimal model. The training results of the model are shown in Fig. 7, where panels (a) and (b) show scatter density plots of ice and water clouds from the CTP retrieval model of POLDER, with R² values of 0.868 and 0.806 and RMSE values of 42.651 and 86.693 hPa, respectively; while panels (c) and (d) show scatter density plots of ice and water clouds from the CTP retrieval model of DPC, with R² values of 0.893 and 0.782 and RMSE values of 61.995 and 80.901 hPa, respectively. The above four models all have strong correlations and low errors. The RF model is set up as follows: the number of water cloud and ice cloud data samples = 3 000 000, N_estimators = 70, criterion = MSE, max_depth = None, MIN_samples_leaf = 5, and random_state = 50.

Figure 7.  The accuracy validation results of the CTP model with 500 hPa as the dividing line to separately sample ice and water clouds, for (a) ice clouds and (b) water clouds.

The algorithm developed to match this study was applied to the POLDER data when comparing it to the L2 Rayleigh pressure and oxygen pressure for validation relative to the same type of satellite product. POLDER maintains the same type of optical sensor as DPC but ceased its observation mission in 2013. Before the comparison, the spatial resolution of the 5 × 5 km of MODIS product was decreased to 16.6 km to match the original 16.6 km of POLDER L2 product. The oceanic region near Southeast Asia and western Australia is chosen for the retrieval experiment. Tropical rainforests in Southeast Asia are located close to the equator and have strong solar radiation, high temperatures, low air pressure, and annual precipitation of approximately 2000 mm. Convective, water, and ice clouds are easily formed in this region and are therefore extremely relevant to the stability of the water-ice cloud inversion model used in this study. Here, the official POLDER cloud detection, cloud phase detection, and optical thickness products are used to preprocess the cloud characteristics to some extent.

The same observation range of POLDER and MODIS is depicted in Fig. 8. In comparing Figs. 8b–d, the distribution of the three CTPs in the medium and high-pressure range is very consistent. Compared with the POLDER Rayleigh pressure, the CTP retrieval results of the current study are more similar to the MODIS CTP. The distribution of the data shows that the coverage of the CTPs in the present study is larger in the northern and southern latitudes than the POLDER Rayleigh pressure, which also indicates the dependence of the CTPs retrieved by polarized information on the existence of an effective observation angle. To further evaluate the correctness of the algorithm, the above three CTP results were qualitatively verified with the CALIOP CTP, and the verification results are shown in Fig. 9. The scatter distribution of several CTPs in Fig. 9a shows that the CTP results of this study are in good agreement with the distributions of MODIS and CALIOP CTP; however, the distribution of POLDER Rayleigh pressure shows greater differences from that of CALIOP CTP, and as the pressure increased, the pressure of some low clouds was significantly overestimated while that of the upper ice clouds was underestimated. Figure 9b shows the frequency histogram of pressure differences between several CTPs and CALIOP CTPs. Though Rayleigh pressure error primarily distributes within 100 hPa, minor peaks also exist at –300 and –600 hPa relative to the radar data, indicating an overestimation. Our CTP pressure difference frequency distributions are largely consistent with MODIS. The MODIS primary peak is at –50 hPa, while our CTP primary peak is closer to the 0 hPa median. From the median distribution, we can see that the distribution is at –5 hPa for our results, –29 hPa for MODIS, and –25 hPa for the Rayleigh pressure; the retrieval results of this study are closer to the CALIOP CTP than the MODIS CTP. Figures 9c–e show the scatter density plots of the three CTPs and CALIOP CTP, with R² = 0.790 and RMSE = 140.521 hPa for the MODIS CTP, R² = 0.281 and RMSE = 335.687 hPa for the POLDER Rayleigh pressure, and R² = 0.827 and RMSE = 121.431 hPa for our CTP. Among the three CTPs, our results correlated better with the CALIOP CTP and showed lower errors. Panel (c) shows that the pressure values of the MODIS product are discontinuous because this product approximates the pressure (minimum unit is 5 hPa). In addition, the scatter point distribution of the ice cloud pressure was in the range of 200–400 hPa of the MODIS CTP, which is slightly lower than the one-to-one line and higher than the CALIOP CTP. The distribution of Rayleigh pressure and CALIOP CTP differences in Fig. 9d is consistent with the conclusions obtained in Fig. 8, which shows that there is an overestimation of water clouds in the range of 500–800 hPa. Figure 9e shows that our CTP and the CALIOP CTP are well correlated and more consistent than MODIS CTP in low-pressure ice cloud areas. Therefore, the effectiveness of the proposed algorithm on the polarized load data was confirmed, and the retrieval accuracy was higher than that of the POLDER Rayleigh pressure.

Figure 8.  Comparison of CTP results on Jan. 1, 2013: (a) POLDER RGB Composite, (b) POLDER Rayleigh pressure, (c) our CTP search results, and (d) MODIS CTP.

Figure 9.  (a) Scatter plot aof the CALIPSO observation trajectory and three CTPs (our CTP retrieval result, MODIS CTP product, and POLDER Rayleigh pressure); (b) Frequency histogram and median values of three CTP differences; (c)–(e) scatter density plot of the CALIPO CTP and three CTPs.

The polarized and O₂ A-band algorithms proposed in the sensitivity analysis in the region of water clouds with COT < 10 have different degrees of inversion errors; therefore, we assess a wide range of optically thin liquid clouds clustering in the western sea of South America on 27 December 2012, through the MODIS COT product, which represents a good sample for testing the algorithm in this study. Figure 10 shows the true color map of POLDER as well as the MODIS water cloud COT range (0–10) and the four CTPs results. We show the MODIS CTP, POLDER oxygen pressure (panel (d)), Rayleigh pressure (panel (e)), and our CTP (panel (f)); noting further that the comparison of the four CTPs shows that the results for several CTPs are not very different. However, the Rayleigh pressure of POLDER is limited by the scattering angle, thus leading to jagged splicing traces, and inversion results are not available for a part of the region. Therefore, we used only oxygen pressure for the comparison in the quantitative validation.

Figure 10.  Comparison of the CTP results on 26 December 2012: (a) DPC RGB composite, (b) MODIS water cloud COT distribution of less than 10 ( Light Blue), and more than 10 (Orange), (c) MODIS CTP, (d) POLDER oxygen pressure, (e) POLDER Rayleigh pressure, and (f) our CTP result.

Figure 11a shows the scatter plot of the range of optically thin water clouds. The red dots in the results indicate the CALIOP CTP, the black dots indicate our CTP, the yellow dots indicate the POLDER oxygen pressure, the blue dots indicate the MODIS CTP, and the green dots indicate the POLDER Rayleigh pressure. The distribution of scattered points in the figure shows that the distribution of the CTP and POLDER oxygen pressure is close to the CALIOP CTP while the MODIS results are underestimated. The histogram distribution of the CTP difference results in Fig. 11b shows that the median of our CTP and CALIOP CTP is 2.57 hPa, and it is centered at the 0-degree line, thus presenting a smaller error compared with that of the MODIS and POLDER oxygen pressure. The results of the scatter density plots in panels (c) and (d) are as follows: R = 0.360 and RMSE = 49.491 hPa for MODIS CTP, R = 0.299 and RMSE = 55.743 for POLDER oxygen pressure, and R = 0.432 and RMSE = 30.661 hPa for our results. The results of our algorithm for thin clouds benefit from the multiple information sources compared with the other products.

Figure 11.  (a) Scatter plot of CALIPSO observation trajectory and three CTPs (our CTP retrieval, MODIS CTP product, POLDER Rayleigh pressure, and POLDER oxygen pressure), (b) Frequency histogram and median values of the three CTP differences. (c)–(e) Scatter density plots of the CALIPO CTP and three CTPs.

To configure relatively similar comparison conditions, the same Southeast Asian and Australian ranges as in section 3.2 were also selected for validating DPC CTP results. In Fig. 12, panel (a) displays the DPC RGB composite, panel (b) our CTP, and panel (c) the MODIS CTP over the region covered by both DPC and MODIS observations. Due to differences in cloud detection algorithms, MODIS CTP exhibits inversion results over DPC-observed ground or water surfaces (non-cloud). The comparison of Figs. 12b and 12c shows that the coverage of MODIS CTP far exceeds our CTP. From the scatter distribution results in Fig. 13a, the DPC CTP is closer to the CTP in the high-pressure water cloud region in the latitude range of 60°S–10°N, while the MODIS results show a slight overestimation, as the CO₂ slicing method exhibits errors of 50 hPa for cirrus and altostratus clouds and are within 200 hPa for other clouds. Our retrieval algorithm demonstrates certain advantages for water clouds at higher pressures. The frequency histogram of pressure difference results in Fig. 13b are similar to that of the POLDER CTP, the difference of the DPC CTP is still centered on the zero lines, and the median value is better than that of MODIS, with a difference of approximately 10 hPa. From the scatter density plots in Figs. 13c and 13d, the MODIS CTP is still overestimated in the range of ice clouds, while the results of the DPC perform well. While MODIS CTP still seems to be overestimated in the ice cloud range, the DPC results perform well. The R² of the MODIS and DPC results are relatively close at 0.769 and 0.779, respectively, and the RMSE values are 148.437 and 142.263 hPa, respectively.

Figure 12.  Comparison of CTP results on 14 Nov. 2019: (a) DPC RGB composite, (b) our CTP, and (c) MODIS CTP.

Figure 13.  (a) Scatter plot of the CALIPSO observation trajectory, MODIS CTP, and our CTP retrieval results from the DPC; (b) frequency histogram and median values of CTP difference; (c) and (d) scatter density plots of the CALIPO CTP, MODIS CTP, and our CTP.

In addition, we retrieved the global CTP from POLDER and DCP data on 2 January 2013 and 2 October 2018, respectively. Figures 14a and 14b show the RGB composite images and CP from POLDER, while Figs. 14c–f compare our CTP, MODIS CTP, POLDER oxygen pressure, and POLDER Rayleigh pressure. For ease of comparison, we masked the four CTPs using the POLDER CP, showing only the cloud areas detected by POLDER. From the spatial distribution, it can be seen that the inversion results of the four CTP products are generally similar. However, the POLDER oxygen pressure has missing values in the Antarctic region. Similarly, the POLDER Rayleigh pressure also has this problem in the large scattering angle region. Figure 15 shows the scatter density plots of the four CTP products against CALIOP CTP. Panel (a) is the comparison with our CTP and CALIOP CTP, with an R² of 0.636 and RMSE of 205.176 (hPa), panel (b) is the comparison with MODIS CTP and CALIOP CTP, with an R² of 0.576 and RMSE of 229.371 (hPa), panel (c) is the comparison with POLDER oxygen pressure and CALIOP CTP, with an R² of 0.452 and RMSE of 247.019 (hPa). Panel (d) is the comparison with POLDER Rayleigh pressure and CALIOP CTP, with an R² of 0.432 and RMSE of 279.501 (hPa). Our POLDER results demonstrate better performance in the comparison. The comparison results of DPC are shown in Fig. 16; panels (a) and (b) show the RGB composite of DPC and MODIS and panels (c) and (d) compare the results of DPC and MODIS cloud phase detection, respectively. As shown in panels (e) and (f), the retrieved DPC and MYD06 CTP distributions are in good agreement. Panel (g) is the comparison with DPC CTP and CALIOP CTP, with an R² of 0.663 and RMSE of 171.141 (hPa), panel (h) shows the comparison with MODIS CTP and CALIOP CTP, with an R² of 0.622 and RMSE of 185.930 (hPa). The RMSE is lower than that of the MODIS CTP (6.173 hPa), with a retrieval performance approaching that of MODIS CTP. The validation results further confirm that the algorithm developed in this study can be applied well not only to POLDER data but also to DPC data.

Figure 14.  Comparison of global CTP on 2 Jan. 2013: (a) POLDER RGB composite; (b) POLDER cloud phase and comparison of the four CTPs; (c) our CTP; (d) MODIS CTP, (e) POLDER oxygen pressure, (f) POLDER Rayleigh pressure.

Figure 15.  The scatter density plot of CALIPO CTP and four CTPs, (a) our CTP, (b) MODIS CTP, (c) POLDER oxygen pressure, and (d) POLDER Rayleigh pressure.

Figure 16.  Comparison of global CTP on 2 Oct. 2018: (a) DPC and (b) MODIS RGB composite, (c) DPC cloud phase, (d) MODIS cloud phase, (e) DPC CTP (our result), and (f) MODIS CTP. The last two panels show scatter density plots of CALIPO CTP and (g) DPC CTP and (h) MODIS CTP.

An algorithm for cooperative data retrieval based on polarized and O₂ A-band data retrievals of CTP is proposed in this work. The following conclusions are drawn based on the results.

(1) The RTM was used to evaluate the effects of COT, CER, and CTH variations on polarized and non-polarized measurements for water and ice clouds. The results show that polarized reflectance has good sensitivity to variations of CTH for both water and ice clouds, indicating that the contribution of atmospheric molecular scattering to polarized radiation decreases with increasing CTH, thus providing good observational information for subsequent retrievals. We also find that the polarized reflectance of water clouds becomes saturated with increasing COT; therefore, specialization is required in this range. We propose the use of a polynomial fitting method for cases of COT < 10. The modeling results for non-polarized reflectance also show good sensitivity to CTH, with the A-band ratio decreasing with increases in the air mass for both water and ice clouds at different heights, thus indicating enhanced oxygen absorption for high air masses. Also, the sensitivity of the correlation between the A-band ratio and CTH and COT shows that this correlation is saturated at COT ≥ 5 for either water or ice clouds. To resolve this phenomenon, a synergistic inversion algorithm is constructed in this paper by training both polarized and O₂ A-band information through an RF model.

(2) The synergistic retrieval algorithm was first used for POLDER data and contrasted with the POLDER Rayleigh pressure and MODIS CTP data products to evaluate the performance. The results showed that the RMSE of the MODIS CTP and our CTP were all within 150 hPa, while the RMSE of POLDER Rayleigh pressure was above 300 hPa. The R² value of 0.827 for our CTP was the best among the three CTPs. Thus, our CTP was more accurate than that of the POLDER Rayleigh pressure. According to the median distribution, all three CTPs were underestimated, and the median of our CTP was located at –5 hPa, which was closest to 0 hPa among the three CTPs. Moreover, the overall error of the synergistic algorithm is small. Subsequently, to address the problem of uncertainty in the single-band algorithm for optically thin water clouds, we searched for optically thin cloud regions with large-scale clustering through the MODIS COT product for validation, and the results showed that our algorithm still had a more desirable performance, with an R = 0.47 and RMSE = 30.611 hPa.

In the next step, the synergistic algorithm was applied to DPC data, and the MODIS product was compared with the CALIOP CTP to further test the correctness of the algorithm established in this work. The RMSE and bias were lower than the MODIS CTP values, and these values had the same error of 150 hPa as the CALIOP CTP. The overall comparison shows that the CTP of this study was lower than the CTP of the hybrid cloud state and the uncertainty of ML may lead to bias in the retrieval results. The effectiveness of the CTP retrieval technique presented in this study, which has a much greater retrieval accuracy than the POLDER Rayleigh pressure, is supported by the aforementioned conclusions. The combined DPC CTP findings indicate that the detection products have potential applications for numerical weather prediction, surface energy estimation, cloud climate influence research, and other relevant fields.