Ensemble Retrieval of Atmospheric Temperature Profiles from AIRS

Atmospheric temperature and humidity are two important parameters in global climate and weather systems. Among all observations, satellite-derived atmospheric temperature and moisture soundings have their own uniqueness. Historical information about severe convective weather at the meso- and micro-scale can be obtained from satellite sounding measurements (Li et al., 2011a), and the profile retrieved from radiance measurements can be assimilated in numerical weather prediction (NWP) models to predict convective weather development (Zavodsky et al., 2007; Reale et al., 2008, 2009; Li and Liu, 2009; Liu and Li, 2010). The atmospheric soundings from geostationary satellites have high temporal resolution (Schmit et al., 2009), while soundings from polar orbiting satellites have the advantage of global coverage; measurements from both polar orbiting and geostationary satellites provide atmospheric soundings for global and regional weather forecasting.

There are two main types of algorithms for atmospheric profile retrieval from satellite radiance measurements: statistical methodology and physical iterative algorithms. Statistical methodology includes regression, neural networks etc. (Smith and Woolf, 1976; Girbanov and Zakharov, 2003; Li et al., 2009), and its advantage is its computational efficiency, numerical stability, and simplicity. It is often used to generate the first guess for physical retrieval algorithms. For example, the Television and Infrared Observation Satellite (TIROS) Operational Vertical Sounder (TOVS) Processing Package (ITPP; Smith et al., 1993), and the International ATOVS Processing Package (IAPP) (Li et al., 2000) use a physical iterative algorithm with its first guess derived from a regression approach. Physical retrieval algorithms are usually based on one-dimensional variational (1DVAR) methodology (Li et al., 2000; Liu and Weng, 2005; Li et al., 2008).

Atmospheric sounding retrievals are affected by many factors, including the spectral coverage, spectral resolution, observational errors (Sokolov et al., 2008), the inverse algorithm (e.g., nonlinearity of the radiative transfer equations), uncertainty of the radiative transfer model, first guess, background, and the background error covariance matrix (Li et al., 2004a; Sharan, 2009; Meibodi et al., 2010). Other factors affecting the retrieval quality include surface properties, clouds in the field of view on the observed infrared radiance, aerosols, and trace gas contamination in the atmosphere (Seemann et al., 2008; Li et al., 2011b; Zhou et al., 2011). Many studies have been conducted to address these problems.

Progress has been made in improving satellite sounding retrievals in the past few decades. For instance, radiance observations from infrared (IR) high spectral resolution sounders, such as the Atmospheric Infrared Sounder (AIRS) (Chahine et al., 2006) onboard NASA's Earth Observing System (EOS) Aqua satellite, the Infrared Atmospheric Sounding Interferometer (IASI) onboard Europe's Metop-A and -B satellites, and the Cross-track Infrared Sounder (CrIS) onboard the Suomi National Polar-orbiting Partnership (NPP) and the Joint Polar Satellite System (JPSS), have shown the capability to derive atmospheric soundings with high vertical resolution and good accuracy (Tobin et al., 2006). In order to better handle the clouds within a sounder sub-pixel, the high spatial resolution imager cloud mask can be used for sounder cloud detection as well as cloud property retrieval (Li et al., 2004a, 2004b). For example, MODIS (Moderate Resolution Imaging Spectroradiometer) and AIRS benefit each other; AIRS is used to inter-calibrate MODIS infrared band radiances (Tobin et al., 2006), while MODIS is used to cloud-clear the AIRS radiances (Li et al., 2005). In the infrared window spectral region, the surface is an important radiative source, which contributes significantly to the observed IR radiance. Therefore, objectively quantifying surface properties (i.e., emissivity) in the radiance calculation is important for the sounding retrieval (Zhou et al., 2011; Li et al., 2011b). The operational MODIS MOD07 atmospheric retrieval algorithms use a set of global profiles and corresponding surface data (surface emissivity, surface skin temperature, and surface pressure) to train the synthetic regression (Seemann et al., 2003, 2008). These studies improved accuracy not only on the regression retrievals, but also on the physical retrievals from the hyperspectral infrared radiance measurements.

Due to the ill-posedness of the retrieval problem (Smith et al., 1993; Hannon et al., 1996; Li et al., 2000), physical retrieval accuracy under cloud-free conditions relies critically on the quality of the first guess. Any errors in the first guess could be amplified if the algorithm does not handle the nonlinear inverse problem well, resulting in significant retrieval errors. Some studies have used NWP forecasts as the first guess; however, interpolation from these data to satellite observations spatially and temporally will cause additional errors that are difficult to quantify (Li and Zeng, 1997). Indeed, regression-based atmospheric temperature, moisture, and ozone retrievals have been proven better than NWP forecasts as a first guess for the physical retrieval (Smith et al., 1993; Li et al., 2000; Li et al., 2008). In order to obtain a good first guess, the AIRS retrieval algorithm (version 5) uses a two-step regression methodology: the so called "cloudy regression" for cloud-clearing and then the so called "clear regression" is used as the first guess for the physical retrieval. The AIRS retrieval algorithm (version 6) uses a neural network scheme to improve the accuracy of the retrieval (Susskind et al., 2012).

The ensemble method has been proven useful in atmospheric science to resolve some error problems, especially in NWP modeling. (Ferreira-Coelho and Rixen, 2008) used ensemble methods to track multi-scale atmosphere errors and classification by running the model several times using different forcings and starting from different initial conditions (Hodur, 1997; Bishop and Toth, 1999; Coelho et al., 2009). Generally, the perturbation of the ensemble members is realized by using a fairly large number of independent runs to resolve error covariance of the state variables (Lermusiaux et al., 2006; Judd et al., 2007). Two general approaches are used to interpret ensemble modeling results: the deterministic approach and the probabilistic approach (Huisman et al., 2009). In the deterministic approach, an optimal combination of ensemble members is used instead of a single ensemble member (Franz et al., 2003). In the probabilistic approach, the ensemble members are treated as possible realizations of the system response. It is used in quantitative ways to determine the uncertainty from an ensemble of predictions (Raftery et al., 2005; Huisman et al., 2009). Because the ensemble method can resolve uncertainty problems and atmosphere errors analysis, it is fit for resolving nonlinear error from retrieval, and the combination of the ensemble method and the retrieval method is called "ensemble retrieval", which is used in the research.

The probabilistic approach, also known as the probability density function (PDF) method, offers compelling advantages for modeling perturbation. It is probably the most widely used technique for uncertainty analysis of mathematical models (Stedinger and Kim, 2010). In particular, it provides an effective solution to the closure problems that arise from averaging or filtering the highly nonlinear source terms (Haworth, 2010). As one form of the PDF, the normal PDF computes the PDF at each of the values using the normal distribution. The PDF technique depends highly on the system's sensitivity to small variations in the initial conditions.

In the present study, an ensemble retrieval method was developed to improve AIRS sounding retrievals, and a normal PDF was used to obtain the optimal profile from the ensemble retrieval members. Just as there are different initial conditions for ensemble NWP model predictions, an ensemble of the first guesses was used for the physical retrieval. The original first guess was perturbed to form a set of new first guesses for the retrievals (the ensemble first guesses). The AIRS data are described in section 2, along with a detailed description of the methodology for generating the first guess perturbations, as well as how the PDF is used to obtain an optimal retrieval profile. Ensemble retrieval results and analysis are given in section 3. A discussion and conclusions are presented in section 4.

The AIRS (Aumann et al., 2003) onboard NASA's EOS Aqua satellite is one of the most advanced high spectral resolution infrared sounders in the world. It is in a polar orbit, and measures the radiances emitted from the atmosphere and surface of the Earth in 2378 channels ranging from 650 cm^-1 (15.4 #cod#956;m) to 2700 cm^-1 (3.7 #cod#956;m), with a spectral resolution (#cod#965;/#cod#916;#cod#965;) over 1200. The AIRS horizontal resolution is 13.5 km at nadir view, with scan to #cod#177;49.5^#cod#x000b0; off nadir, leading to 90 fields-of-view (FOVs) across track and 135 FOVs along track in a 6-minute data granule; it yields a swath width of approximately 1650 km (Nasiri et al., 2010). AIRS has been proven very useful to improve weather forecasting through assimilating either radiances (Singh et al., 2012) or sounding products (Zavodsky et al., 2007; Reale et al., 2008, 2009).

The dataset used in this study includes the AIRS level-1b radiance measurements, MODIS cloud mask product (MYD35), MODIS emissivity product (MYD11), and the ECMWF (European Center for Medium-Range Weather Forecasts) analysis. The granules 65 and 66 on 2 March 2012 were used to test the algorithm. It contains 90#cod#215; 135#cod#215; 2=24300 FOVs. These two granules were chosen because their observation time (0635 and 0629 UTC) is close to the ECMWF analysis time (0600 UTC). The AIRS data cover both land/ocean (5^#cod#x000b0;-50^#cod#x000b0;N, 83^#cod#x000b0;-54^#cod#x000b0;W) and the tropical/mid-latitudes, so the data cover the typical global region and are good for testing the methodology. The high spatial resolution MODIS cloud mask product was used for AIRS sub-pixel cloud detection (Li et al., 2004a). The high spatial resolution MODIS emissivity was used for AIRS sub-pixel surface characters, especially in desert and Gobi regions; the high resolution emissivity is important for atmospheric profiles (Yao et al., 2011). The retrieval was performed for clear-sky footprints only, including 5662 clear sky FOVs. The ECMWF analysis data were used as reference values (true) to test the AIRS profile retrievals. The ECMWF analysis data were spatially and temporally interpolated to the AIRS footprints [see (Liu et al., 2008) for more details on the collocation method].

The ensemble retrieval algorithm has four steps: (2) regression retrieval process for the original first guess——the so called "clear regression" first guess; (3) perturbation of the original first guess to generate the ensemble first guess; (4) nonlinear iterative physical retrieval to generate ensemble results; and (5) the optimal profile from the ensemble physical retrievals. Steps (2) and (4) are similar to other algorithms (Li et al., 2000; Li et al., 2008). The basic idea of the ensemble retrieval method is to run the retrieval process multiple times with different first guesses. Although the first guess is different, the physical retrieval algorithm should be able to bring all first guesses closer to the true state. Therefore, the ensemble physical retrievals should be clustered near the true state. Detailed information is described as follows.

2.2.1 Physical iteration retrieval algorithm

When scattering R' is neglected in the infrared spectra region, the clear spectrum of infrared radiance reaching the satellite from the Earth-atmosphere system can be expressed by

where R is satellite-observed spectral radiance; B is Planck function; #cod#949; is infrared surface emissivity; #cod#964; is the atmospheric transmittance from the satellite to the pressure level p; s is surface; and #cod#964;^*=#cod#964; _s²/#cod#964; is the downwelling transmittance. R is a nonlinear function of the atmospheric temperature profile, water vapor mixing ratio profile, ozone profile, surface skin temperature, and surface emissivity. By using the optimal solution methodology (Rodgers, 1976), a cost function is defined as follows:

J(X)=[Y_m−Y(X)]^TE⁻¹[Y_m−Y(X)]+(X−X_b)E^THE⁻¹(X−X_b). (2)

Here, X is the atmospheric profile to be retrieved; X_b is the first guess from regression for practical purposes; Y _m is the vector of observed bright temperature, m is the meaning of measurement; Y(X) is the vector of calculated bright temperature corresponding to the atmospheric profile X; E is the observation error covariance matrix that includes instrument noise and forward model uncertainty; and H is the background covariance matrix.

With the quasi-Newtonian iteration approach (Eyre, 1989), the physical retrieval can be obtained by

#cod#948;X_n+1=(F′^T_nE⁻¹F′_n−H⁻¹)⁻¹F′T_nE⁻¹(#cod#948;Y_n+F′_n#cod#948;X_n), (3)

where #cod#948; X_n=X_n-X_b; #cod#948; Y_n=Y _m-F(X_n); F' is weighting function; n is the iteration number, T means transpose; F(X) is the tangent model, which is the function of X; and X_n is the matrix at iteration time n. In the retrieval process, the spectrum of radiance is expressed as the weighting function of surface temperature, air temperature and water vapor ratio. Taking the empirical orthogonal functions representation, the number of atmospheric variables is reduced; the detail of the retrieval method was shown in a previous paper (Li, 1994).

2.2.2 The ensemble first guess generation

Originally, the first guess is obtained through regression. The SeeBor (Seemann et al., 2008) dataset containing atmospheric state (temperature/moisture/ozone), surface skin temperature and surface emissivity is used in the training. Synthetic AIRS radiances are calculated (with observation errors added) from the SeeBor database using the Stand-alone AIRS Radiative Transfer Algorithm (SARTA, Strow et al., 2003). SARTA has 101 vertical pressure layer coordinates from 0.05 to 1100 hPa. The calculations take into account the satellite zenith angle, absorption by trace gases (including nitrogen, oxygen, and carbon dioxide), water vapor (including the water vapor continuum) and ozone. The temperature and moisture sounding retrievals are produced when the regression coefficients are applied to the actual clear sky AIRS measurements. The advantage of this approach is that it does not need AIRS radiances collocated in time and space with atmospheric profile data; it requires only the historical profiles (e.g., SeeBor dataset). This regression algorithm is computationally efficient, numerically stable, simple and ideal for deriving the first guess for physical retrieval. In order to provide soundings with the required level of accuracy, proper treatment of surface emissivities in the forward radiative transfer model is very important. In the AIRS sounding algorithm, IR surface emissivity measurements sorted by ecosystem type are used to improve the surface characterization in the training data, and a more physical basis for assigning a skin temperature to each training profile is adopted (Seemann et al., 2008).

Figure 1.  (a) The first 15 temperature profile EV coeffiecients; (b) the first and second EVs (red E1 and blue thick lines E2), EVs 3-15 (thin lines, E3), and their combined EV (thick blue line); and (c) the 27 perturbations by the first two Evs and the combined EVs 3-15.

Because the physical retrieval is dependent on the first guess, it is important to ensure the ensemble first guess is evenly distributed. Eigenvectors (EV) have been used in the sounding retrieval to reduce the number of unknowns (Li et al., 2008); Fig. 1a shows the coefficients of 15 EVs. In this study, the 15 EVs were used to produce perturbation, which were added to the original first guess to produce perturbed first guesses, i.e., the 15 EV profiles shown in Fig. 1b. Because the computation increases with the number of EVs exponentially, only three temperature EVs were used for generating first guess in the retrieval experiments (Fig. 1b; thick lines). The perturbation using a combination of the three EVs can be written as

P= a#cod#923;₁E₁+b#cod#923;₂E₂+c#cod#923;₃E₃, (4)

where P is the perturbation of the first guess;#cod#923;₁,#cod#923;₂,#cod#923;₃ are the coefficients of the first three EVs, equal to 4, 10, and 0.8, as calculated by a sensitivity test; a,b,c all could have three values: -1, 0, +1, respectively. E₁,E₂ are the first two EVs; E₃ is a combination of EVs 3-15 in order to produce large perturbations. A simulation study indicated that the first two EVs and the combination of EVs 3-15 can provide a wide range of temperature profiles, as compared with a combination of only the first three EVs (Fig. 1b). Depending on the value of a,b and c, there are 3#cod#215; 3#cod#215; 3=27 perturbations. Figure 1c shows the 27 perturbations. These perturbations were added to the original first guess to form 27 ensemble first guesses.

2.2.3 Optimal profile selected using PDF

Twenty-seven first guesses yield 27 physical retrievals, or the ensemble retrievals. The PDF technique is used to select the optimal profile from the ensemble retrievals. For a random continuous variable x#cod#8712;(-#cod#8734;,#cod#8734;) the probability density function #cod#961;(x) satisfies the following conditions: #cod#961;(x)#cod#62;0,x#cod#8712;(-#cod#8734;,#cod#8734;), and #cod#x0222b;^#cod#8734;_-#cod#8734;#cod#961;(x)dx=1. The normal PDF uses the normal distribution of x, with a standard deviation of #cod#963; and an average value of #cod#956;; that is:

To use the PDF technique, one needs to specify the reference parameter. Because surface skin temperature has high accuracy, and it has a similar value for 27 retrieval profiles, while atmospheric temperature for 27 retrieval profiles has different values at the same levels. Therefore, the difference between the surface skin temperature and atmospheric temperature is chosen as the reference parameter. Because different levels have different retrieval quality, it is important to perform the PDF analysis for the temperature profile at different heights. In this study, eight reference parameters were chosen, and they were the differences between surface skin temperature and atmospheric temperature at eight pressure levels. The pressure levels were: 100 hPa, 200 hPa, 300 hPa, 400 hPa, 500 hPa, 700 hPa, 850 hPa, and 925 hPa. For every parameter, the PDF analysis finds the profile with the maximum probability in the 27 ensemble retrievals. This profile is chosen as the optimal profile at the corresponding pressure level. So in total, there are eight chosen optimal profiles. The next section further describes how to produce the final optimal profile from the eight optimal profiles.

2.2.4 Ensemble retrieval steps

The flow chart of the ensemble retrieval algorithm is shown in Fig. 2. It includes four steps. In the first step, the first guess is obtained by using statistical regression based on the regression coefficients. In the second step, perturbations are produced to generate 27 ensemble first guesses. In the third step, the nonlinear physical iteration retrieval is performed with the 27 ensemble first guesses. In the fourth step, the optimal profile is obtained from the 27 ensemble retrieval profiles.

Figure 2.  Flow chart depicting the ensemble retrieval algorithm of the temperature profile (T _s is surface temperature, T is temperature profile, h is pressure level, N _c is the number of pressure levels, T _m is MeanOpt profile, T _c is CombOpt profile, F is perturbations, C is coefficient, E is eigenvector, and T₀ is first guess).

The PDF technique finds eight optimal profiles. It is unknown which one of these eight profiles best represents the true state. However, each of these profiles has a certain layer considered optimal. One way to generate the final optimal profile is to combine different layers from the eight profiles to form a single profile (hereafter called the "CombOpt" profile). Because each PDF analysis uses a certain height as reference parameter, the chosen profile is expected to have a better retrieval around that height. This piece of the temperature profile is combined with other pieces from different PDF analyses to form a single temperature profile. Another option is to take the average of all eight selected profiles to form a single profile (hereafter called the "MeanOpt" profile). Experiments were conducted to determine which option is better.

Figure 3 shows ensemble temperature retrieval results of two AIRS granules from different perturbations using the ECMWF analysis as the reference (true). The perturbation method reduces the AIRS temperature retrieval standard deviation (STD) by 0.01-0.15 K (Fig. 3a), the temperature retrieval bias by 0.05-1 K (Fig. 3b), and the temperature root-mean-square error (RMSE) by 0.05-0.7 K (Fig. 3c). Compared with the original retrieved profile using the original first guess, the ensemble temperature retrieval STDs are very close to the original STD; however, absolute values of most of the ensemble retrieval biases are less than the original retrieval bias (most of the lines are closer to zero). Perturbation retrieval RMSEs are distributed on both sides of the original RMSE, and most of the perturbation retrieval RMSEs are less than the original retrieval RMSE. Figure 3 shows that ensemble retrievals can reduce STD, bias and RMSE at some pressure levels, which mainly depend on the error of first guess and ensemble perturbations.

Figure 3.  Ensemble physical results by using 27 ensemble first guesses of two granules on 2 Mar 2012 (thick line is the original profile using original first guess, it is the same as pt14; thin lines are perturbation profiles by 27 perturbations added in original first guess).

Eight profiles were selected with the maximum probability using the PDF according to eight parameters (see section 2.2.3). The CombOpt and MeanOpt profiles were derived from the eight selected profiles (see section 2.2.4). By comparing those eight profiles with the original retrieval profile using ECMWF analysis as the reference, it can be seen that the selected eight profiles have an overall smaller bias and RMSE. This trend is more evident at the levels where the parameters are used for the PDF analysis (not shown). This result confirms that the selected eight parameters are effective and valid. The MeanOpt profile and CombOpt profile were also compared with the ECMWF analysis and the original retrieval profile. Both the MeanOpt profile and CombOpt profile have an overall smaller bias and RMSE. Comparing the MeanOpt profile and CombOpt profile, it can be seen that MeanOpt profiles are overall better than the CombOpt profiles, except for pressure layers between 200 and 300 hPa.

Figure 4 shows the difference of STD, bias and RMSE between the eight PDF selected profiles/the MeanOpt profile/the CombOpt profile and the original retrieval (former minus latter). The bias difference is the difference of the absolute values; a negative value means an improvement. Some STD of the selected profiles shows improvement between 200 and 350 hPa. The improvement is about 0-0.08 K, with the maximum improved value of 0.08 K around the level of 300 hPa. The MeanOpt profile and CombOpt profiles are similar to the eight selected profiles. The PDF selected temperature bias is improved in most layers, with the improved values ranging from 0.1 to 0.69 K. There are three local maximum improved values, 0.55 K, 0.69 K, and 0.5 K, at the pressure levels of 150 hPa, 460 hPa, and 660 hPa, respectively. MeanOpt profiles have more improvement between 300 hPa and the surface, as well as the layer from 120 hPa to 200 hPa. CombOpt profiles have improvement at almost all pressure layers. The pressure layers with local maximum improvement of MeanOpt and CombOpt profiles are the same as those with a maximum improvement of selected profiles.

Figure 4.  Improvements (negative values) of (a) STD, (b) bias (b), and (c) RMSE of the ensemble retrieval (figure shows the difference between the ensemble retrieval profile and original profile; negative values mean improvement of the ensemble retrieval, while bias improvement is equal to the absolute value of the profile).

The eight PDF selected RMSEs are improved between 150 and 200 hPa and below 400 hPa; the improved values are up to 0.3 K. The improved peak values are 0.29 K, 0.23 K, 0.13 K and 0.1 K at the pressure levels of 140 hPa, 460 hPa, 660 hPa and 980 hPa, respectively. The CombOpt profiles are similar to the eight selected profiles; the RMSE are improved between 150 and 320 hPa and below 400h Pa, with the maximum improved value being 0.16 K around 460 hPa. The MeanOpt profile shows improvement at all pressure levels from 100 to 1000 hPa, with local improved peak values being 0.29 K, 0.23 K, 0.13 K and 0.1 K at the levels of 140 hPa, 460 hPa, 660 hPa, and 980 hPa, respectively. Furthermore, the improved values are larger than CombOpt profiles, except for the layer between 200 and 300 hPa. Comparing the STD, bias and RMSE of the MeanOpt profiles with the CombOpt profiles, it can be seen that the MeanOpt profiles are more stable and have greater improvement than the CombOpt profiles. Therefore, the MeanOpt technique was chosen to derive the optimal results from the eight selected profiles. Figure 5 shows a comparison of the statistical results of the MeanOpt profile with the original profile. The results show that the MeanOpt profile has smaller bias and RMSE compared with the original profile, but the STD of the MeanOpt profile is similar to the original profile. These results show that the ensemble retrieval technique is effective in reducing the bias and therefore RMSE.

The bias, RMSE and STD of the final optimal profile of ensemble retrieval depend highly on the eight PDF selected profiles. Figure 6 shows the occurrence frequency distribution of the 27 ensemble members selected by the eight PDF conditions, corresponding to the eight pre-defined pressure levels from 100 to 925 hPa, with level numbers defined as I, II, III, IV, V, VI, VII, and VIII. There are 27 perturbation members selected by the PDF for every level. For each level, the 27 perturbations can be divided into three parts, including the first part (the first nine perturbations), the middle part (from 10 to 18 perturbations), and the last part (from 19 to 27 perturbations) in sequential order. These three parts correspond to -1, 0 and 1 for the coefficient of the first EV in Eq. (5). It is found that the eight PDF frequencies are not the same. The highest frequencies of level numbers III and VIII mostly cover the first part and the middle part; the highest frequency of level number IV occurs mostly in the middle part; the highest frequency of level number II occurs mostly in the middle and last part; and the highest frequency of level numbers I, V, VI and VII occur mostly in the last part. This shows that perturbation is sensitive to the first EV. According to the eight level numbers and their corresponding pressure levels, it can be seen that the perturbation of the negative coefficient of the first EV is favorable for small retrieval errors and bias for pressure layers around 300 hPa (Fig. 6 III) and 925 hPa (Fig. 6 VIII); the positive coefficient of the first EV is favorable for small retrieval errors and bias for pressure layers around 100 hPa (Fig. 6 I ), 500 hPa (Fig. 6 V), 700 hPa (Fig. 6 VI), and 850 hPa (Fig. 6 VII). The zero coefficient of the first EV is favorable for small retrieval errors and bias for pressure layers around 200 hPa (Fig. 6 II) and 400 hPa (Fig. 6 IV). In summary, the first EV contributes more than the other two EVs.

Figure 5.  Comparison of the MeanOpt profile (red lines) with the original profile (blue lines) physical results of two granules on 2 Mar 2012 (thick line is bias; thin line is STD; thick dashed line is RMSE).

Figure 6.  Distribution of the frequency of ensemble members selected by the eight PDFs.

The different frequency histograms of different PDF numbers show that each perturbation is effective in some layers, and less effective in others. The eight selected pressure levels have high uncertainty in their results, which is important for the accuracy of the ensemble retrieval. In order to minimize the effects and uncertainty of the eight selected profiles, and maximize the available information on the selected pressure levels, the MeanOpt profile of the eight selected profiles was used. The above case study also shows that the MeanOpt profile is more stable than the selected profiles and the CombOpt profile, and that is the reason why the MeanOpt profile was regarded as the final optimal result. According to the above analysis, it is concluded that the MeanOpt profiles and CombOpt profiles rely strongly on the eight PDF selected profiles, which are decided by perturbations, the accuracy of the original profile, the PDF method, and so on.

In the above study, the ensemble results were evaluated by ECMWF analysis. However, it should be noted that the ECMWF has its own errors, as it is not observation data but reanalysis data. In order to further test the improvement of the ensemble retrieval, and analyze the information contribution from various errors to the retrieval, a simulation study was performed. In the simulation, all the NOAA-88 profile data samples (http://cimss.ssec.wisc.edu/itwg/groups/rtwg/profiles.html) were used for training, while real values of the profile were used for testing the retrieval results. The perturbation errors added in the simulation were the same as the errors used in the operational algorithm for the physical retrieval. It was found that the perturbation results reduced the temperature RMSE by 0.1-0.4 K between 100 hPa and 300 hPa and 0.15-0.5K below 300 hPa (not shown). Perturbations could decrease the RMSE at some levels in the simulation.

The PDF selected profiles were performed according to the temperature difference between the eight pressure levels (from 100 to 1000 hPa) and the surface temperature (section 1.3). The MeanOpt profile and CombOpt profile were also created according to the eight selected profiles in the simulation. The statistical results of these profiles minus the original profile are shown in Fig. 7. The RMSE of the PDF selected profiles shows improvement between 300 and 800 hPa; the maximum improved value is 0.18 K at 500 hPa and 600 hPa. The RMSE of the MeanOpt profile has improvement below 150 hPa, improving from 0.05 to 0.14 K; the maximum improvement value is 0.14 K at 600 hPa. The RMSE of the CombOpt profile also has improvement below 180 hPa, improving from 0.02 to 0.12 K; the maximum improvement value is 0.12 K at 600 hPa. Just like the results of the ensemble retrieval from real satellite data, the MeanOpt profile shows an obvious improvement, and it is more stable than the CombOpt profile. The MeanOpt profiles and CombOpt profiles rely strongly on the eight PDF selected profiles, which are decided by perturbations, the accuracy of the original profile, the PDF method, and so on.

The depth of the boundary layer over Gobi and desert regions in Northwest China can be more than 4000 m (Zhang et al., 2004, 2011), which results in frequent regional disasters, such as hail storms and sandstorms. The profile over Northwest China is therefore very important for predicting weather disasters, although it is difficult to obtain due to a limited number of observation stations. By using AIRS data for July 2007 (1 July to 15 July; 10 clear days, five cloudy days), atmospheric profiles of ten clear days were retrieved, and MeanOpt profile STD, RMSE and bias were analyzed using observed synchronous radiosonder data, which are shown in Fig. 8. Dust had significant impact on the retrieval (Yao et al., 2012), which is considered in the radiation transfer model. The STD of the MeanOpt profile is less than 1.2 K below 400 hPa, and it is between 1.2 K and 1.8 K below 300 hPa and 400 hPa. The bias of the MeanOpt profile is less than 0.5 K between 300 hPa and 650 hPa, and it is between 1.5 K and 0.5 K below 650 hPa. The RMSE of the MeanOpt profile is less than 1.5 K between 380 hPa and 700 hPa, and it is between 1.5 K and 2 K below 700 hPa and above 380 hPa. Compared with the original retrieval results (not shown), it was found that: the STD and bias of the MeanOpt profile showed improvements below 200 hPa (the maximum improvement value of STD was 0.35 K at 600 hPa and the maximum improvement value of STD was 0.28 K at 800 hPa), and the RMSE showed improvement below 200 hPa (improvement was from 0.05 to 0.44 K and the maximum improvement value was 0.44 K at 600 hPa). Comparing ensemble retrieval of AIRS data with MODIS retrieval results in previous research (Zhang and Zhang, 2008), the ensemble retrieval of AIRS is also better than MODIS results, which means that AIRS data and the ensemble retrieval method potentially have the ability to retrieve atmospheric profiles over Gobi and desert regions.

Figure 7.  Improvements (negative values) of RMSE of the ensemble retrieval in the simulation.

Figure 8.  MeanOpt profile of STD, bias and RMSE over Dunhuang Station in Northwest China in 2007 by comparing with synchronous radiosonde data.

Ensemble first guesses were formed using EVs as perturbation to update the first guess from regression. Then, the nonlinear physical iterative method was used to retrieve atmospheric profiles and obtain ensemble physical retrieval profiles. PDF was used for selecting the optimal profile from these ensemble retrieval results. The results showed that:

Perturbation is an important step for the ensemble retrieval technique; a reasonable perturbation will result in better retrieval. The perturbation's contribution to the ensemble retrieval also relies on the accuracy of the original first guess. Atmospheric temperature profiles can be decomposed using EVs, and the total variation of the first 15 EVs was more than 90%. They were combined to generate 27 perturbations so as to generate the ensemble first guess.

The PDF technique was used to select the optimal profile. Eight parameters of the temperature difference between the atmosphere and the surface skin temperature were used for the PDF analysis. Eight optimal profiles out of the 27 profiles were retained. The final optimal profile was obtained by using the technique of the MeanOpt profile and CombOpt. The MeanOpt showed greater improvement than the CombOpt profile at most pressure levels, and the results from it were considered as the optimal profile of the ensemble retrieval.

Besides the first guess, many factors affect the physical retrieval quality. The ensemble retrieval is a new method developed to minimize the impact from the first guess. The case study also showed that ensemble retrievals had larger than 0.14 K of (RMSE) improvements at many pressure layers below 400 hPa and between 150 hPa and 320 hPa. However, at the other pressure layers (from 320 hPa to 400 hPa, above 150 hPa), the ensemble results did not show improvement, which could be attributed to a number of possible reasons, including: (2) the PDF could not obtain the best information; and (3) the perturbations could not cover the accuracy range of all profiles due to using limited EVs; in other words, the first 15 Evs did not have any detrimental consequences on their results, or limited EVs could not cover all the atmospheric conditions because EVs not only have vertical spatial characteristics, but also represent horizontal spatial ranges. Future efforts to improve the algorithm will include enhancing the perturbation application and testing, particularly over different surfaces. In addition, this perturbation for ensemble retrieval is based on the first guess using regression methodology. It is also appropriate for the first guess from other methodologies, such as the neural network scheme in the operational retrieval algorithm (version 6) for the AIRS instrument, which will be performed in our next study.

Different perturbations will result in different improvements of the retrieval. In this study, in order to emphasize the ensemble methodology and perturbation of the first guess of the temperature profile, water vapor was assumed to remain the same as the original water vapor during the ensemble retrieval. Undoubtedly, the uncertainty in the water vapor content is an important factor affecting the temperature profile accuracy. Therefore, if we provide a reasonable perturbation according to these uncertainty factors, improvement in the ensemble retrieval is possible. Water vapor perturbation can be described by water vapor EVs; improving the water vapor retrieval is our next goal. In addition, a reasonable composition of EVs should be improved so as to improve model speed.

The ensemble retrieval results showed that the temperature bias and RMSE were improved at most of the layers compared with the original AIRS physical retrieval. Both the retrieval of the simulation and application in Northwest China showed improvements by using ensemble retrieval. This has significance to the processing of AIRS data using the ensemble retrieval technique.