HTML
-
In the radar reflectivity factor measurements, the echoes appearing below the cloud are considered clear-air echoes, so the determination of the cloud base height is particularly important. The intention in this paper is to combine the MMCR and laser ceilometer to determine the cloud base height, and then use the L-band radiosonde and the all-sky camera to assist in judging the cloud information to verify the reliability of the estimated cloud base height. In this experiment, the distance between any two of the four instruments is within 200 m and their deployment locations are shown in Fig. 3. The four instruments are introduced in the following subsections.
-
The MMCR used in this study is located in the southern suburb observatory in Beijing. It operates at 8.6 mm wavelength and 35 GHz frequency. Its antenna scans the sky in the vertical direction and its maximum height of observation is 15 km. The vertical spatial resolution is 30 m and the temporal resolution is 60 s. The major technical and subsystem parameters are listed in Table 1. The MMCR uses three vertically pointing observation modes to detect simultaneously, and each observation mode corresponds to a different pulse width. Table 2 shows the specific parameters of the three observation modes. Figure 4 is a schematic diagram of the MMCR’s system configuration (Nashashibi and Ulaby, 2001), in which both indoor and outdoor parts are included.
Item Technical specifications Radar system Coherent, pulsed doppler, solid-statetransmitter, pulse compression Radar frequency 35 GHz (Ka band) Detecting parameters Reflectivity factor (Z), vertical velocity (V), spectral width (W) Transmitter peak power Pt ≥ 50 W Antenna subsystem Antenna type: Cassegrain Antenna diameter: 2.4 m Beam width: 0.25° Range of detection Height: 0.120–15 km Temporal resolution: 60 s (adjustable) Height resolution: 30 m Table 1. Major technical parameters for the MMCR.
Pulse width 2 μs 5 μs 20 μs Minimum range 360 m 960 m 3060 m Maximum range 2460 m 5010 m 18 510 m Dwell time 4 s 4 s 4 s Number of range bins 30–100 65–200 210–617 Range sample volume spacing 30 m 30 m 30 m Table 2. Specific parameters of three observation modes for the MMCR.
The base data and power-spectrum data from the MMCR are used in this study. The base data comprise the reflectivity factor, velocity, and spectral width. The power-spectrum data are obtained from Fast Fourier Transform of the time-domain signal, and reflect the echo power distribution of different Doppler velocity. Each radar range corresponds to a power spectrum, which is composed of 256 spectral points, and each spectral point corresponds to a Doppler velocity. The power-spectrum data are closely related to the microphysics and dynamics in the cloud, and are critical for extracting the features of radar echoes.
-
The laser ceilometer used in this study is Vaisala CL51, which is manufactured by Vaisala in Finland and certified by the CMA. Its maximum detection height is 15 km and its repeat frequency is 10 kHz, with a spatial resolution of about 5 m and temporal resolution of 60 s.
The laser ceilometer emits continuous laser pulses vertically upwards. When the laser pulses enter and pass through the clouds from the cloud base, strong scattering at the cloud border will take place and the backscattered signal is received (He et al., 2011). Thus, the laser ceilometer can be used to assist in determining the cloud base height.
-
The all-sky camera is designed to automatically monitor cloud conditions during daytime. Its camera above the instrument looks down at a hemispherical mirror with a heating device and takes the image of the sky reflected in the mirror. The average operation time of the camera used in this study is 0600–2000 Local standard time (LST hereafter). The all-sky images, with a resolution of 2848 × 4288 and 24 true colors, are taken every 10 min.
-
In meteorological observation, the radiosonde system is important for upper-air observation. The maximum detectable height of the L-band radiosonde system is 30 km. The radiosonde sounding data used in this study are from the southern suburb observatory in Beijing, which are observed twice a day at 0715 and 1915 LST.
-
Discriminating clear-air echoes from cloud echoes becomes extremely challenging when the echoes mix in the lower level of the atmosphere, meaning it is necessary to first check the cloud detection ratios (the ratio between the time when one instrument detects the cloud and the time when both instruments are in operation) of the MMCR and the laser ceilometer for detecting clouds at different heights. The observed clouds are divided into three categories according to their base height. Clouds with a cloud base height of less than 2.5 km are low clouds, 2.5–4.5 km are middle clouds, and higher than 4.5 km are high clouds. Table 3 shows the cloud detection ratios of the MMCR and the laser ceilometer during selected observation times when both instruments were in operation.
Cloud type Selected observation time
when both instruments are in operation (min)Total time when MMCR detects
cloud (min)Total time when laser-ceilometer detects cloud (min) Cloud detection ratio of MMCR (%) Cloud detection
ratio of laser-ceilometer (%)Low cloud 420 265 402 63.1 95.7 Middle cloud 4394 3234 3086 73.6 70.2 High cloud 7808 6502 4115 83.3 52.7 Table 3. Cloud detection ratio of the MMCR and the laser ceilometer in selected observation periods for clouds of different heights.
It can be seen from Table 3 that the detection of high cloud, i.e., the cloud detection ratio of the MMCR, is larger than that of the laser ceilometer, which may be caused by the thin cloud particles of high clouds when backscatter to the laser ceilometer becomes weak and less detectable. For the detection of middle clouds, the MMCR and the laser ceilometer perform similarly, with only a slightly higher detection ratio for the MMCR. For low-cloud detection, the cloud detection ratio of the laser ceilometer is significantly higher than that of the cloud radar. Therefore, it is necessary to combine both instruments to determine the actual cloud base height accurately.
In accordance with commonly used indicators of statistical significance, we employ the t-test method for significance testing, with the level of significance (α) set to 0.05. The cloud base height detected by the MMCR and the laser ceilometer from 1 December 2015 to 31 January 2016 are compared, as shown in the scatterplot in Fig. 5, and a clear linear correlation is shown, with a significant correlation coefficient of R = 92%, which is greater than the t-test critical correlation coefficient and thus is significant. Therefore, the results show that the cloud base height observed by the MMCR and the laser ceilometer is consistent.
-
During 1–18 January 2016 and 1 April to 31 June 2016, a total of 109 days, there were 248 sounding observations from the L-band radiosonde. However, during the observation periods we selected, only 42 observations matched the MMCR observations. Similar as above, comparison of the performances of the MMCR and the L-band radiosonde on the detection of the cloud base indicated that they have consistent cloud base height observations, which is shown in Fig. 6. An example case analysis is presented below.
Figure 7a shows the reflectivity factor profile of the MMCR observed at 0701 LST 3 June 2016. Figure 7b shows vertical profiles of temperature and relative humidity observed by the corresponding radiosonde. As shown in Fig. 7a, the MMCR detected a cloud base at 6150 m, while the radiosonde detected a cloud base at 6517 m. They agree with each other reasonably well.
-
According to section 3.5 and the linear correlation shown in Fig. 5, the threshold of difference between the heights of the laser ceilometer and the MMCR is selected within 1000 m, which is also consistent with the results of Oh et al. (2016) and Zhao et al. (2017). With reference to the above consistent analysis among the instruments, as well as the study of Clothiaux et al. (2000), the benchmark samples for clear-air echoes can be derived: For a given MMCR observation, the cloud base height measured by the laser ceilometer at the same time is marked on the MMCR echoes. If the difference between the cloud base heights measured by the two instruments is within 1000 m, the cloud base height obtained by the MMCR is used as an actual cloud base height. Then, by analyzing the cloud information from the radiosonde data and examining the pictures from the all-sky camera, the reliability of the previously determined cloud base is further verified. If the result is reliable, the obtained cloud base height is used as the benchmark height. Echoes that appear above the benchmark height are considered as cloud echoes, and those appearing below the benchmark height are considered as clear-air echoes.
3.1. MMCR
3.2. Laser ceilometer
3.3. All-sky camera
3.4. L-band radiosonde
3.5. Detectability comparison between the MMCR and laser ceilometer
3.6. Comparison of the MMCR and L-band radiosonde
3.7. Benchmark for clear-air echoes
-
The flowchart in Fig. 8 shows the procedures of the clear-air-echo recognition algorithm. The procedures include feature extraction, feature selection, and clear-air-echo recognition.
Spectral features, including the power spectral kurtosis and spectral width, are effective in distinguishing cloud and clear-air echoes (Luke et al., 2008). The extraction of the spectral features is accomplished in three steps: (1) collection of clear-air and cloud echo samples; (2) extraction of spectral features of the samples; and (3) establishment of a set of features.
As introduced in section 3, the data from September 2015 to August 2016 are used in clear-air-echo recognition. After initial data screening, 54 834 radar echo samples can be obtained to train the neural network. Among them, there are 11 706 clear-air echoes and 43 128 cloud-echo echoes.
From the analysis of the above established dataset, it is found that most of the clear-air echoes display a concentrated area of high-power spectral density along the velocity spectrum. They are multi-peaked with sharp fluctuations and narrow spectral width, compared to that of cloud echoes. Due to the existence of noises, the main spectrum, which might contain meteorological signals, must be extracted first, and then the number of extreme points contained in this main spectrum is calculated. Assuming the radar noise is Gaussian white noise, the piecewise averaging method is used to remove the unwanted noises (Fabry et al., 1997). The specific steps of feature extraction are as follows:
(1) The entire spectrum is divided into 16 segments and the average value of each segment is calculated;
(2) The minimum mean value is used as the noise level;
(3) The signal-to-noise ratio of each spectral point is calculated;
(4) The threshold value of the signal-to-noise ratio is set, the points below this threshold are removed, and the main spectrum is retained.
Figures 9 and 10 show an example of the extracted main spectrum from clear-air echoes and cloud echoes. In order to better distinguish the clear-air echoes and cloud echoes, as many features as possible are extracted for the main spectrum. Presently, 28 features are extracted and are listed in Table 4 in section 4.2.
Feature Weight 1 Reflectivity factor Z 0.213 2 Doppler velocity V 0.028 3 Spectral width W 0.026 4 Spectrum width (SW) 0.049 5 Kurtosis (K) 0.048 6 Skewness (S) 0.035 7 Number of spectral peaks (NSP) 0.007 8 Location of spectral peak (LSP) 0.009 9 Mean of main spectrum (MMS) 0.123 10 Horizontal difference of reflectivity factor (Zx) 0.018 11 Horizontal difference of doppler velocity (Vx) 0.004 12 Horizontal difference of spectral width (Wx) 0.003 13 Horizontal difference of spectrum width (SWx) 0.001 14 Horizontal difference of kurtosis (Kx) 0.007 15 Horizontal difference of skewness (Sx) 0.023 16 Horizontal difference of number of spectral peaks (NSPx) 0.026 17 Horizontal difference of location of spectral peak (LSPx) 0.011 18 Horizontal difference of mean of main spectrum (MMSx) 0.007 19 Vertical difference of reflectivity factor (Zy) 0.010 20 Vertical difference of doppler velocity (Vy) 0.015 21 Vertical difference of spectral width (Wy) 0.001 22 Vertical difference of spectrum width (SWy) 0.001 23 Vertical difference of kurtosis (Ky) 0.004 24 Vertical difference of skewness (Sy) 0.013 25 Vertical difference of number of spectral peaks (NSPy) 0.019 26 Vertical difference of location of spectral peak (LSPy) 0.007 27 Vertical difference of mean of main spectrum (MMSy) 0.005 28 Height (H) 0.296 Table 4. The weight of each feature.
Figure 9. Extracted main spectrum (red) from clear-air echo at 450 m height at 1414 LST 19 April 2016.
Figure 10. Extracted main spectrum (red) from cloud echo at 4.8 km height at 1030 LST 19 April 2016.
For all the extracted features, histograms are made and analyzed. For example, the reflectivity factor histogram is shown in Fig. 11, which shows that separating the cloud echoes from the clear-air echoes by the histogram of the reflectivity factor is possible. In most cases the reflectivity factor values of the two objects are obviously different, while in fewer cases the reflectivity factor values of the two objects are mixed, which makes it difficult to separate them with a high degree of accuracy. Therefore, by analyzing the degree of separation along the histograms of the features, the capability of each feature for distinguishing the two types of echoes can be evaluated.
-
In this study, there are 28 features (Table 4) extracted from clear-air echoes and cloud echoes. The purpose of constructing the sample feature set is to send the features of the clear-air echoes and cloud echoes to the neural network model for training, and let the algorithm model automatically learn the corresponding characteristics of the clear-air echoes and cloud echoes so that more accurate classification of clear-air echoes and cloud echoes can be achieved. The number of features is large and there is often redundancy between them, whose mutual information value is large. At the same time, the correlation between some features of the sample is not high. Therefore, if all the features are used for classification, the effect is not good and it also increases the complexity of subsequent learning algorithms. Therefore, it is necessary to optimize the feature set and to form a feature subset. In this study, the Relief algorithm (Amjady et al., 2010) is used to achieve the feature selection.
The Relief algorithm is a nonlinear instance-based feature selection technique. In this study, the problem is to distinguish which class each sample belongs to—clear-air echoes or non-clear-air (or cloud in this case) echoes. Based on the correlation between the features and the classes, weights are calculated and assigned to the features of each sample. The weight reflects the ability of each feature to distinguish between the two classes.
Let
$X = \{ {x_1},{x_2}, \cdots,{x_n}\} $ represent all$n$ sample points including both clear-air and cloud echoes, and${x_i} = {[{x_{i,1}},{x_{i,2}}, \cdots,{x_{i,N}}]^{\rm{T}}}$ represent the feature vector (a total of$N$ features) of sample${x_i}(i = 1,2,...,n)$ . For a given randomly selected sample xi, the algorithm will obtain the nearest sample to sample xi as measured by the Euclidian distance in the feature space. The nearest sample of sample${x_i}$ in the clear-air echo class (near hit) is denoted as$H(x)$ and the one from the opposite class (near miss) is denoted as$M(x)$ . The weight for the$j{\rm{th}}(j = 1,2,...,N)$ feature of sample${x_i}$ ,${W_{i,j}}$ , is initialized as zero and updated during each iteration by the following:After a given number of iterations, the weight of all candidate features will be updated based on the selected samples. The weight of a feature represents its capability in classifying the samples based on that feature.
Using the above-mentioned Relief algorithm to perform a validity analysis on all the features in the sample feature set, the weight value of each feature can be obtained, which is shown in Table 4. In order to see the difference between each feature weight more visually, the results are shown in a bar chart in Fig. 12. Obviously, there are three features with larger weights than others, and their corresponding features are height
$(H:0.296)$ , mean of main spectrum$({\rm{MMS}}:0.123)$ and reflectivity factor$(Z:0.213)$ , which form the optimal feature subset. This optimal subset of$H$ , MMS, and$Z$ will ultimately be used to achieve the classification of clear-air echoes and cloud echoes. -
In this study, we use a feed-forward neural network architecture and the back propagation of error training algorithm (Bai et al., 2016). Figure 13 shows the framework of the neural network algorithm. The algorithm inputs the data from the input layer, processes it in the hidden layer, outputs the current result from the output layer, then enters the backward propagation, calculates the error between the predicted value and the true value, and sequentially derives the forward direction for each neuron. Then, the eigenvalues are updated to minimize the cost function by means of gradient descent, and the weights and thresholds of the connected neurons are updated according to the errors of the hidden layer neurons. This iterative process will be carried out in a cycle until the pre-set stopping conditions are reached. The details of the classification of cloud echoes and clear-air echoes using this algorithm are as follows.
The neural network includes one input layer, two hidden layers, and one output layer. The first hidden layer contains eight neuron units, the second hidden layer contains six neuron units, and the output layer has two neuron units. We take the three optimal features extracted earlier as inputs for the neural network, and the output is a vector of continuous values with a component for the possible classification. The output ranges from 0.0 to 1.0, representing in the meantime the confidence of a membership classification in each class, and will be treated as the input for the decision criterion. The decision criterion is a method of interpreting the neural network output vector and converting it into a discrete decision state. In this study, we use a “winner-takes-all” function to choose the output with the highest value as the classification result. The two neural network outputs are either clear-air echoes or cloud echoes.
4.1. Procedures of the algorithm
4.2. Feature selection
4.3. Identification algorithm
-
This section presents the results by applying the above algorithm to the MMCR echoes. Due to instrument maintenance, the observation data of the four instruments are not continuous. At the same time, there are almost no clear-air echoes at the bottom of cloud radar echoes in winter in Beijing. Therefore, not every day’s radar echoes can be used as a test sample to verify the effectiveness of the recognition algorithm. The MMCR’s data from March 2016 to August 2016 are used, and there are 82 days of available data. The data samples can be divided into two different weather scenarios. The first is that the regions of cloud echoes and clear-air echoes are separate from each other. The second scenario is when regions of low cloud, clear-air echoes, and precipitation are all intermingled together at low levels of the atmosphere. The first-scenario radar echoes belong to simple weather conditions, and there are 54 days. The second-scenario radar echoes belong to complex conditions, and there are 28 days. In both scenarios, the number of cloud-echo and clear-air-echo points are counted before and after the algorithm recognition is performed, from which the clear-air-echo recognition rate is calculated.
Table 5 shows the statistics of the clear-air-echo recognition rate when the pixel points on the radar echoes are counted. For the simple radar-echo scenario, there are 513 409 points being actual clear-air echoes and 4 821 474 points actual cloud echoes. After applying the recognition algorithm, the number of clear-air echoes being recognized is 492 165 and the number of cloud echoes is 4 842 718 with 21 244 clear-air echo points being recognized as cloud. Therefore, for these simple weather conditions, the clear-air-echo recognition rate is 95.86%. Similarly, in the complex weather scenarios, there are 420 896 points being actual clear-air echoes and 3 321 780 points actual cloud echoes. From the algorithm recognition, 371 998 points are recognized as clear-air echoes and 3 370 678 points as cloud echoes. Thus, the clear-air-echo recognition rate is 88.38% in the complex weather conditions.
Number of actual points before recognition Number of points after recognition Radar echo scenarios Clear-air echo Cloud echo Clear-air echo Cloud echo Clear-air echo
recognition rateSimple 513 409 4 821 474 492 165 4 842 718 95.86% Complex 420 896 3 321 780 371 998 3 370 678 88.38% Table 5. The clear-air-echo recognition rate.
For further evaluation of the new recognition algorithm, we used three cases that include simple and complex weather conditions. The data of 3 April 2016 were representative of a simple weather condition, while the data of 19 April and 27 April 2016 were representative of complex weather conditions. The three cases were excluded from the algorithm training, and are investigated in detail to test the validity and efficiency of recognition. The clear-air echoes and cloud echoes of the two days have already been verified with the data from the MMCR, the laser ceilometer, and the all-sky camera.
-
Figure 14a shows the observed reflectivity factor on 3 April 2016. The day was a cloudy day, and the regions of cloud and clear-air echoes were separate from each other. The echoes at heights above 3 km are cloud echoes while the echoes at altitudes below 3 km are clear-air echoes.
Figure 14. (a) Reflectivity factor of 3 April 2016. (b) Result of reflectivity factor after filtering clear-air echoes.
Because of the radar self-interference, the echo image displays some noises, such as the thin strips of echoes shown in Fig. 14a. Thus, a 3 × 3 convolution kernel of mean is used to smooth the radar echo images after filtering the clear-air echoes, and the outcome of the final filtered clear-air echoes is shown in Fig. 14b.
Overall, Fig. 14b illustrates a very satisfactory identification result, whereas the clear-air echoes are filtered out completely. By statistical analysis, there are 98 162 echoes in the original radar returns, and the number of clear-air echoes is 8931. The number of clear-air echoes in the original image that was correctly identified as clear-air echoes by the recognition algorithm is 8845, indicating that about 99.14% of clear-air echoes are successfully identified and eliminated.
-
Figure 15a shows the reflectivity factor of stratiform clouds and clear-air echoes on 27 April 2016. We saved the reflectivity factor data samples of Fig. 15a and sent them to the trained BP neural network system. The BP neural network system recognizes the samples and divides all the samples into clear-air echoes and cloud echoes. Finally, we filtered out the clear-air echoes and showed only the samples of cloud echoes as shown in Fig. 15b. In this experiment, the new recognition method is very effective in terms of the overall recognition effect. The total number of samples of the echo points is 110 348 and the number of clear-air echo points is 22 376. Through the new algorithm in this paper, 91.25% of clear-air echoes are successfully identified. Comparing Figs. 14b and 15b, the size and thickness of clouds have no effect on the accuracy of the algorithm of identifying clear-air echoes.
-
Figure 16a shows a complicated distribution of reflectivity factor on 19 April 2016. In Fig. 16a, echoes of heights above 3 km are cloud echoes while echoes of heights below 3 km are mixed with precipitation echoes, low cloud echoes, and clear-air echoes. It is known that the echoes from the elliptical circle in the area below the figure are cloud echoes or precipitation echoes by combining the laser ceilometer, the L-band radiosonde, and the all-sky camera. From the statistics, there are 118 635 echoes in the original radar images, and the number of clear-air echoes is 20 219. Like the processes of the filtering and recognition algorithm of clear echoes used for the previous case, the result after filtering out the clear-air echoes is shown in Fig. 16b. After recognition, there are 1887 clear-air echoes left, indicating a 90.67% recognition accuracy. Comparing Figs. 16a and b, we can see that some of the three echo regions of the marker are filtered out while the cloud echoes and precipitation echoes are retained. The part enclosed by the ellipse in Fig. 16b is a clear-air echo that is mistaken for cloud echo. The reason is that the reflectivity factor of the echo is too close to the reflectivity factor of the low cloud echo. Therefore, it is necessary to extract more features that can effectively distinguish cloud echoes from clear-air echoes.
Figure 16. (a) Reflectivity factor of 19 April 2016, in which the circled echoes are confirmed clear-air echoes. (b) Result of reflectivity factor after filtering clear-air echoes.
Through the above case studies, our automatic recognition algorithm can effectively identify the clear-air echoes whether in simple cases or in the complicated situation of cloud-water mixing.