Identification of Convective and Stratiform Clouds Based on the Improved DBSCAN Clustering Algorithm

Rainfall systems are generally composed of cumulus or mixed clouds, and convective clouds are embedded in large-scale stratiform cloud precipitation. Severe weather mainly occurs in convective cloud areas, but the duration and amount of precipitation are determined by the stratiform cloud area (Zhao and Qian, 2008; Zhou et al., 2010; Zhong et al., 2007). Because the generation mechanisms, evolution, dissipation, and moving speed differ between convective and stratiform clouds and their contribution to heat change in the atmosphere also differs, it is beneficial to better understand their mechanisms and improve precipitation estimation ability while identifying and classifying clouds in rainstorms as convective or stratiform systems. This can effectively enhance the accuracy of nowcasting, monitoring and early warning of disastrous weather, and improve aerospace operation and weather modification (Bao, 2007; Xiao and Liu, 2007; Zhong et al., 2007).

To date, many studies have been performed on the identification of convective and stratiform clouds. The early recognition of stratiform cloud precipitation is mostly implemented by means of the 0°C-layer bright band, but this method is limited since stratiform clouds are difficult to identify before the clouds have evolved to the mature stage. Later, many methods were studied based on rain gauge data; in these methods, a threshold reflectivity factor was introduced to judge whether convective or stratiform clouds were present according to the echo intensity in a precipitation area. This approach determines the precipitation center of convective clouds and is addressed as a background-excess technique (BET) (Austin and Houze, 1972; Houze, 1973). Churchill et al. (1984) further applied the BET to a two-dimensional (2D) structure. The convective cloud center is first determined using a threshold of the radar reflectivity factor, and then a fixed-influence radius of the center is given to mark the area of convective clouds (Churchill and Houze, 1984). Steiner improved the identification effect by changing the influence radius of Churchill’s method to a function of the reflectivity factor and modifying the threshold of reflectivity to a function of the average reflectivity in the local background. The method is named the peakness threshold method and is also essentially the BET (Steiner et al., 1995). De Mott et al. (1995) suspected that convective clouds inclined with height may be misclassified if the classification is only performed at the lower level; therefore, they developed the three-dimensional (3D) peakness threshold method and extended the classification to echo tops by implementing the BET for radar volume scanning at each elevation band. Biggerstaff et al. (Biggerstaff and Listemaa, 2000) noted two main misclassifications in Steiner's algorithm: thick stratiform clouds are misclassified as convective clouds, and the periphery of the convective core is misclassified as stratiform clouds. Therefore, the vertical profile of reflectivity (VPR) and the altitude of the 0°C isotherm are applied to improve the algorithm. By means of radar reflectivity, cloud clusters are classified into convective and stratiform clouds according to their 3D structure, and this method is generally utilized in the United States at present.

Most research on cloud classification has been based on satellite and radar data in China (Zhang et al., 2010). Stratiform cloud precipitation identification methods are proposed after the image features of precipitation stratiform clouds are analyzed comprehensively using infrared and visible channel data from the AVHRR satellite (Liu et al., 2004). Li et al. (2003) analyzed the diurnal variation characteristics of stratiform clouds in southern China using satellite, ground observation, and radiosonde data. Using the fuzzy logic method, convective and stratiform clouds are classified according to the 3D morphological distribution of radar reflectivity (Xiao and Liu, 2007).

In recent years, artificial intelligence has developed rapidly. Machine learning, which is one of the core areas of artificial intelligence, has brought new opportunities to meteorological services. Machine learning algorithms, such as neural networks, random forests, decision trees, support vector machines, and Bayesian classifiers, are widely used in the field of meteorology (Kusiak et al., 2009; Behrang et al., 2010; Mellit et al., 2013; Ho et al., 2014; Voyant et al., 2017). Using a neural network, Pérez et al. (2009) retrieved the features of cloud images at night, and Liu et al. (2009) separated clouds from the images observed by the FY2C meteorological satellite. Based on infrared cloud images, Chen et al. (2005) combined neural networks with fuzzy logic methods to classify clouds.

Density-based spatial clustering of applications with noise (DBSCAN) is a typical density-based clustering algorithm applied in machine learning methods that does not assign a cluster center or the number of clusters in advance and only divides different clusters according to density (Feng and Xiao, 2008). Using the data observed with the new-generation S-band Doppler weather radar (CINRAD/SA) in Nanjing, Xuzhou, and Qingdao as a demonstration, a classification method for convective and stratiform clouds is suggested herein based on the improved DBSCAN clustering algorithm.

In section 2 of this paper, the data source and preprocessing approach are introduced briefly. In the third section, the improved DBSCAN clustering algorithm is introduced first, and then the cloud cluster classification steps with 2D and 3D models are illustrated. Three cases are analyzed to verify the addressed models in the fourth section. Finally, some conclusions and a discussion of the cloud classification algorithm are presented in the last section.

The data used in this study were collected by SA radar in Nanjing, Xuzhou, and Qingdao. The maximum detection range of the radar is 460 km, and the radial resolution of the reflectivity factor is 1 km. The VCP21(volume coverage pattern, scan strategy #2, version 1) scan mode was adopted, which comprises nine elevation angles (0.5°, 1.5°, 2.4°, 3.4°, 4.3°, 6.0°, 9.9°, 14.6°, and 19.5°), and the duration of volume scanning was six minutes. First, when the echoes around a point were less than 70%, the point was considered an isolated point clutter and filtered out. Then, the radial velocity and spectral width were simply processed to remove nonmeteorological echoes and ground clutter by adjusting the thresholds of radial velocity, echo intensity, and intensity changes along with range and elevation angles, to reduce the misjudgment of meteorological echoes (Li et al., 2012; Wen et al., 2017). The lightning data were obtained from the ground lightning recorder in Jiangsu Province, and the 0°C-height data were obtained from the fifth generation of European Centre for Medium-Range Weather Forecast (ECMWF) atmospheric reanalysis data (ERA5).

The main definitions of DBSCAN include the following.

Density: the number of objects (points) in the area.

Core point: if the density of one point reaches the threshold set by the algorithm, it is the core point; that is, the number of points in the neighborhood is not less than the value of P_min, where P_min represents the threshold of the number of points in the neighborhood with a distance of ε.

ε neighborhood: the radius of a given core point is called the ε neighborhood of the point.

Direct density reachability: given a set of points D, if point p is in the ε neighborhood of point q and q is a core point, then p is defined as directly density-reachable starting from q.

Density reachability: given a point sequence q₀, q₁, …, q_k and that the density is directly reachable from arbitrary q_i to q_i-1, then the density is reachable from q₀ to q_k, indicating the propagation of the direct density reachability.

Density connection: if starting from a core point and points q and k are density-reachable, then q and k are density-connected.

Boundary point: a noncore point in a cluster that is nonreachable to other points.

Noise point: a point that does not belong to any cluster and is not density-reachable from any core point.

The above definitions are demonstrated in Fig. 1, in which P_min is set to 5, the red points are in the core area because of at least 5 points in the range of the ε neighborhood (black circle), and the black points are in the noncore area for less than 5 points in the ε neighborhood. All red points are direct density reachability in the core area, and these red points connected by green arrows form a density reachable sequence. All points in the ε neighborhood of points in the sequence are density connections. A new cluster with p as a core point is constructed if the points in the ε neighborhood of p contain more than the minimal number P_min; then, a maximal-density-connected cluster is derived according to the density-reachable relation. Some clusters may be merged, and the process ends when no new point can be added to any cluster (Feng and Xiao, 2008).

Figure 1.  Schematic diagram of the DBSCAN algorithm.

By means of reflectivity and echo tops from radar base data, the DBSCAN clustering algorithm in machine learning is improved to realize multiclassification of clouds. 2D and 3D models are built to identify convective and stratiform clouds at different development stages. Figure 2 is the general flow chart of the algorithm.

Figure 2.  Flow chart of cloud cluster identification based on improved DBSCAN algorithm

The steps of the 2D model construction are as follows.

(a) After quality control, reflectivity data within the range of 230 km and 0.5° elevation are selected to classify convective and stratiform clouds using different thresholds.

(b) A point A is arbitrarily selected in the echo data (Fig. 3), and A is marked as a core point of the deep-convective cloud (DCC) if the conditions are satisfied, that is, if there are at least five points whose values are greater than or equal to 45 dBZ within the 3 × 3 window and if the height of the 35 dBZ echo top (ET) at A is more than 6 km.

Figure 3.  A schematic of the identification process with the improved DBSCAN algorithm in which A and B are identified as the core points, C is defined as a boundary point, and N is defined as a noisy point.

(c) Next, each point in the neighborhood of A is checked. Suppose there is a point B that satisfies the conditions given in the previous step, B is also marked as the core point of the DCC.

(d) Continuing to check the points in the neighborhood of point A and supposing there is a point C that does not satisfy the conditions, C is marked as the boundary point of the DCC.

(e) Supposing a point N that cannot be marked as a core point or a boundary point, N is marked as a noise point. This process continues until no new point can be marked.

(f) Repeating the above steps, a point is marked as a shallow-convective cloud (SCC) when Z ≥ 45 dBZ and ET < 6 km or when 37 ≤ Z < 45 dBZ and ET ≥ 5 km, as a hybrid convective-stratiform cloud (HCS) when 37 ≤ Z < 45 dBZ and ET < 5 km or when 30 ≤ Z < 37 dBZ and ET ≥ 3 km, and as a stratiform cloud (SFC) for the remaining echoes.

In general, ET refers to the height of 18 dBZ. However, due to the influence of the radar scanning strategy, this height may not be detected, which causes unrealistic ET values. The reflectivity of convective clouds is almost always greater than 35 dBZ; therefore, the convective clouds are not omitted by using the height of 35 dBZ as the value of ET.

For cloud clusters in different development stages, some misclassification exists if only reflectivity data at a 0.5° elevation are used. Based on the 2D model, a 3D model is further developed in which some parameters and thresholds are improved as follows.

(a) Reflectivity data at three elevation angles, namely, 0.5°, 1.5°, and 2.4°, are used in the identification process. In the third elevation (2.4°) of the VCP21 scan strategy, the height of the echo is 13.78 km at 230 km. Therefore, it is appropriate with three elevations to build a 3D model.

(b) Considering radar beam broadening, the 230 km range is divided into three sections and is modeled separately; these sections range from 0 km to 80 km, from 80 km to 150 km, and from 150 km to 230 km, and the corresponding P_min thresholds are set as 10, 8, and 5, respectively.

Three cases are used to demonstrate the rationality and effectiveness of the improved DBSCAN method in cloud classification. In cases 1 and 2, convective clouds with strong development are embedded in large-scale stratiform clouds, and melting layer echoes are obvious. The contrast images are screenshot from the operational software and classified according to the fuzzy logic method (Xiao and Liu, 2007). The classification results of the two methods are basically the same, but some HCSs near the melting layer are often identified as SFCs by the fuzzy logic method. Instead of judging only by one point in the fuzzy logic method, clouds are marked according to their spatial characteristics of multiple points with the same properties in the improved DBSCAN method. Therefore, the improved DBSCAN method obtains better recognition results than the fuzzy logic method.

Case 3 is a hail and thunderstorm gale weather event, and the identification results of convective clouds can clearly show the development process of disastrous weather.

A multicell precipitation weather process observed with the SA radar in Nanjing at 1000 UTC on 28 April 2015 is taken as an example. Figures 4a–c show images of the 0.5°, 1.5°, and 2.4° PPIs (plan position indicators), which are blocked around the azimuth at 135°. Figures 4d and e are the results of the cloud classifications conducted with the 2D and 3D models corresponding to Fig. 4a, respectively, in which eight colors are used to represent the eight types of clouds, namely, the core and boundary areas of the DCC, SCC, HCS, and SFC types. It can be seen from Figs. 4d and e that there are many convective clouds (marked with purple and red) embedded in large-scope HCSs and SFCs (marked with yellow and green, respectively), and lightning (marked with black plus signs) occurs in the convective cloud regions.

Figure 4.  The reflectivity PPI at 0.5°, 1.5°, and 2.4° observed with the Nanjing SA radar at 1000 UTC on 28 April 2015 (a–c), and the results of the cloud classification performed with the 2D model (d) and 3D model (e) in which the symbol _c represents the core area, and _b represents the bound area of each type of cloud cluster. The black plus signs are the locations of lightning, the black and red lines are the locations of sections corresponding to Fig. 5, and each circle is 50 km apart (same below). To compare the recognition effect, classification PPIs of the convective cloud (red area) and stratiform cloud (blue area) are attached to (f), as identified by Xiao et al. (2007) according to the fuzzy logic method.

Clouds often occur above the 0.5° elevation height of the radar detection and exist at different elevations. Therefore, a 3D model with reflectivity at elevations of 0.5°, 1.5°, and 2.4° is more accurate for identifying cloud clusters than a 2D model. Figures 4a–c show the PPIs of the lowest three elevation angles. The echo height increases with elevation, and the 3D model can distinguish the core and boundary of the cloud cluster well.

Figures 4d and e show that the cloud classifications performed with the 2D and 3D models are basically the same, but differences are seen in some areas. For example, almost no clouds are seen in Fig. 4d, but full, weak SFCs are seen in Fig. 4e in the two boxes indicated by the black dotted line, suggesting that a weak SFC exists above the 0.5° elevation height that cannot be obtained with the 2D model. In addition, developing DCCs are marked by black ellipses in Fig. 4d, in which the core area of the DCC is still relatively small, but significantly increases in Fig. 4e, in which it is almost marked as the core of the DCC. Therefore, the 3D model has great advantages in identifying SFCs and developing DCCs. However, some developing convective clouds of small areas at the low level cannot be identified with the 3D model because they are covered by thick stratiform clouds. Therefore, the classification results may be more reasonable when the two models are combined to identify clouds together.

To further verify the classification process, two VPRs corresponding to the black (Fig. 5a) and red (Fig. 5b) lines in Fig. 4 are demonstrated, in which the horizontal axis denotes the length of the section, the vertical axis represents the height, and the black dotted line shows the height of the 0ºC layer according to the atmospheric reanalysis data. In Fig. 5a, two DCCs are located in the horizontal distance area from 0 km to 35 km; these clouds have obvious columnar structures with large vertical thicknesses and uneven echo tops (Zhang et al., 2010), and echo intensity heights larger than 45 dBZ extend over 7 km. The maximum intensities from 70 km to 95 km and from 105 km to 115 km are over 45 dBZ, and the 35 dBZ height is above 5 km, much higher than the 3.8 km 0ºC-level height; therefore, it is reasonable to identify HCSs using the improved DBSCAN algorithm in these areas with obvious convective structures; however, these areas are identified as SFCs with the fuzzy logic method (Fig. 4f). In addition, the intensities from 50 km to 70 km in Fig. 5a are relatively weak and are correctly identified as SFCs with the improved DBSCAN algorithm.

Figure 5.  The reflectivity of the vertical sections corresponding to the black line (a) and the red line (b) in Fig. 4 (Range is horizontal distance, height is echo height, and the black dotted line is the 3.8 km height of the 0ºC layer in Nanjing at 1200 UTC on 28 April 2015).

In Fig. 5b, there is a strong convective cloud area from 5 km to 35 km in which the reflectivity of the echo center is larger than 65 dBZ and the 45 dBZ height is more than 12 km; this area is identified as DCCs in Fig. 4. In addition, there are several convective bubbles from 60 km to 120 km. The bubble from 60 km to 70 km is weak, and the 35 dBZ echo extends to a height of approximately 5 km and is classified as a SCC in Fig. 4. The bubble at approximately 110 km is relatively strong, showing an obvious columnar structure, and the 35 dBZ height is over 9 km, but the cloud is only marked as the SCC boundary because there are not sufficient core points in the SCC. However, this area further evolves to a core SCC area after six minutes (picture is omitted); therefore, the improved DBSCAN algorithm suggests a developing convective cloud.

According to the above analysis, some differences exist in cloud classifications between the 2D and 3D models. To further illustrate these differences, the number of each cloud type point is calculated and shown in Fig. 6. The total points in the 2D and 3D models are 34 316 and 42 501, respectively. The numbers of convective points are slightly different between the 2D and 3D models, but the total number of points identified with the 3D model is nearly 8 000 larger than that identified with the 2D model, which is mainly caused by the difference in SFCs. There are two reasons why the DCC, HCS, and SFC areas identified with the 3D model are larger than those identified with the 2D model. One is that the 3D model is based on the reflectivity factor at three elevation angles, allowing a larger height scope for identifying points. The other is that the range of the 3D model is divided into several sections to set different density threshold values that can mark more points at far distances. In addition, the number of each type of cluster point under different parameter settings is counted, while two parameters (density threshold and ε neighborhood) of the improved DBSCAN algorithm are set differently. The SCC area identified with the 2D model is slightly higher than that identified with the 3D model, which may be caused by the weak development of the SCC during the event.

Figure 6.  The number of points of each cloud type identified with the 2D and 3D models.

The parameter change in the 2D model is taken as an example to illuminate the impact of the parameters on the classification. In terms of the density threshold (Table 1), the number of echo points is listed, while the density is set to 3, 4, 5, 6, and 7 points within a one-kilometer neighborhood. With the increase in the density threshold, it is found that the more severe the conditions for identifying clouds are, the smaller the echo area of the cloud cluster. In terms of the ε neighborhood (Table 2), the number of points is listed, while the threshold values of ε are given as 1 km, 2 km, 3 km, and 4 km. Contrary to the changing trend of the density threshold, the larger the ε neighborhood is, the larger the echo area of the identified cloud cluster. It is obvious from Tables 1 and 2 that the values of the density threshold and ε neighborhood greatly influence the classification results.

Another radar-based dataset taken as an example was observed with the SA radar in Xuzhou at 0754 UTC on 6 July 2019. The identifications of cloud clusters with the DBSCAN and fuzzy logic algorithms are shown in Fig. 7. The large-scope purple areas indicate that this was a strong convective weather event, and the convective clouds accompanied by lightning were very energetic. Both the 2D and 3D models can obviously classify the convective and stratiform features, and similar to case one, the 2D model marks more SCC points while the 3D model marks more SFC points.

Figure 7.  Similar to Fig. 4, but the radar data were observed in Xuzhou at 0754 UTC 6 July 2019; the black line is the location of the section corresponding to Fig. 8.

Comparing the classifications between adjacent times 0754 UTC and 0800 UTC (figure omitted), the evolution and dissipation of convective clouds can be found in both the 2D and 3D models; these results can improve our understanding of the mechanism of precipitation and allow us to better estimate precipitation.

In the azimuth area from 55° to 95° and ranging from 200 km to 230 km in the 0.5° reflectivity PPI, there is a strong region within the echo over 35 dBZ; this region is marked as an HCS by the 2D and 3D models but as an SFC or as an uncertain cloud cluster by the fuzzy logic method, which considers that these strong echoes are caused by a melting layer. To verify this classification, a vertical section (Fig. 8) is made along the echo region (black line in Fig. 7a). As shown in Fig. 8, most echoes above 35 dBZ are higher than the 4.5-km height of the melting layer; therefore, it is reasonable that the cloud is marked as an HCS by the improved DBSCAN method. In addition, although the intensity is greater than 35 dBZ at the bottom of the black line shown in Fig. 7a, the cloud is marked as an SFC by both the 2D and 3D models. Figure 8 also verifies that the echo in this region is not strong.

Figure 8.  The reflectivity vertical section corresponds to the black line in Fig. 7a, and the black dotted line is the 4.5 km height of the 0ºC layer in Xuzhou at 1200 UTC 6 July 2019.

Affected by the cold vortex, a large-scale severe convective weather event occurred in Shandong Province on 17 May 2020. Disasterous weather, including thunderstorms, gales of magnitude 8–10, and hail, occurred in many places that day. Taking data from the Qingdao SA radar as an example, the clouds are classified by a 3D model. There is no super-refraction within the radar detection range, as found by examining the radial velocity and the intensity vertical texture. The points of the DCC and SCC are drawn in Fig. 9a. From 0940 UTC, thunderstorms and gales appeared, while convective clouds were detected in the radar detection range. Convective clouds developed to their most energetic stage from 1300 UTC to 1500 UTC and then weakened and completely dissipated at approximately 1900 UTC; disastrous weather also dissipated at that time.

Figure 9.  The changing trend of the number of cloud clusters identified by the 3D model; (a) 0940–1904 UTC and (b) 1128–1220 UTC (scale of the point number of the SFC boundary is on the right side of the Y-axis, and the other seven types of clouds are on the left).

To analyze the relationship between the evolution of convection and the size of the cloud boundary area, the variations in the core and boundary points of each type of cloud from 1128 UTC to 1220 UTC are shown in Fig. 9b, and the corresponding PPI images of reflectivity are shown in Fig. 10. As shown in Fig. 10, various types of cloud clusters continued to develop during the eastward movement of the system. From 1128 UTC, the boundary of the SFC increased rapidly, which indicated that the weather system had entered the radar detection range. The core and boundary areas of other clouds increased slowly, and there was an opposite phase change between the curves of the core and boundary with time. In particular, the inverse phase change between the core and boundary curves of the DCC was more obvious, which clearly showed the dynamic evolutionary process of the convective system from occurrence to extinction. In addition, the convective clouds in classification images (Figs. 10e–h) are clearer than those in the PPIs (Figs. 10a–d). At this distance, the strong echoes caused by the melting layer are correctly recognized as stratiform clouds, which can aid in the better location of disastrous weather.

Figure 10.  The reflectivity image of the 0.5° PPI observed with the Qingdao SA radar at 1128 UTC, 1146 UTC, 1203 UTC, and 1220 UTC on 17 May 2020 (a–d), and the results of the cloud classification performed with the 3D model (e–f).

By means of the improved DBSCAN algorithm, a cloud cluster with an arbitrary shape can be identified effectively, and the algorithm has no size or distribution requirements regarding the utilized dataset. A 2D and 3D model are proposed for cloud identification to classify four types of cloud clusters, namely, DCC, SCC, HCS, and SFC types, and each type is further classified into core and edge regions. The classification results can provide more abundant cloud structure information, better indicate the macro characteristics of clouds, and more accurately predict the precipitation intensity and location of disastrous weather. Three typical cases observed with SA radars in Nanjing, Xuzhou, and Qingdao are utilized to verify the effects of this classification method, and the results are compared with the classification results obtained using the fuzzy logic method.

(a) By adjusting the threshold values of the echo intensity, density, echo top of 35 dBZ, and ε neighborhood, clouds with sufficient density can be divided into different clusters to further judge the cloud types. During the clustering process, the cluster center and number do not need to be specified initially, and clusters with arbitrary shapes can be identified in a noisy space.

(b) The cloud clusters of the four types and their core and boundary areas are intuitive. The identified deep convective areas correspond to the locations of lightning. According to the vertical profile of reflectivity, the identification results of energetic convective clouds, weak convective bubbles, and stratiform clouds are also objective and reasonable.

(c) Some differences were observed between the 2D and 3D models. The 3D model identifies larger convective clouds and stratiform clouds than the 2D model. In addition, weaker convective clouds with small areas in the lower layer may be covered by deeper stratiform clouds in the 3D model. Therefore, in practice, it is best to combine the classification results of the two models. According to the continuous recognition results obtained with the adjacent volume scanning data, the evolution of cloud clusters from generation to disappearance can be clearly discovered.

(d) The echo areas are significantly influenced by the density threshold and ε neighborhood. Different parameters have different recognition results, and the classification is inaccurate if the parameter setting is unreasonable, especially for weak clouds. Therefore, the threshold of the improved DBSCAN algorithm should be improved according to more actual observation results in the future.

(e) Traditional algorithms, such as the fuzzy logic method, only divide clouds into two categories, namely, stratiform and convective clouds, but the improved DBSCAN algorithm has a more detailed cloud classification, which can identify the boundaries and cores of four types of clouds. During convection deepening, the boundary of clouds gradually develops into the core of the clouds, and the core also develops into boundary clouds. There is an inverse phase relationship between the two evolutionary curves with time, which can better predict the occurrence, development, and dissipation of strong convective weather.

Acknowledgements. This research was funded by the Key-Area Research and Development Program of Guangdong Province (Grant No. 2020B1111200001), the Key project of monitoring, early warning and prevention of major natural disasters of China (Grant No. 2019YFC1510304), the S&T Program of Hebei (Grant No. 19275408D), and the Scientific Research Projects of Weather Modification in Northwest China (Grant No. RYSY201905). The authors would like to sincerely thank Fen XU of Nanjing Joint Institute for Atmospheric Sciences for providing technical guidance.