Advanced Search
Article Contents

A Test Pattern Identification Algorithm and Its Application to CINRAD/SA(B) Data

Fund Project:

doi: 10.1007/s00376-013-2315-9

  • A variety of faulty radar echoes may cause serious problems with radar data applications, especially radar data assimilation and quantitative precipitation estimates. In this study, ''test pattern'' caused by test signal or radar hardware failures in CINRAD (China New Generation Weather Radar) SA and SB radar operational observations are investigated. In order to distinguish the test pattern from other types of radar echoes, such as precipitation, clear air and other non-meteorological echoes, five feature parameters including the effective reflectivity data percentage (RZ), velocity RF (range folding) data percentage (RRF), missing velocity data percentage (RM), averaged along-azimuth reflectivity fluctuation (RNr,Z) and averaged along-beam reflectivity fluctuation (RNa,Z) are proposed. Based on the fuzzy logic method, a test pattern identification algorithm is developed, and the statistical results from all the different kinds of radar echoes indicate the performance of the algorithm. Analysis of two typical cases with heavy precipitation echoes located inside the test pattern are performed. The statistical results show that the test pattern identification algorithm performs well, since the test pattern is recognized in most cases. Besides, the algorithm can effectively remove the test pattern signal and retain strong precipitation echoes in heavy rainfall events.
    摘要: A variety of faulty radar echoes may cause serious problems with radar data applications, especially radar data assimilation and quantitative precipitation estimates. In this study, ``test pattern'' caused by test signal or radar hardware failures in CINRAD (China New Generation Weather Radar) SA and SB radar operational observations are investigated. In order to distinguish the test pattern from other types of radar echoes, such as precipitation, clear air and other non-meteorological echoes, five feature parameters including the effective reflectivity data percentage (RZ), velocity RF (range folding) data percentage (R RF), missing velocity data percentage (R M), averaged along-azimuth reflectivity fluctuation (RN_ r,Z) and averaged along-beam reflectivity fluctuation (RN_ a,Z) are proposed. Based on the fuzzy logic method, a test pattern identification algorithm is developed, and the statistical results from all the different kinds of radar echoes indicate the performance of the algorithm. Analysis of two typical cases with heavy precipitation echoes located inside the test pattern are performed. The statistical results show that the test pattern identification algorithm performs well, since the test pattern is recognized in most cases. Besides, the algorithm can effectively remove the test pattern signal and retain strong precipitation echoes in heavy rainfall events.
  • 加载中
  • da Silveira, R. B., and A. R. Holt, 2001:An automatic identification of clutter and anomalous propagation in polarization diversity weather radar data using neural networks. IEEE Trans. Geosci. Remote Sens., 39, 1777-1788.
    Gao, J., and K. K. Droegemeier, 2004:A variational technique for dealiasing doppler radial velocity data. J. Appl. Meteor., 43, 934-940.
    Hu, D. M., and Z. F. Wu, 2003:CINRAD/SA radar daily maintenance and fault diagnosis method. Meteorological Monthly, 29(10), 26-28. (in Chinese)
    Jiang, Y., L. P. Liu, and W. Zhuang, 2009:Statistical characteristics of clutter and improvements of ground clutter identification technique with dopple weather radar. Journal of Applied Meteorological Science, 20(2), 203-213. (in Chinese)
    Kessinger, C., S. Ellis, and J. Van Andel, 2001:NEXRAD data quality: The AP clutter mitigation scheme. Preprints, 30th International Conference on Radar Meteorology, Munich, Germany, 707-709.
    Lakshmanan, V., K. Hondl, G. Stumpf, and T. Smith, 2003:Quality control of weather radar data using texture features and a neural network. 5th Int'l Conf. on. Adv. in Patt. Recogn., IEEE, Kolkota, 522-525.
    Liu, L. P., L. L. Wu, and Y. M. Yang, 2007:Development of fuzzy-logical two step ground clutter detection algorithm. Acta Meteorologica Sinica, 65(2), 252-260. (in Chinese)
    Ray, P., and C. Ziegler, 1977:De-Aliasing First-Moment Doppler Estimates. J. Appl. Meteor., 16, 563-564.
    Wang, G. L., P. L. Liu, and Z. Ruan, 2007:Application of Doppler radar data to nowcasting of heavy rainfall. Journal of Applied Meteorological Science, 18(3), 285-295. (in Chinese)
    Wang, H. Y., L. P. Liu, G. L. Wang, W. Zhuang, Z. Z. Zhang, and X. L. Chen, 2009:Decelopment and application of the Doppler weather radar 3-D digital mosaic system. Journal of Applied Meteorological Science, 20(2), 214-224. (in Chinese)
    Wang, J., and L. P. Liu, 2009:Assimilation of microphysical adjustments using reflectivity of CINRAD/CD Doppler radar for mesoscale model. Plateau Meteorology, 28(1), 173-185. (in Chinese)
    Wang, Z. W., H. G. Zhou, and Z. N. Lin, 2005:Fault analysis of the CINRAD SA&B. Modern Radar, 27(1), 16-18. (in Chinese)
    Xiao, Y. J., and L. P. Liu, 2006:Study of method for three-dimensional multiple-radar reflectivity mosaics. Acta Meteorologica Sinica, 64(5), 648-656. (in Chinese)
    Xu, Q., 2009:Bayesian perspective of the unconventional approach for assimilating aliased radar radial velocities. Tellus, 61A, 631-634.
    Xu, Q., K. Nai, L. Wei, and Q. Zhao, 2009:An unconventional approach for assimilating aliased radar radial velocities. Tellus, 61A, 621-630.
    Yang, C. F., X. S. Huang, and X. G. Diao, 2005:Fault diagnosis of Ji Nan CINRAD/SA radar's lanuch high-power. Meteorological Monthly, 31(1), 88-89. (in Chinese)
    Zhang, J., and S. Wang, 2006:An automated 2D multipass doppler radar velocity dealiasing scheme. J. Atmos. Oceanic Technol., 23, 1239-1248.
    Zhang, Z. Q., L. P. Liu, M. Y. Xie, Y. J. Xiao, and H. Y. Wang, 2007:A display system of CINRAD 3D mosaic products. Meteorological Monthly, 33(9), 19-24. (in Chinese)
    Zhou, H. G., M. H. Zhu, S. L. Duan, and Y. M. Shi, 2005:CINRAD/SA radar fault analysis. Meteorological Monthly, 31(10), 39-42. (in Chinese)
    Zhou, H. G., X. J. Zhou, X. Qi, Y. J. Wang, J. G. Tang, and D. Y. Cao, 2007:Special fault analysis of CINRAR weather radar servo system. Meteorological Monthly, 33(2), 98-101. (in Chinese)
    Zhou, H. G., X. M. Cai, F. Hu, Q. Cai, L. Zhou, and T. Wu, 2008:Abnormal radar echo analysis of CINRAD. Meteorological Monthly, 34(6), 112-115. (in Chinese)
  • [1] Bo-Young YE, GyuWon LEE, Hong-Mok PARK, 2015: Identification and Removal of Non-meteorological Echoes in Dual-polarization Radar Data Based on a Fuzzy Logic Algorithm, ADVANCES IN ATMOSPHERIC SCIENCES, 32, 1217-1230.  doi: 10.1007/s00376-015-4092-0
    [2] Eun-Han KWON, Jinlong LI, B. J. SOHN, Elisabeth WEISZ, 2012: Use of Total Precipitable Water Classification of A Priori Error and Quality Control in Atmospheric Temperature and Water Vapor Sounding Retrieval, ADVANCES IN ATMOSPHERIC SCIENCES, 29, 263-273.  doi: 10.1007/s00376-011-1119-z
    [3] LIAO Jie, WANG Bin, LI Qingxiang, 2014: A New Method for Quality Control of Chinese Rawinsonde Wind Observations, ADVANCES IN ATMOSPHERIC SCIENCES, 31, 1293-1304.  doi: 10.1007/s00376-014-4030-6
    [4] Jo-Han LEE, Dong-Kyou LEE, Hyun-Ha LEE, Yonghan CHOI, Hyung-Woo KIM, 2010: Radar Data Assimilation for the Simulation of Mesoscale Convective Systems, ADVANCES IN ATMOSPHERIC SCIENCES, 27, 1025-1042.  doi: 10.1007/s00376-010-9162-8
    [5] HOU Tuanjie, Fanyou KONG, CHEN Xunlai, LEI Hengchi, HU Zhaoxia, 2015: Evaluation of Radar and Automatic Weather Station Data Assimilation for a Heavy Rainfall Event in Southern China, ADVANCES IN ATMOSPHERIC SCIENCES, 32, 967-978.  doi: 10.1007/s00376-014-4155-7
    [6] Xingchao CHEN, Kun ZHAO, Juanzhen SUN, Bowen ZHOU, Wen-Chau LEE, 2016: Assimilating Surface Observations in a Four-Dimensional Variational Doppler Radar Data Assimilation System to Improve the Analysis and Forecast of a Squall Line Case, ADVANCES IN ATMOSPHERIC SCIENCES, 33, 1106-1119.  doi: 10.1007/s00376-016-5290-0
    [7] ZHUO Jinqing, HUANG Jianping, WANG Jiemin, ZHANG Wu, BI Jianrong, WANG Guoyin, LI Weijing, FU Peijian, 2009: Surface Turbulent Flux Measurements over the Loess Plateau for a Semi-Arid Climate Change Study, ADVANCES IN ATMOSPHERIC SCIENCES, 26, 679-691.  doi: 10.1007/s00376-009-8188-2
    [8] Xin LI, Mingjian ZENG, Yuan WANG, Wenlan WANG, Haiying WU, Haixia MEI, 2016: Evaluation of Two Momentum Control Variable Schemes and Their Impact on the Variational Assimilation of Radar Wind Data: Case Study of a Squall Line, ADVANCES IN ATMOSPHERIC SCIENCES, 33, 1143-1157.  doi: 10.1007/s00376-016-5255-3
    [9] Jie HE, Xulin MA, Xuyang GE, Juanjuan LIU, Wei CHENG, Man-Yau CHAN, Ziniu XIAO, 2021: Variational Quality Control of Non-Gaussian Innovations in the GRAPES m3DVAR System: Mass Field Evaluation of Assimilation Experiments, ADVANCES IN ATMOSPHERIC SCIENCES, 38, 1510-1524.  doi: 10.1007/s00376-021-0336-3
    [10] HU Zhiqun, LIU Liping, WANG Lirong, 2012: A Quality Assurance Procedure and Evaluation of Rainfall Estimates for C-Band Polarimetric Radar, ADVANCES IN ATMOSPHERIC SCIENCES, 29, 144-156.  doi: 10.1007/s00376-011-0172-y
    [11] HUO Juan, ZHANG Wenxing, ZENG Xiaoxia, Lü Daren, LIU Yi, 2013: Examination of the Quality of GOSAT/CAI Cloud Flag Data over Beijing Using Ground-based Cloud Data, ADVANCES IN ATMOSPHERIC SCIENCES, 30, 1526-1534.  doi: 10.1007/s00376-013-2267-0
    [12] Jidong GAO, Keith BREWSTER, Ming XUE, 2006: A Comparison of the Radar Ray Path Equations and Approximations for Use in Radar Data Assimilation, ADVANCES IN ATMOSPHERIC SCIENCES, 23, 190-198.  doi: 10.1007/s00376-006-0190-3
    [13] LIU Xiaoyang, MAO Jietai, ZHU Yuanjing, LI Jiren, 2003: Runoff Simulation Using Radar and Rain Gauge Data, ADVANCES IN ATMOSPHERIC SCIENCES, 20, 213-218.  doi: 10.1007/s00376-003-0006-7
    [14] Fabien CARMINATI, Nigel ATKINSON, Brett CANDY, Qifeng LU, 2021: Insights into the Microwave Instruments Onboard the Fengyun 3D Satellite: Data Quality and Assimilation in the Met Office NWP System, ADVANCES IN ATMOSPHERIC SCIENCES, 38, 1379-1396.  doi: 10.1007/s00376-020-0010-1
    [15] ZHOU Lian-Tong, HUANG Ronghui, 2010: An Assessment of the Quality of Surface Sensible Heat Flux Derived from Reanalysis Data through Comparison with Station Observations in Northwest China, ADVANCES IN ATMOSPHERIC SCIENCES, 27, 500-512.  doi: 10.1007/s00376-009-9081-8
    [16] LIU Hongya, XUE Jishan, GU Jianfeng, XU Haiming, 2012: Radar Data Assimilation of the GRAPES Model and Experimental Results in a Typhoon Case, ADVANCES IN ATMOSPHERIC SCIENCES, 29, 344-358.  doi: 10.1007/s00376-011-1063-y
    [17] T. S. Spassova, 1992: A Theoretical Test of the Geostrophic Momentum Approximation, ADVANCES IN ATMOSPHERIC SCIENCES, 9, 251-255.  doi: 10.1007/BF02657516
    [18] Juan HUO, Yongheng BI, Daren Lü, Shu DUAN, 2019: Cloud Classification and Distribution of Cloud Types in Beijing Using Ka-Band Radar Data, ADVANCES IN ATMOSPHERIC SCIENCES, , 793-803.  doi: 10.1007/s00376-019-8272-1
    [19] Shibo GAO, Haiqiu YU, Chuanyou REN, Limin LIU, Jinzhong MIN, 2021: Assimilation of Doppler Radar Data with an Ensemble 3DEnVar Approach to Improve Convective Forecasting, ADVANCES IN ATMOSPHERIC SCIENCES, 38, 132-146.  doi: 10.1007/s00376-020-0081-z
    [20] Liping LIU, Jiafeng ZHENG, Jingya WU, 2017: A Ka-band Solid-state Transmitter Cloud Radar and Data Merging Algorithm for Its Measurements, ADVANCES IN ATMOSPHERIC SCIENCES, 34, 545-558.  doi: 10.1007/s00376-016-6044-8

Get Citation+

Export:  

Share Article

Manuscript History

Manuscript received: 22 December 2012
Manuscript revised: 09 April 2013
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

A Test Pattern Identification Algorithm and Its Application to CINRAD/SA(B) Data

    Corresponding author: LIU Liping
  • 1. State Key Laboratory of Severe Weather, Chinese Academy of Meteorological Sciences, Beijing 100081;
  • 2. Nanjing University of Information Science and Technology, Nanjing 210044;
  • 3. National Meteorological Center, Beijing 100081
Fund Project:  The authors are thankful to Pengfei ZHANG of NSSL and the anonymous reviewers for their comments and suggestions, which helped improve the presentation of the paper. Also, thanks are extended to CAO Jie of IAP in Beijing for assistance with the English. This paper is supported by the National Key Program for Developing Basic Sciences under Grant 2012CB417202, the National Natural Science Foundation of China under Grant No. 41175038, No. 41305088 and No. 41075023, the Meteorological Special Project Radar network observation technology and QC, the CMA Key project Radar Operational Software Engineering and the Chinese Academy of Meteorological Sciences Basic Scientific and Operational Projects Observation and retrieval methods of micro-physics and dynamic parameters of cloud and precipitation with multi-wavelength Remote Sensing, and also by Project of the State Key Laboratory of Severe Weather grant 2012LASW-B04.

Abstract: A variety of faulty radar echoes may cause serious problems with radar data applications, especially radar data assimilation and quantitative precipitation estimates. In this study, ''test pattern'' caused by test signal or radar hardware failures in CINRAD (China New Generation Weather Radar) SA and SB radar operational observations are investigated. In order to distinguish the test pattern from other types of radar echoes, such as precipitation, clear air and other non-meteorological echoes, five feature parameters including the effective reflectivity data percentage (RZ), velocity RF (range folding) data percentage (RRF), missing velocity data percentage (RM), averaged along-azimuth reflectivity fluctuation (RNr,Z) and averaged along-beam reflectivity fluctuation (RNa,Z) are proposed. Based on the fuzzy logic method, a test pattern identification algorithm is developed, and the statistical results from all the different kinds of radar echoes indicate the performance of the algorithm. Analysis of two typical cases with heavy precipitation echoes located inside the test pattern are performed. The statistical results show that the test pattern identification algorithm performs well, since the test pattern is recognized in most cases. Besides, the algorithm can effectively remove the test pattern signal and retain strong precipitation echoes in heavy rainfall events.

摘要: A variety of faulty radar echoes may cause serious problems with radar data applications, especially radar data assimilation and quantitative precipitation estimates. In this study, ``test pattern'' caused by test signal or radar hardware failures in CINRAD (China New Generation Weather Radar) SA and SB radar operational observations are investigated. In order to distinguish the test pattern from other types of radar echoes, such as precipitation, clear air and other non-meteorological echoes, five feature parameters including the effective reflectivity data percentage (RZ), velocity RF (range folding) data percentage (R RF), missing velocity data percentage (R M), averaged along-azimuth reflectivity fluctuation (RN_ r,Z) and averaged along-beam reflectivity fluctuation (RN_ a,Z) are proposed. Based on the fuzzy logic method, a test pattern identification algorithm is developed, and the statistical results from all the different kinds of radar echoes indicate the performance of the algorithm. Analysis of two typical cases with heavy precipitation echoes located inside the test pattern are performed. The statistical results show that the test pattern identification algorithm performs well, since the test pattern is recognized in most cases. Besides, the algorithm can effectively remove the test pattern signal and retain strong precipitation echoes in heavy rainfall events.

1. Introduction
  • Since CINRAD (China New Generation Weather Radar) became operational, much work on radar failure analysis and safety maintenance has been carried out (Zhou et al., 2007). Through diagnosing the reasons for radar launch system breakdowns (Yang et al., 2005), or by continually improving radar automatic calibration systems and the reliability of failure diagnosis systems, effective routine maintenance and better identification of malfunctions have been established (Hu and Wu, 2003). Many methods that can improve radar performance by tackling all kinds of hardware issues have been proposed (Wang et al., 2005), and experts have also proposed a number of correction schemes through analyzing the various causes of hardware and software problems (Zhou et al., 2005). Nevertheless, contaminated radar data caused by hardware failure remains an issue during routine operations, and one such example is the appearance of "test pattern" in radar data. Because test pattern shows up in many different forms and the reasons are unknown, it is not currently possible to totally avoid them from CINRAD operations simply by improving the hardware. Often, the occurrence of test pattern is accompanied by receiver power failure, or a failure of the receiver main channel or frequency synthesizer (Zhou et al., 2008). The presence of test pattern in radar observations causes many problems in operational applications. Despite radar operators and forecasters being able to limit the usage of this information manually, it can still damage quantitative precipitation estimates and radar data assimilation during real-time operations.

    Acquisition of high quality radar data has always been the key to successful application of radar data, and associated quality control techniques have made great progress in recent years. For example, the fuzzy logic algorithm that uses reflectivity, radial velocity and spectrum width simultaneously has been widely applied to identify clutter and anomalous propagation in the WSR-88D weather radar ORPG (Open Radar Product Generator) system (Kessinger et al., 2001). Furthermore, neural network technique has been proposed and applied in radar data quality control by (da Silveira and Holt, 2001) and Lakshmanman et al. (2003). For Doppler velocity dealiasing, most technologies are based on the conventional temporal spatial continuity principle (Ray and Ziegler, 1977; Gao and Droegemeier, 2004; Zhang and Wang, 2006), while recently an unconventional method to dealias radar radial velocities based on a Bayesian approach has been proposed (Xu, 2009; Xu et al., 2009). However, despite the aforementioned progress, test pattern identification method has yet to be developed. The successful deployment of CINRAD is central not only to severe weather nowcasting and short- and long-term forecasting (Wang et al., 2007; Wang et al., 2009), but also application systems (Zhang et al., 2007; Wang et al., 2009) developed for using radar data in real-time operations in many cities throughout China. (Liu et al., 2007), (Xiao and Liu, 2006) and (Jiang et al., 2009) have also developed a ground clutter echo identification algorithm by using the fuzzy logic method. The algorithm has not only been applied in SWAN (severe weather auto nowforecasting) and ROSE (radar operational software engineering) systems, but also in the rapid update cycle system of the Shanghai Meteorological Administration and Guangzhou Institute of Tropical Marine Meteorology. The positive impact of this work has been very clear to see.

    The paper is organized as follows. The collections and analysis of test pattern that occurred during operation of CINRAD/SA(B) radar in Shenzhen, Hefei, Yancheng, Qingdao, Zhumadian and Beijing are described in the next Section. Section 3 examines the characteristics of reflectivity and radial velocity fields that contain the test pattern, meteorological and non-meteorological echoes, and several statistical parameters that were calculated to identify those echoes. Based on these properties, a test pattern identification algorithm is developed in section 4. Statistical results and application of the method for test pattern identification are presented in section 5. And finally, conclusions follow in section 6.

2. Data collection
  • A set of 38 volume scans data including a total of 144 PPIs (plane position indictors) contaminated by the test pattern was selected. These data were collected from CINRAD/SA(B) in Shenzhen, Hefei, Yancheng, Qingdao, Zhumadian and Beijing in 2008 and 2010. Another dataset that includes precipitation echoes (1800 PPIs), clear air echoes (565 PPIs), and other non-meteorological echoes (such as clutter and electromagnetic interference; 721 PPIs) was also collected. In this study, the test pattern was defined as a round or scalloped (more than 20 continuous radials) echo that covered the entire PPI plane and for which reflectivity enhanced with distance, but did not change much azimuthally. Test pattern caused by various factors, most of which are unclear hardware faults. Because there is no way to control them absolutely through hardware improvements, an algorithm that can automatically recognize such pattern is needed.

    Test pattern occurs irregularly. Sometimes one can appear in seven to ten successive volumes, and then disappears. It can appear at lower tilts, and sometimes at higher tilts. When one does occur, the radial velocity might not be contaminated. If the radial velocity is contaminated, the value indicates they are range folding (RF), and so RF rings are observed. For higher tilts (higher than 4°), the value of contaminated radial velocity data might be zero. A mixture of test pattern and precipitation echoes are often observed together. The reflectivity field is the sum of precipitation signals and test pattern signals, but radial velocity is the true value surrounded by the RF value. Spectrum width is linked to radial velocity, i.e., it is contaminated when radial velocity is contaminated by the test pattern. All the aforementioned features of test pattern can occur alone or together; several typical test pattern PPIs are shown as examples in Fig. 1.

    Figure 1.  Test pattern PPIs: first and third columns are reflectivity (dBZ); second and fourth columns are Doppler velocity (m s-1) (range circle: 100 km).

    Figures 1a and d show test pattern PPIs at tilt of 0.5° observed by the Yancheng SA radar at 0920 UTC 21 July 2008. Meteorological echoes can be seen near the radar site (no further than 50 km), while the rest of the area is filled with test pattern. Reflectivity is composited by two semicircular of test pattern, while radial velocity is filled with RF values. Figures 1b and e show test pattern PPIs at tilt of 0.5° observed by Yancheng SA radar at 0837 UTC 21 July 21 2008, in which reflectivity is contaminated but radial velocity is normal. Figures 1c and f show test pattern PPIs at tilt of 9.9° observed by Shenzhen SA radar at 0032 UTC 28 July 2010. Here, a test pattern appears at high tilt, while both reflectivity and radial velocity are contaminated, with the latter showing a value of zero. Figures 1g and j show 0.5° tilt PPIs from Shenzhen SA radar at 0418 UTC 22 April 2010. Precipitation echoes are mixed with the test pattern; or, in other words, the test pattern signal is mixed with the precipitation signal, but the radial velocity value of the precipitation echoes are true. Figures 1h and k show semicircular test pattern PPIs from Shenzhen SA radar at tilt of 0.5° at 0912 UTC 7 April 2010, and radial velocity is also contaminated. Figures 1i and l display a fan-shaped test pattern observed by Shenzhen SA radar at tilt of 0.5° at 2000 UTC 24 April 2010 and, although the test pattern is not filled with all 360 azimuths, only 20 continuous azimuths are contaminated. Besides, it can be seen from Fig. 1 that the reflectivity of the test pattern does not change much azimuthally, but increases with range.

3. Feature analysis
  • As described above, two types of test pattern were observed in collected data: (1) contaminated reflectivity, but normal radial velocity and spectrum width; (2) both radial velocity and spectrum width contaminated, as per reflectivity. When a test pattern occurs, reflectivity data will be contaminated and spread extensively over the radar coverage area. And for the first type of pattern, good reflectivity data might be obtained inside the corresponding radial velocity coverage range, while contaminated reflectivity data occur over the rest of the area. For the other type, both reflectivity and radial velocity are contaminated and spread over their coverage areas, and contaminated velocity will be observed as RF value. Test pattern has another obvious feature, which is that reflectivity is almost the same azimuthally, but increases with radial distance. Based on our understanding of the possible causes of test pattern, the contaminated signal of test pattern should be constant. Thus, to reflect the feature of the test pattern signal, removing the distance correction from the reflectivity first, then, the subsequent reflectivity (i.e., after removing the distance correction), which in this case reflects the test pattern signal, has small fluctuations in specific area range. Therefore, five feature parameters that can identify test pattern are proposed: effective reflectivity data percentage (RZ, subscript Z indicate reflectivity data); velocity RF data percentage (R RF, subscript RF indicates range fold); missing velocity data percentage (R M, subscript M indicates missing velocity data); averaged along-azimuth reflectivity fluctuation (RN_ r, Z, see section 3.4.1 for subscript meaning); averaged along-beam reflectivity fluctuation (RN_ a, Z, see section 3.4.2 for subscript meaning). It should be noted that identification of test pattern is for each PPI, so these feature parameters are calculated to reflect the characteristic of the entire PPI. When a PPI has been flagged as test pattern, each echo pixel in that PPI will be classified only if quality control for each echo pixel in it is still needed.

  • When the test pattern appears, compared to all other kinds of echoes (precipitation, clear air, and other non-meteorological echoes), it fills the entire radar coverage. Its reflectivity field has the most data pixels covered in a PPI. The effective reflectivity data percentage (RZ) is thus proposed to compute the coverage of total obtained data pixels in the total detection range. This feature parameter represents the reflectivity characteristic of the test pattern, and can be expressed as where Zi, j is the reflectivity at gate (i, j) [units in dBZ; i and j represent the pixel's azimuth and radius, respectively (radar coordinates)]; MZ counts the total number of effective observations in a PPI; N a and N r indicate total numbers of azimuths and gates along each beam, and N a, Z and N r, Z are N a and N r for reflectivity, respectively; and I v indicates invalid-value at gate (i, j).

    Being almost fully covered with effective data is one obvious feature of a test pattern PPI. In order to avoid classifying heavy precipitation cases as test pattern based on this feature, heavy stratiform precipitation and typhoon precipitation cases were chosen as precipitation echo samples in this study: namely, torrential rain events in Henan, Anhui and Hunan in 2010; typhoon rain in Haitang and Longwang in 2005; and typhoon rain in Saomei in 2006. However, no misclassifications caused by this parameter occurred during testing of these precipitation cases. Statistical results (Fig. 2a) show that the ratio of PPIs of heavy rain with RZ larger than 65% to total precipitation PPIs is 56/1800, and for RZ exceeding 60% the result is 154/1800. However, the RZ of test pattern PPIs larger than 65% to total test pattern PPIs is 114/114, and for RZ exceeding 60% the result is 117/144. As one of the parameters for recognizing a test pattern, RZ performs well, and therefore will not cause any problems for other kinds of echoes.

    Figure 2.  Comparison plots of Z and ZD in a random radial from (a) test pattern case and (b) precipitation echo case.

  • Both normal observations and RF ones can be obtained in the radial velocity field when it is contaminated by a test pattern. Furthermore, as mentioned above, RF mostly appears when the test pattern occurs. RF considered as a contaminated value is therefore helpful in calculating the percentage of RF pixels to the total number of effective detective pixels in a PPI, and is defined as where Vi, j is the radial velocity at gate (i, j) (radar coordinates); RF v represents the RF mark value; N a, V and N r, V are N a and N r for velocity, respectively; M RF counts the RF pixels in a PPI; and MV counts all the effective velocity pixels, including normal observations and RF.

    Generally, R RF is less than 50% when other kinds of echoes (precipitation, clear air, and other non-meteorological echoes) are observed (Fig. 2b). When radial velocity is contaminated by a test pattern, this parameter can be used to identify it.

  • When Doppler radar is operated, reflectivity and radial velocity data should correspond at the same location, pixel-to-pixel. In other words, both reflectivity and radial velocity should be effective data when meteorological echoes are observed. In this case, R M is defined as and should be very small. MZ and MV in Eq. (3) are the same as in Eqs. (1) and (2), respectively.

    Because reflectivity and radial velocity data from SA/SB radar have different gate widths (1 km for reflectivity data, 0.25 km for radial velocity data) and gate numbers (460 for reflectivity data, 920 for radial velocity data), the effective detection range is smaller for radial velocity. The azimuth and detection range of velocity should be used as the calculation domain, and then computation can be performed with all the radial velocity data and related reflectivity data at the same location. Statistical results (Fig. 2c) of all samples show that when R M is larger than 70%, the PPIs are test pattern; and when R M is between 50%-70%, they are mainly test pattern and clear air echo PPIs.

  • As mentioned above, the echo power returned from test pattern does not change much in certain fields. In order to describe this feature, a value ZD with a constant gap from the echo power is computed from observed data as where Zi, j means the same as in Eq. (1); Di, j (units: km) represents the distance between the radar center and the pixel at position (i, j).

    The variance of Z and ZD with distance plotted shows the radial at azimuth 56 (true north as azimuth 0, clockwise) for the test pattern case observed by the radar at Shenzhen at 0912 UTC 7 April 2010 (Fig. 3a), and the radial at azimuth 264 (true north as azimuth 0, clockwise) for the precipitation case observed by the radar at Zhengzhou at 0141 UTC 8 June 2010 (Fig. 3b), respectively. It is also shown in Fig. 2 that reflectivity of the test pattern increases with distance, but reflectivity of precipitation echoes changes little. ZD has no obvious changes with distance after removing the constant related to distance from Z, showing the true echo signal of the test pattern; while in contrast, the ZD of precipitation echoes changes a lot with distance.

    The reflectivity Z of the test pattern changes little with azimuth; neither does ZD, the reflectivity after correction (Fig. 3a). This means the fluctuations of the echo signal of the test pattern in a specified field is small for a given pixel.

    Therefore, Z f, the reflectivity fluctuation in a specific field is calculated as

    Here, subscript f indicates fluctuation of ZD. As shown in Fig. 4, each pixel of the test pattern has a small Z f in the specified field (11×11 in this study). Comparing the Z f value between the test pattern case and the precipitation case from the data in Fig. 3, it is obvious in Fig. 4 that the Z f of the test pattern is smaller than 1, while the precipitation echo is the opposite. Because of this feature of test pattern, these two parameters are proposed as candidates for identifying test pattern.

    Figure 3.  Comparison plot of mean variation from corrected reflectivity Z f of the same case in Fig. 2.

    Figure 4.  Scatter diagram of the five feature parameters from all samples: test pattern PPI (green dots); precipitation echo PPI (yellow dots); clear air echo PPI (blue dots); and other non-meteorological echo PPI (red dots). (a) RZ (%); (b) R RF (%); (c) R M (%); (d) RN_ r, Z (%); (e) RN_ a, Z (%).

    Figure 5.  Probability distribution [P (%)] of five feature parameters collected from the test pattern (long/short dashed line), precipitation echo (solid line), clear air echo (dot-dashed line) and other non-meteorological echo samples (dashed line): (a) RZ (%); (b) R RF (%); (c) R M (%); (d) RN_ r, Z (%); (e) RN_ a, Z (%).

    Figure 6.  Five test pattern identified membership functions plots: (a) RZ (%); (b) R RF (%); (c) R M (%); (d) RN_ r, Z (%); (e) RN_ a, Z (%).42mm

    The first three parameters mentioned above are computed directly using the whole PPI data. Two more are computed as follows, first in a radial (or a range circle), and then in the entire PPI. The reason for doing this are: (1) test pattern identification is aimed at each PPI, and therefore the calculated parameters must represent the feature of the entire PPI; and (2) a test pattern does not always show as a full circle all of the time, but as a semicircle, as fan-shaped etc. Thus, it would be very useful to identify these typical test pattern using the radial (or range circle) parameter to present the feature of the entire PPI. When PPIs that have been identified as test pattern need to be processed, especially for those that have heavy precipitation information inlayed, it would be helpful to determine whether a radial is contaminated or not by calculating the radial parameter. Then, combined with other parameters, even whether or not a pixel is contaminated can be determined.

    3.4.1. Averaged along-azimuth reflectivityfluctuation(RNr, Z)

    From the above analysis, the average fluctuation of Z f in the field from corrected reflectivity (ZD) has been computed, and Z f is small for test pattern. In order to decide whether the PPI is a test pattern, the operation should be performed on the total PPI. First, the gate with small Z f (smaller than 1) is counted along-azimuth, presented by MZ_ f in Eq. (6); and M v in Eq. (6) counts the valid Z f along-azimuth; then, comes the percentage R a of small Z f in a circle (the total pixel number is 360 because of organizing radar data within interval of 1°); for lower tilts in SA/B radar, 460 (460 gates in a radial mean 460 range circles) R a values obtained. To represent the feature of the whole PPI, R a values are counted as percentages in range circle as long the value is larger than 60%, while RN_ r, Z presents the feature of the whole PPI. Mathematically, R a and RN_ r, Z are defined as

    The subscript of RN_ r, Z indicates that the last step of calculating RN_ r, Z is along-bean. Because the corrected reflectivity (ZD) of the test pattern changes little with azimuth, as mentioned above, the RN_ r, Z of the test pattern is large for precipitation and other kinds of echoes. RN_ r, Z becomes small, and even reaches 0 at some gates.

    3.4.2. Averaged along-beam reflectivity fluctuation (RNa, z)

    Likewise, M v in Eq. (8) counts the valid Z f along-beam, then data pixels with small Z f(<1) can be counted along a radial to percentage small Z f along-beam, which is presented by R r (for lower tilts in SA/B radar, the total along-beam number is 460 because of 460 gates in a radial). Three hundred and sixty R r values are obtained because organizational radar data have azimuth intervals of 1°. R r with values larger than 50% are counted, which leads to the RNa, Z that can present the feature of the whole PPI. R r and RNa, Z are defined as MZf is the same as in Eq. (6). The subscript of RNa, Z indicates that the last step of calculating RN_ a, Z is along-azimuth. Also, RN_ a, Z for the test pattern is much larger than other kinds of echoes.

    In the above formulas, R a and R r are the percentages of small averaged reflectivity fluctuation (Z f) along-azimuth and along-beam, respectively; these then lead to the percentage of the whole PPI. For those flagged as test pattern, R r (R a) is used to determine whether or not a beam (azimuth) is contaminated; together with other parameters, such as R a(R r), Z f and R M, an echo pixel can be determined as to whether or not it is contaminated by a test pattern. The steps for checking and removing test pattern contamination in pixels will be detailed in section 5.2.

    Five feature parameters were statistically analyzed for the PPIs, which included 144 test pattern PPIs, 1800 precipitation PPIs, 565 clear air PPIs, 723 other kinds of clutter PPIs (Fig. 2). As shown in Fig. 2, the five feature parameters of the test pattern have larger values compared to all other kinds of echoes, but the test pattern could not be recognized by only one of the parameters. Therefore, the purpose of the study is to recognize test pattern PPIs as much as possible without misjudging all other kinds of echo (precipitation echo, clear air echo, electromagnetic interference and ground clutter echo) PPIs.

4. Description of the method
  • Based on contaminated echo's features, the fuzzy logic method is adapted. The five membership functions are RZ, R RF, R M, RN_ r, Z, and RN_ a, Z.

  • One hundred and forty-four test pattern PPIs, 1800 precipitation echo PPIs, 565 clear air echo PPIs, and 721 other kinds of echo (clutter, electromagnetic interference) PPIs were selected in this study. The selected data were randomly divided into two parts, and feature parameters (Fig. 5) of test pattern came from a statistical analysis of the five feature fields by using one part of the data (72 test pattern PPIs, 900 precipitation echo PPIs, 282 clear air echo PPIs and 360 other echo PPIs); the other part of the data was used to test the effect of the identification method.

    Feature fields of test pattern, as shown in Fig. 5, are quite different from all the other kinds of echoes, which is in accordance with the earlier analysis. In order to recognize a test pattern well, the fuzzy logic method is used, and membership functions (Fig. 6) of each feature field are obtained by their probability distribution. Membership functions scale each feature field to match the features of the test pattern. An output value of between 0-1 is obtained after scaling feature fields by stepwise linear membership functions. An output with value 1.0 indicates a high possibility that the undetermined echo is test pattern; likewise, an output value of 0.0 means it is impossible that the undetermined echo is test pattern.

  • Based on the features of test pattern and their membership functions in Fig. 6, a flowchart to identify test pattern is displayed in Fig. 7. The flowchart shows the classification procedure; that is, to use the fuzzy logic to give different weights to membership functions, and then use the threshold T based on the output of each PPI to make a judgment as to whether the PPI is a test pattern PPI. When a precipitation process occurs, especially a strong torrential rain event, it is important to retain meteorological information from the test pattern PPI. Therefore, each echo pixel in the test pattern PPI should be inspected to make sure contaminated echoes are recognized and then removed. The steps to do this are explained in section 5.2.

    4.2.1. Data preprocessing

    Because reflectivity and radial velocity at lower tilts observed by SA/SB radar are obtained respectively-or, in other words, these data do not correspond, pixel-to-pixel- they should be processed radially, and strictly scaled in 1° intervals to make sure they are at the same location all of the time.

    4.2.2. Reflectivity effective detection threshold

    The most obvious feature of the test pattern is filling the entire PPI, which means the test pattern will have more pixels in a PPI. Figure 8 shows a possible minimum test pattern pixels model, which has all effective echoes filling the fields of nearby radar 20 km and any 20 successive beams. This means 16000 pixels (for lowest reflectivity data) in total. The model is set up since clutter or clear air echoes always appear nearby radar 20 km, and also a test pattern is defined as a sector that has more than 20 radials successively. Therefore, 16 000 is considered as the threshold. If reflectivity pixels are smaller than this threshold, the PPI is not a test pattern, and any other steps for test pattern identification after this can be skipped. This threshold can therefore reduce computation time.

    4.2.3. Fuzzy logic method

    After the outputs of all feature fields from membership functions are obtained, a weighted mean value of all outputs is calculated. Different weights are adopted based on the performance of each feature field. The weights used here for membership functions RZ, R RF, R M, RN_ r, Z and RN_ a, Z are 3, 1, 1, 1 and 1, respectively.

    Two thresholds are used in this step. Statistical results show that the final outputs of precipitation echoes, clear air echoes and other non-meteorological echoes are lower than 0.48. Therefore, the first threshold for the test pattern is 0.48. Specifically, when the final output of each PPI is higher than 0.48, this PPI will be flagged as the test pattern. The other threshold is related to R M. Statistical results (Fig. 2c) show that when the output for the feature field of R M is higher than 55%, test pattern and clear air echo PPIs occur. Since the final output of clear air echoes is much smaller than for the test pattern, the threshold for a test pattern can be lower than 0.48 when the output value for the feature field of R M is higher than 55%. Furthermore, this threshold is set to 0.28 based on statistical results. Those PPIs that have output values for the feature field of R M higher than 55% are test pattern when the final output is higher than 0.28.

    Figure 7.  Flowchart showing the test pattern identification algorithm.

    Figure 8.  The possible test pattern model with minimum pixels. The radius of the gray shaded circle is 20 km, and the angle of the gray shaded sector is 20°.

5. Test pattern identification results
  • Test pattern PPIs cause a lot of problems for radar data applications, and so they should be identified before operational applications. For those that are identified as test pattern PPIs, they should be flagged and should not be used in radar data retrieval and assimilation. If test pattern PPIs appear in severe precipitation weather events, radar quality control should be considered, and precipitation information should be retained.

  • Test pattern can be identified based on the procedure described above. An analysis of the identification results for the other part of the PPI data (including 72 test pattern PPIs, 900 precipitation PPIs, 283 clear air PPIs, and 361 other echo PPIs) is presented below. Because this test pattern identification method is based on no misjudgment of clear air, precipitation and other non-meteorological echo PPIs by the thresholds mentioned in section 4, the results show that the identified PPI percentage for test pattern PPIs, precipitation PPIs, clear air PPIs and the other non-meteorological echo PPIs are 71/72, 900/900, 283/283, 361/361, respectively. And the accuracy percentage for all kinds of samples is 99.65%.

    In this study, the identification accuracy percentage means the proportion of how many test pattern PPIs are identified as test pattern and how many of all the other kinds of echo are not identified as test pattern. Statistical results show that this method can identify almost all test pattern PPIs without misjudging all of the other kinds of echo PPIs. It should be noted that there is one test pattern PPI in the test data that passed through the test pattern identification process. It was found that the density of pixels in this PPI was low, even though it covered the entire PPI, and it did not fit the criteria used. Similarly, the criteria cannot be lowered because this might cause some other kinds of echo PPIs to be misclassified as test pattern, which is exactly what we are trying to avoid.

  • For almost all PPIs that are identified as test pattern (especially those including little meteorological information), the flagged PPIs do not need to go through any other quality control procedure or follow radar product calculations. However, for those test pattern PPIs that mingled into precipitation echoes, the test pattern signal will be removed, and precipitation information will be retained.

    In this study, after test pattern PPIs are flagged, we need to remove test pattern pixels and also extract the precipitation information that is mingled with the test pattern pixels. The above analysis shows that, for test pattern pixels, the corrected reflectivity (ZD) actually represents the true test pattern contamination value. The specific procedure for retaining precipitation information that has been mingled with test pattern PPIs is as follows: (1) Check from pixel to pixel if the reflectivity fluctuation (Z f) is smaller than 1. If not, it means the pixel is not contaminated, so move on to another pixel; if it is, it means it is a test pattern pixel, so flag it and go to the next step. (2) Average the corrected reflectivity (ZD) along-azimuth and along-beam of the flagged pixel, respectively. (3) Average the two values from the last step. (4) After Z minus the value from the last step, the remaining part should be the true precipitation echo signal without test pattern contamination. Two test pattern cases are presented below, both of which have test pattern pixels mingled into precipitation echoes.

    5.2.1. Case one

    Figure 9 shows the test pattern PPIs and quality control results of a precipitation event observed by the radar at Shenzhen at 0418 UTC 22 April 2010. Figures 9a and c show reflectivity PPIs from tilts of 0.5° and 1.5°, respectively. It can be seen that the blockage to the north of the radar is more serious in the lower tilt, and the sector area of the test pattern appears to the south of the radar, while precipitation echoes are observed to the west of the radar. At the higher tilt, the test pattern PPI appears, while precipitation echoes are embedded to the north corner of the radar, and the reflectivity signals there are stronger than the contaminated pixels nearby, which are the compositions of precipitation signals and test pattern signals. The quality control results of these two PPIs are shown in Figs. 9b and d. In Fig. 9b, precipitation pixels observed to the west of the radar are the same as in Fig. 9a, and the test pattern sector area to the south of the radar is identified and removed. At the 1.5° tilt PPI (Fig. 9d), test pattern pixels are removed, while precipitation pixels are determined. Besides, the test pattern signal that added to the precipitation pixels are removed, which means the true precipitation information is retained. Compared to normal precipitation echoes observed at the lower tilt, an accurate shape and intensity of precipitation echoes are obtained when the tilt is raised.

    5.2.2. Case two

    Figure 10 shows the test pattern PPIs and quality control results of a precipitation event observed by the radar at Shenzhen at 0912 UTC 7 April 2010. Figures 10a and c show reflectivity PPIs from tilts of 0.5° and 1.5°, respectively. Semicircular test pattern pixels are observed at the lower tilt, while precipitation pixels appear to the west of the radar. When raising the tilt, test pattern PPIs show, and precipitation pixels are mingled with the test pattern signal to the west of the radar where the echo signal is stronger than contaminated pixels nearby. The quality control results of these two PPIs are shown in Figs. 10b and d. In Fig. 10b, precipitation echoes observed to the west of the radar are retained, and semicircular test pattern pixels to the south of the radar are identified and removed. For the 1.5° tilt PPI, test pattern contamination is removed, while precipitation pixels are determined, and the test pattern signal added to precipitation echoes are removed too. Compared to precipitation echoes observed at the low tilt, an accurate shape and intensity of precipitation echoes are obtained when the tilt is raised. Meanwhile, it should be noted in Fig. 10b that a few precipitation pixels located southwest of the radar around 200-300 km are removed as test pattern. In general, identifying and removing test pattern signals performed well, and precipitation echoes were retained.

    Figure 9.  Reflectivity observed by the radar at Shenzhen at 0418 UTC 22 April 2010 (units: dBZ, range circle: 100 km): (a) 0.5° tilt before quality control; (b) 0.5° tilt after quality control; (c) 1.5° tilt before quality control; (d) 1.5° tilt after quality control.

    Figure 10.  Reflectivity observed by the radar at Shenzhen at 0912 UTC 7 April 2010 (units: dBZ, range circle: 100 km): (a) 0.5° tilt before quality control; (b) 0.5° tilt after quality control; (c) 1.5° tilt before quality control; (d) 1.5° tilt after quality control.

6. Conclusions
  • In this study, test pattern observed from operational CINRAD/SA(B) were analyzed. After comparing with precipitation, clear air and other non-meteorological echo (clutter, electromagnetic interference) PPIs by using both reflectivity and radial velocity data, statistical results of five feature parameters that could identify test pattern were studied. An identification method based on the fuzzy logic was applied to obtain membership functions that could identify test pattern, and observations were used for test pattern identification tests. The main findings can be summarized as follows.

    (1) Thirty-eight volumes of test pattern data (total of 144 test pattern PPIs), collected from CINRAD/SA(B) in Shenzhen, Hefei, Yancheng, Qingdao, Zhumadian and Beijing in 2008 and 2010, were analyzed. The test pattern is a fake signal caused by uncertain hardware failures. There are no certain rules as to when they appear; they may occur in several continuous volumes, or only one or several tilts in a volume. Both reflectivity and radial velocity data may be contaminated or only reflectivity data. When radial velocity is also contaminated by the test pattern, the contaminated pixels have an RF value, and sometimes at higher tilts ( >4°) the contaminated velocity might have a value of zero. Test pattern may spread all over the detection plane, or cover just half, or even display as a fan shape.

    (2) After comparing test pattern PPIs with precipitation echo PPIs, clear air echo PPIs, and other kinds of non-meteorological echo (clutter, electromagnetic interference) PPIs, five feature parameters that can distinguish test pattern were proposed: RZ, R RF, R M, and two other parameters (RN_ r, Z and RN_ a, Z) that arise from the reality that test pattern have small reflectivity fluctuations. These five feature parameters can present the characteristics of test pattern well, such as: more effective pixels of reflectivity pixels in a PPI, velocity pixels may be contaminated as well while the contaminated velocity has the value of RF, and small reflectivity fluctuations. As a result of the above, test pattern PPIs were able to be identified effectively.

    (3) Procedures for identifying test pattern PPIs were established based on their features. Statistical results of the test pattern identification method were obtained by analyzing the various types of echoes. Identification accuracy is calculated when test pattern PPIs are identified as test pattern and all other types of echo PPIs are not identified as test pattern. It turned out that the method is able to identify almost all test pattern PPIs without misjudging precipitation, clear air and other kinds of non-meteorological echo PPIs.

    (4) Case studies showed that the position and intensity of a pixel contaminated by the test pattern in a PPI could be confirmed by the test pattern identification feature parameters RN_ r, Z and RN_ a, Z, and the reflectivity fluctuations Z f and corrected reflectivity ZD. Especially when some precipitation pixels are contaminated by test pattern signal, the test pattern signal can be removed, and precipitation information can be retained.

    There are no certain rules as to when test pattern appears. Sometimes there is no contamination at lower tilts, while high tilts are contaminated, or vice versa. Therefore, relying solely on radar operators to identify test pattern is not sufficient. Especially, in order to avoid using contaminated radar data in subsequent applications, such as radar data 3D gridding, it is very important to identify test pattern PPIs. Furthermore, it is also very useful to accurately retain meteorological information mixed with test pattern for obtaining radar productions in real-time operation and for radar data assimilation. This research will eventually be applied to SWAN, ROSE and radar data quality control systems for radar data assimilation in numerical models after more testing, and will then further enhance the capability of radar data application in operations, as well as the capability of forecasting in numerical models.

Reference

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint