Forecast Evaluation of Short-Term Heavy Precipitation from Operational Models by the Fractions Skill Score Method Based on the Cumulative Climatological Probability

In this paper, to gain insight into the performance of numerical weather prediction models on refined heavy rainfall, a neighborhood verification method named Fractions Skill Score (FSS) is introduced to explore the capability of six operational models on 3-h accumulated precipitation during the warm season in 2019 in Zhejiang Province, with an emphasis on short-term torrential rain and the impacts of different weather backgrounds. (1) According to the cumulative climatological probability of 3-h accumulated precipitation at the station, the FSS method is improved by determining the skill score thresholds for 5-grade precipitation with thresholds of 0.1, 3.0, 10.0, 20.0, and 50.0 mm/3 h to be 0.583, 0.522, 0.506, 0.502, and 0.500, respectively, and applied in the evaluation of their prediction skill scales in the long time series. (2) The regional models perform better than the global ones in heavy rain forecasts, and only the mean scores of the Shanghai Regional Numerical Weather Prediction System and Zhejiang Regional Numerical Weather Prediction System realize the forecast skills for torrential rain, and their corresponding skill scales are 159, 159, and 183 km, respectively. There are approximately 60% of forecasts in these models that achieve the prediction skills, and the cumulative frequency of the skill scale is increased by nearly 50% from 3 to 183 km. These scale-selective evaluation results can provide guidance for the application of model products on different scales. (3) There are evident differences in the model performance of precipitation forecasts under various weather backgrounds. The best models for predicting torrential rains under the backgrounds of tropical cyclones, Meiyu fronts, and weak synoptic forcing are the Global Forecast Model from the European Centre for Medium-Range Weather Forecasts (EC-GFS), Shanghai Regional Numerical Weather Prediction System, and Zhejiang Regional Numerical Weather Prediction System, with skill scales of 27, 99, and 135 km, respectively. Therefore, the application of model products should be treated differently in terms of weather background type.