Oct.  2019

Article Contents

# A Model Output Machine Learning Method for Grid Temperature Forecasts in the Beijing Area

Funds:

National Key Research and Development Program of China (Grant Nos. 2018YFF0300104 and 2017YFC0209804) and the National Natural Science Foundation of China (Grant No. 11421101) and Beijing Academy of Artifical Intelligence (BAAI)

• In this paper, the model output machine learning (MOML) method is proposed for simulating weather consultation, which can improve the forecast results of numerical weather prediction (NWP). During weather consultation, the forecasters obtain the final results by combining the observations with the NWP results and giving opinions based on their experience. It is obvious that using a suitable post-processing algorithm for simulating weather consultation is an interesting and important topic. MOML is a post-processing method based on machine learning, which matches NWP forecasts against observations through a regression function. By adopting different feature engineering of datasets and training periods, the observational and model data can be processed into the corresponding training set and test set. The MOML regression function uses an existing machine learning algorithm with the processed dataset to revise the output of NWP models combined with the observations, so as to improve the results of weather forecasts. To test the new approach for grid temperature forecasts, the 2-m surface air temperature in the Beijing area from the ECMWF model is used. MOML with different feature engineering is compared against the ECMWF model and modified model output statistics (MOS) method. MOML shows a better numerical performance than the ECMWF model and MOS, especially for winter. The results of MOML with a linear algorithm, running training period, and dataset using spatial interpolation ideas, are better than others when the forecast time is within a few days. The results of MOML with the Random Forest algorithm, year-round training period, and dataset containing surrounding gridpoint information, are better when the forecast time is longer.
摘要: 数值天气预报的预报结果可以通过天气会商来进行提高，本文提出了模式输出机器学习(MOML)方法对天气会商过程进行模拟，从而提高数值预报结果。通过天气会商，预报员利用预报经验知识结合数值预报结果和观测数据得到最终的天气预报结果。显然，利用合适的模式后处理算法模拟预报员天气会商的过程是一个有趣和重要的课题。MOML方法是一个基于机器学习的模式后处理方法，它通过一个回归函数将数值预报结果跟观测数据进行配置。对数据集和训练期采用不同的特征工程技术，我们把观测数据和模式数据处理为不同的训练集和测试集，之后再将已有的机器学习回归算法应用到处理后的数据集中，从而提高模式结果。我们把这个方法应用到北京地区2米格点地表气温的ECMWF模式后处理中来进行检验。我们设计了各种特征工程方案，得到了不同的MOML算法模型，并和ECMWF模式结果以及模式输出统计(MOS)方法进行比较。数值结果表明，MOML方法的结果比ECMWF模式结果和MOS方法更好，尤其是冬季更明显。其中最好的MOML特征工程混合方案是短期预报用线性回归、滑动训练器和基于空间插值思想的数据集的组合，中期预报用随机森林、全年训练器和包含周围格点的数据集的组合。
• Figure 1.  Diagram of a regression tree generation algorithm, where xj is the optimal splitting features and aj is the optimal splitting point.

Figure 2.  Flow diagram of the MOML method. The blue cuboids are the original data in the Beijing area, and the green cuboids are the dataset with proper feature engineering. The yellow cuboid represents the process of machine learning, and the orange rectangle represents the output.

Figure 3.  Diagram of datasets 1−3. Dataset 1 focuses on the fixed spatial point, and dataset 2 adds the surrounding eight grid points. Dataset 3 takes all the 30 spatial points of the Beijing area into account in a unified way.

Figure 4.  Results of the ${\rm{lr}}\_3\_{\rm{r}}$, ${\rm{rf}}\_2\_{\rm{y}}$ and ${\rm{mos}}\_{\rm{r}}$ models, using one-year temperature grid data in the Beijing area as the test set. Left: TRMSE (RMSE; units: °C). Right: Fa (forecast accuracy; units: %). (a) shows ${\rm{lr}}\_3\_{\rm{r}}$ has obvious advantages when the forecast time is 1−9 days, and (b) shows ${\rm{rf}}\_2\_{\rm{y}}$ is superior to other models in the whole forecast period, especially in the longer period.

Figure 5.  A feasible solution fMOML to the grid temperature correction in the Beijing area. fMOML uses the ${\rm{lr}}\_3\_{\rm{r}}$ method for days 1−6 of the forecast lead time and the ${\rm{rf}}\_2\_{\rm{y}}$ method for days 7−15, and it has a lot of advantages in the whole forecast period.

Figure 6.  Results of grid temperature forecasts in the Beijing area in November (a), December (b), January (c) and February (d). In these months, the forecast results of the ECMWF model do not work well, and the linear methods ${\rm{lr}}\_3\_{\rm{r}}$ are better than other methods.

Figure 7.  Results of grid temperature forecasts in the Beijing area in March (a), June (b), July (c), August (d) and October (e). In these five months, the forecast results of the ECMWF model are better than those in winter months. The linear methods are better than other methods when the forecast lead time is short, and Random Forest algorithm are better when the forecast lead time is relatively long.

Figure 8.  Results of grid temperature forecasts in the Beijing area in April (a), May (b) and September (c). In these three months, the forecast results of the ECMWF model in these three months are better than those in the other months. The multiple linear regression algorithm is best in the first few days of the forecast period, and the Random Forest algorithm is better than for other methods in the next few days.

•  Alessandrini, S., L. D. Monache, S. Sperati, and G. Cervone, 2015: An analog ensemble for short-term probabilistic solar power forecast. Applied Energy, 157, 95−110, https://doi.org/10.1016/j.apenergy.2015.08.011. Alpaydin, E., 2014: Introduction to Machine Learning. 3rd ed., The MIT Press, 640 pp. Bishop, C. M., 2006: Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag, 738 pp. Bogoslovskiy, N. N., S. I. Erin, I. A. Borodina, L. I. Kizhner, and K. A. Alipova, 2016: Satellite data assimilation in global numerical weather prediction model using kalman filter. Proceedings of SPIE 10035, 22nd International Symposium Atmospheric and Ocean Optics: Atmospheric Physics, Tomsk, Russian Federation, SPIE, 100356Z, https://doi.org/10.1117/12.2249275. Breiman, L., 2001a: Random forests. Machine Learning, 45(1), 5−32, https://doi.org/10.1023/A:1010933404324. Breiman, L., 2001b: Statistical modeling: The two cultures. Statistical Science, 16(3), 199−215. Buehner, M., R. McTaggart-Cowan, and S. Heilliette, 2017: An ensemble Kalman filter for numerical weather prediction based on variational data assimilation: VarEnKF. Mon. Wea. Rev., 145(2), 617−635, https://doi.org/10.1175/MWR-D-16-0106.1. Cabos, R., P. Hecker, N. Kneuper, and J. Schiefele, 2017: Wind forecast uncertainty prediction using machine learning techniques on big weather data. Proceedings of the 17th AIAA Aviation Technology, Integration, and Operations Conference, Denver, Colorado, AIAA. Cassola, F., and M. Burlando, 2012: Wind speed and wind energy forecast through Kalman filtering of numerical weather prediction model output. Applied Energy, 99, 154−166, https://doi.org/10.1016/j.apenergy.2012.03.054. Chattopadhyay, R., A. Vintzileos, and C. D. Zhang, 2013: A description of the Madden-Julian oscillation based on a self-organizing map. J. Climate, 26(5), 1716−1732, https://doi.org/10.1175/JCLI-D-12-00123.1. Cheng, W. Y. Y., and W. J. Steenburgh, 2007: Strengths and weaknesses of MOS, running-mean bias removal, and Kalman filter techniques for improving model forecasts over the western United States. Wea. Forecasting, 22(6), 1304−1318, https://doi.org/10.1175/2007WAF2006084.1. Delle Monache, L., T. Nipen, Y. B. Liu, G. Roux, and R. Stull, 2011: Kalman filter and analog schemes to postprocess numerical weather predictions. Mon. Wea. Rev., 139(11), 3554−3570, https://doi.org/10.1175/2011MWR3653.1. Domingos, P., 2012: A few useful things to know about machine learning. Communications of the ACM, 55, 78−87. Glahn, B., 2014: Determining an optimal decay factor for bias-correcting MOS temperature and dewpoint forecasts. Wea. Forecasting, 29(4), 1076−1090, https://doi.org/10.1175/WAF-D-13-00123.1. Glahn, B., M. Peroutka, J. Wiedenfeld, J. Wagner, G. Zylstra, B. Schuknecht, and B. Jackson, 2009: MOS uncertainty estimates in an ensemble framework. Mon. Wea. Rev., 137(1), 246−268, https://doi.org/10.1175/2008MWR2569.1. Glahn, H. R., and D. A. Lowry, 1972: The use of model output statistics (MOS) in objective weather forecasting. J. Appl. Meteor., 11(8), 1203−1211, https://doi.org/10.1175/1520-0450(1972)011<1203:TUOMOS>2.0.CO;2. Hart, K. A., W. J. Steenburgh, D. J. Onton, and A. J. Siffert, 2003: An evaluation of mesoscale-model-based model output statistics (MOS) during the 2002 Olympic and Paralympic winter games. Wea. Forecasting, 19(2), 200−218, https://doi.org/10.1175/1520-0434(2004)019<0200:AEOMMO>2.0.CO;2. Haupt, S. E., and B. Kosovic, 2016: Big data and machine learning for applied weather forecasts: Forecasting solar power for utility operations. Proceedings of 2015 IEEE Symposium Series on Computational Intelligence, Cape Town, South Africa, IEEE, 496−501, https://doi.org/10.1109/SSCI.2015.79. Haupt, S. E., A. Pasini, and C. Marzban, 2009: Artificial Intelligence Methods in the Environmental Sciences. Springer, https://doi.org/10.1007/978-1-4020-9119-3. Jacks, E., J. Brent Bower, V. J. Dagostaro, J. Paul Dallavalle, M. C. Erickson, and J. C. Su, 2009: New NGM-based MOS guidance for maximum/minimum temperature, probability of precipitation, cloud amount, and surface wind. Wea. Forecasting, 5(5), 128−138, https://doi.org/10.1175/1520-0434(1990)005<0128:NNBMGF>2.0.CO;2. Junk, C., L. Delle Monache, and S. Alessandrini, 2015: Analog-based ensemble model output statistics. Mon. Wea. Rev., 143(7), 2909−2917, https://doi.org/10.1175/MWR-D-15-0095.1. Lakshmanan, V., E. Gilleland, A. McGovern, and M. Tingley, 2015: Machine Learning and Data Mining Approaches to Climate Science. Springer International Publishing, https://doi.org/10.1007/978-3-319-17220-0. Marzban, C., S. Sandgathe, and E. Kalnay, 2006: MOS, perfect prog, and reanalysis. Mon. Wea. Rev., 134(2), 657−663, https://doi.org/10.1175/MWR3088.1. Mass, C. F., D. Ovens, K. Westrick, and B. A. Colle, 2002: Does increasing horizontal resolution produce more skillful forecasts? Bull. Amer. Meteor. Soc., 83(3), 407−430, https://doi.org/10.1175/1520-0477(2002)083<0407:DIHRPM>2.3.CO;2. Mirkin, B., 2011: Data analysis, mathematical statistics, machine learning, data mining: Similarities and differences. Proceedings of 2011 International Conference on Advanced Computer Science and Information Systems, Jakarta, Indonesia, IEEE, 1−8. Mjolsness, E., and D. Decoste, 2001: Machine learning for science: State of the art and future prospects. Science, 293(5537), 2051−2055, https://doi.org/10.1126/science.293.5537.2051. Molteni, F., R. Buizza, T. N. Palmer, and T. Petroliagis, 1996: The ECMWF ensemble prediction system: Methodology and validation. Quart. J. Roy. Meteor. Soc., 122(529), 73−119, https://doi.org/10.1002/qj.49712252905. Monache, L. D., F. A. Eckel, D. L. Rife, B. Nagarajan, and K. Searight, 2013: Probabilistic weather prediction with an analog ensemble. Mon. Wea. Rev., 141(10), 3498−3516, https://doi.org/10.1175/MWR-D-12-00281.1. Paegle, J., Q. Yang, and M. Wang, 1997: Predictability in limited area and global models. Meteor. Atmos. Phys., 63(1−2), 53−69, https://doi.org/10.1007/BF01025364. Pelosi, A., H. Medina, J. Van Den Bergh, S. Vannitsem, and G. B. Chirico, 2017: Adaptive Kalman filtering for postprocessing ensemble numerical weather predictions. Mon. Wea. Rev., 145(12), 4837−4584, https://doi.org/10.1175/MWR-D-17-0084.1. Peng, X. D., Y. Z. Che, and J. Chang, 2013: A novel approach to improve numerical weather prediction skills by using anomaly integration and historical data. J. Geophys. Res., 118(16), 8814−8826, https://doi.org/10.1002/jgrd.50682. Peng, X. D., Y. Z. Che, and J. Chang, 2014: Observational calibration of numerical weather prediction with anomaly integration. Proceedings of the EGU General Assembly Conference Abstracts, Vienna, Austria, EGU. Plenković, I. O., L. D. Monache, K. Horvath, M. Hrastinski, and A. Bajić, 2016: Probabilistic wind speed predictions with an analog ensemble. Proceedings of the 6th EMS Annual Meeting & 11th European Conference on Applied Climatology, Trst, Italija, ECAC. Rudack, D. E., and J. E. Ghirardelli, 2010: A comparative verification of localized aviation model output statistics program (LAMP) and numerical weather prediction (NWP) model forecasts of ceiling height and visibility. Wea. Forecasting, 25(4), 1161−1178, https://doi.org/10.1175/2010WAF2222383.1. Schiller, H., and R. Doerffer, 1999: Neural network for emulation of an inverse model operational derivation of case II water properties from MERIS data. Int. J. Remote Sens., 20(9), 1735−1746, https://doi.org/10.1080/014311699212443. Sperati, S., S. Alessandrini, and L. Delle Monache, 2017: Gridded probabilistic weather forecasts with an analog ensemble. Quart. J. Roy. Meteor. Soc., 143, 2874−2885, https://doi.org/10.1002/qj.3137. Toth, Z., and E. Kalnay, 1997: Ensemble forecasting at NCEP and the breeding method. Mon. Wea. Rev., 125(12), 3297−3319, https://doi.org/10.1175/1520-0493(1997)125<3297:EFANAT>2.0.CO;2. Tsang, L., Z. Chen, S. Oh, R. J. Marks, and A. T. C. Chang, 1992: Inversion of snow parameters from passive microwave remote sensing measurements by a neural network trained with a multiple scattering model. IEEE Trans. Geosci. Remote Sens., 30, 1015−1024, https://doi.org/10.1109/36.175336. Veenhuis, B. A., 2013: Spread calibration of ensemble MOS forecasts. Mon. Wea. Rev., 141(7), 2467−2482, https://doi.org/10.1175/MWR-D-12-00191.1. Wilks, D. S., and T. M. Hamill, 2007: Comparison of ensemble-MOS methods using gfs reforecasts. Mon. Wea. Rev., 135(6), 2379−2390, https://doi.org/10.1175/MWR3402.1. Woo, W. C., and W. K. Wong, 2017: Operational application of optical flow techniques to radar-based rainfall nowcasting. Atmosphere, 8(3), 48, https://doi.org/10.3390/atmos8030048. Wu, J., H. Q. Pei, Y. Shi, J. Y. Zhang, and Q. H. Wang, 2007: The forecasting of surface air temperature using BP-MOS method based on the numerical forecasting results. Scientia Meteorologica Sinica, 27(4), 430−435, https://doi.org/10.3969/j.issn.1009-0827.2007.04.012. (in Chinese) Wu, Q., M. Han, H. Guo, and T. Su, 2016: The optimal training period scheme of MOS temperature forecast. Journal of Applied Meteorological Science, 27(4), 426−434, https://doi.org/10.11898/1001-7313.20160405. (in Chinese) Zhang, X. N., J. Cao, S. Y. Yang, and M. H. Qi, 2011: Multi-model compositive MOS method application of fine temperature forecast. Journal of Yunnan University, 33(1), 67−71. (in Chinese)

Export:

## Manuscript History

Manuscript revised: 15 April 2019
Manuscript accepted: 13 May 2019
###### 通讯作者: 陈斌, bchen63@163.com
• 1.

沈阳化工大学材料科学与工程学院 沈阳 110142

## A Model Output Machine Learning Method for Grid Temperature Forecasts in the Beijing Area

###### Corresponding author: Pingwen ZHANG, pzhang@pku.edu.cn
• 1. School of Mathematical Sciences, Peking University, Beijing 100871, China
• 2. Institute of Atmospheric Physics Chinese Academy of Sciences, Beijing 100029, China
• 3. Beijing Meteorological Service, Beijing 100089, China
• 4. School of Science, Beijing University of Posts and Telecommunications, Beijing 100876, China

Abstract: In this paper, the model output machine learning (MOML) method is proposed for simulating weather consultation, which can improve the forecast results of numerical weather prediction (NWP). During weather consultation, the forecasters obtain the final results by combining the observations with the NWP results and giving opinions based on their experience. It is obvious that using a suitable post-processing algorithm for simulating weather consultation is an interesting and important topic. MOML is a post-processing method based on machine learning, which matches NWP forecasts against observations through a regression function. By adopting different feature engineering of datasets and training periods, the observational and model data can be processed into the corresponding training set and test set. The MOML regression function uses an existing machine learning algorithm with the processed dataset to revise the output of NWP models combined with the observations, so as to improve the results of weather forecasts. To test the new approach for grid temperature forecasts, the 2-m surface air temperature in the Beijing area from the ECMWF model is used. MOML with different feature engineering is compared against the ECMWF model and modified model output statistics (MOS) method. MOML shows a better numerical performance than the ECMWF model and MOS, especially for winter. The results of MOML with a linear algorithm, running training period, and dataset using spatial interpolation ideas, are better than others when the forecast time is within a few days. The results of MOML with the Random Forest algorithm, year-round training period, and dataset containing surrounding gridpoint information, are better when the forecast time is longer.

Reference

/