Hybrid Bayesian-Machine Learning Framework for Multi-Profile Atmospheric Retrieval from Hyperspectral Infrared Observations

Accurate retrieval of atmospheric vertical profiles is critical for improving weather prediction and climate monitoring. However, the complexity of atmospheric processes in cloudy regions pose challenges as compared to clear sky scenarios. This study presents a novel framework integrating Bayesian optimization and machine learning approaches to retrieve atmospheric vertical profiles—including temperature, humidity, ozone concentration, cloud fraction, ice water content (IWC), and liquid water content (LWC)—from hyperspectral infrared observations. Specifically, a Bayesian method was used to refine ERA5 reanalysis data by minimizing brightness temperature (BT) discrepancies against FY-4B Geostationary Interferometric Infrared Sounder (GIIRS) observations, generating a high-quality profile database (approximately, 2.8 million profiles) across diverse weather systems. The optimized profiles improve radiative consistency, reducing BT biases from >40 K to <10 K in cloudy regions. To further overcome the limitations of the Bayesian method, we developed a Transformer-Resnet hybrid model (TERNet), achieving superior performance with RMSE values of 1.61 K (temperature), 5.77% (humidity), and 2.25×10⁻⁶/6.09×10⁻⁶ kg/kg (IWC/LWC) across the entire vertical levels in all-sky conditions. The TERNet outperforms both ERA5 in cloud parameter retrieval and GIIRS L2 product in thermodynamic profiling. Independent verification with radiosonde and Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) datasets confirms the framework’s reliability across various meteorological regimes. This work demonstrates the capability of combining physics-informed Bayesian methods with data-driven machine learning to fully exploit hyperspectral IR data.