Quality Assessment and Forecast Sensitivity of Global Remote Sensing Observations

The accuracy of numerical weather prediction (NWP) largely depends on the initial condition (analysis). The four-dimensional variational (4D-Var) data assimilation system used in the UKMO Unified Model (UM), which prepares the analysis for the GCM, uses observations from a variety of surface-based and space-based platforms. The 4D-Var assimilation system combines the information from observations and background fields (model state) and uses a linearized forecast model to ensure that the observations are dynamically consistent in the analysis field (Talagrand and Courtier, 1987; Courtier et al., 1994; Rawlins et al., 2007). The effective monitoring of such a system with a large number of observations per 6-h assimilation cycle is an important task, requiring great efficiency. During the past few years, efforts have been made to assimilate more satellite observations (Dee and Uppala, 2009; Polkinghorne and Vukicevic, 2011), but it is important to know the usefulness of these observations in the analysis and forecast. The assessment of the contribution of individual observations to the analysis field is the most challenging task in data assimilation and NWP systems (Cardinali, 2009a).

Recently, adjoint-based forecast sensitivity to observation (FSO) methods have been used to measure the observation contribution to the forecast (Zhu and Gelaro, 2008; Cardinali, 2009a; Joo et al., 2013). The adjoint-based FSO method has the advantage of measuring the impact of all the observations together and present in the assimilation system, and this methodology can infer the impact of observations at different pressure levels in detail. In general, this methodology can be used to estimate the sensitivity with respect to any important parameter in the assimilation system (Cardinali, 2009b). The use of adjoint-based FSO has been increasing over the last few years. Many studies have discussed the importance of adjoint-based FSO regarding the impact of individual observations used in global NWP systems (Joo et al., 2013; Lupu et al., 2015). In one study, (Gelaro et al., 2010) used adjoint-based FSO to compare the observation impact on 24-h forecasts in three different forecast systems: the Goddard Earth Observing System-5 (GEOS-5) of the NASA Global Modeling and Assimilation Office, the Navy Operational Global Atmospheric Prediction System (NOGAPS) of the Naval Research Laboratory, and the Global Deterministic Prediction System (GDPS) of Environment Canada. (Bonavita, 2014) studied the forecasting impact of global positioning system radio occultation (GPSRO) observations in a data denial assimilation experiment with respect to the full observation system. They found a statistically significant positive impact on the tropospheric synoptic skill scores. (Cardinali, 2009b) showed that GPSRO observations have a stronger impact in the stratosphere than in the troposphere.

(Le Marshall et al., 2005) studied the impact of atmospheric infrared sounder (AIRS) observations on the NWP system and showed that the AIRS radiance observations significantly improved global forecasting skill. The impacts of various subsets of space-based observations were studied by (Joo et al., 2013), particularly those from instruments on board the European Organisation for the Exploitation of Meteorological Satellites Meteorological Operational-A (MetOp-A) platform. They found that satellite data accounted for 64% of the short-range global forecasting error reduction, with the remaining 36% coming from the assimilation of surface-based observations. In addition, they showed that the GPSRO technique had the largest mean impact per profile of observations among satellite types.

In this study, our focus is mainly on the satellite-derived wind (SATWIND) from four different types of geostationary satellites: GOES [National Oceanic and Atmospheric Administration/National Environmental Satellite, Data and Information System (NOAA/NESDIS) in the United States], METEOSAT-7, 10 [European Organisation for Meteorological Satellites (EUMETSAT)], and MTSAT-2 [Multifunction Transport Satellite, Japanese Meteorological Agency (JMA)]. In the first part of our study, we calculate the statistics of the observation increments (observation minus background; hereafter, O-B) from SATWIND observations and the short-range forecast from the UM background. The aim of this study is to analyze the SATWIND observation increment (O-B) values at the different pressure levels (lower level, >700 hPa; middle level, 700-400 hPa; and upper level, <400 hPa). The O-B values can tell where the large values come from and how they vary from one time to another. They help us to understand the observation error characteristics, to improve error treatment in the operational NWP system. To further investigate the impacts of these SATWIND observations in the short-range 24-h forecast, we use adjoint-based FSO. The adjoint-based FSO can tell what type of observations or subsets of observations are actually improving the forecast skill. The impact of the SATWIND observations within the National Centre for Medium Range Weather Forecasting (NCMRWF) NWP system for 20 days (1-20 January 2015), with four assimilation cycles per day (0000, 0600, 1200 and 1800 UTC), is diagnosed using the adjoint-based FSO technique with 24-h short-range forecasts. This study also discusses the use of adjoint-based FSO to identify the impact of various satellite wind observations on the 24-h forecast error reduction in different regions. In addition, we also illustrate the impact of radiances from hyperspectral sensors like the infrared atmospheric sounding interferometer (IASI) and AIRS, and the impact of GPSRO in 24-h short-range forecasts.

In section 2 we describe the model configuration, the data, and the FSO method used in this study; section 3 presents the results, and section 4 the summary and discussion.

Figure 1.  Spatial distribution of four different types of SATWIND observations before quality control and thinning (gray dots) and the observations actually assimilated (black dots) in the NWP system for a typical day, 1 January 2015 at the 0000 UTC assimilation cycle.

We use the NCMRWF global UM (NCUM), which is based on the PS28 version of the UKMO UM (Rajagopal et al., 2012). It includes an atmospheric forecast model (UM), an observation processing system (OPS), and a 4D-Var data assimilation system (VAR). The UM forecast model has a horizontal resolution of N512 (∼25 km in the midlatitudes), with 70 vertical levels up to 80 km height. In the OPS, data quality control is performed based on an internal consistency check against the model background and against neighboring observations. The quality control in OPS is based on the estimation of probability gross error (PGE; Lorenc and Hammon, 1988; Ingleby and Lorenc, 1993) of each observation. The threshold value of PGE is set to 0.5. Only those observations having a PGE value of less than 0.5 are used in the 4D-Var assimilation system. The pre-processing of satellite data in the NCUM system (like GOES, IASI and Aqua-AIRS radiances), including retrievals and bias correction applied, are discussed by (Prasad, 2014). The VAR system is based on an incremental 4D-Var assimilation technique (Rawlins et al., 2007), which prepares the atmospheric initial condition for the UM model. The VAR system is capable of assimilating various conventional as well as satellite observations. The analysis increments are computed from N216 state variables (resolution of ∼60 km). The 4D-Var assimilation is conducted on a 6-h data assimilation cycle. Each cycle includes observations during a 6-h period. The background error covariance matrix, a climatological file, is used in this study, including four control variables. The four control variables are stream function, velocity potential, unbalance pressure, and humidity. The latest version of the NCMRWF variational data assimilation system, along with details of its analysis product, including a surface analysis system, is discussed by (George et al., 2016). The UM 4D-Var system requires a linear perturbation forecast (PF) model rather than a tangent linear model (Lorenc, 2003). The PF model is designed to approximate the perturbation to the nonlinear forecast due to a finite change to the initial conditions with similar amplitude to the analysis increment (Rawlins et al., 2007). The study by (Stiller and Ballard, 2009) discussed how the model's parameterizations are constructed. The observation increment statistics (O-B) are computed against the model background, produced at N512 resolution, by the UM global model short-range forecast (forecast with 6-h lead time). The background fields are valid at the same time and location as the observations. The observation increment statistics computations for this study are carried out for all four data assimilation cycles per day for the period 1-20 January 2015.

The atmospheric motion vector (AMV) wind fields from satellites are derived from the tracking of cloud and water vapor features (Hayashi and Shimoji, 2012). Wind observations are very important over the tropics, where they cannot be inferred from temperature because of the weak geostrophic wind relationship. The AMV from geostationary satellites provides tropospheric wind information with near global coverage at a high temporal frequency. There have been significant updates to SATWIND observations, which come from the geostationary satellites, such as METEOSAT, GOES, and MTSAT, and the polar orbiting satellites containing moderate resolution imaging spectroradiometers (MODIS), like the NASA spacecraft TERRA and AQUA. Many changes in the use of SATWIND in the NWP system have been implemented in recent years. These include temporal and spatial thinning schemes, observation error statistics, and bias correction methods (Cotton, 2014). Before the OPS in the UM system, all SATWIND data are packed in an OBSTORE file format with different batch numbers and individual element lists. A single SATWIND OBSTORE file contains observations from all the geostationary as well as polar satellites. Details of the SATWIND data pre-processing with UM and the list of observations used can be found in (Prasad and Indira Rani, 2014). A spatial distribution of the four different types of geostationary SATWIND observations before quality control and thinning, along with the observations actually assimilated in the NWP system, for a typical day (1 January 2015 for the 0000 UTC assimilation cycle), is shown in Fig. 1. It is noted that less than 10% of the wind data are actually used in the assimilation system after quality control and thinning to prepare the analysis field for the single assimilation cycle. Among all the SATWIND observations, only about 1.25% of GOES and 2.5% of MTSAT wind data are assimilated in the UM system (Prasad and Indira Rani, 2014). This is because of the different quality control and thinning criteria of the UM system depending on the types of SATWIND observations. It results in a large variation in number of assimilated observations from one assimilation cycle to another.

The impact of GPSRO observations from the Global Navigation Satellite System Receiver for Atmospheric Sounding (GRAS), the Constellation Observing System for Meteorology, Ionosphere, and Climate (COSMIC), and the Gravity Recovery and Climate Experiment (GRACE-A/B) satellite data are studied here. These datasets are regularly assimilated in the 4D-Var UM system. The GPSRO observations have high accuracy with a relatively high vertical resolution of the moisture profile (Kursinski et al., 1997). In a study by (Eyre, 2008), it was shown that these observations have low systematic errors and can deliver similar information to that available from satellite radiances (Collard and Healy, 2003). The GPSRO observations were found to improve the NWP forecast skill and to decrease temperature biases (model-induced) in the analysis (Bonavita, 2014). In addition, the impact of hyperspectral radiances from IASI on board MetOp-A (Prunet et al., 1998) and AIRS on board the NASA-AQUA satellite (Aumann et al., 2003) was also studied.

The hyperspectral radiances from IASI and AIRS are processed through the OPS and assimilated in the VAR System. AIRS is a grating spectrometer with 2378 channels covering the thermal infrared spectrum between 3 and 15 μm. A total of 324 channels are selected for assimilation (out of 2378 channels) according to the peak weighting function and meteorological importance (Collard, 2004). Also, according to the surface type and day-night conditions, some of the channels are not assimilated in the VAR. An OSSE (observing system simulation experiment) was conducted to identify the impact of the AIRS radiance in the NCUM forecast system (Srinivas et al., 2016). The impact of AIRS is positive in the hyperspectral radiances, like IASI, CrIS (Cross-track Infrared Sounder), and in infrared and microwave sensors.

In addition, IASI provides information on the vertical structure of the atmospheric temperature and humidity with an accuracy of 1 K and a vertical resolution of 1 km. It measures the radiance emitted from Earth in 8641 channels, covering the spectral interval 645-2760 cm^-1. The high volume of data resulting from IASI presents many challenges, particularly in the area of assimilation (Collard, 2007). A total of 314 channels are selected (out of these 8641), depending on the relevance of information in each channel to be assimilated in the 4D-Var system. The assimilation of hyperspectral radiances like AIRS and IASI improves the analysis field, with a positive impact in the forecast (Sharma et al., 2016; Mallick et al., 2016).

Many studies have discussed the satellite retrievals methods of temperature and water vapor (Nerry et al., 1988; Salisbury and D'Aria, 1992). These satellite-retrieved atmospheric variables depend on surface parameter estimation. The surface contribution to the simulated radiances is limited by the uncertainties regarding skin temperature and land emissivity. The hyperspectral radiances from infrared instruments are exploited more over ocean than over land (Vincensini et al., 2012). In addition, the surface temperature varies much more over land than over sea and the land surface emissivity varies with wavelength, surface type, roughness, soil moisture and viewing angle (Snyder et al., 1998; Seemann et al., 2008). It should be noted that, over the ocean, the emissivity is provided by the Infrared Surface Emissivity Model (ISEM-6) (Sherlock, 1999). These parameterized emissivity values are accurate to 0.0002 for the ISEM-6 model (Saunders, 2001). Further, the UM OPS computes the brightness temperature from the first guess field using RTTOV-9 (Radiative Transfer for TOVS) with ISEM-6 emissivity (Saunders et al., 2010).

This study uses an adjoint-based FSO method in the NCUM system, which is based on the UKMO FSO system (Marriott et al., 2012; Lorenc and Marriott, 2014). Full details of the FSO system are documented in (Lorenc and Marriott, 2014). The mathematical expression of FSO is given by $$ \hat{y}_o=(\frac{∂e}{∂y_o})^T ,$$ and the impact due to observations by $$ E_o =(y_o)^T(\frac{∂e}{∂y_o})^T ,$$ where e is a measure of the error in the forecast and y _o are the observations. The hat above the \(\hat{y}_o\) indicates the derivative of the forecast error with respect to that variable. The mean impact per observation is defined as \(\sum^n_{i=1} E_{o,i}/n\), where n is the number of observations and \(\sum^n_{i=1} E_{o,i}\) is the total impact. The forecasting error originates from the initial condition error and the model error. The adjoint sensitivity of the forecasting error to the initial conditions can be obtained by a single backward integration of the adjoint model, assuming that a forecast aspect of interest is an input to the adjoint model and there is no contribution of model error to the forecast error. Therefore, it is considered that the forecasting error is entirely caused by the initial condition error. One of the drawbacks with the FSO method is the tangent linear assumption, which is valid up to 3 days (Cardinali and Prates, 2011). (Isaksen et al., 2005) showed that in adjoint-based FSO models the analysis uncertainties obtained throughout the adjoint integration can be incorrect if the propagating back signal is weak. The accuracy of the linear forecast model, including the error contributed from linear approximation against the full nonlinear model, was discussed by (Lorenc and Marriott, 2014). The study showed that the linear PF model is capable of producing reasonable forecasts of prognostic model fields up to 27 h. The linearizing about the average trajectory improves the accuracy of the PF model increments for different variables with relative RMSEs lower than 1.0, as compared to the nonlinear model increments. Also, the PF model with moist physics processes has the effect of increasing the magnitude of analysis impact. They concluded that the linear PF model has adequate overall skill to produce useful observation impacts. The linearity assumption in the FSO method works better with a smaller time step in the backward integration of the linear model (Joo et al., 2012). Our study is performed by the hourly averaged linearization trajectory of the PF model, which is a limitation discussed by (Lorenc and Marriott, 2014). It is important to investigate the increased nonlinearity of both the model forecast and the observations, but this is left for future study.

Figure 2.  Daily mean observation increment (O-B) of eastward wind (top panel) and northward wind (bottom panel) for four different satellites over the globe during 1-20 January 2015. The numbers in parentheses in the figure legend represent the mean O-B for the whole period.

The observation impacts give an estimate of the change in the 24-h forecasting error due to the assimilation of the observations. The term, observation impact, will refer to the partial sums of the observation impact over different observation types (subset). We calculate the observation impact to assess the relative importance of each observation type within the context of this experiment. Although our main focus is on the mean impact of SATWIND observations and their impact at different levels, we also calculate the mean impact per observation for three other satellite subtypes separately (GPSRO, IASI, and AIRS), as shown in Table 1. It should be noted that the observation impact depends on the data accumulation period, and it should not be compared directly with that of experiments looking at a different period (Joo et al., 2013). Therefore, it is more appropriate to compare the mean impact per observation. It should be noted that we calculate the mean impact per observation and for the different types of observations separately. These results are described in section 3. Finally, a comparison is made between the UKMO and NCMRWF GPSRO observation impacts with the same model configuration but for the period 1800 UTC 22 August to 1200 UTC 29 September 2010. In our experiment, observation impacts are produced for the period 0000 UTC 1 January to 1800 UTC 20 January 2015.

Figure 3.  Kernel density distribution of O-B for (U,V) component winds (units: m s^-1) for the four types of SATWIND and at three different levels.

The SATWIND observation increment (O-B) is used in quality control, as well as the forecast implications. During the 4D-Var assimilation process, SATWIND observation increments are used to correct the model fields, which produce a more accurate and dynamically consistent analysis for a new forecast cycle. The daily mean observation increment (O-B) of the eastward wind component (top panel) and northward wind component (bottom panel) for the four satellites during 1-20 January 2015 are shown in Fig. 2. The number in parentheses represents the 20-day mean O-B. It can be seen that the mean value is negative in the case of the eastward wind. In addition, METEOSAT-7 shows a higher negative value (-0.65 m s^-1) compared to GOES (-0.33 m s-1), MTSAT (-0.21 m s^-1), and METEOSAT-10 (-0.24 m s^-1). In the case of the northward wind, only GOES shows a positive mean value (0.03 m s^-1), with the others having negative values. The daily variation of the mean negative O-B lies between -1 and 1 m s^-1, with the errors almost randomly distributed. This indicates that the model background from UM has larger easterly components than those observed from SATWIND.

The kernel density distribution (Wilks, 2006) of the observation increment is computed at three different levels: a lower level (LL, >700 hPa); a middle level (ML, 700-400 hPa); and a higher level (HL, <400 hPa). It is noted that the error distribution follows a Gaussian curve with mean value limits at zero (Fig. 3). The negative O-B spread at the HL is more pronounced, and is reduced towards the LL. In addition, the O-B distribution for both eastward and northward winds follows a similar pattern for MTSAT-2, METEOSAT-7 and METEOSAT-10, and the error range is higher for all levels (-30 to 30 m s^-1) compared to GOES (-15 to 15 m s^-1). Figure 4 shows that the mean eastward wind observation increment increases from LL to HL for MTSAT-2, METEOSAT-7 and METEOSAT-10; whereas, for GOES, the mean observation increment is higher at the ML (-0.7 m s^-1). This may be because, in the case of GOES, the total number of observations assimilated is less than for the other satellites due to quality control and thinning during the observation pre-processing. In the case of the northward wind components, the error is positive at the ML. For the HL O-B, there are very similar negative values in all cases.

Figure 4.  The 20-day mean O-B eastward wind (U, top panel) and northward wind (V, bottom panel) at the high level (<400 hPa), middle level (700-400 hPa) and lower level (>700 hPa) for GOES, MTSAT-2, METEOSAT-7, and METEOSAT-10.

In this section, we present the satellite observation impact results calculated from the UM forecast system. The percentages of beneficial observations for the eastward wind and the northward wind from GOES, MTSAT, METEOSAT-7 and METEOSAT-10 are depicted in Fig. 5. The results show that only just over 50% of the observations have a beneficial impact, which has been confirmed with many systems (Gelaro et al., 2010; Lorenc and Marriott, 2014). In addition, large variability of GOES' beneficial observations is observed compared to the other satellites. The geographical distributions of the 24-h SATWIND observation impact at the HL (top panels), ML (middle panels), and LL (bottom panels) for the satellite-derived eastward wind (U-wind, left panels) and northward wind (V-wind, right panels) are shown in Fig. 6. The largest U-wind and V-wind impacts are seen in the HL winds. In the ML and over the tropical region, the number of observations assimilated is very small, compared to the subtopical region. So, the impact is less over the tropical region compared to subtropical regions at that level. The percentage of the impact per observation contributed by each satellite's wind for the U-wind and V-wind components corresponding to the levels is shown in Fig. 7. In terms of the individual contribution at all levels for both U-wind and V-wind, METEOSAT-7 shows a higher percentage of impact (∼50%) compared to GEOS, MTSAT-2 and METEOSAT-10. The others are all below 25% of the total SATWIND impact. At the ML, GOES shows a 60% U-wind impact compared to MTSAT-2 and METEOSAT-10. Note that the impact is calculated by taking the sum of the impact at level wise separately. The total impact value is divided by the number of observations within that level, as discussed in the previous section. The mean impact of METEOSAT-7 is the highest for all levels, but the results in this case may slightly skewed. It should be noted that no observations are assimilated at the ML.

Figure 5.  Percentages of beneficial observations for eastward wind (U, top panel) and northward wind (V, bottom panel) from GOES, MTSAT, METEOSAT-7 and METEOSAT-10.

Figure 6.  Geographical distribution of 24-h SATWIND observation impact per day (units: 10^-6 J kg^-1) at the (a, b) upper level, (b, c) middle level and (e, f) lower level, for satellite-derived eastward wind (U, left panels) and northward wind (V, right panels). The average impact value is calculated over 20 days and for all four assimilation cycles (0000, 0600, 1200 and 1800 UTC).

In terms of the percentage of beneficial observation impacts by observation type (Fig. 8a) for the full trial period, just over 50% of all satellite observations show beneficial observations. In addition, the SATWIND observations have a higher percentage of beneficial observations, at just over 53%, as compared to the radiances from AIRS (51.5%) and IASI (52.1%). The GPSRO shows fewer beneficial observations (50.6%) compared to the other sets. It should be noted that during the last few years many authors have expressed different opinions in terms of this finding of "just over 50% have beneficial observations" (Gelaro et al., 2010). A study by (Johnson et al., 2006) showed that even a perfect observation in a perfect data assimilation system can degrade the forecast. Figure 8b shows the percentage of mean impact per observation by different observation types for the full trial period. Note that, to calculate the mean AMV impact per observation in this case, the U-wind and V-wind components are not counted separately. The (U,V) pair is considered to be a single observation. METOSAT-7 shows the highest mean impact at above 20%, as compared to METOSAT-10, MTSAT-2 and GOES. In addition, IASI shows a higher percentage of mean impact compared to AIRS and GPSRO.

Figure 7.  Percentage of the total impact contributed by each satellite’s eastward wind (U) and the northward wind (V) components corresponding to the levels are shown. The 20-day total impact of U and V calculated separately for all the satellite wind for with all assimilation cycles (00, 06, 12, and 18 UTC).

Figure 8.  The (a) percentage of beneficial observations of 24-h observation impact, and (b) percentage of impact per observation with different observation types for the full 20-day trial period.

Figure 9a shows a comparison of the impact of the GPSRO observations by height. The results are similar to those in (Lorenc and Marriott, 2014), because the configuration and energy norm calculation used in this study follows (Lorenc and Marriott, 2014), but for a different time period. The beneficial impacts are seen from the GPSRO observations up to 50 km, above which they are not assimilated. (Lorenc and Marriott, 2014) discussed reasons for the quick drop-off of the impacts at a height of around 15 km and small detrimental impacts at 20 km and above. Although our results are similar to those in (Lorenc and Marriott, 2014), the drop-off of the impact is more pronounced from 20 km and above in our case. It should also be noted that in our experiment no detrimental (positive value) impacts are observed above 20 km. Figure 9b shows the latitude-averaged plots of the GPSRO total impacts percentage to latitude above and below 20 km. In the tropics (30^°N-30^°S), the percentage of GPSRO impacts is higher in all cases. In terms of the percentage of total GPSRO impacts above 20 km, 51% are from the Southern Hemisphere and 49% from the Northern Hemisphere. However, at the lower levels (below 20 km), 55% of total impacts are from the Southern Hemisphere and 45% from the Northern Hemisphere.

The geographical distribution of the 24-h observation impact of the radiances from the hyperspectral sensors like IASI (top panel) and AIRS (bottom panel) are shown in Fig. 10. The IASI observation impact has higher values from 30^°-60^°S and 30^°-60^°N. In terms of the percentage of the total IASI observation impact, 45% are from 30^°-60^°S and 32.7% from 30^°N-30^°S. It is interesting to note that over the Southern Hemisphere the percentage of impact is high (73%), compared to the Northern Hemisphere. In the case of AIRS, although the geographical distribution of the impact is mixed, the higher percentage of impacts is from the Southern Hemisphere, especially from 30^°-60^°S, where the percentage of AIRS total impact is 36%; and from 30^°S to the equator, where the value is 33%. Only 5% of the impacts are from above 30^°N (Fig. 11). Also, over the Southern Hemisphere, the percentage of AIRS impact is high, which is very similar to the IASI impact. This may be due to the fact that the land area over the Southern Hemisphere is only 32%, whereas over the Northern Hemisphere it is 68%. Over the land area, there are good amounts of conventional observations (e.g., surface synoptic observations, automatic weather station, wind profiler and dropsonde) available and they are used in the assimilation system. In other words, the data-sparse area shows a higher impact compared to the data-dense area. It is also interesting to note that the impact value is higher over the oceanic regions than over land. This may be due to the fact that satellite-derived radiance depends on surface parameter estimation. In addition, only radiances that are not affected by the surface are assimilated, as discussed in the last paragraph of section 2.2 satellite data.

Figure 9.  (a) GPSRO observation impact by height. (b) Percentage GPSRO observation impact in six different regions.

Figure 10.  Geographical distribution of 24-h IASI (top panel) and AIRS (bottom panel) observation impact per day (units: 10$-6 J kg^-1$).

Figure 11.  Percentage IASI and AIRS observation impact in six different regions.

This study describes the importance of the satellite winds in the UM NWP system and their impact on short-term 24-h forecasts. The quality of the observations is compared to the short-range forecast from the model background. Less than 10% of the wind data are actually used in the 4D-Var assimilation system after quality control and thinning to prepare the analysis field for each assimilation cycle. Observation increments (O-B) are computed as the satellite-derived wind minus the model forecast with a 6-h lead time. The O-B statistics are produced for error pattern analysis from the first 20 days of a winter month (January 2015). The 4D-Var assimilation system, including observation processing, has the advantage of monitoring and analyzing O-B statistics between co-located observations and the UM background at individual observation times. The adjoint-based FSO method is used to measure the potential forecast impacts for each type of observation.

The result shows that the mean O-B has negative values at the higher level, while the observation increment has similar negative values for all cases. This means that the model background has larger easterly components than from observations. In terms of the percentage mean impact of SATWIND from FSO, METEOSAT-7 shows a higher percentage mean impact compared to GEOS, MTSAT-2 and METEOSAT-10. At the mid-level, GOES shows a high mean impact compared to MTSAT-2 and METEOSAT-10. The SATWIND observations have a higher percentage of beneficial observations as compared to the radiances from AIRS and IASI. The GPSRO shows fewer beneficial observations as compared to the other sets. Over the Southern Hemisphere, the impact of high spectral radiances from IASI and AIRS shows higher values as compared to the Northern Hemisphere. This is due to the fact that the land area over the Southern Hemisphere is only 32% (and there are fewer observations), whereas over the Northern Hemisphere it is 68% (and more observations).

The FSO method is found to be useful for the quality control of bad observations and determining the relative impact of observations. The adjoint-based FSO method has increasingly been used at many NWP centers in the last few years to study the impact of all types of observations, including satellite radiances and winds. The present study is important in operational aspects because FSO allows us to visualize the performance of the satellite data in the NWP system.

Although our study is based on 20-day mean O-B values of four assimilation cycles during the winter period, the same work can be carried out for the summer period. It is important to identify the seasonal variation of O-B; for example, during the monsoon season the behaviour of O-B may differ compared to the post-monsoon and pre-monsoon period. Another possibility for research in this field could be to compare different regions, e.g., the tropics and subtropics. The next step could be to determine where the large O-B values come from, i.e., whether there are problems with the observations or the model background. The impacts of SATWIND observations can vary depending on cloud interaction, observation time, and land/sea properties. In this study, these varying conditions are not explored and are left for future work. It is also possible to use FSO with any type of observation as a function of time or domain. This may provide information for the use of any new satellite observations in an operational NWP system.