Correlated observation errors in data assimilation

Data assimilation provides techniques for combining observations and prior model forecasts to create initial conditions for numerical weather prediction (NWP). The relative weighting assigned to each observation in the analysis is determined by its associated error. Remote sensing data usually has correlated errors, but the correlations are typically ignored in NWP. Here, we describe three approaches to the treatment of observation error correlations. For an idealized data set, the information content under each simplified assumption is compared with that under correct correlation specification. Treating the errors as uncorrelated results in a significant loss of information. However, retention of an approximated correlation gives clear benefits.

Sensitivity of tropospheric forecasts to stratospheric initial conditions

A number of recent papers in the atmospheric science literature have suggested that a dynamical link exists between the stratosphere and troposphere. Numerical modelling studies have shown that the troposphere has a time-mean response to changes to the stratospheric climatological state. In this study the response of the troposphere to an imposed transient stratospheric change is examined. The study uses a high horizontal and vertical resolution numerical weather-prediction model. Experiments compare the tropospheric forecasts of two medium-range forecast ensembles which have identical tropospheric initial conditions and different stratospheric initial conditions. In three case studies described here, stratospheric initial conditions have a statistically significant impact on the tropospheric flow. The mechanism for this change involves, in its most basic step, a change to tropospheric synoptic-scale systems. A consistent change to the tropospheric synoptic-scale systems occurs in response to the stratospheric initial conditions. The aggregated impact of changes to individual synoptic systems maps strongly onto the structure of the Arctic Oscillation, particularly over the North Atlantic storm track. The relationship between the stratosphere and troposphere, while apparent in Arctic Oscillation diagnostics, does not occur on coherent, hemispheric scales.

The accuracy of Doppler radar wind retrievals using insects as targets

Insect returns from the UK's Doppler weather radars were collected in the summers of 2007 and 2008, to ascertain their usefulness in providing information about boundary layer winds. Such observations could be assimilated into numerical weather prediction models to improve forecasts of convective showers before precipitation begins. Significant numbers of insect returns were observed during daylight hours on a number of days through this period, when they were detected at up to 30 km range from the radars, and up to 2 km above sea level. The range of detectable insect returns was found to vary with time of year and temperature. There was also a very weak correlation with wind speed and direction. Use of a dual-polarized radar revealed that the insects did not orient themselves at random, but showed distinct evidence of common orientation on several days, sometimes at an angle to their direction of travel. Observation minus model background residuals of wind profiles showed greater bias and standard deviation than that of other wind measurement types, which may be due to the insects' headings/airspeeds and to imperfect data extraction. The method used here, similar to the Met Office's procedure for extracting precipitation returns, requires further development as clutter contamination remained one of the largest error contributors. Wind observations derived from the insect returns would then be useful for data assimilation applications.

How many facets are needed to represent the surface energy balance of an urban area?

We investigate the question of how many facets are needed to represent the energy balance of an urban area by developing simplified 3-, 2- and 1-facet versions of a 4-facet energy balance model of two-dimensional streets and buildings. The 3-facet model simplifies the 4-facet model by averaging over the canyon orientation, which results in similar net shortwave and longwave balances for both wall facets, but maintains the asymmetry in the heat fluxes within the street canyon. For the 2-facet model, on the assumption that the wall and road temperatures are equal, the road and wall facets can be combined mathematically into a single street-canyon facet with effective values of the heat transfer coefficient, albedo, emissivity and thermodynamic properties, without further approximation. The 1-facet model requires the additional assumption that the roof temperature is also equal to the road and wall temperatures. Idealised simulations show that the geometry and material properties of the walls and road lead to a large heat capacity of the combined street canyon, whereas the roof behaves like a flat surface with low heat capacity. This means that the magnitude of the diurnal temperature variation of the street-canyon facets are broadly similar and much smaller than the diurnal temperature variation of the roof facets. Consequently, the approximation that the street-canyon facets have similar temperatures is sound, and the road and walls can be combined into a single facet. The roof behaves very differently and a separate roof facet is required. Consequently, the 2-facet model performs similarly to the 4-facet model, while the 1-facet model does not. The models are compared with previously published observations collected in Mexico City. Although the 3- and 2-facet models perform better than the 1-facet model, the present models are unable to represent the phase of the sensible heat flux. This result is consistent with previous model comparisons, and we argue that this feature of the data cannot be produced by a single column model. We conclude that a 2-facet model is necessary, and for numerical weather prediction sufficient, to model an urban surface, and that this conclusion is robust and therefore applicable to more general geometries.

Can desert dust explain the outgoing longwave radiation anomaly over the Sahara during July 2003?

Measurements of the top‐of‐the‐atmosphere outgoing longwave radiation (OLR) for July 2003 from Meteosat‐7 are used to assess the performance of the numerical weather prediction version of the Met Office Unified Model. A significant difference is found over desert regions of northern Africa where the model emits too much OLR by up to 35 Wm−2 in the monthly mean. By cloud‐screening the data we find an error of up to 50 Wm−2 associated with cloud‐free areas, which suggests an error in the model surface temperature, surface emissivity, or atmospheric transmission. By building up a physical model of the radiative properties of mineral dust based on in situ, and surface‐based and satellite remote sensing observations we show that the most plausible explanation for the discrepancy in OLR is due to the neglect of mineral dust in the model. The calculations suggest that mineral dust can exert a longwave radiative forcing by as much as 50 Wm−2 in the monthly mean for 1200 UTC in cloud‐free regions, which accounts for the discrepancy between the model and the Meteosat‐7 observations. This suggests that inclusion of the radiative effects of mineral dust will lead to a significant improvement in the radiation balance of numerical weather prediction models with subsequent improvements in performance.

A case study of the radiative forcing of persistent contrails evolving into contrail-induced cirrus

The radiative forcing due to a distinct pattern of persistent contrails that form into contrail-induced cirrus near and over the UK is investigated in detail for a single case study during March 2009. The development of the contrail-induced cirrus is tracked using a number of high-resolution polar orbiting and lower-resolution geostationary satellite instruments and is found to persist for a period of around 18 h, and at its peak, it covers over 50,000 km2. The shortwave (SW) and longwave (LW) radiative forcing of the contrail-induced cirrus is estimated using a combination of geostationary satellite instruments, numerical weather prediction models, and surface observation sites. As expected, the net radiative effect is a relatively small residual of the much stronger but opposing SW and LW effects, locally totaling around 10 W m−2 during daylight hours and 30 W m−2 during nighttime. A simple estimate indicates that this single localized event may have generated a global-mean radiative forcing of around 7% of recent estimates of the persistent contrail radiative forcing due to the entire global aircraft fleet on a diurnally averaged basis. A single aircraft operating in conditions favorable for persistent contrail formation appears to exert a contrail-induced radiative forcing some 5000 times greater (in W m−2 km−1) than recent estimates of the average persistent contrail radiative forcing from the entire civil aviation fleet. This study emphasizes the need to establish whether similar events are common or highly unusual for a confident assessment of the total climate effect of aviation to be made.

A case study of the radiative forcing of persistent contrails evolving into contrail-induced cirrus

The radiative forcing due to a distinct pattern of persistent contrails that form into contrail-induced cirrus near and over the UK is investigated in detail for a single case study during March 2009. The development of the contrail-induced cirrus is tracked using a number of high-resolution polar orbiting and lower-resolution geostationary satellite instruments and is found to persist for a period of around 18 h, and at its peak, it covers over 50,000 km2. The shortwave (SW) and longwave (LW) radiative forcing of the contrail-induced cirrus is estimated using a combination of geostationary satellite instruments, numerical weather prediction models, and surface observation sites. As expected, the net radiative effect is a relatively small residual of the much stronger but opposing SW and LW effects, locally totaling around 10 W m−2 during daylight hours and 30 W m−2 during nighttime. A simple estimate indicates that this single localized event may have generated a global-mean radiative forcing of around 7% of recent estimates of the persistent contrail radiative forcing due to the entire global aircraft fleet on a diurnally averaged basis. A single aircraft operating in conditions favorable for persistent contrail formation appears to exert a contrail-induced radiative forcing some 5000 times greater (in W m−2 km−1) than recent estimates of the average persistent contrail radiative forcing from the entire civil aviation fleet. This study emphasizes the need to establish whether similar events are common or highly unusual for a confident assessment of the total climate effect of aviation to be made.

A fast radiative transfer model for the assimilation of water vapor radiances from the Kalpana very high resolution radiometer

A fast radiative transfer model (RTM) to compute emitted infrared radiances for a very high resolution radiometer (VHRR), onboard the operational Indian geostationary satellite Kalpana has been developed and verified. This work is a step towards the assimilation of Kalpana water vapor (WV) radiances into numerical weather prediction models. The fast RTM uses a regression‐based approach to parameterize channel‐specific convolved level to space transmittances. A comparison between the fast RTM and the line‐by‐line RTM demonstrated that the fast RTM can simulate line‐by‐line radiances for the Kalpana WV channel to an accuracy better than the instrument noise, while offering more rapid radiance calculations. A comparison of clear sky radiances of the Kalpana WV channel with the ECMWF model first guess radiances is also presented, aiming to demonstrate the fast RTM performance with the real observations. In order to assimilate the radiances from Kalpana, a simple scheme for bias correction has been suggested.

The ASSET intercomparison of ozone analyses: method and first results

This paper aims to summarise the current performance of ozone data assimilation (DA) systems, to show where they can be improved, and to quantify their errors. It examines 11 sets of ozone analyses from 7 different DA systems. Two are numerical weather prediction (NWP) systems based on general circulation models (GCMs); the other five use chemistry transport models (CTMs). The systems examined contain either linearised or detailed ozone chemistry, or no chemistry at all. In most analyses, MIPAS (Michelson Interferometer for Passive Atmospheric Sounding) ozone data are assimilated; two assimilate SCIAMACHY (Scanning Imaging Absorption Spectrometer for Atmospheric Chartography) observations instead. Analyses are compared to independent ozone observations covering the troposphere, stratosphere and lower mesosphere during the period July to November 2003. Biases and standard deviations are largest, and show the largest divergence between systems, in the troposphere, in the upper-troposphere/lower-stratosphere, in the upper-stratosphere and mesosphere, and the Antarctic ozone hole region. However, in any particular area, apart from the troposphere, at least one system can be found that agrees well with independent data. In general, none of the differences can be linked to the assimilation technique (Kalman filter, three or four dimensional variational methods, direct inversion) or the system (CTM or NWP system). Where results diverge, a main explanation is the way ozone is modelled. It is important to correctly model transport at the tropical tropopause, to avoid positive biases and excessive structure in the ozone field. In the southern hemisphere ozone hole, only the analyses which correctly model heterogeneous ozone depletion are able to reproduce the near-complete ozone destruction over the pole. In the upper-stratosphere and mesosphere (above 5 hPa), some ozone photochemistry schemes caused large but easily remedied biases. The diurnal cycle of ozone in the mesosphere is not captured, except by the one system that includes a detailed treatment of mesospheric chemistry. These results indicate that when good observations are available for assimilation, the first priority for improving ozone DA systems is to improve the models. The analyses benefit strongly from the good quality of the MIPAS ozone observations. Using the analyses as a transfer standard, it is seen that MIPAS is similar to 5% higher than HALOE (Halogen Occultation Experiment) in the mid and upper stratosphere and mesosphere (above 30 hPa), and of order 10% higher than ozonesonde and HALOE in the lower stratosphere (100 hPa to 30 hPa). Analyses based on SCIAMACHY total column are almost as good as the MIPAS analyses; analyses based on SCIAMACHY limb profiles are worse in some areas, due to problems in the SCIAMACHY retrievals.

Observations of the depth of ice particle evaporation beneath frontal cloud to improve NWP modelling

The evaporation (sublimation) of ice particles beneath frontal ice cloud can provide a significant source of diabatic cooling which can lead to enhanced slantwise descent below the frontal surface. The strength and vertical extent of the cooling play a role in determining the dynamic response of the atmosphere, and an adequate representation is required in numerical weather-prediction (NWP) models for accurate forecasts of frontal dynamics. In this paper, data from a vertically pointing 94 GHz radar are used to determine the characteristic depth-scale of ice particle sublimation beneath frontal ice cloud. A statistical comparison is made with equivalent data extracted from the NWP mesoscale model operational at the Met Office, defining the evaporation depth-scale as the distance for the ice water content to fall to 10% of its peak value in the cloud. The results show that the depth of the ice evaporation zone derived from observations is less than 1 km for 90% of the time. The model significantly overestimates the sublimation depth-scales by a factor of between two and three, and underestimates the local ice water content by a factor of between two and four. Consequently the results suggest the model significantly underestimates the strength of the evaporative cooling, with implications for the prediction of frontal dynamics. A number of reasons for the model discrepancy are suggested. A comparison with radiosonde relative humidity data suggests part of the overestimation in evaporation depth may be due to a high RH bias in the dry slot beneath the frontal cloud, but other possible reasons include poor vertical resolution and deficiencies in the evaporation rate or ice particle fall-speed parametrizations.

Assimilation of stratospheric ozone from MIPAS into a global general-circulation model: The September 2002 vortex split

Ozone and temperature profiles from the Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) have been assimilated, using three-dimensional variational assimilation, into a stratosphere troposphere version of the Met Office numerical weather-prediction system. Analyses are made for the month of September 2002, when there was an unprecedented split in the southern hemisphere polar vortex. The analyses are validated against independent ozone observations from sondes, limb-occultation and total column ozone satellite instruments. Through most of the stratosphere, precision varies from 5 to 15%, and biases are 15% or less of the analysed field. Problems remain in the vortex and below the 60 hPa. level, especially at the tropopause where the analyses have too much ozone and poor agreement with independent data. Analysis problems are largely a result of the model rather than the data, giving confidence in the MIPAS ozone retrievals, though there may be a small high bias in MIPAS ozone in the lower stratosphere. Model issues include an excessive Brewer-Dobson circulation, which results both from known problems with the tracer transport scheme and from the data assimilation of dynamical variables. The extreme conditions of the vortex split reveal large differences between existing linear ozone photochemistry schemes. Despite these issues, the ozone analyses are able to successfully describe the ozone hole split and compare well to other studies of this event. Recommendations are made for the further development of the ozone assimilation system.

Modelling convective processes during the suppressed phase of the a Madden-Julian Oscillation: Comparing single-column models with cloud resolving models

The role of convective processes in moistening the atmosphere during suppressed periods of the suppressed phase of a Madden-Julian oscillation is investigated in cloud-resolving model (CRM) simulations, and the impact of moistening on the subsequent evolution of convection is assessed as part of a Global Energy and Water Cycle Experiment Cloud System Study (GCSS) intercomparison project. The ability of single-column model (SCM) versions of a number of state-of-the-art climate and numerical weather prediction models to capture these convective processes is also evaluated. During the suppressed periods, the CRMs are found to simulate a maximum moistening around 3 km, which is associated with a predominance of shallow convection. All SCMs produce adequate amounts of shallow convection during the suppressed periods, comparable to that seen in CRMs, but the relatively drier SCMs have higher precipitation rates than the relatively wetter SCMs and CRMs. The relatively drier SCMs dry, rather than moisten, the lower troposphere below the melting level. During the transition periods, convective processes act to moisten the atmosphere above the level at which mean advection changes from moistening to drying, despite an overall drying effect for the column. The SCMs capture some essence of this moistening at upper levels. A gradual transition from shallow to deep convection is simulated by the CRMs and the wetter SCMs during the transition periods, but the onset of deep convection is delayed in the drier SCMs. This results in lower precipitation rates for these SCMs during the active periods, although much better agreement exists between the models at this time.

Influence-matrix diagnostic of a data assimilation system

The influence matrix is used in ordinary least-squares applications for monitoring statistical multiple-regression analyses. Concepts related to the influence matrix provide diagnostics on the influence of individual data on the analysis - the analysis change that would occur by leaving one observation out, and the effective information content (degrees of freedom for signal) in any sub-set of the analysed data. In this paper, the corresponding concepts have been derived in the context of linear statistical data assimilation in numerical weather prediction. An approximate method to compute the diagonal elements of the influence matrix (the self-sensitivities) has been developed for a large-dimension variational data assimilation system (the four-dimensional variational system of the European Centre for Medium-Range Weather Forecasts). Results show that, in the boreal spring 2003 operational system, 15% of the global influence is due to the assimilated observations in any one analysis, and the complementary 85% is the influence of the prior (background) information, a short-range forecast containing information from earlier assimilated observations. About 25% of the observational information is currently provided by surface-based observing systems, and 75% by satellite systems. Low-influence data points usually occur in data-rich areas, while high-influence data points are in data-sparse areas or in dynamically active regions. Background-error correlations also play an important role: high correlation diminishes the observation influence and amplifies the importance of the surrounding real and pseudo observations (prior information in observation space). Incorrect specifications of background and observation-error covariance matrices can be identified, interpreted and better understood by the use of influence-matrix diagnostics for the variety of observation types and observed variables used in the data assimilation system. Copyright © 2004 Royal Meteorological Society

A swarm intelligence method for feature tracking in AMV derivation

Feature tracking is a key step in the derivation of Atmospheric Motion Vectors (AMV). Most operational derivation processes use some template matching technique, such as Euclidean distance or cross-correlation, for the tracking step. As this step is very expensive computationally, often shortrange forecasts generated by Numerical Weather Prediction (NWP) systems are used to reduce the search area. Alternatives, such as optical flow methods, have been explored, with the aim of improving the number and quality of the vectors generated and the computational efficiency of the process. This paper will present the research carried out to apply Stochastic Diffusion Search, a generic search technique in the Swarm Intelligence family, to feature tracking in the context of AMV derivation. The method will be described, and we will present initial results, with Euclidean distance as reference.

Verification of cloud-fraction forecasts

Cloud radar and lidar can be used to evaluate the skill of numerical weather prediction models in forecasting the timing and placement of clouds, but care must be taken in choosing the appropriate metric of skill to use due to the non- Gaussian nature of cloud-fraction distributions. We compare the properties of a number of different verification measures and conclude that of existing measures the Log of Odds Ratio is the most suitable for cloud fraction. We also propose a new measure, the Symmetric Extreme Dependency Score, which has very attractive properties, being equitable (for large samples), difficult to hedge and independent of the frequency of occurrence of the quantity being verified. We then use data from five European ground-based sites and seven forecast models, processed using the ‘Cloudnet’ analysis system, to investigate the dependence of forecast skill on cloud fraction threshold (for binary skill scores), height, horizontal scale and (for the Met Office and German Weather Service models) forecast lead time. The models are found to be least skillful at predicting the timing and placement of boundary-layer clouds and most skilful at predicting mid-level clouds, although in the latter case they tend to underestimate mean cloud fraction when cloud is present. It is found that skill decreases approximately inverse-exponentially with forecast lead time, enabling a forecast ‘half-life’ to be estimated. When considering the skill of instantaneous model snapshots, we find typical values ranging between 2.5 and 4.5 days. Copyright c 2009 Royal Meteorological Society