Conditioning of incremental variational data assimilation, with application to the Met Office system

Implementations of incremental variational data assimilation require the iterative minimization of a series of linear least-squares cost functions. The accuracy and speed with which these linear minimization problems can be solved is determined by the condition number of the Hessian of the problem. In this study, we examine how different components of the assimilation system influence this condition number. Theoretical bounds on the condition number for a single parameter system are presented and used to predict how the condition number is affected by the observation distribution and accuracy and by the specified lengthscales in the background error covariance matrix. The theoretical results are verified in the Met Office variational data assimilation system, using both pseudo-observations and real data.

Conditioning and preconditioning of the variational data assimilation problem

Numerical weather prediction (NWP) centres use numerical models of the atmospheric flow to forecast future weather states from an estimate of the current state. Variational data assimilation (VAR) is used commonly to determine an optimal state estimate that miminizes the errors between observations of the dynamical system and model predictions of the flow. The rate of convergence of the VAR scheme and the sensitivity of the solution to errors in the data are dependent on the condition number of the Hessian of the variational least-squares objective function. The traditional formulation of VAR is ill-conditioned and hence leads to slow convergence and an inaccurate solution. In practice, operational NWP centres precondition the system via a control variable transform to reduce the condition number of the Hessian. In this paper we investigate the conditioning of VAR for a single, periodic, spatially-distributed state variable. We present theoretical bounds on the condition number of the original and preconditioned Hessians and hence demonstrate the improvement produced by the preconditioning. We also investigate theoretically the effect of observation position and error variance on the preconditioned system and show that the problem becomes more ill-conditioned with increasingly dense and accurate observations. Finally, we confirm the theoretical results in an operational setting by giving experimental results from the Met Office variational system.

The ERA-Interim reanalysis: configuration and performance of the data assimilation system

ERA-Interim is the latest global atmospheric reanalysis produced by the European Centre for Medium-Range Weather Forecasts (ECMWF). The ERA-Interim project was conducted in part to prepare for a new atmospheric reanalysis to replace ERA-40, which will extend back to the early part of the twentieth century. This article describes the forecast model, data assimilation method, and input datasets used to produce ERA-Interim, and discusses the performance of the system. Special emphasis is placed on various difficulties encountered in the production of ERA-40, including the representation of the hydrological cycle, the quality of the stratospheric circulation, and the consistency in time of the reanalysed fields. We provide evidence for substantial improvements in each of these aspects. We also identify areas where further work is needed and describe opportunities and objectives for future reanalysis projects at ECMWF

Impact of income on nutrient intakes: implications for undernourishment and obesity

The relationship between income and nutrient intake is explored. Nonparametric, panel, and quantile regressions are used. Engle curves for calories, fat, and protein are approximately linear in logs with carbohydrate intakes exhibiting diminishing elasticities as incomes increase. Elasticities range from 0.10 to 0.25, with fat having the highest elasticities. Countries in higher quantiles have lower elasticities than those in lower quantiles. Results predict significant cumulative increases in calorie consumption which are increasingly composed of fats. Though policies aimed at poverty alleviation and economic growth may assuage hunger and malnutrition, they may also exacerbate problems associated with obesity.

Hillslope scale surface runoff, sediment, and nutrient losses associated with tramline wheelings

Research on arable sandy loam and silty clay loam soils on 4° slopes in England has shown that tramlines (i.e. the unseeded wheeling areas used to facilitate spraying operations in cereal crops) can represent the most important pathway for phosphorus and sediment loss from moderately sloping fields. Detailed monitoring over the October–March period in winters 2005–2006 and 2006–2007 included event-based sampling of surface runoff, suspended and particulate sediment, and dissolved and particulate phosphorus from hillslope segments (each ∼300–800 m2) established in a randomized block design with four replicates of each treatment at each of two sites on lighter and heavier soils. Experimental treatments assessed losses from the cropped area without tramlines, and from the uncropped tramline area, and were compared to losses from tramlines which had been disrupted once in the autumn with a shallow tine. On the lighter soil, the effects of removal or shallow incorporation of straw residues was also determined. Research on both sandy and silty clay loam soils across two winters showed that tramline wheelings represented the dominant pathway for surface runoff and transport of sediment, phosphorus and nitrogen from cereal crops on moderate slopes. Results indicated 5·5–15·8% of rainfall lost as runoff, and losses of 0·8–2·9 kg TP ha−1 and 0·3–4·8 t ha−1 sediment in tramline treatments, compared to only 0·2–1·7% rainfall lost as runoff, and losses of 0·0–0·2 kg TP ha−1 and 0·003–0·3 t ha−1 sediment from treatments without tramlines or those where tramlines had been disrupted. The novel shallow disruption of tramline wheelings using a tine once following the autumn spray operation consistently and dramatically reduced (p < 0·001) surface runoff and loads of sediment, total nitrogen and total phosphorus to levels similar to those measured in cropped areas between tramlines. Results suggest that options for managing tramline wheelings warrant further refinement and evaluation with a view to incorporating them into spatially-targeted farm-level management planning using national or catchment-based agri-environment policy instruments aimed at reducing diffuse pollution from land to surface water systems. Copyright © 2010 John Wiley & Sons, Ltd.

On the equivalence between radiance and retrieval assimilation

The need for consistent assimilation of satellite measurements for numerical weather prediction led operational meteorological centers to assimilate satellite radiances directly using variational data assimilation systems. More recently there has been a renewed interest in assimilating satellite retrievals (e.g., to avoid the use of relatively complicated radiative transfer models as observation operators for data assimilation). The aim of this paper is to provide a rigorous and comprehensive discussion of the conditions for the equivalence between radiance and retrieval assimilation. It is shown that two requirements need to be satisfied for the equivalence: (i) the radiance observation operator needs to be approximately linear in a region of the state space centered at the retrieval and with a radius of the order of the retrieval error; and (ii) any prior information used to constrain the retrieval should not underrepresent the variability of the state, so as to retain the information content of the measurements. Both these requirements can be tested in practice. When these requirements are met, retrievals can be transformed so as to represent only the portion of the state that is well constrained by the original radiance measurements and can be assimilated in a consistent and optimal way, by means of an appropriate observation operator and a unit matrix as error covariance. Finally, specific cases when retrieval assimilation can be more advantageous (e.g., when the estimate sought by the operational assimilation system depends on the first guess) are discussed.

Biotic carbon feedbacks in a materially-closed soil-vegetation-atmosphere system

The magnitude and direction of the coupled feedbacks between the biotic and abiotic components of the terrestrial carbon cycle is a major source of uncertainty in coupled climate–carbon-cycle models1, 2, 3. Materially closed, energetically open biological systems continuously and simultaneously allow the two-way feedback loop between the biotic and abiotic components to take place4, 5, 6, 7, but so far have not been used to their full potential in ecological research, owing to the challenge of achieving sustainable model systems6, 7. We show that using materially closed soil–vegetation–atmosphere systems with pro rata carbon amounts for the main terrestrial carbon pools enables the establishment of conditions that balance plant carbon assimilation, and autotrophic and heterotrophic respiration fluxes over periods suitable to investigate short-term biotic carbon feedbacks. Using this approach, we tested an alternative way of assessing the impact of increased CO2 and temperature on biotic carbon feedbacks. The results show that without nutrient and water limitations, the short-term biotic responses could potentially buffer a temperature increase of 2.3 °C without significant positive feedbacks to atmospheric CO2. We argue that such closed-system research represents an important test-bed platform for model validation and parameterization of plant and soil biotic responses to environmental changes.

Ensemble data assimilation in the presence of cloud

In numerical weather prediction (NWP) data assimilation (DA) methods are used to combine available observations with numerical model estimates. This is done by minimising measures of error on both observations and model estimates with more weight given to data that can be more trusted. For any DA method an estimate of the initial forecast error covariance matrix is required. For convective scale data assimilation, however, the properties of the error covariances are not well understood. An effective way to investigate covariance properties in the presence of convection is to use an ensemble-based method for which an estimate of the error covariance is readily available at each time step. In this work, we investigate the performance of the ensemble square root filter (EnSRF) in the presence of cloud growth applied to an idealised 1D convective column model of the atmosphere. We show that the EnSRF performs well in capturing cloud growth, but the ensemble does not cope well with discontinuities introduced into the system by parameterised rain. The state estimates lose accuracy, and more importantly the ensemble is unable to capture the spread (variance) of the estimates correctly. We also find, counter-intuitively, that by reducing the spatial frequency of observations and/or the accuracy of the observations, the ensemble is able to capture the states and their variability successfully across all regimes.

Regularization techniques for ill-posed inverse problems in data assimilation

Optimal state estimation from given observations of a dynamical system by data assimilation is generally an ill-posed inverse problem. In order to solve the problem, a standard Tikhonov, or L2, regularization is used, based on certain statistical assumptions on the errors in the data. The regularization term constrains the estimate of the state to remain close to a prior estimate. In the presence of model error, this approach does not capture the initial state of the system accurately, as the initial state estimate is derived by minimizing the average error between the model predictions and the observations over a time window. Here we examine an alternative L1 regularization technique that has proved valuable in image processing. We show that for examples of flow with sharp fronts and shocks, the L1 regularization technique performs more accurately than standard L2 regularization.

L1-regularisation for ill-posed problems in variational data assimilation

We consider four-dimensional variational data assimilation (4DVar) and show that it can be interpreted as Tikhonov or L2-regularisation, a widely used method for solving ill-posed inverse problems. It is known from image restoration and geophysical problems that an alternative regularisation, namely L1-norm regularisation, recovers sharp edges better than L2-norm regularisation. We apply this idea to 4DVar for problems where shocks and model error are present and give two examples which show that L1-norm regularisation performs much better than the standard L2-norm regularisation in 4DVar.

Estimation of systematic error in an equatorial ocean model using data assimilation

Assimilation of temperature observations into an ocean model near the equator often results in a dynamically unbalanced state with unrealistic overturning circulations. The way in which these circulations arise from systematic errors in the model or its forcing is discussed. A scheme is proposed, based on the theory of state augmentation, which uses the departures of the model state from the observations to update slowly evolving bias fields. Results are summarized from an experiment applying this bias correction scheme to an ocean general circulation model. They show that the method produces more balanced analyses and a better fit to the temperature observations.

Adjoint methods in data assimilation for estimating model error

Data assimilation aims to incorporate measured observations into a dynamical system model in order to produce accurate estimates of all the current (and future) state variables of the system. The optimal estimates minimize a variational principle and can be found using adjoint methods. The model equations are treated as strong constraints on the problem. In reality, the model does not represent the system behaviour exactly and errors arise due to lack of resolution and inaccuracies in physical parameters, boundary conditions and forcing terms. A technique for estimating systematic and time-correlated errors as part of the variational assimilation procedure is described here. The modified method determines a correction term that compensates for model error and leads to improved predictions of the system states. The technique is illustrated in two test cases. Applications to the 1-D nonlinear shallow water equations demonstrate the effectiveness of the new procedure.

Data assimilation for state and parameter estimation: application to morphodynamic modelling

Data assimilation is predominantly used for state estimation; combining observational data with model predictions to produce an updated model state that most accurately approximates the true system state whilst keeping the model parameters fixed. This updated model state is then used to initiate the next model forecast. Even with perfect initial data, inaccurate model parameters will lead to the growth of prediction errors. To generate reliable forecasts we need good estimates of both the current system state and the model parameters. This paper presents research into data assimilation methods for morphodynamic model state and parameter estimation. First, we focus on state estimation and describe implementation of a three dimensional variational(3D-Var) data assimilation scheme in a simple 2D morphodynamic model of Morecambe Bay, UK. The assimilation of observations of bathymetry derived from SAR satellite imagery and a ship-borne survey is shown to significantly improve the predictive capability of the model over a 2 year run. Here, the model parameters are set by manual calibration; this is laborious and is found to produce different parameter values depending on the type and coverage of the validation dataset. The second part of this paper considers the problem of model parameter estimation in more detail. We explain how, by employing the technique of state augmentation, it is possible to use data assimilation to estimate uncertain model parameters concurrently with the model state. This approach removes inefficiencies associated with manual calibration and enables more effective use of observational data. We outline the development of a novel hybrid sequential 3D-Var data assimilation algorithm for joint state-parameter estimation and demonstrate its efficacy using an idealised 1D sediment transport model. The results of this study are extremely positive and suggest that there is great potential for the use of data assimilation-based state-parameter estimation in coastal morphodynamic modelling.