The equivalent-weights particle filter in a high-dimensional system

In general, particle filters need large numbers of model runs in order to avoid filter degeneracy in high-dimensional systems. The recently proposed, fully nonlinear equivalent-weights particle filter overcomes this requirement by replacing the standard model transition density with two different proposal transition densities. The first proposal density is used to relax all particles towards the high-probability regions of state space as defined by the observations. The crucial second proposal density is then used to ensure that the majority of particles have equivalent weights at observation time. Here, the performance of the scheme in a high, 65 500 dimensional, simplified ocean model is explored. The success of the equivalent-weights particle filter in matching the true model state is shown using the mean of just 32 particles in twin experiments. It is of particular significance that this remains true even as the number and spatial variability of the observations are changed. The results from rank histograms are less easy to interpret and can be influenced considerably by the parameter values used. This article also explores the sensitivity of the performance of the scheme to the chosen parameter values and the effect of using different model error parameters in the truth compared with the ensemble model runs.

Convective updraught evaluation in high-resolution NWP simulations using single-Doppler radar measurements

This study presents an evaluation of the size and strength of convective updraughts in high-resolution simulations by the UK Met Office Unified Model (UM). Updraught velocities have been estimated from range–height indicator (RHI) Doppler velocity measurements using the Chilbolton advanced meteorological radar, as part of the Dynamical and Microphysical Evolution of Convective Storms (DYMECS) project. Based on mass continuity and the vertical integration of the observed radial convergence, vertical velocities tend to be underestimated for convective clouds due to the undetected cross-radial convergence. Velocity fields from the UM at a resolution corresponding to the radar observations are used to scale such estimates to mitigate the inherent biases. The analysis of more than 100 observed and simulated storms indicates that the horizontal scale of updraughts in simulations tend to decrease with grid length; the 200 m grid length agreed most closely with the observations. Typical updraught mass fluxes in the 500 m grid length simulations were up to an order of magnitude greater than observed, and greater still in the 1.5 km grid length simulations. The effect of increasing the mixing length in the sub-grid turbulence scheme depends on the grid length. For the 1.5 km simulations, updraughts were weakened though their horizontal scale remained largely unchanged. Progressively more so for the sub-kilometre grid lengths, updraughts were broadened and intensified; horizontal scale was now determined by the mixing length rather than the grid length. In general, simulated updraughts were found to weaken too quickly with height. The findings were supported by the analysis of the widths of reflectivity patterns in both the simulations and observations.

The representation of the West-African Monsoon vertical cloud structure in the Met Office Unified Model: an evaluation with CloudSat

Weather and climate model simulations of the West African Monsoon (WAM) have generally poor representation of the rainfall distribution and monsoon circulation because key processes, such as clouds and convection, are poorly characterized. The vertical distribution of cloud and precipitation during the WAM are evaluated in Met Office Unified Model simulations against CloudSat observations. Simulations were run at 40-km and 12-km horizontal grid length using a convection parameterization scheme and at 12-km, 4-km, and 1.5-km grid length with the convection scheme effectively switched off, to study the impact of model resolution and convection parameterization scheme on the organisation of tropical convection. Radar reflectivity is forward-modelled from the model cloud fields using the CloudSat simulator to present a like-with-like comparison with the CloudSat radar observations. The representation of cloud and precipitation at 12-km horizontal grid length improves dramatically when the convection parameterization is switched off, primarily because of a reduction in daytime (moist) convection. Further improvement is obtained when reducing model grid length to 4 km or 1.5 km, especially in the representation of thin anvil and mid-level cloud, but three issues remain in all model configurations. Firstly, all simulations underestimate the fraction of anvils with cloud top height above 12 km, which can be attributed to too low ice water contents in the model compared to satellite retrievals. Secondly, the model consistently detrains mid-level cloud too close to the freezing level, compared to higher altitudes in CloudSat observations. Finally, there is too much low-level cloud cover in all simulations and this bias was not improved when adjusting the rainfall parameters in the microphysics scheme. To improve model simulations of the WAM, more detailed and in-situ observations of the dynamics and microphysics targeting these non-precipitating cloud types are required.

Serial clustering of extratropical cyclones in a multi-model ensemble of historical and future simulations

This study has investigated serial (temporal) clustering of extra-tropical cyclones simulated by 17 climate models that participated in CMIP5. Clustering was estimated by calculating the dispersion (ratio of variance to mean) of 30 December-February counts of Atlantic storm tracks passing nearby each grid point. Results from single historical simulations of 1975-2005 were compared to those from historical ERA40 reanalyses from 1958-2001 ERA40 and single future model projections of 2069-2099 under the RCP4.5 climate change scenario. Models were generally able to capture the broad features in reanalyses reported previously: underdispersion/regularity (i.e. variance less than mean) in the western core of the Atlantic storm track surrounded by overdispersion/clustering (i.e. variance greater than mean) to the north and south and over western Europe. Regression of counts onto North Atlantic Oscillation (NAO) indices revealed that much of the overdispersion in the historical reanalyses and model simulations can be accounted for by NAO variability. Future changes in dispersion were generally found to be small and not consistent across models. The overdispersion statistic, for any 30 year sample, is prone to large amounts of sampling uncertainty that obscures the climate change signal. For example, the projected increase in dispersion for storm counts near London in the CNRMCM5 model is 0.1 compared to a standard deviation of 0.25. Projected changes in the mean and variance of NAO are insufficient to create changes in overdispersion that are discernible above natural sampling variations.

A route to systematic error in forecasts of Rossby waves

Recent work has shown that both the amplitude of upper-level Rossby waves and the tropopause sharpness decrease with forecast lead time for several days in some operational weather forecast systems. In this contribution, the evolution of error growth in a case study of this forecast error type is diagnosed through analysis of operational forecasts and hindcast simulations. Potential vorticity (PV) on the 320-K isentropic surface is used to diagnose Rossby waves. The Rossby-wave forecast error in the operational ECMWF high-resolution forecast is shown to be associated with errors in the forecast of a warm conveyor belt (WCB) through trajectory analysis and an error metric for WCB outflows. The WCB forecast error is characterised by an overestimation of WCB amplitude, a location of the WCB outflow regions that is too far to the southeast, and a resulting underestimation of the magnitude of the negative PV anomaly in the outflow. Essentially the same forecast error development also occurred in all members of the ECMWF Ensemble Prediction System and the Met Office MOGREPS-15 suggesting that in this case model error made an important contribution to the development of forecast error in addition to initial condition error. Exploiting this forecast error robustness, a comparison was performed between the realised flow evolution, proxied by a sequence of short-range simulations, and a contemporaneous forecast. Both the proxy to the realised flow and the contemporaneous forecast a were produced with the Met Office Unified Model enhanced with tracers of diabatic processes modifying potential temperature and PV. Clear differences were found in the way potential temperature and PV are modified in the WCB between proxy and forecast. These results demonstrate that differences in potential temperature and PV modification in the WCB can be responsible for forecast errors in Rossby waves.

Microphysical properties of cold frontal rainbands

Observations have been obtained within an intense (precipitation rates > 50 mm h−1 ) narrow cold-frontal rainband (NCFR) embedded within a broader region of stratiform precipitation. In situ data were obtained from an aircraft which flew near a steerable dual-polarisation Doppler radar. The observations were obtained to characterise the microphysical properties of cold frontal clouds, with an emphasis on ice and precipitation formation and development. Primary ice nucleation near cloud top (−55◦ C) appeared to be enhanced by convective features. However, ice multiplication led to the largest ice particle number concentrations being observed at relatively high temperatures (> −10◦ C). The multiplication process (most likely rime splintering) occurs when stratiform precipitation interacts with supercooled water generated in the NCFR. Graupel was notably absent in the data obtained. Ice multiplication processes are known to have a strong impact in glaciating isolated convective clouds, but have rarely been studied within larger organised convective systems such as NCFRs. Secondary ice particles will impact on precipitation formation and cloud dynamics due to their relatively small size and high number density. Further modelling studies are required to quantify the effects of rime splintering on precipitation and dynamics in frontal rainbands. Available parametrizations used to diagnose the particle size distributions do not account for the influence of ice multiplication. This deficiency in parametrizations is likely to be important in some cases for modelling the evolution of cloud systems and the precipitation formation. Ice multiplication has significant impact on artefact removal from in situ particle imaging probes.

Rayleigh scattering by hexagonal ice crystals and the interpretation of dual-polarisation radar measurements

Dual-polarisation radar measurements provide valuable information about the shapes and orientations of atmospheric ice particles. For quantitative interpretation of these data in the Rayleigh regime, common practice is to approximate the true ice crystal shape with that of a spheroid. Calculations using the discrete dipole approximation for a wide range of crystal aspect ratios demonstrate that approximating hexagonal plates as spheroids leads to significant errors in the predicted differential reflectivity, by as much as 1.5 dB. An empirical modification of the shape factors in Gans's spheroid theory was made using the numerical data. The resulting simple expressions, like Gans's theory, can be applied to crystals in any desired orientation, illuminated by an arbitrarily polarised wave, but are much more accurate for hexagonal particles. Calculations of the scattering from more complex branched and dendritic crystals indicate that these may be accurately modelled using the new expression, but with a reduced permittivity dependent on the volume of ice relative to an enclosing hexagonal prism.

The role of index trading in price formation in the grains and oilseeds markets

We use both Granger-causality and instrumental variables (IV) methods to examine the impact of index fund positions on price returns for the main US grains and oilseed futures markets. Our analysis supports earlier conclusions that Granger-causal impacts are generally not discernible. However, market microstructure theory suggests trading impacts should be instantaneous. IV-based tests for contemporaneous causality provide stronger evidence of price impact. We find even stronger evidence that changes in index positions can help predict future changes in aggregate commodity price indices. This result suggests that changes in index investment are in part driven by information which predicts commodity price changes over the coming months.

Improved seasonal prediction of the hot summer of 2003 over Europe through better representation of uncertainty in the land surface

Methods to explicitly represent uncertainties in weather and climate models have been developed and refined over the past decade, and have reduced biases and improved forecast skill when implemented in the atmospheric component of models. These methods have not yet been applied to the land surface component of models. Since the land surface is strongly coupled to the atmospheric state at certain times and in certain places (such as the European summer of 2003), improvements in the representation of land surface uncertainty may potentially lead to improvements in atmospheric forecasts for such events. Here we analyse seasonal retrospective forecasts for 1981–2012 performed with the European Centre for Medium-Range Weather Forecasts’ (ECMWF) coupled ensemble forecast model. We consider two methods of incorporating uncertainty into the land surface model (H-TESSEL): stochastic perturbation of tendencies, and static perturbation of key soil parameters. We find that the perturbed parameter approach considerably improves the forecast of extreme air temperature for summer 2003, through better representation of negative soil moisture anomalies and upward sensible heat flux. Averaged across all the reforecasts the perturbed parameter experiment shows relatively little impact on the mean bias, suggesting perturbations of at least this magnitude can be applied to the land surface without any degradation of model climate. There is also little impact on skill averaged across all reforecasts and some evidence of overdispersion for soil moisture. The stochastic tendency experiments show a large overdispersion for the soil temperature fields, indicating that the perturbation here is too strong. There is also some indication that the forecast of the 2003 warm event is improved for the stochastic experiments, however the improvement is not as large as observed for the perturbed parameter experiment.

Aspects of designing and evaluating seasonal-to-interannual Arctic sea-ice prediction systems

Using lessons from idealised predictability experiments, we discuss some issues and perspectives on the design of operational seasonal to inter-annual Arctic sea-ice prediction systems. We first review the opportunities to use a hierarchy of different types of experiment to learn about the predictability of Arctic climate. We also examine key issues for ensemble system design, such as: measuring skill, the role of ensemble size and generation of ensemble members. When assessing the potential skill of a set of prediction experiments, using more than one metric is essential as different choices can significantly alter conclusions about the presence or lack of skill. We find that increasing both the number of hindcasts and ensemble size is important for reliably assessing the correlation and expected error in forecasts. For other metrics, such as dispersion, increasing ensemble size is most important. Probabilistic measures of skill can also provide useful information about the reliability of forecasts. In addition, various methods for generating the different ensemble members are tested. The range of techniques can produce surprisingly different ensemble spread characteristics. The lessons learnt should help inform the design of future operational prediction systems.

Twin experiments with the equivalent weights particle filter and HadCM3

This paper investigates the use of a particle filter for data assimilation with a full scale coupled ocean–atmosphere general circulation model. Synthetic twin experiments are performed to assess the performance of the equivalent weights filter in such a high-dimensional system. Artificial 2-dimensional sea surface temperature fields are used as observational data every day. Results are presented for different values of the free parameters in the method. Measures of the performance of the filter are root mean square errors, trajectories of individual variables in the model and rank histograms. Filter degeneracy is not observed and the performance of the filter is shown to depend on the ability to keep maximum spread in the ensemble.

Mobile phones and radios: effects on transactions costs and market participation for households in Northern Ghana

The literature on agricultural markets suggests that transactions costs are the main obstacles preventing households from participating in agricultural markets. We examine the impact of the recent massive penetration of information communication technologies (ICTs), particularly mobile phones and radios, in developing countries to investigate the role of information in economic transactions and participation in food crop markets. To fully capture market participation behaviours, the current theoretical framework on market participation and transactions costs is extended to include those households that sell and buy in the same time period. We correct for endogeneity and selectivity throughout our models. We used a novel dataset of 393 households in northern Ghana with detailed information on market transactions and ICTs usage. Results show that receiving market information via mobile phones has a positive and significant impact on market participation, with a greater impact for households with a surplus of food crops. We find that radios have a larger impact on the quantity traded. This may reflect the nature of mobile phones in reducing searching costs, whereas radios provide an updated and regular flow of information which affects the pattern of crops consumed and sold. We also emphasise that the most significant factor is how ICTs are used, rather than their ownership.

The interaction of moist convection and mid-level dry air in the advance of the onset of the Indian monsoon

The advance of the onset of the Indian monsoon is here explained in terms of a balance between the low-level monsoon flow and an over-running intrusion of mid-tropospheric dry air. The monsoon advances, over a period of about 6 weeks, from the south of the country to the northwest. Given that the low-level monsoon winds are westerly or southwesterly, and the midlevel winds northwesterly, the monsoon onset propagates upwind relative to midlevel flow, and perpendicular to the low-level flow, and is not directly caused by moisture flux toward the northwest. Lacking a conceptual model for the advance means that it has been hard to understand and correct known biases in weather and climate prediction models. The mid-level northwesterlies form a wedge of dry air that is deep in the far northwest of India and over-runs the monsoon flow. The dry layer is moistened from below by shallow cumulus and congestus clouds, so that the profile becomes much closer to moist adiabatic, and the dry layer is much shallower in the vertical, toward the southeast of India. The profiles associated with this dry air show how the most favourable environment for deep convection occurs in the south, and onset occurs here first. As the onset advances across India, the advection of moisture from the Arabian Sea becomes stronger, and the mid-level dry air is increasingly moistened from below. This increased moistening makes the wedge of dry air shallower throughout its horizontal extent, and forces the northern limit of moist convection to move toward the northwest. Wetting of the land surface by rainfall will further reinforce the north-westward progression, by sustaining the supply of boundary layer moisture and shallow cumulus. The local advance of the monsoon onset is coincident with weakening of the mid-level northwesterlies, and therefore weakened mid-level dry advection.

Breaching the social contract: crises of democratic representation and patterns of extreme right party support

Why has the extreme right Greek Golden Dawn, a party with clear links to fascism experienced a rise defying all theories that claim that such a party is unlikely to win in post-WWII Europe? And, if we accept that economic crisis is an explanation for this, why has such a phenomenon not occurred in other countries that have similar conducive conditions, such as Portugal and Spain? This article addresses this puzzle by (a) carrying out a controlled comparison of Greece, Portugal and Spain and (b) showing that the rise of the extreme right is not a question of intensity of economic crisis. Rather it is the nature of the crisis, i.e. economic versus overall crisis of democratic representation that facilitates the rise of the extreme right. We argue that extreme right parties are more likely to experience an increase in their support when economic crisis culminates into an overall crisis of democratic representation. Economic crisis is likely to become a political crisis when severe issues of governability impact upon the ability of the state to fulfil its social contract obligations. This breach of the social contract is accompanied by declining levels of trust in state institutions, resulting in party system collapse.

Theoretical insight into diagnosing observation error correlations using observation-minus-background and observation-minus-analysis statistics

To improve the quantity and impact of observations used in data assimilation it is necessary to take into account the full, potentially correlated, observation error statistics. A number of methods for estimating correlated observation errors exist, but a popular method is a diagnostic that makes use of statistical averages of observation-minus-background and observation-minus-analysis residuals. The accuracy of the results it yields is unknown as the diagnostic is sensitive to the difference between the exact background and exact observation error covariances and those that are chosen for use within the assimilation. It has often been stated in the literature that the results using this diagnostic are only valid when the background and observation error correlation length scales are well separated. Here we develop new theory relating to the diagnostic. For observations on a 1D periodic domain we are able to the show the effect of changes in the assumed error statistics used in the assimilation on the estimated observation error covariance matrix. We also provide bounds for the estimated observation error variance and eigenvalues of the estimated observation error correlation matrix. We demonstrate that it is still possible to obtain useful results from the diagnostic when the background and observation error length scales are similar. In general, our results suggest that when correlated observation errors are treated as uncorrelated in the assimilation, the diagnostic will underestimate the correlation length scale. We support our theoretical results with simple illustrative examples. These results have potential use for interpreting the derived covariances estimated using an operational system.