Evaluating multivariate forecast densities: a comparison of two approaches

We consider methods of evaluating multivariate density forecasts. A recently proposed method is found to lack power when the correlation structure is mis-specified. Tests that have good power to detect mis-specifications of this sort are described. We also consider the properties of the tests in the presence of more general mis-specifications.

Comparison of a trained sensory panel and an electronic tongue in the assessment of bitter dairy protein hydrolsates

The bitter taste elicited by dairy protein hydrolysates (DPH) is a renowned issue for their acceptability by consumers and therefore incorporation into foods. The traditional method of assessment of taste in foods is by sensory analysis but this can be problematic due to the overall unpleasantness of the samples. Thus, there is a growing interest into the use of electronic tongues (e-tongues) as an alternative method to quantify the bitterness in such samples. In the present study the response of the e-tongue to the standard bitter agent caffeine and a range of both casein and whey based hydrolysates was compared to that of a trained sensory panel. Partial least square regression (PLS) was employed to compare the response of the e-tongue and the sensory panel. There was strong correlation shown between the two methods in the analysis of caffeine (R2 of 0.98) and DPH samples with R2 values ranging from 0.94-0.99. This study exhibits potential for the e-tongue to be used in bitterness screening in DPHs to reduce the reliance on expensive and time consuming sensory panels.

Studying the contact point and interface moving in a sinusoidal tube with lattice Boltzmann method

The multicomponent nonideal gas lattice Boltzmann model by Shan and Chen (S-C) is used to study the immiscible displacement in a sinusoidal tube. The movement of interface and the contact point (contact line in three-dimension) is studied. Due to the roughness of the boundary, the contact point shows "stick-slip" mechanics. The "stick-slip" effect decreases as the speed of the interface increases. For fluids that are nonwetting, the interface is almost perpendicular to the boundaries at most time, although its shapes at different position of the tube are rather different. When the tube becomes narrow, the interface turns a complex curves rather than remains simple menisci. The velocity is found to vary considerably between the neighbor nodes close to the contact point, consistent with the experimental observation that the velocity is multi-values on the contact line. Finally, the effect of three boundary conditions is discussed. The average speed is found different for different boundary conditions. The simple bounce-back rule makes the contact point move fastest. Both the simple bounce-back and the no-slip bounce-back rules are more sensitive to the roughness of the boundary in comparison with the half-way bounce-back rule. The simulation results suggest that the S-C model may be a promising tool in simulating the displacement behaviour of two immiscible fluids in complex geometry.

Open solar flux estimates from near-Earth measurements of the interplanetary magnetic field: comparison of the first two perihelion passes of the Ulysses spacecraft

Results from all phases of the orbits of the Ulysses spacecraft have shown that the magnitude of the radial component of the heliospheric field is approximately independent of heliographic latitude. This result allows the use of near- Earth observations to compute the total open flux of the Sun. For example, using satellite observations of the interplanetary magnetic field, the average open solar flux was shown to have risen by 29% between 1963 and 1987 and using the aa geomagnetic index it was found to have doubled during the 20th century. It is therefore important to assess fully the accuracy of the result and to check that it applies to all phases of the solar cycle. The first perihelion pass of the Ulysses spacecraft was close to sunspot minimum, and recent data from the second perihelion pass show that the result also holds at solar maximum. The high level of correlation between the open flux derived from the various methods strongly supports the Ulysses discovery that the radial field component is independent of latitude. We show here that the errors introduced into open solar flux estimates by assuming that the heliospheric field’s radial component is independent of latitude are similar for the two passes and are of order 25% for daily values, falling to 5% for averaging timescales of 27 days or greater. We compare here the results of four methods for estimating the open solar flux with results from the first and second perehelion passes by Ulysses. We find that the errors are lowest (1–5% for averages over the entire perehelion passes lasting near 320 days), for near-Earth methods, based on either interplanetary magnetic field observations or the aa geomagnetic activity index. The corresponding errors for the Solanki et al. (2000) model are of the order of 9–15% and for the PFSS method, based on solar magnetograms, are of the order of 13–47%. The model of Solanki et al. is based on the continuity equation of open flux, and uses the sunspot number to quantify the rate of open flux emergence. It predicts that the average open solar flux has been decreasing since 1987, as Correspondence to: M. Lockwood (m.lockwood@rl.ac.uk) is observed in the variation of all the estimates of the open flux. This decline combines with the solar cycle variation to produce an open flux during the second (sunspot maximum) perihelion pass of Ulysses which is only slightly larger than that during the first (sunspot minimum) perihelion pass.

A comparison of methods for treatment selection in seamless phase II/III clinical trials incorporating information on short-term endpoints

In an adaptive seamless phase II/III clinical trial interim analysis, data are used for treatment selection, enabling resources to be focused on comparison of more effective treatment(s) with a control. In this paper, we compare two methods recently proposed to enable use of short-term endpoint data for decision-making at the interim analysis. The comparison focuses on the power and the probability of correctly identifying the most promising treatment. We show that the choice of method depends on how well short-term data predict the best treatment, which may be measured by the correlation between treatment effects on short- and long-term endpoints.

Intercomparison of methods of coupling between convection and large-scale circulation: 1. Comparison over uniform surface conditions

As part of an international intercomparison project, a set of single column models (SCMs) and cloud-resolving models (CRMs) are run under the weak temperature gradient (WTG) method and the damped gravity wave (DGW) method. For each model, the implementation of the WTG or DGW method involves a simulated column which is coupled to a reference state defined with profiles obtained from the same model in radiative-convective equilibrium. The simulated column has the same surface conditions as the reference state and is initialized with profiles from the reference state. We performed systematic comparison of the behavior of different models under a consistent implementation of the WTG method and the DGW method and systematic comparison of the WTG and DGW methods in models with different physics and numerics. CRMs and SCMs produce a variety of behaviors under both WTG and DGW methods. Some of the models reproduce the reference state while others sustain a large-scale circulation which results in either substantially lower or higher precipitation compared to the value of the reference state. CRMs show a fairly linear relationship between precipitation and circulation strength. SCMs display a wider range of behaviors than CRMs. Some SCMs under the WTG method produce zero precipitation. Within an individual SCM, a DGW simulation and a corresponding WTG simulation can produce different signed circulation. When initialized with a dry troposphere, DGW simulations always result in a precipitating equilibrium state. The greatest sensitivities to the initial moisture conditions occur for multiple stable equilibria in some WTG simulations, corresponding to either a dry equilibrium state when initialized as dry or a precipitating equilibrium state when initialized as moist. Multiple equilibria are seen in more WTG simulations for higher SST. In some models, the existence of multiple equilibria is sensitive to some parameters in the WTG calculations.

EMD performance comparison: single vs double floating points

Empirical mode decomposition (EMD) is a data-driven method used to decompose data into oscillatory components. This paper examines to what extent the defined algorithm for EMD might be susceptible to data format. Two key issues with EMD are its stability and computational speed. This paper shows that for a given signal there is no significant difference between results obtained with single (binary32) and double (binary64) floating points precision. This implies that there is no benefit in increasing floating point precision when performing EMD on devices optimised for single floating point format, such as graphical processing units (GPUs).

An annotation-free method for evaluating privacy protection techniques in videos

While several privacy protection techniques are pre- sented in the literature, they are not complemented with an established objective evaluation method for their assess- ment and comparison. This paper proposes an annotation- free evaluation method that assesses the two key aspects of privacy protection that are privacy and utility. Unlike some existing methods, the proposed method does not rely on the use of subjective judgements and does not assume a spe- cific target type in the image data. The privacy aspect is quantified as an appearance similarity and the utility aspect is measured as a structural similarity between the original raw image data and the privacy-protected image data. We performed an extensive experimentation using six challeng- ing datasets (including two new ones) to demonstrate the effectiveness of the evaluation method by providing a per- formance comparison of four state-of-the-art privacy pro- tection techniques.

Comparison of terrain following and cut cell grids using a non-hydrostatic model

Terrain following coordinates are widely used in operational models but the cut cell method has been proposed as an alternative that can more accurately represent atmospheric dynamics over steep orography. Because the type of grid is usually chosen during model implementation, it becomes necessary to use different models to compare the accuracy of different grids. In contrast, here a C-grid finite volume model enables a like-for-like comparison of terrain following and cut cell grids. A series of standard two-dimensional tests using idealised terrain are performed: tracer advection in a prescribed horizontal velocity field, a test starting from resting initial conditions, and orographically induced gravity waves described by nonhydrostatic dynamics. In addition, three new tests are formulated: a more challenging resting atmosphere case, and two new advection tests having a velocity field that is everywhere tangential to the terrain following coordinate surfaces. These new tests present a challenge on cut cell grids. The results of the advection tests demonstrate that accuracy depends primarily upon alignment of the flow with the grid rather than grid orthogonality. A resting atmosphere is well-maintained on all grids. In the gravity waves test, results on all grids are in good agreement with existing results from the literature, although terrain following velocity fields lead to errors on cut cell grids. Due to semi-implicit timestepping and an upwind-biased, explicit advection scheme, there are no timestep restrictions associated with small cut cells. We do not find the significant advantages of cut cells or smoothed coordinates that other authors find.

A new method for the characterisation and verification of local spatial predictability for convective scale ensembles

The use of kilometre-scale ensembles in operational forecasting provides new challenges for forecast interpretation and evaluation to account for uncertainty on the convective scale. A new neighbourhood based method is presented for evaluating and characterising the local predictability variations from convective scale ensembles. Spatial scales over which ensemble forecasts agree (agreement scales, S^A) are calculated at each grid point ij, providing a map of the spatial agreement between forecasts. By comparing the average agreement scale obtained from ensemble member pairs (S^A(mm)_ij), with that between members and radar observations (S^A(mo)_ij), this approach allows the location-dependent spatial spread-skill relationship of the ensemble to be assessed. The properties of the agreement scales are demonstrated using an idealised experiment. To demonstrate the methods in an operational context the S^A(mm)_ij and S^A(mo)_ij are calculated for six convective cases run with the Met Office UK Ensemble Prediction System. The S^A(mm)_ij highlight predictability differences between cases, which can be linked to physical processes. Maps of S^A(mm)_ij are found to summarise the spatial predictability in a compact and physically meaningful manner that is useful for forecasting and for model interpretation. Comparison of S^A(mm)_ij and S^A(mo)_ij demonstrates the case-by-case and temporal variability of the spatial spread-skill, which can again be linked to physical processes.

Intercomparison of methods of coupling between convection and large-scale circulation. 2: Comparison over non-uniform surface conditions

As part of an international intercomparison project, the weak temperature gradient (WTG) and damped gravity wave (DGW) methods are used to parameterize large-scale dynamics in a set of cloud-resolving models (CRMs) and single column models (SCMs). The WTG or DGW method is implemented using a configuration that couples a model to a reference state defined with profiles obtained from the same model in radiative-convective equilibrium. We investigated the sensitivity of each model to changes in SST, given a fixed reference state. We performed a systematic comparison of the WTG and DGW methods in different models, and a systematic comparison of the behavior of those models using the WTG method and the DGW method. The sensitivity to the SST depends on both the large-scale parameterization method and the choice of the cloud model. In general, SCMs display a wider range of behaviors than CRMs. All CRMs using either the WTG or DGW method show an increase of precipitation with SST, while SCMs show sensitivities which are not always monotonic. CRMs using either the WTG or DGW method show a similar relationship between mean precipitation rate and column-relative humidity, while SCMs exhibit a much wider range of behaviors. DGW simulations produce large-scale velocity profiles which are smoother and less top-heavy compared to those produced by the WTG simulations. These large-scale parameterization methods provide a useful tool to identify the impact of parameterization differences on model behavior in the presence of two-way feedback between convection and the large-scale circulation.