Self-play: statistical significance

Heinz recently completed a comprehensive experiment in self-play using the FRITZ chess engine to establish the ‘decreasing returns’ hypothesis with specific levels of statistical confidence. This note revisits the results and recalculates the confidence levels of this and other hypotheses. These appear to be better than Heinz’ initial analysis suggests.

Self-play: statistical significance

Heinz recently completed a comprehensive experiment in self-play using the FRITZ chess engine to establish the ‘decreasing returns’ hypothesis with specific levels of statistical confidence. This note revisits the results and recalculates the confidence levels of this and other hypotheses. These appear to be better than Heinz’ initial analysis suggests.

Testing the statistical significance of sector & regional diversification

The question as to whether it is better to diversify a real estate portfolio within a property type across the regions or within a region across the property types is one of continuing interest for academics and practitioners alike. The current study, however, is somewhat different from the usual sector/regional analysis taking account of the fact that holdings in the UK real estate market are heavily concentrated in a single region, London. As a result this study is designed to investigate whether a real estate fund manager can obtain a statistically significant improvement in risk/return performance from extending out of a London based portfolio into firstly the rest of the South East of England and then into the remainder of the UK, or whether the manger would be better off staying within London and diversifying across the various property types. The results indicating that staying within London and diversifying across the various property types may offer performance comparable with regional diversification, although this conclusion largely depends on the time period and the fund manager’s ability to diversify efficiently.

Testing for significance of phase synchronisation dynamics in the EEG

A number of tests exist to check for statistical significance of phase synchronisation within the Electroencephalogram (EEG); however, the majority suffer from a lack of generality and applicability. They may also fail to account for temporal dynamics in the phase synchronisation, regarding synchronisation as a constant state instead of a dynamical process. Therefore, a novel test is developed for identifying the statistical significance of phase synchronisation based upon a combination of work characterising temporal dynamics of multivariate time-series and Markov modelling. We show how this method is better able to assess the significance of phase synchronisation than a range of commonly used significance tests. We also show how the method may be applied to identify and classify significantly different phase synchronisation dynamics in both univariate and multivariate datasets.

The mean evolution and variability of the Asian summer monsoon: Comparison of ECMWF and NCEP-NCAR reanalyses

The behavior of the Asian summer monsoon is documented and compared using the European Centre for Medium-Range Weather Forecasts (ECMWF) Reanalysis (ERA) and the National Centers for Environmental Prediction-National Center for Atmospheric Research (NCEP-NCAR) Reanalysis. In terms of seasonal mean climatologies the results suggest that, in several respects, the ERA is superior to the NCEP-NCAR Reanalysis. The overall better simulation of the precipitation and hence the diabatic heating field over the monsoon domain in ERA means that the analyzed circulation is probably nearer reality. In terms of interannual variability, inconsistencies in the definition of weak and strong monsoon years based on typical monsoon indices such as All-India Rainfall (AIR) anomalies and the large-scale wind shear based dynamical monsoon index (DMI) still exist. Two dominant modes of interannual variability have been identified that together explain nearly 50% of the variance. Individually, they have many features in common with the composite flow patterns associated with weak and strong monsoons, when defined in terms of regional AIR anomalies and the large-scale DMI. The reanalyses also show a common dominant mode of intraseasonal variability that describes the latitudinal displacement of the tropical convergence zone from its oceanic-to-continental regime and essentially captures the low-frequency active/break cycles of the monsoon. The relationship between interannual and intraseasonal variability has been investigated by considering the probability density function (PDF) of the principal component of the dominant intraseasonal mode. Based on the DMI, there is an indication that in years with a weaker monsoon circulation, the PDF is skewed toward negative values (i,e., break conditions). Similarly, the PDFs for El Nino and La Nina years suggest that El Nino predisposes the system to more break spells, although the sample size may limit the statistical significance of the results.

Global observed changes in daily climate extremes of temperature and precipitation

A suite of climate change indices derived from daily temperature and precipitation data, with a primary focus on extreme events, were computed and analyzed. By setting an exact formula for each index and using specially designed software, analyses done in different countries have been combined seamlessly. This has enabled the presentation of the most up-to-date and comprehensive global picture of trends in extreme temperature and precipitation indices using results from a number of workshops held in data-sparse regions and high-quality station data supplied by numerous scientists world wide. Seasonal and annual indices for the period 1951-2003 were gridded. Trends in the gridded fields were computed and tested for statistical significance. Results showed widespread significant changes in temperature extremes associated with warming, especially for those indices derived from daily minimum temperature. Over 70% of the global land area sampled showed a significant decrease in the annual occurrence of cold nights and a significant increase in the annual occurrence of warm nights. Some regions experienced a more than doubling of these indices. This implies a positive shift in the distribution of daily minimum temperature throughout the globe. Daily maximum temperature indices showed similar changes but with smaller magnitudes. Precipitation changes showed a widespread and significant increase, but the changes are much less spatially coherent compared with temperature change. Probability distributions of indices derived from approximately 200 temperature and 600 precipitation stations, with near-complete data for 1901-2003 and covering a very large region of the Northern Hemisphere midlatitudes (and parts of Australia for precipitation) were analyzed for the periods 1901-1950, 1951-1978 and 1979-2003. Results indicate a significant warming throughout the 20th century. Differences in temperature indices distributions are particularly pronounced between the most recent two periods and for those indices related to minimum temperature. An analysis of those indices for which seasonal time series are available shows that these changes occur for all seasons although they are generally least pronounced for September to November. Precipitation indices show a tendency toward wetter conditions throughout the 20th century.

Validating ECMWF forecasts for the occurrence of ice supersaturation using visual observations of persistent contrails and radiosonde measurements over England

One of the largest uncertainties in quantifying the impact of aviation on climate concerns the formation and spreading of persistent contrails. The inclusion of a cloud scheme that allows for ice supersaturation into the integrated forecast system (IFS) of the European Centre for Medium-Range Weather Forecasts (ECMWF) can be a useful tool to help reduce these uncertainties. This study evaluates the quality of the ECMWF forecasts with respect to ice super saturation in the upper troposphere by comparing them to visual observations of persistent contrails and radiosonde measurements of ice supersaturation over England. The performance of 1- to 3-day forecasts is compared including also the vertical accuracy of the supersaturation forecasts. It is found that the operational forecasts from the ECMWF are able to predict cold ice supersaturated regions very well. For the best cases Peirce skill scores of 0.7 are obtained, with hit rates at times exceeding 80% and false-alarm rates below 20%. Results are very similar for comparisons with visual observations and radiosonde measurements, the latter providing the better statistical significance.

The measurement of maternal plasma corticotropin-releasing factor (CRF) and CRF-binding protein improves the early prediction of preeclampsia

In the present study we measured maternal plasma concentrations of two placental neurohormones, corticotropin-releasing factor (CRF) and CRF-binding protein (CRF-BP), in 58 at-risk pregnant women consecutively enrolled between 28 and 29 wk of pregnancy to evaluate whether their evaluation may predict third trimester-onset preeclampsia ( PE). The statistical significance was assessed by t test. The cut-off points for defining altered CRF and CRF-BP levels for prediction of PE were chosen by receiving operator characteristics curve analysis, and the probability of developing PE was calculated for several combinations of hormone testing results. CRF and CRF-BP levels were significantly ( both P < 0.0001) higher and lower, respectively, in the patients (n = 20) who later developed PE than in those who did not present PE at follow-up. CRF at the cut-off 425.95 pmol/liter achieved a sensitivity of 94.8% and a specificity of 96.9%, whereas CRF-BP at the cut-off 125.8 nmol/liter combined a sensitivity of 92.5% and a specificity of 82.5% as single markers for prediction of PE. The probability of PE was 34.5% in the whole study population, 93.75% when both CRF and CRF-BP levels were changed, and 0% if both hormone markers were unaltered. The measurement of CRF and CRF-BP levels may add significant prognostic information for predicting PE in at-risk pregnant women.

Detection of neural correlates of self-paced motor activity using empirical mode decomposition phase locking analysis

Transient episodes of synchronisation of neuronal activity in particular frequency ranges are thought to underlie cognition. Empirical mode decomposition phase locking (EMDPL) analysis is a method for determining the frequency and timing of phase synchrony that is adaptive to intrinsic oscillations within data, alleviating the need for arbitrary bandpass filter cut-off selection. It is extended here to address the choice of reference electrode and removal of spurious synchrony resulting from volume conduction. Spline Laplacian transformation and independent component analysis (ICA) are performed as pre-processing steps, and preservation of phase synchrony between synthetic signals. combined using a simple forward model, is demonstrated. The method is contrasted with use of bandpass filtering following the same preprocessing steps, and filter cut-offs are shown to influence synchrony detection markedly. Furthermore, an approach to the assessment of multiple EEG trials using the method is introduced, and the assessment of statistical significance of phase locking episodes is extended to render it adaptive to local phase synchrony levels. EMDPL is validated in the analysis of real EEG data, during finger tapping. The time course of event-related (de)synchronisation (ERD/ERS) is shown to differ from that of longer range phase locking episodes, implying different roles for these different types of synchronisation. It is suggested that the increase in phase locking which occurs just prior to movement, coinciding with a reduction in power (or ERD) may result from selection of the neural assembly relevant to the particular movement. (C) 2009 Elsevier B.V. All rights reserved.

Markers of intestinally-derived lipoproteins: application to studies of altered diet and meal fatty acid compositions

BACKGROUND AND AIM: The atherogenic potential of dietary derived lipids, chylomicrons (CM) and their remnants (CMr) is now becoming more widely recognised. To investigate factors effecting levels of CM and CMr and their importance in coronary heart disease risk it is essential to use a specific method of quantification. Two studies were carried out to investigate: (i) effects of increased daily intake of long chain n-3 polyunsaturated fatty acid (LC n-3 PUFA), and (ii) effects of increasing meal monounsaturated fatty acid (MUFA) content on the postprandial response of intestinally-derived lipoproteins. The contribution of the intestinally-derived lipoproteins to total lipaemia was assessed by triacylglycerol-rich lipoprotein (TRL) apolipoprotein B-48 (apo B-48) and retinyl ester (RE) concentrations. METHODS AND RESULTS: In a randomised controlled crossover trial (placebo vs LC n-3 PUFA) a mean daily intake of 1.4 g/day of LC n-3 PUFA failed to reduce fasting and postprandial triacylglycerol (TAG) response in 9 healthy male volunteers. Although the pattern and nature of the apo B-48 response was consistent with the TAG response following the two diets, the postprandial RE response differed on the LC n-3 PUFA diet with a lower early RE response and a delayed and more marked increase in RE in the late postprandial period compared with the control diet, but the differences did not reach levels of statistical significance. In the meal study there was no effect of MUFA/SFA content on the total lipaemic response to the meals nor on the contribution of intestinally derived lipoproteins evaluated as TAG, apo B-48 and RE responses in the TRL fraction. In both studies, the RE and apo B-48 measurements provided broadly similar information with respect to lack of effects of dietary or meal fatty acid composition and the presence of single or multiple peak responses. However the apo B-48 and RE measurements differed with respect to the timing of their peak response times, with a delayed RE peak, relalive to apo B-48, of approximately 2-3 hours for the LC n-3 PUFA diet (p = 0.002) study and 1-1.5 hours for the meal MUFA/SFA study. CONCLUSIONS: It was concluded that there are limitations of using RE as a specific CM marker, apo B-48 quantitation was found to be a more appropriate method for CM and CMr quantitation. However it was still considered of value to measure RE as it provided additional information regarding the incorporation of other constituents into the CM particle.

The influence of dietary fatty acid composition on N-ethyl-Nnitrosourea-induced mammary tumour incidence in the rat and on the composition of inositol- and ethanolamine-phospholipids of normal and tumour mammary tissue

This study has investigated the influence of dietary fatty acid composition on mammary tumour incidence in N-ethyl-N-nitrosourea (ENU)-treated rats and has compared the susceptibility to dietary fatty acid modification of the membrane phospholipids phosphatidyliuositol (PI) and phosphatidylethanolamine (PE) from normal and tumour tissue of rat mammary gland. The incidence of mammary tumours was significantly lower in fish oil- (29%), compared with olive oil- (75%; P < 0.04) but not maize oil- (63%; P < 0.1) fed animals. No differences in PI fatty acid composition were found in normal or tumour tissue between rats fed on maize oil, olive oil or fish oil in diets from weaning. When normal and tumour tissue PI fatty acids were compared, significantly higher amounts of stearic acid (18:O) were found in tumour than normal tissue in rats given olive oil (P < 0.05). A similar trend was found in animals fed on maize oil, although differences between normal and tumour tissue did not reach a level of statistical significance (P < 0.1). In mammary PE, maize oil-fed control animals had significantly higher levels of linoleic acid (18:2n-6) than either olive oil- or fish oil-fed animals (P < 0.05, both cases) and levels of arachidonic acid were also higher in maize oil- compared with fish oil-fed animals (P < 0.05). In tumourbearing animals no differences in PE fatty acid composition were found between the three dietary groups. When normal and tumour tissue PE fatty acids were compared, significantly lower amounts of liuoleic acid (18:2n-6; P < 0.01) and significantly greater amounts of arachidonic acid (20:4n-6; P < 0.05) were found in tumour than normal tissue of rats fed on maize oil. The present study shows that the fatty acid composition of PI from both normal and tumour tissue of the mammary gland is resistant to dietary fatty acid modification. The PE fraction is more susceptible to dietary modification and in this fraction there is evidence of increased conversion of linoleic acid to arachidonic acid in tumour compared with normal tissue. Lower tumour incidence rates in rats given fish oils may in part be due to alteration in prostanoid metabolism secondary to displacement of arachidonic acid by eicosapentaenoic acid, but PE rather than PI would appear to be the most likely locus for diet-induced alteration in prostanoid synthesis in this tissue. Effects of dietary fatty acids other than on the balance of n-6 and n-3 fatty acids, and on prostanoid metabolism, should also be considered. The significance of increased stearic acid content of PI in tumours of olive oil-fed animals and the possible influence of dietary fatty acids on the capacity for stearic acid accumulation requires further study.

Advances in sequential data assimilation and numerical weather forecasting: an Ensemble Transform Kalman-Bucy Filter, a study on clustering in deterministic Ensemble Square Root Filters, and a test of a new time stepping scheme in an atmospheric model

This dissertation deals with aspects of sequential data assimilation (in particular ensemble Kalman filtering) and numerical weather forecasting. In the first part, the recently formulated Ensemble Kalman-Bucy (EnKBF) filter is revisited. It is shown that the previously used numerical integration scheme fails when the magnitude of the background error covariance grows beyond that of the observational error covariance in the forecast window. Therefore, we present a suitable integration scheme that handles the stiffening of the differential equations involved and doesn’t represent further computational expense. Moreover, a transform-based alternative to the EnKBF is developed: under this scheme, the operations are performed in the ensemble space instead of in the state space. Advantages of this formulation are explained. For the first time, the EnKBF is implemented in an atmospheric model. The second part of this work deals with ensemble clustering, a phenomenon that arises when performing data assimilation using of deterministic ensemble square root filters in highly nonlinear forecast models. Namely, an M-member ensemble detaches into an outlier and a cluster of M-1 members. Previous works may suggest that this issue represents a failure of EnSRFs; this work dispels that notion. It is shown that ensemble clustering can be reverted also due to nonlinear processes, in particular the alternation between nonlinear expansion and compression of the ensemble for different regions of the attractor. Some EnSRFs that use random rotations have been developed to overcome this issue; these formulations are analyzed and their advantages and disadvantages with respect to common EnSRFs are discussed. The third and last part contains the implementation of the Robert-Asselin-Williams (RAW) filter in an atmospheric model. The RAW filter is an improvement to the widely popular Robert-Asselin filter that successfully suppresses spurious computational waves while avoiding any distortion in the mean value of the function. Using statistical significance tests both at the local and field level, it is shown that the climatology of the SPEEDY model is not modified by the changed time stepping scheme; hence, no retuning of the parameterizations is required. It is found the accuracy of the medium-term forecasts is increased by using the RAW filter.

From ensemble forecasts to predictive distribution functions

The translation of an ensemble of model runs into a probability distribution is a common task in model-based prediction. Common methods for such ensemble interpretations proceed as if verification and ensemble were draws from the same underlying distribution, an assumption not viable for most, if any, real world ensembles. An alternative is to consider an ensemble as merely a source of information rather than the possible scenarios of reality. This approach, which looks for maps between ensembles and probabilistic distributions, is investigated and extended. Common methods are revisited, and an improvement to standard kernel dressing, called ‘affine kernel dressing’ (AKD), is introduced. AKD assumes an affine mapping between ensemble and verification, typically not acting on individual ensemble members but on the entire ensemble as a whole, the parameters of this mapping are determined in parallel with the other dressing parameters, including a weight assigned to the unconditioned (climatological) distribution. These amendments to standard kernel dressing, albeit simple, can improve performance significantly and are shown to be appropriate for both overdispersive and underdispersive ensembles, unlike standard kernel dressing which exacerbates over dispersion. Studies are presented using operational numerical weather predictions for two locations and data from the Lorenz63 system, demonstrating both effectiveness given operational constraints and statistical significance given a large sample.

Response of the northern stratospheric polar vortex to the seasonal alignment of QBO phase transitions

This study considers the strength of the Northern Hemisphere Holton-Tan effect (HTE) in terms of the phase alignment of the quasi-biennial oscillation (QBO) with respect to the annual cycle. Using the ERA-40 Reanalysis, it is found that the early winter (Nov–Dec) and late winter (Feb–Mar) relation between QBO phase and the strength of the stratospheric polar vortex is optimized for subsets of the 44-year record that are chosen on the basis of the seasonality of QBO phase transitions at the 30 hPa level. The timing of phase transitions serves as a proxy for changes in the vertical structure of the QBO over the whole depth of the tropical stratosphere. The statistical significance of the Nov–Dec (Feb–Mar) HTE is greatest when 30 hPa QBO phase transitions occur 9–14 (4–9) months prior to the January of the NH winter in question. This suggests that there exists for both early and late winter a vertical structure of tropical stratospheric winds that is most effective at influencing the interannual variability of the polar vortex, and that an early (late) winter HTE is associated with an early (late) progression of QBO phase towards that structure. It is also shown that the seasonality of QBO phase transitions at 30 hPa varies on a decadal timescale, with transitions during the first half of the calendar year being relatively more common during the first half of the tropical radiosonde wind record. Combining these two results suggests that decadal changes in HTE strength could result from the changing seasonality of QBO phase transitions. Citation: Anstey, J. A., and T. G. Shepherd (2008), Response of the northern stratospheric polar vortex to the seasonal alignment of QBO phase transitions, Geophys. Res. Lett., 35, L22810, doi:10.1029/2008GL035721.

Simple uncertainty frameworks for selecting weighting schemes and interpreting multimodel ensemble climate change experiments

Future climate change projections are often derived from ensembles of simulations from multiple global circulation models using heuristic weighting schemes. This study provides a more rigorous justification for this by introducing a nested family of three simple analysis of variance frameworks. Statistical frameworks are essential in order to quantify the uncertainty associated with the estimate of the mean climate change response. The most general framework yields the “one model, one vote” weighting scheme often used in climate projection. However, a simpler additive framework is found to be preferable when the climate change response is not strongly model dependent. In such situations, the weighted multimodel mean may be interpreted as an estimate of the actual climate response, even in the presence of shared model biases. Statistical significance tests are derived to choose the most appropriate framework for specific multimodel ensemble data. The framework assumptions are explicit and can be checked using simple tests and graphical techniques. The frameworks can be used to test for evidence of nonzero climate response and to construct confidence intervals for the size of the response. The methodology is illustrated by application to North Atlantic storm track data from the Coupled Model Intercomparison Project phase 5 (CMIP5) multimodel ensemble. Despite large variations in the historical storm tracks, the cyclone frequency climate change response is not found to be model dependent over most of the region. This gives high confidence in the response estimates. Statistically significant decreases in cyclone frequency are found on the flanks of the North Atlantic storm track and in the Mediterranean basin.