Mean field methods for classification with Gaussian processes

We discuss the Application of TAP mean field methods known from Statistical Mechanics of disordered systems to Bayesian classification with Gaussian processes. In contrast to previous applications, no knowledge about the distribution of inputs is needed. Simulation results for the Sonar data set are given.

Learning curves for Gaussian processes models: fluctuations and universality

Based on a statistical mechanics approach, we develop a method for approximately computing average case learning curves and their sample fluctuations for Gaussian process regression models. We give examples for the Wiener process and show that universal relations (that are independent of the input distribution) between error measures can be derived.

Conditional distribution modeling for estimating and exploiting uncertainty in control systems

This paper presents a general methodology for estimating and incorporating uncertainty in the controller and forward models for noisy nonlinear control problems. Conditional distribution modeling in a neural network context is used to estimate uncertainty around the prediction of neural network outputs. The developed methodology circumvents the dynamic programming problem by using the predicted neural network uncertainty to localize the possible control solutions to consider. A nonlinear multivariable system with different delays between the input-output pairs is used to demonstrate the successful application of the developed control algorithm. The proposed method is suitable for redundant control systems and allows us to model strongly non Gaussian distributions of control signal as well as processes with hysteresis.

Regularisation of mixture density networks

Mixture Density Networks are a principled method to model conditional probability density functions which are non-Gaussian. This is achieved by modelling the conditional distribution for each pattern with a Gaussian Mixture Model for which the parameters are generated by a neural network. This thesis presents a novel method to introduce regularisation in this context for the special case where the mean and variance of the spherical Gaussian Kernels in the mixtures are fixed to predetermined values. Guidelines for how these parameters can be initialised are given, and it is shown how to apply the evidence framework to mixture density networks to achieve regularisation. This also provides an objective stopping criteria that can replace the `early stopping' methods that have previously been used. If the neural network used is an RBF network with fixed centres this opens up new opportunities for improved initialisation of the network weights, which are exploited to start training relatively close to the optimum. The new method is demonstrated on two data sets. The first is a simple synthetic data set while the second is a real life data set, namely satellite scatterometer data used to infer the wind speed and wind direction near the ocean surface. For both data sets the regularisation method performs well in comparison with earlier published results. Ideas on how the constraint on the kernels may be relaxed to allow fully adaptable kernels are presented.

Bayesian classification with Gaussian processes

We consider the problem of assigning an input vector to one of m classes by predicting P(c|x) for c=1,...,m. For a two-class problem, the probability of class one given x is estimated by s(y(x)), where s(y)=1/(1+e-y). A Gaussian process prior is placed on y(x), and is combined with the training data to obtain predictions for new x points. We provide a Bayesian treatment, integrating over uncertainty in y and in the parameters that control the Gaussian process prior the necessary integration over y is carried out using Laplace's approximation. The method is generalized to multiclass problems (m>2) using the softmax function. We demonstrate the effectiveness of the method on a number of datasets.

Detection of signals corrupted by the combination of Gaussian and Non-Gaussian noise

The detection of signals in the presence of noise is one of the most basic and important problems encountered by communication engineers. Although the literature abounds with analyses of communications in Gaussian noise, relatively little work has appeared dealing with communications in non-Gaussian noise. In this thesis several digital communication systems disturbed by non-Gaussian noise are analysed. The thesis is divided into two main parts. In the first part, a filtered-Poisson impulse noise model is utilized to calulate error probability characteristics of a linear receiver operating in additive impulsive noise. Firstly the effect that non-Gaussian interference has on the performance of a receiver that has been optimized for Gaussian noise is determined. The factors affecting the choice of modulation scheme so as to minimize the deterimental effects of non-Gaussian noise are then discussed. In the second part, a new theoretical model of impulsive noise that fits well with the observed statistics of noise in radio channels below 100 MHz has been developed. This empirical noise model is applied to the detection of known signals in the presence of noise to determine the optimal receiver structure. The performance of such a detector has been assessed and is found to depend on the signal shape, the time-bandwidth product, as well as the signal-to-noise ratio. The optimal signal to minimize the probability of error of; the detector is determined. Attention is then turned to the problem of threshold detection. Detector structure, large sample performance and robustness against errors in the detector parameters are examined. Finally, estimators of such parameters as. the occurrence of an impulse and the parameters in an empirical noise model are developed for the case of an adaptive system with slowly varying conditions.

What determines the size frequency distribution of β-amyloid (Aβ) deposits in Alzheimer's disease patients?

The factors determining the size of individual β-amyloid (A,8) deposits and their size frequency distribution in tissue from Alzheimer's disease (AD) patients have not been established. In 23/25 cortical tissues from 10 AD patients, the frequency of Aβ deposits declined exponentially with increasing size. In a random sample of 400 Aβ deposits, 88% were closely associated with one or more neuronal cell bodies. The frequency distribution of (Aβ) deposits which were associated with 0,1,2,...,n neuronal cell bodies deviated significantly from a Poisson distribution, suggesting a degree of clustering of the neuronal cell bodies. In addition, the frequency of Aβ deposits declined exponentially as the number of associated neuronal cell bodies increased. Aβ deposit area was positively correlated with the frequency of associated neuronal cell bodies, the degree of correlation being greater for pyramidal cells than smaller neurons. These data suggested: (1) the number of closely adjacent neuronal cell bodies which simultaneously secrete Aβ was an important factor determining the size of an Aβ deposit and (2) the exponential decline in larger Aβ deposits reflects the low probability that larger numbers of adjacent neurons will secrete Aβ simultaneously to form a deposit. © 1995.

Describing and communicating uncertainty within the semantic web

The Semantic Web relies on carefully structured, well defined, data to allow machines to communicate and understand one another. In many domains (e.g. geospatial) the data being described contains some uncertainty, often due to incomplete knowledge; meaningful processing of this data requires these uncertainties to be carefully analysed and integrated into the process chain. Currently, within the SemanticWeb there is no standard mechanism for interoperable description and exchange of uncertain information, which renders the automated processing of such information implausible, particularly where error must be considered and captured as it propagates through a processing sequence. In particular we adopt a Bayesian perspective and focus on the case where the inputs / outputs are naturally treated as random variables. This paper discusses a solution to the problem in the form of the Uncertainty Markup Language (UncertML). UncertML is a conceptual model, realised as an XML schema, that allows uncertainty to be quantified in a variety of ways i.e. realisations, statistics and probability distributions. UncertML is based upon a soft-typed XML schema design that provides a generic framework from which any statistic or distribution may be created. Making extensive use of Geography Markup Language (GML) dictionaries, UncertML provides a collection of definitions for common uncertainty types. Containing both written descriptions and mathematical functions, encoded as MathML, the definitions within these dictionaries provide a robust mechanism for defining any statistic or distribution and can be easily extended. Universal Resource Identifiers (URIs) are used to introduce semantics to the soft-typed elements by linking to these dictionary definitions. The INTAMAP (INTeroperability and Automated MAPping) project provides a use case for UncertML. This paper demonstrates how observation errors can be quantified using UncertML and wrapped within an Observations & Measurements (O&M) Observation. The interpolation service uses the information within these observations to influence the prediction outcome. The output uncertainties may be encoded in a variety of UncertML types, e.g. a series of marginal Gaussian distributions, a set of statistics, such as the first three marginal moments, or a set of realisations from a Monte Carlo treatment. Quantifying and propagating uncertainty in this way allows such interpolation results to be consumed by other services. This could form part of a risk management chain or a decision support system, and ultimately paves the way for complex data processing chains in the Semantic Web.

Statnote 36: Do the data fit the Poisson distribution

In previous Statnotes, many of the statistical tests described rely on the assumption that the data are a random sample from a normal or Gaussian distribution. These include most of the tests in common usage such as the ‘t’ test ), the various types of analysis of variance (ANOVA), and Pearson’s correlation coefficient (‘r’) . In microbiology research, however, not all variables can be assumed to follow a normal distribution. Yeast populations, for example, are a notable feature of freshwater habitats, representatives of over 100 genera having been recorded . Most common are the ‘red yeasts’ such as Rhodotorula, Rhodosporidium, and Sporobolomyces and ‘black yeasts’ such as Aurobasidium pelculans, together with species of Candida. Despite the abundance of genera and species, the overall density of an individual species in freshwater is likely to be low and hence, samples taken from such a population will contain very low numbers of cells. A rare organism living in an aquatic environment may be distributed more or less at random in a volume of water and therefore, samples taken from such an environment may result in counts which are more likely to be distributed according to the Poisson than the normal distribution. The Poisson distribution was named after the French mathematician Siméon Poisson (1781-1840) and has many applications in biology, especially in describing rare or randomly distributed events, e.g., the number of mutations in a given sequence of DNA after exposure to a fixed amount of radiation or the number of cells infected by a virus given a fixed level of exposure. This Statnote describes how to fit the Poisson distribution to counts of yeast cells in samples taken from a freshwater lake.

Statnote 39: The binomial distribution: comparing two proportions

Many populations consist of two classes only, e.g., alive or dead, present or absent, clean or dirty, infected or non-infected, and it is the proportion or percentage of observations that fall into one of these classes that is of interest to an investigator. An observation that falls into one of the two classes is considered a ‘success’ (S), and ‘p’ is defined as the proportion of observations falling into that class. If a random sample of size ‘n’ is obtained from a population, the probability of obtaining 0, 1, 2, 3, etc., successes is then given by the binomial distribution. The binomial distribution can be used as the basis of a number of statistical tests but is most useful when comparing two proportions. This statnote describes two such scenarios in which the binomial distribution is used to compare: (1) two proportions when the samples are independent and (2) two proportions when the samples are paired.

Revealing statistical properties of quasi-CW fibre lasers in bandwidth-limited measurements

We introduce a general technique how to reveal in experiments of limited electrical bandwidth which is lower than the optical bandwidth of the optical signal under study, whether the statistical properties of the light source obey Gaussian distribution or mode correlations do exist. To do that one needs to perform measurements by decreasing the measurement bandwidth. We develop a simple model of bandwidth-limited measurements and predict universal laws how intensity probability density function and intensity auto-correlation function of ideal completely stochastic source of Gaussian statistics depend on limited measurement bandwidth and measurement noise level. Results of experimental investigation are in good agreement with model predictions. In particular, we reveal partial mode correlations in the radiation of quasi-CW Raman fibre laser.

Sums of a Random Number of Random Variables and their Approximations with ν- Accompanying Infinitely Divisible Laws

* Research supported by NATO GRANT CRG 900 798 and by Humboldt Award for U.S. Scientists.

On the Coherence Between Probability and Possibility Measures

* This paper is supported by CICYT (Spain) under Project TIN 2005-08943-C02-01.

Mixed Negative Binomial Distribution by Weighted Gamma Mixing Distribution

Павел Т. Стойнов - В тази работа се разглежда отрицателно биномното разпределение, известно още като разпределение на Пойа. Предполагаме, че смесващото разпределение е претеглено гама разпределение. Изведени са вероятностите в някои частни случаи. Дадени са рекурентните формули на Панжер.

An Estimate of the Probability Pr(X

2000 Mathematics Subject Classification: 33C90, 62E99