Prediction with Gaussian processes: from linear regression to linear prediction and beyond

The main aim of this paper is to provide a tutorial on regression with Gaussian processes. We start from Bayesian linear regression, and show how by a change of viewpoint one can see this method as a Gaussian process predictor based on priors over functions, rather than on priors over parameters. This leads in to a more general discussion of Gaussian processes in section 4. Section 5 deals with further issues, including hierarchical modelling and the setting of the parameters that control the Gaussian process, the covariance functions for neural network models and the use of Gaussian processes in classification problems.

Bayesian classification with Gaussian processes

We consider the problem of assigning an input vector bfx to one of m classes by predicting P(c|bfx) for c = 1, ldots, m. For a two-class problem, the probability of class 1 given bfx is estimated by s(y(bfx)), where s(y) = 1/(1 + e^-y). A Gaussian process prior is placed on y(bfx), and is combined with the training data to obtain predictions for new bfx points. We provide a Bayesian treatment, integrating over uncertainty in y and in the parameters that control the Gaussian process prior; the necessary integration over y is carried out using Laplace's approximation. The method is generalized to multi-class problems (m >2) using the softmax function. We demonstrate the effectiveness of the method on a number of datasets.

General bounds on Bayes errors for regression with Gaussian processes

Based on a simple convexity lemma, we develop bounds for different types of Bayesian prediction errors for regression with Gaussian processes. The basic bounds are formulated for a fixed training set. Simpler expressions are obtained for sampling from an input distribution which equals the weight function of the covariance kernel, yielding asymptotically tight results. The results are compared with numerical experiments.

Mean field methods for classification with Gaussian processes

We discuss the Application of TAP mean field methods known from Statistical Mechanics of disordered systems to Bayesian classification with Gaussian processes. In contrast to previous applications, no knowledge about the distribution of inputs is needed. Simulation results for the Sonar data set are given.

Bayesian classification with Gaussian processes

We consider the problem of assigning an input vector to one of m classes by predicting P(c|x) for c=1,...,m. For a two-class problem, the probability of class one given x is estimated by s(y(x)), where s(y)=1/(1+e-y). A Gaussian process prior is placed on y(x), and is combined with the training data to obtain predictions for new x points. We provide a Bayesian treatment, integrating over uncertainty in y and in the parameters that control the Gaussian process prior the necessary integration over y is carried out using Laplace's approximation. The method is generalized to multiclass problems (m>2) using the softmax function. We demonstrate the effectiveness of the method on a number of datasets.

Diagenetic processes in ore formation with special reference to the Zambian copperbelt and permian marl slate.

This thesis is concerned with the role of diagenesis in forming ore deposits. Two sedimentary 'ore-types' have been examined; the Proterozoic copper-cobalt orebodies of the Konkola Basin on the Zambian Copperbelt, and the Permian Marl Slate of North East England. Facies analysis of the Konkola Basin shows the Ore-Shale to have formed in a subtidal to intertidal environment. A sequence of diagenetic events is outlined from which it is concluded that the sulphide ores are an integral part of the diagenetic process. Sulphur isotope data establish that the sulphides formed as a consequence of the bacterial reduction of sulphate, while the isotopic and geochemical composition of carbonates is shown to reflect changes in the compositions of diagenetic pore fluids. Geochemical studies indicate that the copper and cobalt bearing mineralising fluids probably had different sources. Veins which crosscut the orebodies contain hydrocarbon inclusions, and are shown to be of late diagenetic lateral secretion origin. RbiSr dating indicates that the Ore-Shale was subject to metamorphism at 529 A- 20 myrs. The sedimentology and petrology of the Marl Slate are described. Textural and geochemical studies suggest that much of the pyrite (framboidal) in the Marl Slate formed in an anoxic water column, while euhedral pyrite and base metal sulphides formed within the sediment during early diagenesis. Sulphur isotope data confirm that conditions were almost "ideal" for sulphide formation during Marl Slate deposition, the limiting factors in ore formation being the restricted supply of chalcophile elements. Carbon and oxygen isotope data, along with petrographic observations, indicate that much of the calcite and dolomite occurring in the Marl Slate is primary, and probably formed in isotopic equilibrium. A depositional model is proposed which explains all of the data presented and links the lithological variations with fluctuations in the anoxicioxic boundary layer of the water column.

A simulation of an ERP implementation with informal processes

This paper presents a discrete event simulation study to examine tenancy service performance in a shopping centre. The study aims to provide an understanding of how informal management mechanisms could enhance existing ERP systems. The research shows the potential benefits of combining the traditional strengths of ERP in providing better performance in terms of efficiency with the ability to react with flexibility to customer's requests. © 2012 SIMULATION COUNCILS, INC.

Modeling high-speed network traffic with truncated α-stable processes

It has been reported that high-speed communication network traffic exhibits both long-range dependence (LRD) and burstiness, which posed new challenges in network engineering. While many models have been studied in capturing the traffic LRD, they are not capable of capturing efficiently the traffic impulsiveness. It is desirable to develop a model that can capture both LRD and burstiness. In this letter, we propose a truncated a-stable LRD process model for this purpose, which can characterize both LRD and burstiness accurately. A procedure is developed further to estimate the model parameters from real traffic. Simulations demonstrate that our proposed model has a higher accuracy compared to existing models and is flexible in capturing the characteristics of high-speed network traffic. © 2012 Springer-Verlag GmbH.

Aggregation and caking processes of granular materials:continuum model and simulation with application to sugar

Aggregation and caking of particles are common severe problems in many operations and processing of granular materials, where granulated sugar is an important example. Prevention of aggregation and caking of granular materials requires a good understanding of moisture migration and caking mechanisms. In this paper, the modeling of solid bridge formation between particles is introduced, based on moisture migration of atmospheric moisture into containers packed with granular materials through vapor evaporation and condensation. A model for the caking process is then developed, based on the growth of liquid bridges (during condensation), and their hardening and subsequent creation of solid bridges (during evaporation). The predicted caking strengths agree well with some available experimental data on granulated sugar under storage conditions.

Application of the Artificial Intelligence Algorithms with a Cognitive Graphic as a Tool for a System Analysis of Electrophysiological Processes

Summarizing the accumulated experience for a long time in the polyparametric cognitive modeling of different physiological processes (electrocardiogram, electroencephalogram, electroreovasogram and others) and the development on this basis some diagnostics methods give ground for formulating a new methodology of the system analysis in biology. The gist of the methodology consists of parametrization of fractals of electrophysiological processes, matrix description of functional state of an object with a unified set of parameters, construction of the polyparametric cognitive geometric model with artificial intelligence algorithms. The geometry model enables to display the parameter relationships are adequate to requirements of the system approach. The objective character of the elements of the models and high degree of formalization which facilitate the use of the mathematical methods are advantages of these models. At the same time the geometric images are easily interpreted in physiological and clinical terms. The polyparametric modeling is an object oriented tool possessed advances functional facilities and some principal features.

Monte Carlo Algorithm for Solving Integral Equations with Polynomial Non-Linearity. Parallel Implementation

An iterative Monte Carlo algorithm for evaluating linear functionals of the solution of integral equations with polynomial non-linearity is proposed and studied. The method uses a simulation of branching stochastic processes. It is proved that the mathematical expectation of the introduced random variable is equal to a linear functional of the solution. The algorithm uses the so-called almost optimal density function. Numerical examples are considered. Parallel implementation of the algorithm is also realized using the package ATHAPASCAN as an environment for parallel realization.The computational results demonstrate high parallel efficiency of the presented algorithm and give a good solution when almost optimal density function is used as a transition density.

Branching Stochastic Processes: Regulation, Regeneration, Estimation, Applications

2000 Mathematics Subject Classification: 60J80.

Conflict-Controlled Processes Involving Fractional Differential Equations with Impulses

MSC 2010: 34A08, 34A37, 49N70

Projected sequential Gaussian processes:a C++ tool for interpolation of large datasets with heterogeneous noise

Heterogeneous datasets arise naturally in most applications due to the use of a variety of sensors and measuring platforms. Such datasets can be heterogeneous in terms of the error characteristics and sensor models. Treating such data is most naturally accomplished using a Bayesian or model-based geostatistical approach; however, such methods generally scale rather badly with the size of dataset, and require computationally expensive Monte Carlo based inference. Recently within the machine learning and spatial statistics communities many papers have explored the potential of reduced rank representations of the covariance matrix, often referred to as projected or fixed rank approaches. In such methods the covariance function of the posterior process is represented by a reduced rank approximation which is chosen such that there is minimal information loss. In this paper a sequential Bayesian framework for inference in such projected processes is presented. The observations are considered one at a time which avoids the need for high dimensional integrals typically required in a Bayesian approach. A C++ library, gptk, which is part of the INTAMAP web service, is introduced which implements projected, sequential estimation and adds several novel features. In particular the library includes the ability to use a generic observation operator, or sensor model, to permit data fusion. It is also possible to cope with a range of observation error characteristics, including non-Gaussian observation errors. Inference for the covariance parameters is explored, including the impact of the projected process approximation on likelihood profiles. We illustrate the projected sequential method in application to synthetic and real datasets. Limitations and extensions are discussed. © 2010 Elsevier Ltd.