Prediction with Gaussian processes: from linear regression to linear prediction and beyond

The main aim of this paper is to provide a tutorial on regression with Gaussian processes. We start from Bayesian linear regression, and show how by a change of viewpoint one can see this method as a Gaussian process predictor based on priors over functions, rather than on priors over parameters. This leads in to a more general discussion of Gaussian processes in section 4. Section 5 deals with further issues, including hierarchical modelling and the setting of the parameters that control the Gaussian process, the covariance functions for neural network models and the use of Gaussian processes in classification problems.

Upper and lower bounds on the learning curve for Gaussian processes

In this paper we introduce and illustrate non-trivial upper and lower bounds on the learning curves for one-dimensional Gaussian Processes. The analysis is carried out emphasising the effects induced on the bounds by the smoothness of the random process described by the Modified Bessel and the Squared Exponential covariance functions. We present an explanation of the early, linearly-decreasing behavior of the learning curves and the bounds as well as a study of the asymptotic behavior of the curves. The effects of the noise level and the lengthscale on the tightness of the bounds are also discussed.

Gaussian processes:iterative sparse approximations

In recent years there has been an increased interest in applying non-parametric methods to real-world problems. Significant research has been devoted to Gaussian processes (GPs) due to their increased flexibility when compared with parametric models. These methods use Bayesian learning, which generally leads to analytically intractable posteriors. This thesis proposes a two-step solution to construct a probabilistic approximation to the posterior. In the first step we adapt the Bayesian online learning to GPs: the final approximation to the posterior is the result of propagating the first and second moments of intermediate posteriors obtained by combining a new example with the previous approximation. The propagation of em functional forms is solved by showing the existence of a parametrisation to posterior moments that uses combinations of the kernel function at the training points, transforming the Bayesian online learning of functions into a parametric formulation. The drawback is the prohibitive quadratic scaling of the number of parameters with the size of the data, making the method inapplicable to large datasets. The second step solves the problem of the exploding parameter size and makes GPs applicable to arbitrarily large datasets. The approximation is based on a measure of distance between two GPs, the KL-divergence between GPs. This second approximation is with a constrained GP in which only a small subset of the whole training dataset is used to represent the GP. This subset is called the em Basis Vector, or BV set and the resulting GP is a sparse approximation to the true posterior. As this sparsity is based on the KL-minimisation, it is probabilistic and independent of the way the posterior approximation from the first step is obtained. We combine the sparse approximation with an extension to the Bayesian online algorithm that allows multiple iterations for each input and thus approximating a batch solution. The resulting sparse learning algorithm is a generic one: for different problems we only change the likelihood. The algorithm is applied to a variety of problems and we examine its performance both on more classical regression and classification tasks and to the data-assimilation and a simple density estimation problems.

Studies on the generalisation of Gaussian processes and Bayesian neural networks

The assessment of the reliability of systems which learn from data is a key issue to investigate thoroughly before the actual application of information processing techniques to real-world problems. Over the recent years Gaussian processes and Bayesian neural networks have come to the fore and in this thesis their generalisation capabilities are analysed from theoretical and empirical perspectives. Upper and lower bounds on the learning curve of Gaussian processes are investigated in order to estimate the amount of data required to guarantee a certain level of generalisation performance. In this thesis we analyse the effects on the bounds and the learning curve induced by the smoothness of stochastic processes described by four different covariance functions. We also explain the early, linearly-decreasing behaviour of the curves and we investigate the asymptotic behaviour of the upper bounds. The effect of the noise and the characteristic lengthscale of the stochastic process on the tightness of the bounds are also discussed. The analysis is supported by several numerical simulations. The generalisation error of a Gaussian process is affected by the dimension of the input vector and may be decreased by input-variable reduction techniques. In conventional approaches to Gaussian process regression, the positive definite matrix estimating the distance between input points is often taken diagonal. In this thesis we show that a general distance matrix is able to estimate the effective dimensionality of the regression problem as well as to discover the linear transformation from the manifest variables to the hidden-feature space, with a significant reduction of the input dimension. Numerical simulations confirm the significant superiority of the general distance matrix with respect to the diagonal one.In the thesis we also present an empirical investigation of the generalisation errors of neural networks trained by two Bayesian algorithms, the Markov Chain Monte Carlo method and the evidence framework; the neural networks have been trained on the task of labelling segmented outdoor images.

Transient Receptor Potential Vanilloid 4 Ion Channel Functions as a Pruriceptor in Epidermal Keratinocytes to Evoke Histaminergic Itch.

TRPV4 ion channels function in epidermal keratinocytes and in innervating sensory neurons; however, the contribution of the channel in either cell to neurosensory function remains to be elucidated. We recently reported TRPV4 as a critical component of the keratinocyte machinery that responds to ultraviolet B (UVB) and functions critically to convert the keratinocyte into a pain-generator cell after excess UVB exposure. One key mechanism in keratinocytes was increased expression and secretion of endothelin-1, which is also a known pruritogen. Here we address the question of whether TRPV4 in skin keratinocytes functions in itch, as a particular form of "forefront" signaling in non-neural cells. Our results support this novel concept based on attenuated scratching behavior in response to histaminergic (histamine, compound 48/80, endothelin-1), not non-histaminergic (chloroquine) pruritogens in Trpv4 keratinocyte-specific and inducible knock-out mice. We demonstrate that keratinocytes rely on TRPV4 for calcium influx in response to histaminergic pruritogens. TRPV4 activation in keratinocytes evokes phosphorylation of mitogen-activated protein kinase, ERK, for histaminergic pruritogens. This finding is relevant because we observed robust anti-pruritic effects with topical applications of selective inhibitors for TRPV4 and also for MEK, the kinase upstream of ERK, suggesting that calcium influx via TRPV4 in keratinocytes leads to ERK-phosphorylation, which in turn rapidly converts the keratinocyte into an organismal itch-generator cell. In support of this concept we found that scratching behavior, evoked by direct intradermal activation of TRPV4, was critically dependent on TRPV4 expression in keratinocytes. Thus, TRPV4 functions as a pruriceptor-TRP in skin keratinocytes in histaminergic itch, a novel basic concept with translational-medical relevance.

Floating s- and p-type Gaussian Orbitals

The advantages of including a small number of p-type gaussian functions in a floating spherical gaussian orbital calculation are pointed out and illustrated by calculations on molecules which previously have proved to be troublesome. These include molecules such as F2 with multiple lone pairs and C2H2 with multiple bonds. A feature of the results is the excellent correlation between the orbital energies and those of a double zeta calculation reported by Snyder and Basch.

The Expanded Human Kallikrein (KLK) gene family : genomic organisation, tissue specific expression and potential functions

The tissue kallikreins are serine proteases encoded by highly conserved multigene families. The rodent kallikrein (KLK) families are particularly large, consisting of 13 26 genes clustered in one chromosomal locus. It has been recently recognised that the human KLK gene family is of a similar size (15 genes) with the identification of another 12 related genes (KLK4-KLK15) within and adjacent to the original human KLK locus (KLK1-3) on chromosome 19q13.4. The structural organisation and size of these new genes is similar to that of other KLK genes except for additional exons encoding 5 or 3 untranslated regions. Moreover, many of these genes have multiple mRNA transcripts, a trait not observed with rodent genes. Unlike all other kallikreins, the KLK4-KLK15 encoded proteases are less related (25–44%) and do not contain a conventional kallikrein loop. Clusters of genes exhibit high prostatic (KLK2-4, KLK15) or pancreatic (KLK6-13) expression, suggesting evolutionary conservation of elements conferring tissue specificity. These genes are also expressed, to varying degrees, in a wider range of tissues suggesting a functional involvement of these newer human kallikrein proteases in a diverse range of physiological processes.