Projected sequential Gaussian processes: flexible interpolation for large data sets

Recently within the machine learning and spatial statistics communities many papers have explored the potential of reduced rank representations of the covariance matrix, often referred to as projected or fixed rank approaches. In such methods the covariance function of the posterior process is represented by a reduced rank approximation which is chosen such that there is minimal information loss. In this paper a sequential framework for inference in such projected processes is presented, where the observations are considered one at a time. We introduce a C++ library for carrying out such projected, sequential estimation which adds several novel features. In particular we have incorporated the ability to use a generic observation operator, or sensor model, to permit data fusion. We can also cope with a range of observation error characteristics, including non-Gaussian observation errors. Inference for the variogram parameters is based on maximum likelihood estimation. We illustrate the projected sequential method in application to synthetic and real data sets. We discuss the software implementation and suggest possible future extensions.

Projected sequential Gaussian processes:a C++ tool for interpolation of large datasets with heterogeneous noise

Heterogeneous datasets arise naturally in most applications due to the use of a variety of sensors and measuring platforms. Such datasets can be heterogeneous in terms of the error characteristics and sensor models. Treating such data is most naturally accomplished using a Bayesian or model-based geostatistical approach; however, such methods generally scale rather badly with the size of dataset, and require computationally expensive Monte Carlo based inference. Recently within the machine learning and spatial statistics communities many papers have explored the potential of reduced rank representations of the covariance matrix, often referred to as projected or fixed rank approaches. In such methods the covariance function of the posterior process is represented by a reduced rank approximation which is chosen such that there is minimal information loss. In this paper a sequential Bayesian framework for inference in such projected processes is presented. The observations are considered one at a time which avoids the need for high dimensional integrals typically required in a Bayesian approach. A C++ library, gptk, which is part of the INTAMAP web service, is introduced which implements projected, sequential estimation and adds several novel features. In particular the library includes the ability to use a generic observation operator, or sensor model, to permit data fusion. It is also possible to cope with a range of observation error characteristics, including non-Gaussian observation errors. Inference for the covariance parameters is explored, including the impact of the projected process approximation on likelihood profiles. We illustrate the projected sequential method in application to synthetic and real datasets. Limitations and extensions are discussed. © 2010 Elsevier Ltd.

Efficient estimation using the characteristic function : theory and applications with high frequency data

The attached file is created with Scientific Workplace Latex

Bayes quadratic unbiased estimator of spatial covariance parameters

This work presents Bayes invariant quadratic unbiased estimator, for short BAIQUE. Bayesian approach is used here to estimate the covariance functions of the regionalized variables which appear in the spatial covariance structure in mixed linear model. Firstly a brief review of spatial process, variance covariance components structure and Bayesian inference is given, since this project deals with these concepts. Then the linear equations model corresponding to BAIQUE in the general case is formulated. That Bayes estimator of variance components with too many unknown parameters is complicated to be solved analytically. Hence, in order to facilitate the handling with this system, BAIQUE of spatial covariance model with two parameters is considered. Bayesian estimation arises as a solution of a linear equations system which requires the linearity of the covariance functions in the parameters. Here the availability of prior information on the parameters is assumed. This information includes apriori distribution functions which enable to find the first and the second moments matrix. The Bayesian estimation suggested here depends only on the second moment of the prior distribution. The estimation appears as a quadratic form y'Ay , where y is the vector of filtered data observations. This quadratic estimator is used to estimate the linear function of unknown variance components. The matrix A of BAIQUE plays an important role. If such a symmetrical matrix exists, then Bayes risk becomes minimal and the unbiasedness conditions are fulfilled. Therefore, the symmetry of this matrix is elaborated in this work. Through dealing with the infinite series of matrices, a representation of the matrix A is obtained which shows the symmetry of A. In this context, the largest singular value of the decomposed matrix of the infinite series is considered to deal with the convergence condition and also it is connected with Gerschgorin Discs and Poincare theorem. Then the BAIQUE model for some experimental designs is computed and compared. The comparison deals with different aspects, such as the influence of the position of the design points in a fixed interval. The designs that are considered are those with their points distributed in the interval [0, 1]. These experimental structures are compared with respect to the Bayes risk and norms of the matrices corresponding to distances, covariance structures and matrices which have to satisfy the convergence condition. Also different types of the regression functions and distance measurements are handled. The influence of scaling on the design points is studied, moreover, the influence of the covariance structure on the best design is investigated and different covariance structures are considered. Finally, BAIQUE is applied for real data. The corresponding outcomes are compared with the results of other methods for the same data. Thereby, the special BAIQUE, which estimates the general variance of the data, achieves a very close result to the classical empirical variance.

A tunable radial basis function model for nonlinear system identification using particle swarm optimisation

A tunable radial basis function (RBF) network model is proposed for nonlinear system identification using particle swarm optimisation (PSO). At each stage of orthogonal forward regression (OFR) model construction, PSO optimises one RBF unit's centre vector and diagonal covariance matrix by minimising the leave-one-out (LOO) mean square error (MSE). This PSO aided OFR automatically determines how many tunable RBF nodes are sufficient for modelling. Compared with the-state-of-the-art local regularisation assisted orthogonal least squares algorithm based on the LOO MSE criterion for constructing fixed-node RBF network models, the PSO tuned RBF model construction produces more parsimonious RBF models with better generalisation performance and is computationally more efficient.

Nonlinear system identification using particle swarm optimisation tuned radial basis function models

A novel particle swarm optimisation (PSO) tuned radial basis function (RBF) network model is proposed for identification of non-linear systems. At each stage of orthogonal forward regression (OFR) model construction process, PSO is adopted to tune one RBF unit's centre vector and diagonal covariance matrix by minimising the leave-one-out (LOO) mean square error (MSE). This PSO aided OFR automatically determines how many tunable RBF nodes are sufficient for modelling. Compared with the-state-of-the-art local regularisation assisted orthogonal least squares algorithm based on the LOO MSE criterion for constructing fixed-node RBF network models, the PSO tuned RBF model construction produces more parsimonious RBF models with better generalisation performance and is often more efficient in model construction. The effectiveness of the proposed PSO aided OFR algorithm for constructing tunable node RBF models is demonstrated using three real data sets.

Construction of tunable radial basis function networks using orthogonal forward selection

An orthogonal forward selection (OFS) algorithm based on leave-one-out (LOO) criteria is proposed for the construction of radial basis function (RBF) networks with tunable nodes. Each stage of the construction process determines an RBF node, namely, its center vector and diagonal covariance matrix, by minimizing the LOO statistics. For regression application, the LOO criterion is chosen to be the LOO mean-square error, while the LOO misclassification rate is adopted in two-class classification application. This OFS-LOO algorithm is computationally efficient, and it is capable of constructing parsimonious RBF networks that generalize well. Moreover, the proposed algorithm is fully automatic, and the user does not need to specify a termination criterion for the construction process. The effectiveness of the proposed RBF network construction procedure is demonstrated using examples taken from both regression and classification applications.

Radial basis function classifier construction using particle swarm optimisation aided orthogonal forward regression

We develop a particle swarm optimisation (PSO) aided orthogonal forward regression (OFR) approach for constructing radial basis function (RBF) classifiers with tunable nodes. At each stage of the OFR construction process, the centre vector and diagonal covariance matrix of one RBF node is determined efficiently by minimising the leave-one-out (LOO) misclassification rate (MR) using a PSO algorithm. Compared with the state-of-the-art regularisation assisted orthogonal least square algorithm based on the LOO MR for selecting fixednode RBF classifiers, the proposed PSO aided OFR algorithm for constructing tunable-node RBF classifiers offers significant advantages in terms of better generalisation performance and smaller model size as well as imposes lower computational complexity in classifier construction process. Moreover, the proposed algorithm does not have any hyperparameter that requires costly tuning based on cross validation.

Cosmological forecasts from photometric measurements of the angular correlation function

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Estimates of covariance functions for growth from birth to 630 days of age in Nelore cattle

Weight records of Brazilian Nelore cattle, from birth to 630 d of age, recorded every 3 mo, were analyzed using random regression models. Independent variables were Legendre polynomials of age at recording. The model of analysis included contemporary groups as fixed effects and age of dam as a linear and quadratic covariable. Mean trends were modeled through a cubic regression on orthogonal polynomials of age. Up to four sets of random regression coefficients were fitted for animals' direct and maternal, additive genetic, and permanent environmental effects. Changes in measurement error variances with age were modeled through a variance function. Orders of polyno-mial fit from three to six were considered, resulting in up to 77 parameters to be estimated. Models fitting random regressions modeled the pattern of variances in the data adequately, with estimates similar to those from corresponding univariate analysis. Direct heritability estimates decreased after birth and tended to be lowest at ages at which maternal effect estimates tended to be highest. Maternal heritability estimates increased after birth to a peak around 110 to 120 d of age and decreased thereafter. Additive genetic direct correlation estimates between weights at standard ages (birth, weaning, yearling, and final weight) were moderate to high and maternal genetic and environmental correlations were consistently high. © 2001 American Society of Animal Science. All rights reserved.

Estimates of covariance functions for growth of Nelore cattle applying a parametric correlation structure to model within-animal correlations

A total of 20,065 weights recorded on 3016 Nelore animals were used to estimate covariance functions for growth from birth to 630 days of age, assuming a parametric correlation structure to model within-animal correlations. The model of analysis included fixed effects of contemporary groups and age of dam as quadratic covariable. Mean trends were taken into account by a cubic regression on orthogonal polynomials of animal age. Genetic effects of the animal and its dam and maternal permanent environmental effects were modelled by random regressions on Legendre polynomials of age at recording. Changes in direct permanent environmental effect variances were modelled by a polynomial variance function, together with a parametric correlation function to account for correlations between ages. Stationary and nonstationary models were used to model within-animal correlations between different ages. Residual variances were considered homogeneous or heterogeneous, with changes modelled by a step or polynomial function of age at recording. Based on Bayesian information criterion, a model with a cubic variance function combined with a nonstationary correlation function for permanent environmental effects, with 49 parameters to be estimated, fitted best. Modelling within-animal correlations through a parametric correlation structure can describe the variation pattern adequately. Moreover, the number of parameters to be estimated can be decreased substantially compared to a model fitting random regression on Legendre polynomial of age. © 2004 Elsevier B.V. All rights reserved.

Estimates of covariance functions for growth from birth to 630 days of age in Nelore cattle

Weight records of Brazilian Nelore cattle, from birth to 630 d of age, recorded every 3 mo, were analyzed using random regression models. Independent variables were Legendre polynomials of age at recording. The model of analysis included contemporary groups as fixed effects and age of dam as a linear and quadratic covariable. Mean trends were modeled through a cubic regression on orthogonal polynomials of age. Up to four sets of random regression coefficients were fitted for animals' direct and maternal, additive genetic, and permanent environmental effects. Changes in measurement error variances with age were modeled through a variance function. Orders of polynomial fit from three to six were considered, resulting in up to 77 parameters to be estimated. Models fitting random regressions modeled the pattern of variances in the data adequately, with estimates similar to those from corresponding univariate analysis. Direct heritability estimates decreased after birth and tended to be lowest at ages at which maternal effect estimates tended to be highest. Maternal heritability estimates increased after birth to a peak around 110 to 120 d of age and decreased thereafter. Additive genetic direct correlation estimates between weights at standard ages (birth, weaning, yearling, and final weight) were moderate to high and maternal genetic and environmental correlations were consistently high.

Relationship between glomerular filtration rate and the adipokines adiponectin, resistin and leptin in coronary patients with predominantly normal or mildly impaired renal function

BACKGROUND: Due to their molecular weight, it is possible that the adipokines adiponectin, resistin and leptin accumulate when glomerular filtration rate (GFR) is decreased. In reduced renal clearance, altered serum concentrations of these proteins might affect cardiovascular risk. The objective of the study was to investigate the relationship between adipokine concentrations and GFR. METHODS: The association between GFR, as determined by the abbreviated MDRD equation, and the concentrations of the adipokines adiponectin, resistin and leptin was assessed in a cohort of coronary patients (n=538; 363 male, 165 female). After calculation of correlations between GFR and adipokine concentrations, the association was further assessed by analysis of covariance following adjustment for age, gender, BMI, presence of type 2 diabetes, presence of hypertension, history of smoking as well as for serum lipid concentrations. RESULTS: Mean GFR in our study population was 68.74+/-15.27 ml/min/1.73 m(2). 74.3% of the patients had a GFR >60 ml/min/1.73 m(2), 24% of the patients had a GFR between 30 and 60 ml/min/1.73 m(2), and 1.7% of the patients had a GFR <30 ml/min/1.73 m(2). There were significant inverse correlations between adiponectin (r=-0.372; p<0.001), resistin (r=-0.227; p<0.001) and leptin (r=-0.151; p=0.009) concentrations and GFR. After multivariate adjustment, the associations remained significant for adiponectin and resistin. Subgroup analysis in patients with GFR >60 ml/min/1.73 m(2) showed a significant correlation between GFR and adiponectin as well as leptin concentrations. However, after adjustment, these associations no longer were significant. CONCLUSIONS: There is an independent association between GFR and the serum concentrations of adiponectin and resistin. However, this association is not present at GFR >60 ml/min/1.73 m(2). This finding suggests that adipokine concentrations in mildly impaired and normal renal function are influenced by factors other than GFR.

A covariance analysis to optimize the optical follow-up strategies

The Astronomical Institute of the University of Bern (AIUB) is conducting several search campaigns for space debris using optical sensors. The debris objects are discovered during systematic survey observations. In general, the result of a discovery consists in only a short observation arc, or tracklet, which is used to perform a first orbit determination in order to be able to observe t he object again in subsequent follow-up observations. The additional observations are used in the orbit improvement process to obtain accurate orbits to be included in a catalogue. In order to obtain the most accurate orbit within the time available it is necessary to optimize the follow-up observations strategy. In this paper an in‐depth study, using simulations and covariance analysis, is performed to identify the optimal sequence of follow-up observations to obtain the most accurate orbit propagation to be used for the space debris catalogue maintenance. The main factors that determine the accuracy of the results of an orbit determination/improvement process are: tracklet length, number of observations, type of orbit, astrometric error of the measurements, time interval between tracklets, and the relative position of the object along its orbit with respect to the observing station. The main aim of the covariance analysis is to optimize the follow-up strategy as a function of the object-observer geometry, the interval between follow-up observations and the shape of the orbit. This an alysis can be applied to every orbital regime but particular attention was dedicated to geostationary, Molniya, and geostationary transfer orbits. Finally the case with more than two follow-up observations and the influence of a second observing station are also analyzed.

A Thalamic-Fronto-Parietal Structural Covariance Network Emerging in the Course of Recovery from Hand Paresis after Ischemic Stroke.

AIM To describe structural covariance networks of gray matter volume (GMV) change in 28 patients with first-ever stroke to the primary sensorimotor cortices, and to investigate their relationship to hand function recovery and local GMV change. METHODS Tensor-based morphometry maps derived from high-resolution structural images were subject to principal component analyses to identify the networks. We calculated correlations between network expression and local GMV change, sensorimotor hand function and lesion volume. To verify which of the structural covariance networks of GMV change have a significant relationship to hand function, we performed an additional multivariate regression approach. RESULTS Expression of the second network, explaining 9.1% of variance, correlated with GMV increase in the medio-dorsal (md) thalamus and hand motor skill. Patients with positive expression coefficients were distinguished by significantly higher GMV increase of this structure during stroke recovery. Significant nodes of this network were located in md thalamus, dorsolateral prefrontal cortex, and higher order sensorimotor cortices. Parameter of hand function had a unique relationship to the network and depended on an interaction between network expression and lesion volume. Inversely, network expression is limited in patients with large lesion volumes. CONCLUSION Chronic phase of sensorimotor cortical stroke has been characterized by a large scale co-varying structural network in the ipsilesional hemisphere associated specifically with sensorimotor hand skill. Its expression is related to GMV increase of md thalamus, one constituent of the network, and correlated with the cortico-striato-thalamic loop involved in control of motor execution and higher order sensorimotor cortices. A close relation between expression of this network with degree of recovery might indicate reduced compensatory resources in the impaired subgroup.