Priors Stabilizers and Basis Functions: From Regularization to Radial, Tensor and Additive Splines

We had previously shown that regularization principles lead to approximation schemes, as Radial Basis Functions, which are equivalent to networks with one layer of hidden units, called Regularization Networks. In this paper we show that regularization networks encompass a much broader range of approximation schemes, including many of the popular general additive models, Breiman's hinge functions and some forms of Projection Pursuit Regression. In the probabilistic interpretation of regularization, the different classes of basis functions correspond to different classes of prior probabilities on the approximating function spaces, and therefore to different types of smoothness assumptions. In the final part of the paper, we also show a relation between activation functions of the Gaussian and sigmoidal type.

Smoothness of scale functions for spectrally negative Lévy processes

Scale functions play a central role in the fluctuation theory of spectrally negative Lévy processes and often appear in the context of martingale relations. These relations are often require excursion theory rather than Itô calculus. The reason for the latter is that standard Itô calculus is only applicable to functions with a sufficient degree of smoothness and knowledge of the precise degree of smoothness of scale functions is seemingly incomplete. The aim of this article is to offer new results concerning properties of scale functions in relation to the smoothness of the underlying Lévy measure. We place particular emphasis on spectrally negative Lévy processes with a Gaussian component and processes of bounded variation. An additional motivation is the very intimate relation of scale functions to renewal functions of subordinators. The results obtained for scale functions have direct implications offering new results concerning the smoothness of such renewal functions for which there seems to be very little existing literature on this topic.

Fast identification algorithms for Gaussian process model

A class identification algorithms is introduced for Gaussian process(GP)models.The fundamental approach is to propose a new kernel function which leads to a covariance matrix with low rank,a property that is consequently exploited for computational efficiency for both model parameter estimation and model predictions.The objective of either maximizing the marginal likelihood or the Kullback–Leibler (K–L) divergence between the estimated output probability density function(pdf)and the true pdf has been used as respective cost functions.For each cost function,an efficient coordinate descent algorithm is proposed to estimate the kernel parameters using a one dimensional derivative free search, and noise variance using a fast gradient descent algorithm. Numerical examples are included to demonstrate the effectiveness of the new identification approaches.

Gaussian quadrature rules with simple node-weight relations

In this paper, we consider the symmetric Gaussian and L-Gaussian quadrature rules associated with twin periodic recurrence relations with possible variations in the initial coefficient. We show that the weights of the associated Gaussian quadrature rules can be given as rational functions in terms of the corresponding nodes where the numerators and denominators are polynomials of degree at most 4. We also show that the weights of the associated L-Gaussian quadrature rules can be given as rational functions in terms of the corresponding nodes where the numerators and denominators are polynomials of degree at most 5. Special cases of these quadrature rules are given. Finally, an easy to implement procedure for the evaluation of the nodes is described.

Gaussian basis sets to the theoretical study of the electronic structure of perovskite (LaMnO3)

The Generator Coordinate Hartree-Fock (GCHF) method is applied to generate extended (20s14p), (30s19p13d), and (31s23p18d) Gaussian basis sets for the 0, Mn, and La atoms, respectively. The role of the weight functions (WFs) in the assessment of the numerical integration range of the GCHF equations is shown. These basis sets are then contracted to [5s3p] and [11s6p6d] for 0 and Mn atoms, respectively, and [17s11p7d] for La atom by a standard procedure. For quality evaluation of contracted basis sets in molecular calculations, we have accomplished calculations of total and orbital energies in the Hartree-Fock-Roothaan (HFR) method for (MnO1+)-Mn-5 and (LaO1+)-La-1 fragments. The results obtained with the contracted basis sets are compared with values obtained with the extended basis sets. The addition of one d polarization function in the contracted basis set for 0 atom and its utilization with the contracted basis sets for Mn and La atoms leads to the calculations of dipole moment and total atomic charges of perovskite (LaMnO3). The calculations were performed at the HFR level with the crystal [LaMnO3](2) fragment in space group C-2v the values of dipole moment, total energy, and total atomic charges showed that it is reasonable to believe that LaMnO3 presents behaviour of piezoelectric material. (C) 2003 Elsevier B.V. All rights reserved.

The generator coordinate Hartree-Fock method as strategy for building Gaussian basis sets to ab initio study of the process of adsorption of sulfur on platinum (200) surface

The scheme named generator coordinate Hartree-Fock method (GCHF) is used to build (22s14p) and (33s22p16d9f) gaussian basis sets to S ((3)P) and Pt ((3)D) atoms, respectively. Theses basis sets are contracted to [13s10p] and [19s13p9d5f] through of Dunning's segmented contraction scheme and are enriched with d and g polarization functions, [13s10p1d] and [19s13p9d5flg]. Finally, the [19s13p9d5f1g] basis Set to Pt ((3)D) was supplemented with s and d diffuse functions, [20s13p10d5flg], and used in combination with [13s10p1d] to study the effects of adsorption of S ((3)D) atom on a pt ((3)D) atom belonged to infinite Pt (200) surface. Atom-atom overlap population, bond order, and infrared spectrum of [pt(_)S](2 -) were calculated properties and were carried out at Hartree-Fock-Roothaan level. The results indicate that the process of adsorption of S ((3)P) on pt ((3)D) in the infinite Pt (200) surface is mainly caused by a strong contribution of sigma between the 3p(z) orbital of S ((3)P) and the 6s orbital of pt ((3)D). (c) 2004 Elsevier B.V. All rights reserved.

Integration of polyharmonic functions

The results in this paper are motivated by two analogies. First, m-harmonic functions in R(n) are extensions of the univariate algebraic polynomials of odd degree 2m-1. Second, Gauss' and Pizzetti's mean value formulae are natural multivariate analogues of the rectangular and Taylor's quadrature formulae, respectively. This point of view suggests that some theorems concerning quadrature rules could be generalized to results about integration of polyharmonic functions. This is done for the Tchakaloff-Obrechkoff quadrature formula and for the Gaussian quadrature with two nodes.

Gaussian basis sets by generator coordinate Hartree-Fock method to ab initio calculations of electron affinities of enolates

The Generator Coordinate Hartree-Fock (GCHF) method is employed to generate uncontracted 15s and 18s11p gaussian basis sets for the H, C and O atoms, respectively. These basis sets are then contracted to 3s and 4s H atom and 6s5p, for C and O atoms by a standard procedure. For quality evaluation of contracted basis sets in molecular calculations, we have accomplished calculations of total and orbital energies in the Hartree-Fock-Roothaaii (HFR) approach for CH, C(2) and CO molecules. The results obtained with the uncontracted basis sets are compared with values obtained with the standard D95, 6-311G basis sets and with values reported in the literature. The 4s and 6s5p basis sets are enriched with polarization and diffuse functions for atoms of the parent neutral systems and of the enolates anions (cycloheptanone enolate, 2,5-dimethyleyelopentanone enolate, 4-heptanone enolate, and di-isopropyl ketone enolate) from the literature, in order to assess their performance in ab initio molecular calculations, and applied for calculations of electron affinities of the enolates. The calculations were performed at the DFT (BLYP and B3LYP) and HF levels and compared with the corresponding experimental values and with those obtained by using other 6-3 1 + +G((*)) and 6-311 + +G((*)) basis sets from literature. For the enolates studied, the differences between the electron affinities obtained with GCHF basis sets, at the B3LYP level, and the experimental values are -0.001, -0,014, -0.001, and -0.001 eV. (C) 2002 Elsevier B.V. B.V. All rights reserved.

Influence diagnostics in Gaussian spatial linear models

Spatial linear models have been applied in numerous fields such as agriculture, geoscience and environmental sciences, among many others. Spatial dependence structure modelling, using a geostatistical approach, is an indispensable tool to estimate the parameters that define this structure. However, this estimation may be greatly affected by the presence of atypical observations in the sampled data. The purpose of this paper is to use diagnostic techniques to assess the sensitivity of the maximum-likelihood estimators, covariance functions and linear predictor to small perturbations in the data and/or the spatial linear model assumptions. The methodology is illustrated with two real data sets. The results allowed us to conclude that the presence of atypical values in the sample data have a strong influence on thematic maps, changing the spatial dependence structure.

Hydrodynamic correlation functions of a driven granular fluid in steady state

We study a homogeneously driven granular fluid of hard spheres at intermediate volume fractions and focus on time-delayed correlation functions in the stationary state. Inelastic collisions are modeled by incomplete normal restitution, allowing for efficient simulations with an event-driven algorithm. The incoherent scattering function Fincoh(q,t ) is seen to follow time-density superposition with a relaxation time that increases significantly as the volume fraction increases. The statistics of particle displacements is approximately Gaussian. For the coherent scattering function S(q,ω), we compare our results to the predictions of generalized fluctuating hydrodynamics, which takes into account that temperature fluctuations decay either diffusively or with a finite relaxation rate, depending on wave number and inelasticity. For sufficiently small wave number q we observe sound waves in the coherent scattering function S(q,ω) and the longitudinal current correlation function Cl(q,ω). We determine the speed of sound and the transport coefficients and compare them to the results of kinetic theory.

Estimating and quantifying uncertainties on level sets using the Vorob'ev expectation and deviation with Gaussian process models

Several methods based on Kriging have recently been proposed for calculating a probability of failure involving costly-to-evaluate functions. A closely related problem is to estimate the set of inputs leading to a response exceeding a given threshold. Now, estimating such a level set—and not solely its volume—and quantifying uncertainties on it are not straightforward. Here we use notions from random set theory to obtain an estimate of the level set, together with a quantification of estimation uncertainty. We give explicit formulae in the Gaussian process set-up and provide a consistency result. We then illustrate how space-filling versus adaptive design strategies may sequentially reduce level set estimation uncertainty.

A new Bayesian approach to the reconstruction of spectral functions

We present a novel approach for the reconstruction of spectra from Euclidean correlator data that makes close contact to modern Bayesian concepts. It is based upon an axiomatically justified dimensionless prior distribution, which in the case of constant prior function m(ω) only imprints smoothness on the reconstructed spectrum. In addition we are able to analytically integrate out the only relevant overall hyper-parameter α in the prior, removing the necessity for Gaussian approximations found e.g. in the Maximum Entropy Method. Using a quasi-Newton minimizer and high-precision arithmetic, we are then able to find the unique global extremum of P[ρ|D] in the full Nω » Nτ dimensional search space. The method actually yields gradually improving reconstruction results if the quality of the supplied input data increases, without introducing artificial peak structures, often encountered in the MEM. To support these statements we present mock data analyses for the case of zero width delta peaks and more realistic scenarios, based on the perturbative Euclidean Wilson Loop as well as the Wilson Line correlator in Coulomb gauge.

Quantifying uncertainty on Pareto fronts with Gaussian process conditional simulations

Multi-objective optimization algorithms aim at finding Pareto-optimal solutions. Recovering Pareto fronts or Pareto sets from a limited number of function evaluations are challenging problems. A popular approach in the case of expensive-to-evaluate functions is to appeal to metamodels. Kriging has been shown efficient as a base for sequential multi-objective optimization, notably through infill sampling criteria balancing exploitation and exploration such as the Expected Hypervolume Improvement. Here we consider Kriging metamodels not only for selecting new points, but as a tool for estimating the whole Pareto front and quantifying how much uncertainty remains on it at any stage of Kriging-based multi-objective optimization algorithms. Our approach relies on the Gaussian process interpretation of Kriging, and bases upon conditional simulations. Using concepts from random set theory, we propose to adapt the Vorob’ev expectation and deviation to capture the variability of the set of non-dominated points. Numerical experiments illustrate the potential of the proposed workflow, and it is shown on examples how Gaussian process simulations and the estimated Vorob’ev deviation can be used to monitor the ability of Kriging-based multi-objective optimization algorithms to accurately learn the Pareto front.

On degeneracy and invariances of random fields paths with applications in Gaussian process modelling

We study pathwise invariances and degeneracies of random fields with motivating applications in Gaussian process modelling. The key idea is that a number of structural properties one may wish to impose a priori on functions boil down to degeneracy properties under well-chosen linear operators. We first show in a second order set-up that almost sure degeneracy of random field paths under some class of linear operators defined in terms of signed measures can be controlled through the two first moments. A special focus is then put on the Gaussian case, where these results are revisited and extended to further linear operators thanks to state-of-the-art representations. Several degeneracy properties are tackled, including random fields with symmetric paths, centred paths, harmonic paths, or sparse paths. The proposed approach delivers a number of promising results and perspectives in Gaussian process modelling. In a first numerical experiment, it is shown that dedicated kernels can be used to infer an axis of symmetry. Our second numerical experiment deals with conditional simulations of a solution to the heat equation, and it is found that adapted kernels notably enable improved predictions of non-linear functionals of the field such as its maximum.