Quantifying uncertainty on Pareto fronts with Gaussian process conditional simulations

Multi-objective optimization algorithms aim at finding Pareto-optimal solutions. Recovering Pareto fronts or Pareto sets from a limited number of function evaluations are challenging problems. A popular approach in the case of expensive-to-evaluate functions is to appeal to metamodels. Kriging has been shown efficient as a base for sequential multi-objective optimization, notably through infill sampling criteria balancing exploitation and exploration such as the Expected Hypervolume Improvement. Here we consider Kriging metamodels not only for selecting new points, but as a tool for estimating the whole Pareto front and quantifying how much uncertainty remains on it at any stage of Kriging-based multi-objective optimization algorithms. Our approach relies on the Gaussian process interpretation of Kriging, and bases upon conditional simulations. Using concepts from random set theory, we propose to adapt the Vorob’ev expectation and deviation to capture the variability of the set of non-dominated points. Numerical experiments illustrate the potential of the proposed workflow, and it is shown on examples how Gaussian process simulations and the estimated Vorob’ev deviation can be used to monitor the ability of Kriging-based multi-objective optimization algorithms to accurately learn the Pareto front.

On degeneracy and invariances of random fields paths with applications in Gaussian process modelling

We study pathwise invariances and degeneracies of random fields with motivating applications in Gaussian process modelling. The key idea is that a number of structural properties one may wish to impose a priori on functions boil down to degeneracy properties under well-chosen linear operators. We first show in a second order set-up that almost sure degeneracy of random field paths under some class of linear operators defined in terms of signed measures can be controlled through the two first moments. A special focus is then put on the Gaussian case, where these results are revisited and extended to further linear operators thanks to state-of-the-art representations. Several degeneracy properties are tackled, including random fields with symmetric paths, centred paths, harmonic paths, or sparse paths. The proposed approach delivers a number of promising results and perspectives in Gaussian process modelling. In a first numerical experiment, it is shown that dedicated kernels can be used to infer an axis of symmetry. Our second numerical experiment deals with conditional simulations of a solution to the heat equation, and it is found that adapted kernels notably enable improved predictions of non-linear functionals of the field such as its maximum.

Estimating and quantifying uncertainties on level sets using the Vorob'ev expectation and deviation with Gaussian process models

Several methods based on Kriging have recently been proposed for calculating a probability of failure involving costly-to-evaluate functions. A closely related problem is to estimate the set of inputs leading to a response exceeding a given threshold. Now, estimating such a level set—and not solely its volume—and quantifying uncertainties on it are not straightforward. Here we use notions from random set theory to obtain an estimate of the level set, together with a quantification of estimation uncertainty. We give explicit formulae in the Gaussian process set-up and provide a consistency result. We then illustrate how space-filling versus adaptive design strategies may sequentially reduce level set estimation uncertainty.

A Nonstationary Space-Time Gaussian Process Model for Partially Converged Simulations

In the context of expensive numerical experiments, a promising solution for alleviating the computational costs consists of using partially converged simulations instead of exact solutions. The gain in computational time is at the price of precision in the response. This work addresses the issue of fitting a Gaussian process model to partially converged simulation data for further use in prediction. The main challenge consists of the adequate approximation of the error due to partial convergence, which is correlated in both design variables and time directions. Here, we propose fitting a Gaussian process in the joint space of design parameters and computational time. The model is constructed by building a nonstationary covariance kernel that reflects accurately the actual structure of the error. Practical solutions are proposed for solving parameter estimation issues associated with the proposed model. The method is applied to a computational fluid dynamics test case and shows significant improvement in prediction compared to a classical kriging model.

Global Optimization with Sparse and Local Gaussian Process Models

We present a novel surrogate model-based global optimization framework allowing a large number of function evaluations. The method, called SpLEGO, is based on a multi-scale expected improvement (EI) framework relying on both sparse and local Gaussian process (GP) models. First, a bi-objective approach relying on a global sparse GP model is used to determine potential next sampling regions. Local GP models are then constructed within each selected region. The method subsequently employs the standard expected improvement criterion to deal with the exploration-exploitation trade-off within selected local models, leading to a decision on where to perform the next function evaluation(s). The potential of our approach is demonstrated using the so-called Sparse Pseudo-input GP as a global model. The algorithm is tested on four benchmark problems, whose number of starting points ranges from 102 to 104. Our results show that SpLEGO is effective and capable of solving problems with large number of starting points, and it even provides significant advantages when compared with state-of-the-art EI algorithms.

On trend estimation under monotone Gaussian subordination with long-memory: application to fossil pollen series

Fossil pollen data from stratigraphic cores are irregularly spaced in time due to non-linear age-depth relations. Moreover, their marginal distributions may vary over time. We address these features in a nonparametric regression model with errors that are monotone transformations of a latent continuous-time Gaussian process Z(T). Although Z(T) is unobserved, due to monotonicity, under suitable regularity conditions, it can be recovered facilitating further computations such as estimation of the long-memory parameter and the Hermite coefficients. The estimation of Z(T) itself involves estimation of the marginal distribution function of the regression errors. These issues are considered in proposing a plug-in algorithm for optimal bandwidth selection and construction of confidence bands for the trend function. Some high-resolution time series of pollen records from Lago di Origlio in Switzerland, which go back ca. 20,000 years are used to illustrate the methods.

Comparative Analysis of Evolving Software Systems Using the Gini Coefficient

Software metrics offer us the promise of distilling useful information from vast amounts of software in order to track development progress, to gain insights into the nature of the software, and to identify potential problems. Unfortunately, however, many software metrics exhibit highly skewed, non-Gaussian distributions. As a consequence, usual ways of interpreting these metrics --- for example, in terms of "average" values --- can be highly misleading. Many metrics, it turns out, are distributed like wealth --- with high concentrations of values in selected locations. We propose to analyze software metrics using the Gini coefficient, a higher-order statistic widely used in economics to study the distribution of wealth. Our approach allows us not only to observe changes in software systems efficiently, but also to assess project risks and monitor the development process itself. We apply the Gini coefficient to numerous metrics over a range of software projects, and we show that many metrics not only display remarkably high Gini values, but that these values are remarkably consistent as a project evolves over time.

Fast parallel kriging-based stepwise uncertainty reduction with application to the identification of an excursion set

Stepwise uncertainty reduction (SUR) strategies aim at constructing a sequence of points for evaluating a function  f in such a way that the residual uncertainty about a quantity of interest progressively decreases to zero. Using such strategies in the framework of Gaussian process modeling has been shown to be efficient for estimating the volume of excursion of f above a fixed threshold. However, SUR strategies remain cumbersome to use in practice because of their high computational complexity, and the fact that they deliver a single point at each iteration. In this article we introduce several multipoint sampling criteria, allowing the selection of batches of points at which f can be evaluated in parallel. Such criteria are of particular interest when f is costly to evaluate and several CPUs are simultaneously available. We also manage to drastically reduce the computational cost of these strategies through the use of closed form formulas. We illustrate their performances in various numerical experiments, including a nuclear safety test case. Basic notions about kriging, auxiliary problems, complexity calculations, R code, and data are available online as supplementary materials.

Differentiating the Multipoint Expected Improvement for Optimal Batch Design

This work deals with parallel optimization of expensive objective functions which are modelled as sample realizations of Gaussian processes. The study is formalized as a Bayesian optimization problem, or continuous multi-armed bandit problem, where a batch of q > 0 arms is pulled in parallel at each iteration. Several algorithms have been developed for choosing batches by trading off exploitation and exploration. As of today, the maximum Expected Improvement (EI) and Upper Confidence Bound (UCB) selection rules appear as the most prominent approaches for batch selection. Here, we build upon recent work on the multipoint Expected Improvement criterion, for which an analytic expansion relying on Tallis’ formula was recently established. The computational burden of this selection rule being still an issue in application, we derive a closed-form expression for the gradient of the multipoint Expected Improvement, which aims at facilitating its maximization using gradient-based ascent algorithms. Substantial computational savings are shown in application. In addition, our algorithms are tested numerically and compared to state-of-the-art UCB-based batchsequential algorithms. Combining starting designs relying on UCB with gradient-based EI local optimization finally appears as a sound option for batch design in distributed Gaussian Process optimization.

Is computer-assisted telephone triage safe? A prospective surveillance study in walk-in patients with non-life-threatening medical conditions

Background Patients often establish initial contact with healthcare institutions by telephone. During this process they are frequently medically triaged. Purpose To investigate the safety of computer-assisted telephone triage for walk-in patients with non-life-threatening medical conditions at an emergency unit of a Swiss university hospital. Methods This prospective surveillance study compared the urgency assessments of three different types of personnel (call centre nurses, hospital physicians, primary care physicians) who were involved in the patients' care process. Based on the urgency recommendations of the hospital and primary care physicians, cases which could potentially have resulted in an avoidable hazardous situation (AHS) were identified. Subsequently, the records of patients with a potential AHS were assessed for risk to health or life by an expert panel. Results 208 patients were enrolled in the study, of whom 153 were assessed by all three types of personnel. Congruence between the three assessments was low. The weighted κ values were 0.115 (95% CI 0.038 to 0.192) (hospital physicians vs call centre), 0.159 (95% CI 0.073 to 0.242) (primary care physicians vs call centre) and 0.377 (95% CI 0.279 to 0.480) (hospital vs primary care physicians). Seven of 153 cases (4.57%; 95% CI 1.85% to 9.20%) were classified as a potentially AHS. A risk to health or life was adjudged in one case (0.65%; 95% CI 0.02% to 3.58%). Conclusion Medical telephone counselling is a demanding task requiring competent specialists with dedicated training in communication supported by suitable computer technology. Provided these conditions are in place, computer-assisted telephone triage can be considered to be a safe method of assessing the potential clinical risks of patients' medical conditions.

Early non-invasive cardiac output monitoring in hemodynamically unstable intensive care patients: A multi-center randomized controlled trial

Introduction Acute hemodynamic instability increases morbidity and mortality. We investigated whether early non-invasive cardiac output monitoring enhances hemodynamic stabilization and improves outcome. Methods A multicenter, randomized controlled trial was conducted in three European university hospital intensive care units in 2006 and 2007. A total of 388 hemodynamically unstable patients identified during their first six hours in the intensive care unit (ICU) were randomized to receive either non-invasive cardiac output monitoring for 24 hrs (minimally invasive cardiac output/MICO group; n = 201) or usual care (control group; n = 187). The main outcome measure was the proportion of patients achieving hemodynamic stability within six hours of starting the study. Results The number of hemodynamic instability criteria at baseline (MICO group mean 2.0 (SD 1.0), control group 1.8 (1.0); P = .06) and severity of illness (SAPS II score; MICO group 48 (18), control group 48 (15); P = .86)) were similar. At 6 hrs, 45 patients (22%) in the MICO group and 52 patients (28%) in the control group were hemodynamically stable (mean difference 5%; 95% confidence interval of the difference -3 to 14%; P = .24). Hemodynamic support with fluids and vasoactive drugs, and pulmonary artery catheter use (MICO group: 19%, control group: 26%; P = .11) were similar in the two groups. The median length of ICU stay was 2.0 (interquartile range 1.2 to 4.6) days in the MICO group and 2.5 (1.1 to 5.0) days in the control group (P = .38). The hospital mortality was 26% in the MICO group and 21% in the control group (P = .34). Conclusions Minimally-invasive cardiac output monitoring added to usual care does not facilitate early hemodynamic stabilization in the ICU, nor does it alter the hemodynamic support or outcome. Our results emphasize the need to evaluate technologies used to measure stroke volume and cardiac output--especially their impact on the process of care--before any large-scale outcome studies are attempted.

Magnetization exchange observed in human skeletal muscle by non-water-suppressed proton magnetic resonance spectroscopy

Many metabolites in the proton magnetic resonance spectrum undergo magnetization exchange with water, such as those in the downfield region (6.0-8.5 ppm) and the upfield peaks of creatine, which can be measured to reveal additional information about the molecular environment. In addition, these resonances are attenuated by conventional water suppression techniques complicating detection and quantification. To characterize these metabolites in human skeletal muscle in vivo at 3 T, metabolite cycled non-water-suppressed spectroscopy was used to conduct a water inversion transfer experiment in both the soleus and tibialis anterior muscles. Resulting median exchange-independent T(1) times for the creatine methylene resonances were 1.26 and 1.15 s, and for the methyl resonances were 1.57 and 1.74 s, for soleus and tibialis anterior muscles, respectively. Magnetization transfer rates from water to the creatine methylene resonances were 0.56 and 0.28 s(-1) , and for the methyl resonances were 0.39 and 0.30 s(-1) , with the soleus exhibiting faster transfer rates for both resonances, allowing speculation about possible influences of either muscle fibre orientation or muscle composition on the magnetization transfer process. These water magnetization transfer rates observed without water suppression are in good agreement with earlier reports that used either postexcitation water suppression in rats, or short CHESS sequences in human brain and skeletal muscle.

Expression of antagonists of WNT and BMP signaling after non-rigid fixation of osteotomies

Delayed fracture healing and non-unions represent rare but severe complications in orthopedic surgery. Further knowledge on the mechanisms of the bone repair process and of the development of a pseudoarthrosis is essential to predict and prevent impaired healing of fractures. The present study aimed at elucidating differences in gene expression during the repair of rigidly and non-rigidly fixed osteotomies. For this purpose, the MouseFix™ and the FlexiPlate™ systems (AO Development Institute, Davos, CH), allowing the creation of well defined osteotomies in mouse femora, were employed. A time course following the healing process of the osteotomy was performed and bones and periimplant tissues were analyzed by high-resolution X-ray, MicroCT and by histology. For the assessment of gene expression, Low Density Arrays (LDA) were done. In animals with rigid fixation, X-ray and MicroCT revealed healing of the osteotomy within 3 weeks. Using the FlexiPlate™ system, the osteotomy was still visible by X-ray after 3 weeks and a stabilizing cartilaginous callus was formed. After 4.5 weeks, the callus was remodeled and the osteotomy was, on a histological level, healed. Gene expression studies revealed levels of transcripts encoding proteins associated with inflammatory processes not to be altered in tissues from bones with rigid and non-rigid fixation, respectively. Levels of transcripts encoding proteins of the extracellular matrix and essential for bone cell functions were not increased in the rigidly fixed group when compared to controls without osteotomy. In the FlexiPlate™ group, levels of transcripts encoding the same set of genes were significantly increased 3 weeks after surgery. Expression of transcripts encoding BMPs and BMP antagonists was increased after 3 weeks in repair tissues from bones fixed with FlexiPlate™, as were inhibitors of the WNT signaling pathways. Little changes only were detected in transcript levels of tissues from rigidly fixed bones. The data of the present study suggest that rigid fixation enables accelerated healing of an experimental osteotomy as compared to non-rigid fixation. The changes in the healing process after non-rigid fixation are accompanied by an increase in the levels of transcripts encoding inhibitors of osteogenic pathways and, probably as a consequence, by temporal changes in bone matrix synthesis.