Regression with input-dependent noise: A Gaussian process treatment

Gaussian processes provide natural non-parametric prior distributions over regression functions. In this paper we consider regression problems where there is noise on the output, and the variance of the noise depends on the inputs. If we assume that the noise is a smooth function of the inputs, then it is natural to model the noise variance using a second Gaussian process, in addition to the Gaussian process governing the noise-free output value. We show that prior uncertainty about the parameters controlling both processes can be handled and that the posterior distribution of the noise rate can be sampled from using Markov chain Monte Carlo methods. Our results on a synthetic data set give a posterior noise variance that well-approximates the true variance.

Regression with input-dependent noise: A Bayesian treatment

In most treatments of the regression problem it is assumed that the distribution of target data can be described by a deterministic function of the inputs, together with additive Gaussian noise having constant variance. The use of maximum likelihood to train such models then corresponds to the minimization of a sum-of-squares error function. In many applications a more realistic model would allow the noise variance itself to depend on the input variables. However, the use of maximum likelihood to train such models would give highly biased results. In this paper we show how a Bayesian treatment can allow for an input-dependent variance while overcoming the bias of maximum likelihood.

Internal versus external R&D:a study of R&D choice with sample selection

This paper extends previous analyses of the choice between internal and external R&D to consider the costs of internal R&D. The Heckman two-stage estimator is used to estimate the determinants of internal R&D unit cost (i.e. cost per product innovation) allowing for sample selection effects. Theory indicates that R&D unit cost will be influenced by scale issues and by the technological opportunities faced by the firm. Transaction costs encountered in research activities are allowed for and, in addition, consideration is given to issues of market structure which influence the choice of R&D mode without affecting the unit cost of internal or external R&D. The model is tested on data from a sample of over 500 UK manufacturing plants which have engaged in product innovation. The key determinants of R&D mode are the scale of plant and R&D input, and market structure conditions. In terms of the R&D cost equation, scale factors are again important and have a non-linear relationship with R&D unit cost. Specificities in physical and human capital also affect unit cost, but have no clear impact on the choice of R&D mode. There is no evidence of technological opportunity affecting either R&D cost or the internal/external decision.

State-dependent microstructural white matter changes in bipolar I depression

Abnormalities in fronto-limbic-striatal white matter (WM) have been reported in bipolar disorder (BD), but results have been inconsistent across studies. Furthermore, there have been no detailed investigations as to whether acute mood states contribute to microstructural changes in WM tracts. In order to compare fiber density and structural integrity within WM tracts between BD depression and remission, whole-brain fractional anisotropy (FA) and mean diffusivity (MD) were assessed in 37 bipolar I disorder (BD-I) patients (16 depressed and 21 remitted), and 26 healthy individuals with diffusion tensor imaging. Significantly decreased FA and increased MD in bilateral prefronto-limbic-striatal white matter and right inferior fronto-occipital, superior and inferior longitudinal fasciculi were shown in all BD-I patients versus controls, as well as in depressed BD-I patients compared to both controls and remitted BD-I patients. Depressed BD-I patients also exhibited increased FA in the ventromedial prefrontal cortex. Remitted BD-I patients did not differ from controls in FA or MD. These findings suggest that BD-I depression may be associated with acute microstructural WM changes.

Optimal design for correlated processes with input-dependent noise

Optimal design for parameter estimation in Gaussian process regression models with input-dependent noise is examined. The motivation stems from the area of computer experiments, where computationally demanding simulators are approximated using Gaussian process emulators to act as statistical surrogates. In the case of stochastic simulators, which produce a random output for a given set of model inputs, repeated evaluations are useful, supporting the use of replicate observations in the experimental design. The findings are also applicable to the wider context of experimental design for Gaussian process regression and kriging. Designs are proposed with the aim of minimising the variance of the Gaussian process parameter estimates. A heteroscedastic Gaussian process model is presented which allows for an experimental design technique based on an extension of Fisher information to heteroscedastic models. It is empirically shown that the error of the approximation of the parameter variance by the inverse of the Fisher information is reduced as the number of replicated points is increased. Through a series of simulation experiments on both synthetic data and a systems biology stochastic simulator, optimal designs with replicate observations are shown to outperform space-filling designs both with and without replicate observations. Guidance is provided on best practice for optimal experimental design for stochastic response models. © 2013 Elsevier Inc. All rights reserved.

Fully probabilistic control for stochastic nonlinear control systems with input dependent noise

Robust controllers for nonlinear stochastic systems with functional uncertainties can be consistently designed using probabilistic control methods. In this paper a generalised probabilistic controller design for the minimisation of the Kullback-Leibler divergence between the actual joint probability density function (pdf) of the closed loop control system, and an ideal joint pdf is presented emphasising how the uncertainty can be systematically incorporated in the absence of reliable systems models. To achieve this objective all probabilistic models of the system are estimated from process data using mixture density networks (MDNs) where all the parameters of the estimated pdfs are taken to be state and control input dependent. Based on this dependency of the density parameters on the input values, explicit formulations to the construction of optimal generalised probabilistic controllers are obtained through the techniques of dynamic programming and adaptive critic methods. Using the proposed generalised probabilistic controller, the conditional joint pdfs can be made to follow the ideal ones. A simulation example is used to demonstrate the implementation of the algorithm and encouraging results are obtained.

Enhanced information transmission with signal-dependent noise in an array of nonlinear elements

We have investigated information transmission in an array of threshold units that have signal-dependent noise and a common input signal. We demonstrate a phenomenon similar to stochastic resonance and suprathreshold stochastic resonance with additive noise and show that information transmission can be enhanced by a nonzero level of noise. By comparing system performance to one with additive noise we also demonstrate that the information transmission of weak signals is significantly better with signal-dependent noise. Indeed, information rates are not compromised even for arbitrary small input signals. Furthermore, by an appropriate selection of parameters, we observe that the information can be made to be (almost) independent of the level of the noise, thus providing a robust method of transmitting information in the presence of noise. These result could imply that the ability of hair cells to code and transmit sensory information in biological sensory systems is not limited by the level of signal-dependent noise. © 2007 The American Physical Society.

Regression with Gaussian processes

The Bayesian analysis of neural networks is difficult because the prior over functions has a complex form, leading to implementations that either make approximations or use Monte Carlo integration techniques. In this paper I investigate the use of Gaussian process priors over functions, which permit the predictive Bayesian analysis to be carried out exactly using matrix operations. The method has been tested on two challenging problems and has produced excellent results.

Neural network regression with input uncertainty

It is generally assumed when using Bayesian inference methods for neural networks that the input data contains no noise or corruption. For real-world (errors in variable) problems this is clearly an unsafe assumption. This paper presents a Bayesian neural network framework which allows for input noise given that some model of the noise process exists. In the limit where this noise process is small and symmetric it is shown, using the Laplace approximation, that there is an additional term to the usual Bayesian error bar which depends on the variance of the input noise process. Further, by treating the true (noiseless) input as a hidden variable and sampling this jointly with the network's weights, using Markov Chain Monte Carlo methods, it is demonstrated that it is possible to infer the unbiassed regression over the noiseless input.

General bounds on Bayes errors for regression with Gaussian processes

Based on a simple convexity lemma, we develop bounds for different types of Bayesian prediction errors for regression with Gaussian processes. The basic bounds are formulated for a fixed training set. Simpler expressions are obtained for sampling from an input distribution which equals the weight function of the covariance kernel, yielding asymptotically tight results. The results are compared with numerical experiments.

Enhanced information transmission with signal dependent noise in an array of LIF neurons

Sensory cells usually transmit information to afferent neurons via chemical synapses, in which the level of noise is dependent on an applied stimulus. Taking into account such dependence, we model a sensory system as an array of LIF neurons with a common signal. We show that information transmission is enhanced by a nonzero level of noise. Moreover, we demonstrate a phenomenon similar to suprathreshold stochastic resonance with additive noise. We remark that many properties of information transmission found for the LIF neurons was predicted by us before with simple binary units [Phys. Rev. E 75, 021121 (2007)]. This confirmation of our predictions allows us to point out identical roots of the phenomena found in the simple threshold systems and more complex LIF neurons.

A probabilistic indirect adaptive control for systems with input-dependent noise

A probabilistic indirect adaptive controller is proposed for the general nonlinear multivariate class of discrete time system. The proposed probabilistic framework incorporates input–dependent noise prediction parameters in the derivation of the optimal control law. Moreover, because noise can be nonstationary in practice, the proposed adaptive control algorithm provides an elegant method for estimating and tracking the noise. For illustration purposes, the developed method is applied to the affine class of nonlinear multivariate discrete time systems and the desired result is obtained: the optimal control law is determined by solving a cubic equation and the distribution of the tracking error is shown to be Gaussian with zero mean. The efficiency of the proposed scheme is demonstrated numerically through the simulation of an affine nonlinear system.

Patient anxiety and experiences associated with an outpatient "one-stop" "see and treat" hysteroscopy clinic

BACKGROUND: "One-stop" outpatient hysteroscopy clinics have become well established for the investigation and treatment of women with abnormal uterine bleeding. However, the advantages of these clinics may be offset by patient factors such as anxiety, pain, and dissatisfaction. This study aimed to establish patients' views and experiences of outpatient service delivery in the context of a one-stop diagnostic and therapeutic hysteroscopy clinic, to determine the amount of anxiety experienced by these women and compare this with other settings, and to determine any predictors for patient preferences. METHODS: The 20-item State-Trait Anxiety Inventory was given to 240 women attending a one-stop hysteroscopy clinic: to 73 consecutive women before their appointment in a general gynecology clinic and to 36 consecutive women attending a chronic pelvic pain clinic. The results were compared with published data for the normal female population, for women awaiting major surgery, and for women awaiting a colposcopy clinic appointment. In addition, a questionnaire designed to ascertain patients' views and experiences was used. Logistic regression analysis was used to delineate the predictive values of diagnostic or therapeutic hysteroscopy, and to determine their effect on the preference of patients to have the procedure performed under general anesthesia in the future. RESULTS: Women attending the hysteroscopy clinic in this study reported significantly higher levels of anxiety than those attending the general gynecology clinic (median, 45 vs 39; p = 0.004), but the levels of anxiety were comparable with those of women attending the chronic pelvic pain clinic (median, 45 vs 46; p = 0.8). As compared with the data from the normal female population (mean, 35.7) and those reported for women awaiting major surgery (mean, 41.2), the levels of anxiety experienced before outpatient hysteroscopy clinic treatment were found to be higher (mean, 45.7). Only women awaiting colposcopy (6-item mean score, 51.1 +/- 13.3) experienced significantly higher anxiety scores than the women awaiting outpatient hysteroscopy (6-item mean score, 47.3 +/- 13.9; p = 0.002). Despite their anxiety, most women are satisfied with the outpatient hysteroscopy "see and treat" service. High levels of anxiety, particularly concerning pain but not operative intervention, were significant predictors of patients desiring a future procedure to be performed under general anesthesia. CONCLUSIONS: Outpatient hysteroscopy is associated with significant anxiety, which increases the likelihood of intolerance for the outpatient procedure. However, among those undergoing operative therapeutic procedures, dissatisfaction was not associated with the outpatient setting.

Epidemiology of myopia in a United Kingdom urban child population

The Aston Eye Study (AES) was instigated in October 2005 to determine the distribution of refractive error and associated ocular biometry in a sample of UK urban school children. The AES is the first study to compare outcome measures separately in White, South Asian and Black children. Children were selected from two age groups (Year 2 children aged 6/7 years, Year8 children aged 12/13 years of age) using random cluster sampling of schools in Birmingham, West Midlands UK. To date, the AES has examined 598 children (302 Year 2,296 Year 8). Using open-field cycloplegic autorefraction, the overall prevalence of myopia (=-0.50D SER in either eye) determined was 19.6%, with a higher prevalence in older (29.4%) compared to younger (9.9%) children (p<0.001). Using multiple logistic regression models, the risk of myopia was higher in Year 8 South Asian compared to White children and higher in children attending grammar schools relative to comprehensive schools. In addition, the prevalence of uncorrected ametropia was found to be high (Year 8: 12.84%, Year 2: 15.23%), which will be of concern to bodies responsible for the implementation of school vision screening strategies. Biometric data using non-contact partial coherence interferometry revealed a contributory effect of axial length (AL) and central corneal radius (CR) on myopic refraction, resulting in a strong coefficient of determination of the AL/CR ratio on refractive error. Ocular biometric measures did not vary significantly as a function of ethnicity, suggesting a greater miscorrelation of components in susceptible ethnic groups to account for their higher myopia prevalence. Corneal radius was found to be steeper in myopes in both age groups, but was found to flatten with increasing axial length. Due to the inextricable link between myopia and axial elongation, the paradoxical finding of the cornea demands further longitudinal investigation, particularly in relation to myopia onset. Questionnaire analysis revealed a history of myopia in parents and siblings to be significantly associated with myopia in Year 8 children, with a dose-dependent rise in the odds ratio of myopia evident with increasing number of myopic parents. By classifying socioeconomic status (SES) using Index of Multiple Deprivation values, it was found that Year 8 children from moderately deprived backgrounds were more at risk of myopia compared with children located at both extremities of the deprivation spectrum. However, the main effect of SES weakened following multivariate analysis, with South Asian ethnicity and grammar schooling remaining associated with Year 8 myopia after adjustment.

Provision of primary care to patients with chronic cough in the community pharmacy setting

BACKGROUND: Community pharmacies are at the forefront of primary care providers and have an important role in the referral of patients to a medical practitioner for review when necessary. Chronic cough is a common disorder in the community and requires medical assessment. The proficiency of community pharmacy staff to refer patients with chronic cough is currently unknown. OBJECTIVE: To assess the ability of community pharmacy staff to recognize and medically refer patients with a chronic nonproductive cough. METHODS: Following ethics approval, a simulated patient study of 156 community pharmacies in Perth, Western Australia, was conducted over a 3-month period. Simulated patients presented to the pharmacy requesting treatment for a cough. The simulated patient required a referral based on a designated scenario. Demographic details, assessment questions, and advice provided were recorded by the simulated patient immediately postvisit. A logistic regression analysis was performed, with referral for medical assessment as the dependent variable. RESULTS: Of the 155 community pharmacies included in the analysis, 38% provided appropriate medical referral. Cough suppressants were provided as therapy in 72% of all visits. Predictors of medical referral were assessment of symptom duration, medical history, current medications being taken, frequency of reliever use, and the position of the pharmacy staff member conducting the consultation. A third of community pharmacies provided appropriate primary care by recommending medical referral advice to patients with chronic cough. The majority of pharmacy staff members acquired information from the patient that suggested a need for medical referral, yet did not provide referral advice. CONCLUSIONS: Appropriate medical referral is more likely when adequate assessment is undertaken and when a pharmacist is directly involved in the consultation. This highlights the need for pharmacies to ensure that processes are in place for patients to access the pharmacist.