A novel neural network technique for exploratory data analysis

This paper surveys the context of feature extraction by neural network approaches, and compares and contrasts their behaviour as prospective data visualisation tools in a real world problem. We also introduce and discuss a hybrid approach which allows us to control the degree of discriminatory and topographic information in the extracted feature space.

Point-wise confidence interval estimation by neural networks: A comparative study based on automotive engine calibration.

In developing neural network techniques for real world applications it is still very rare to see estimates of confidence placed on the neural network predictions. This is a major deficiency, especially in safety-critical systems. In this paper we explore three distinct methods of producing point-wise confidence intervals using neural networks. We compare and contrast Bayesian, Gaussian Process and Predictive error bars evaluated on real data. The problem domain is concerned with the calibration of a real automotive engine management system for both air-fuel ratio determination and on-line ignition timing. This problem requires real-time control and is a good candidate for exploring the use of confidence predictions due to its safety-critical nature.

A novel approach to modelling and exploiting uncertainty in stochastic control systems

We consider an inversion-based neurocontroller for solving control problems of uncertain nonlinear systems. Classical approaches do not use uncertainty information in the neural network models. In this paper we show how we can exploit knowledge of this uncertainty to our advantage by developing a novel robust inverse control method. Simulations on a nonlinear uncertain second order system illustrate the approach.

Improved robust control of nonlinear stochastic systems using uncertain models

We introduce a technique for quantifying and then exploiting uncertainty in nonlinear stochastic control systems. The approach is suboptimal though robust and relies upon the approximation of the forward and inverse plant models by neural networks, which also estimate the intrinsic uncertainty. Sampling from the resulting Gaussian distributions of the inversion based neurocontroller allows us to introduce a control law which is demonstrably more robust than traditional adaptive controllers.

Robust control of nonlinear stochastic systems by modelling conditional distributions of control signals

We introduce a novel inversion-based neuro-controller for solving control problems involving uncertain nonlinear systems that could also compensate for multi-valued systems. The approach uses recent developments in neural networks, especially in the context of modelling statistical distributions, which are applied to forward and inverse plant models. Provided that certain conditions are met, an estimate of the intrinsic uncertainty for the outputs of neural networks can be obtained using the statistical properties of networks. More generally, multicomponent distributions can be modelled by the mixture density network. In this work a novel robust inverse control approach is obtained based on importance sampling from these distributions. This importance sampling provides a structured and principled approach to constrain the complexity of the search space for the ideal control law. The performance of the new algorithm is illustrated through simulations with example systems.

Conditional distribution modeling for estimating and exploiting uncertainty in control systems

This paper presents a general methodology for estimating and incorporating uncertainty in the controller and forward models for noisy nonlinear control problems. Conditional distribution modeling in a neural network context is used to estimate uncertainty around the prediction of neural network outputs. The developed methodology circumvents the dynamic programming problem by using the predicted neural network uncertainty to localize the possible control solutions to consider. A nonlinear multivariable system with different delays between the input-output pairs is used to demonstrate the successful application of the developed control algorithm. The proposed method is suitable for redundant control systems and allows us to model strongly non Gaussian distributions of control signal as well as processes with hysteresis.

Probability distribution modelling to improve stability in nonlinear MIMO control

We consider the direct adaptive inverse control of nonlinear multivariable systems with different delays between every input-output pair. In direct adaptive inverse control, the inverse mapping is learned from examples of input-output pairs. This makes the obtained controller sub optimal, since the network may have to learn the response of the plant over a larger operational range than necessary. Moreover, in certain applications, the control problem can be redundant, implying that the inverse problem is ill posed. In this paper we propose a new algorithm which allows estimating and exploiting uncertainty in nonlinear multivariable control systems. This approach allows us to model strongly non-Gaussian distribution of control signals as well as processes with hysteresis. The proposed algorithm circumvents the dynamic programming problem by using the predicted neural network uncertainty to localise the possible control solutions to consider.

Evolutionary neural networks : models and applications

The scaling problems which afflict attempts to optimise neural networks (NNs) with genetic algorithms (GAs) are disclosed. A novel GA-NN hybrid is introduced, based on the bumptree, a little-used connectionist model. As well as being computationally efficient, the bumptree is shown to be more amenable to genetic coding lthan other NN models. A hierarchical genetic coding scheme is developed for the bumptree and shown to have low redundancy, as well as being complete and closed with respect to the search space. When applied to optimising bumptree architectures for classification problems the GA discovers bumptrees which significantly out-perform those constructed using a standard algorithm. The fields of artificial life, control and robotics are identified as likely application areas for the evolutionary optimisation of NNs. An artificial life case-study is presented and discussed. Experiments are reported which show that the GA-bumptree is able to learn simulated pole balancing and car parking tasks using only limited environmental feedback. A simple modification of the fitness function allows the GA-bumptree to learn mappings which are multi-modal, such as robot arm inverse kinematics. The dynamics of the 'geographic speciation' selection model used by the GA-bumptree are investigated empirically and the convergence profile is introduced as an analytical tool. The relationships between the rate of genetic convergence and the phenomena of speciation, genetic drift and punctuated equilibrium arc discussed. The importance of genetic linkage to GA design is discussed and two new recombination operators arc introduced. The first, linkage mapped crossover (LMX) is shown to be a generalisation of existing crossover operators. LMX provides a new framework for incorporating prior knowledge into GAs.Its adaptive form, ALMX, is shown to be able to infer linkage relationships automatically during genetic search.

Control of the endocrine pancreas by gastrointestinal peptides

This thesis concerns the mechanism through which enteral delivery of glucose results in a larger insulin response than an equivalent parenteral glucose load. Preliminary studies in which mice received a glucose solution either intragastrically or intraperitoneally confirmed this phenomenon. An important regulatory system in this respect is the entero-insular axis, through which insulin secretion is influenced by neural and endocrine communication between the gastrointestinal tract and the pancreatic islets of Langerhans. Using an in vitro system involving static incubation of isolated (by collagenase digestion) islets of Langerhans, the effect of a variety of gastrointestinal peptides on the secretion of the four main islet hormones, namely insulin, glucagon, somatostatin and pancreatic polypeptide, was studied. The gastrointestinal peptides investigated in this study were the secretin family, comprising secretin, glucagon, gastric inhibitory polypeptide (GIP), vasoactive intestinal polypeptide (VIP), peptide histidine isoleucine (PHI) and growth hormone releasing factor (GRF). Gastrin releasing peptide (GRP) was also studied. The results showed that insulin release was stimulated by all peptides studied except PHI, glucagon release was stimulated by all peptides tested, except GRF which suppressed glucagon release, somatostatin release was stimulated by GIP and GRF but suppressed by VIP, PHI, glucagon and secretin, and PP release was stimulated by GIP and GRF, but suppressed by PHI. The insulinotropic effect of GRP was investigated further. A perifusion system was used to examine the time-course of insulin release from isolated islets after stimulation with GRP. GRP was shown to be insulinotropic only in the presence of physiologically elevated glucose concentrations and both first and second phases of insulin release were augmented. There was no effect at substimulatory or very high glucose concentrations. Studies using a cultured insulin-secreting islet cell line, the RINm5F cell line, were undertaken to elucidate the intracellular mechanism of action of GRP. This peptide did not enhance insulin release via an augmentation of glucose metabolism, or via the adenylate cyclase/cyclic AMP secondary messenger system. The pattern of changes of cytosolic free calcium in response to GRP, which involved both mobilization of intracellular stores and an influx of extracellular calcium, suggested the involvement of phosphatidylinositol bisphosphate breakdown as a mediator of the effect of GRP on insulin secretion.

Exploiting uncertainty in nonlinear stochastic control problem

This work introduces a novel inversion-based neurocontroller for solving control problems involving uncertain nonlinear systems which could also compensate for multi-valued systems. The approach uses recent developments in neural networks, especially in the context of modelling statistical distributions, which are applied to forward and inverse plant models. Provided that certain conditions are met, an estimate of the intrinsic uncertainty for the outputs of neural networks can be obtained using the statistical properties of networks. More generally, multicomponent distributions can be modelled by the mixture density network. Based on importance sampling from these distributions a novel robust inverse control approach is obtained. This importance sampling provides a structured and principled approach to constrain the complexity of the search space for the ideal control law. The developed methodology circumvents the dynamic programming problem by using the predicted neural network uncertainty to localise the possible control solutions to consider. Convergence of the output error for the proposed control method is verified by using a Lyapunov function. Several simulation examples are provided to demonstrate the efficiency of the developed control method. The manner in which such a method is extended to nonlinear multi-variable systems with different delays between the input-output pairs is considered and demonstrated through simulation examples.

Development of reasoning:Behavioral evidence to support reinforcement over cognitive control accounts

Speed's theory makes two predictions for the development of analogical reasoning. Firstly, young children should not be able to reason analogically due to an undeveloped PFC neural network. Secondly, category knowledge enables the reinforcement of structural features over surface features, and thus the development of sophisticated, analogical, reasoning. We outline existing studies that support these predictions and highlight some critical remaining issues. Specifically, we argue that the development of inhibition must be directly compared alongside the development of reasoning strategies in order to support Speed's account. © 2010 Psychology Press.

Differential patterns of activity and functional connectivity in emotion processing neural circuitry to angry and happy faces in adolescents with and without suicide attempt

Background - Neural substrates of emotion dysregulation in adolescent suicide attempters remain unexamined. Method - We used functional magnetic resonance imaging to measure neural activity to neutral, mild or intense (i.e. 0%, 50% or 100% intensity) emotion face morphs in two separate emotion-processing runs (angry and happy) in three adolescent groups: (1) history of suicide attempt and depression (ATT, n = 14); (2) history of depression alone (NAT, n = 15); and (3) healthy controls (HC, n = 15). Post-hoc analyses were conducted on interactions from 3 group × 3 condition (intensities) whole-brain analyses (p < 0.05, corrected) for each emotion run. Results - To 50% intensity angry faces, ATT showed significantly greater activity than NAT in anterior cingulate gyral–dorsolateral prefrontal cortical attentional control circuitry, primary sensory and temporal cortices; and significantly greater activity than HC in the primary sensory cortex, while NAT had significantly lower activity than HC in the anterior cingulate gyrus and ventromedial prefrontal cortex. To neutral faces during the angry emotion-processing run, ATT had significantly lower activity than NAT in the fusiform gyrus. ATT also showed significantly lower activity than HC to 100% intensity happy faces in the primary sensory cortex, and to neutral faces in the happy run in the anterior cingulate and left medial frontal gyri (all p < 0.006,corrected). Psychophysiological interaction analyses revealed significantly reduced anterior cingulate gyral–insula functional connectivity to 50% intensity angry faces in ATT v. NAT or HC. Conclusions - Elevated activity in attention control circuitry, and reduced anterior cingulate gyral–insula functional connectivity, to 50% intensity angry faces in ATT than other groups suggest that ATT may show inefficient recruitment of attentional control neural circuitry when regulating attention to mild intensity angry faces, which may represent a potential biological marker for suicide risk.

Dissociable patterns of neural activity during response inhibition in depressed adolescents with and without suicidal behavior

Objectives - Impaired attentional control and behavioral control are implicated in adult suicidal behavior. Little is known about the functional integrity of neural circuitry supporting these processes in suicidal behavior in adolescence. Method - Functional magnetic resonance imaging was used in 15 adolescent suicide attempters with a history of major depressive disorder (ATTs), 15 adolescents with a history of depressive disorder but no suicide attempt (NATs), and 14 healthy controls (HCs) during the performance of a well-validated go-no-go response inhibition and motor control task that measures attentional and behavioral control and has been shown to activate prefrontal, anterior cingulate, and parietal cortical circuitries. Questionnaires assessed symptoms and standardized interviews characterized suicide attempts. Results - A 3 group by 2 condition (go-no-go response inhibition versus go motor control blocks) block-design whole-brain analysis (p < .05, corrected) showed that NATs showed greater activity than ATTs in the right anterior cingulate gyrus (p = .008), and that NATs, but not ATTs, showed significantly greater activity than HCs in the left insula (p = .004) to go-no-go response inhibition blocks. Conclusions - Although ATTs did not show differential patterns of neural activity from HCs during the go-no-go response inhibition blocks, ATTs and NATs showed differential activation of the right anterior cingulate gyrus during response inhibition. These findings indicate that suicide attempts during adolescence are not associated with abnormal activity in response inhibition neural circuitry. The differential patterns of activity in response inhibition neural circuitry in ATTs and NATs, however, suggest different neural mechanisms for suicide attempt versus major depressive disorder in general in adolescence that should be a focus of further study.

Familial and disease specific abnormalities in the neural correlates of the Stroop Task in Bipolar Disorder

Patients with Bipolar Disorder (BD) perform poorly on tasks of selective attention and inhibitory control. Although similar behavioural deficits have been noted in their relatives, it is yet unclear whether they reflect dysfunction in the same neural circuits. We used functional magnetic resonance imaging and the Stroop Colour Word Task to compare task related neural activity between 39 euthymic BD patients, 39 of their first-degree relatives (25 with no Axis I disorders and 14 with Major Depressive Disorder) and 48 healthy controls. Compared to controls, all individuals with familial predisposition to BD, irrespective of diagnosis, showed similar reductions in neural responsiveness in regions involved in selective attention within the posterior and inferior parietal lobules. In contrast, hypoactivation within fronto-striatal regions, implicated in inhibitory control, was observed only in BD patients and MDD relatives. Although striatal deficits were comparable between BD patients and their MDD relatives, right ventrolateral prefrontal dysfunction was uniquely associated with BD. Our findings suggest that while reduced parietal engagement relates to genetic risk, fronto-striatal dysfunction reflects processes underpinning disease expression for mood disorders. © 2011 Elsevier Inc.