Context-free and context-sensitive dynamics in recurrent neural networks

Continuous-valued recurrent neural networks can learn mechanisms for processing context-free languages. The dynamics of such networks is usually based on damped oscillation around fixed points in state space and requires that the dynamical components are arranged in certain ways. It is shown that qualitatively similar dynamics with similar constraints hold for a(n)b(n)c(n), a context-sensitive language. The additional difficulty with a(n)b(n)c(n), compared with the context-free language a(n)b(n), consists of 'counting up' and 'counting down' letters simultaneously. The network solution is to oscillate in two principal dimensions, one for counting up and one for counting down. This study focuses on the dynamics employed by the sequential cascaded network, in contrast to the simple recurrent network, and the use of backpropagation through time. Found solutions generalize well beyond training data, however, learning is not reliable. The contribution of this study lies in demonstrating how the dynamics in recurrent neural networks that process context-free languages can also be employed in processing some context-sensitive languages (traditionally thought of as requiring additional computation resources). This continuity of mechanism between language classes contributes to our understanding of neural networks in modelling language learning and processing.

Identifying rate-limiting nodes in large-scale cortical networks for visuospatial processing: an illustration using fMRI

With the advent of functional neuroimaging techniques, in particular functional magnetic resonance imaging (fMRI), we have gained greater insight into the neural correlates of visuospatial function. However, it may not always be easy to identify the cerebral regions most specifically associated with performance on a given task. One approach is to examine the quantitative relationships between regional activation and behavioral performance measures. In the present study, we investigated the functional neuroanatomy of two different visuospatial processing tasks, judgement of line orientation and mental rotation. Twenty-four normal participants were scanned with fMRI using blocked periodic designs for experimental task presentation. Accuracy and reaction time (RT) to each trial of both activation and baseline conditions in each experiment was recorded. Both experiments activated dorsal and ventral visual cortical areas as well as dorsolateral prefrontal cortex. More regionally specific associations with task performance were identified by estimating the association between (sinusoidal) power of functional response and mean RT to the activation condition; a permutation test based on spatial statistics was used for inference. There was significant behavioral-physiological association in right ventral extrastriate cortex for the line orientation task and in bilateral (predominantly right) superior parietal lobule for the mental rotation task. Comparable associations were not found between power of response and RT to the baseline conditions of the tasks. These data suggest that one region in a neurocognitive network may be most strongly associated with behavioral performance and this may be regarded as the computationally least efficient or rate-limiting node of the network.

Visualization of learning in multilayer perceptron networks using principal component analysis

This paper is concerned with the use of scientific visualization methods for the analysis of feedforward neural networks (NNs). Inevitably, the kinds of data associated with the design and implementation of neural networks are of very high dimensionality, presenting a major challenge for visualization. A method is described using the well-known statistical technique of principal component analysis (PCA). This is found to be an effective and useful method of visualizing the learning trajectories of many learning algorithms such as back-propagation and can also be used to provide insight into the learning process and the nature of the error surface.

Prediction of carbon monoxide in fires by conditional moment closure

Carbon monoxide is the chief killer in fires. Dangerous levels of CO can occur when reacting combustion gases are quenched by heat transfer, or by mixing of the fire plume in a cooled under- or overventilated upper layer. In this paper, carbon monoxide predictions for enclosure fires are modeled by the conditional moment closure (CMC) method and are compared with laboratory data. The modeled fire situation is a buoyant, turbulent, diffusion flame burning under a hood. The fire plume entrains fresh air, and the postflame gases are cooled considerably under the hood by conduction and radiation, emulating conditions which occur in enclosure fires and lead to the freezing of CO burnout. Predictions of CO in the cooled layer are presented in the context of a complete computational fluid dynamics solution of velocity, temperature, and major species concentrations. A range of underhood equivalence ratios, from rich to lean, are investigated. The CMC method predicts CO in very good agreement with data. In particular, CMC is able to correctly predict CO concentrations in lean cooled gases, showing its capability in conditions where reaction rates change considerably.

Synthesis and Characterization of Semi-Interpenetrating Networks Based on Poly(dimethylsiloxane) and Poly(vinyl alcohol)

Semi-interpenetrating networks (Semi-IPNs) with different compositions were prepared from poly(dimethylsiloxane) (PDMS), tetraethylorthosilicate (TEOS), and poly (vinyl alcohol) (PVA) by the sol-gel process in this study. The characterization of the PDMS/PVA semi-IPN was carried out using Fourier transform infrared spectroscopy (FTIR), thermogravimetric analysis (TGA), differential scanning calorimetry (DSC), scanning electron microscopy (SEM), and swelling measurements. The presence of PVA domains dispersed in the PDMS network disrupted the network and allowed PDMS to crystallize, as observed by the crystallization and melting peaks in the DSC analyses. Because of the presence of hydrophilic (-OH) and hydrophobic (Si-(CH(3))(2)) domains, there was an appropriate hydrophylic/hydrophobic balance in the semi-IPNs prepared, which led to a maximum equilibrium water content of similar to 14 wt % without a loss in the ability to swell less polar solvents. (C) 2009 Wiley Periodicals, Inc. J Appl Polym Sci 115: 158-166, 2010

Use of the q-Gaussian mutation in evolutionary algorithms

This paper proposes the use of the q-Gaussian mutation with self-adaptation of the shape of the mutation distribution in evolutionary algorithms. The shape of the q-Gaussian mutation distribution is controlled by a real parameter q. In the proposed method, the real parameter q of the q-Gaussian mutation is encoded in the chromosome of individuals and hence is allowed to evolve during the evolutionary process. In order to test the new mutation operator, evolution strategy and evolutionary programming algorithms with self-adapted q-Gaussian mutation generated from anisotropic and isotropic distributions are presented. The theoretical analysis of the q-Gaussian mutation is also provided. In the experimental study, the q-Gaussian mutation is compared to Gaussian and Cauchy mutations in the optimization of a set of test functions. Experimental results show the efficiency of the proposed method of self-adapting the mutation distribution in evolutionary algorithms.

Expectation Propagation with Factorizing Distributions: A Gaussian Approximation and Performance Results for Simple Models

We discuss the expectation propagation (EP) algorithm for approximate Bayesian inference using a factorizing posterior approximation. For neural network models, we use a central limit theorem argument to make EP tractable when the number of parameters is large. For two types of models, we show that EP can achieve optimal generalization performance when data are drawn from a simple distribution.

Modelling of species in hood fires by conditional moment closure

Carbon monoxide, the chief killer in fires, and other species are modelled for a series of enclosure fires. The conditions emulate building fires where CO is formed in the rich, turbulent, nonpremixed flame and is transported frozen to lean mixtures by the ceiling jet which is cooled by radiation and dilution. Conditional moment closure modelling is used and computational domain minimisation criteria are developed which reduce the computational cost of this method. The predictions give good agreement for CO and other species in the lean, quenched-gas stream, holding promise that this method may provide a practical means of modelling real, three-dimensional fire situations. (c) 2005 The Combustion Institute. Published by Elsevier Inc. All rights reserved.

Visuospatial processing and the function of prefrontal-parietal networks in autism spectrum disorders: a functional MRI study

Objective: Individuals with autism spectrum disorders typically have normal visuospatial abilities but impaired executive functioning, particularly in abilities related to working memory and attention. The aim of this study was to elucidate the functioning of frontoparietal networks underlying spatial working memory processes during mental rotation in persons with autism spectrum disorders. Method: Seven adolescent males with normal IQ with an autism spectrum disorder and nine age- and IQ-matched male comparison subjects underwent functional magnetic resonance imaging scans while performing a mental rotation task. Results: The autism spectrum disorders group showed less activation in lateral and medial premotor cortex, dorsolateral prefrontal cortex, anterior cingulate gyrus, and caudate nucleus. Conclusions: The finding of less activation in prefrontal regions but not in parietal regions supports a model of dysfunction of frontostriatal networks in autism spectrum disorders.

Conditional variance reduction by measurements on correlated field modes

We consider the case of two cavity modes of the electromagnetic field, which are coupled via the action of a parametric amplifier. The fields are allowed to leak from the cavity and homodyne measurement is performed on one of the modes. Because of the correlations between the modes, this leads to a reduction of the variance in a quadrature of the other mode, although no measurement is performed on it directly. We discuss how this relates to the Einstein-Podolky-Rosen Gedankenexperiment.

Limiting conditional distributions for birth-death processes

In a recent paper [16], one of us identified all of the quasi-stationary distributions for a non-explosive, evanescent birth-death process for which absorption is certain, and established conditions for the existence of the corresponding limiting conditional distributions. Our purpose is to extend these results in a number of directions. We shall consider separately two cases depending on whether or not the process is evanescent. In the former case we shall relax the condition that absorption is certain. Furthermore, we shall allow for the possibility that the minimal process might be explosive, so that the transition rates alone will not necessarily determine the birth-death process uniquely. Although we shall be concerned mainly with the minimal process, our most general results hold for any birth-death process whose transition probabilities satisfy both the backward and the forward Kolmogorov differential equations.

An fMRI Normative Database for Connectivity Networks Using One-Class Support Vector Machines

The application of functional magnetic resonance imaging (fMRI) in neuroscience studies has increased enormously in the last decade. Although primarily used to map brain regions activated by specific stimuli, many studies have shown that fMRI can also be useful in identifying interactions between brain regions (functional and effective connectivity). Despite the widespread use of fMRI as a research tool, clinical applications of brain connectivity as studied by fMRI are not well established. One possible explanation is the lack of normal pattern, and intersubject variability-two variables that are still largely uncharacterized in most patient populations of interest. In the current study, we combine the identification of functional connectivity networks extracted by using Spearman partial correlation with the use of a one-class support vector machine in order construct a normative database. An application of this approach is illustrated using an fMRI dataset of 43 healthy Subjects performing a visual working memory task. In addition, the relationships between the results obtained and behavioral data are explored. Hum Brain Mapp 30:1068-1076, 2009. (C) 2008 Wiley-Liss. Inc.