Neural Networks as Cybernetic Systems - Part II

Neural Networks as Cybernetic Systems is a textbox that combines classical systems theory with artificial neural network technology.

Neural Networks as Cybernetic Systems

This book on biological cybernetics combines system theory and artificial neural networks. Following a 'holistic' approach, the book examines the role of simulation in biology. Mainly addressed to students of Biology, the book tries to avoid the use of mathematical formula as far as possible. Exercises can be performed with a related software tool (tkCybernetics) for part 1 as well as with educational simulation on neural networks for part two (in prepration.)

Food Recognition for Dietary Assessment Using Deep Convolutional Neural Networks

Diet management is a key factor for the prevention and treatment of diet-related chronic diseases. Computer vision systems aim to provide automated food intake assessment using meal images. We propose a method for the recognition of already segmented food items in meal images. The method uses a 6-layer deep convolutional neural network to classify food image patches. For each food item, overlapping patches are extracted and classified and the class with the majority of votes is assigned to it. Experiments on a manually annotated dataset with 573 food items justified the choice of the involved components and proved the effectiveness of the proposed system yielding an overall accuracy of 84.9%.

Algoritmos para mineria de datos con redes de neuronas

Este trabajo propone una serie de algoritmos con el objetivo de extraer información de conjuntos de datos con redes de neuronas. Se estudian dichos algoritmos con redes de neuronas Enhenced Neural Networks (ENN), debido a que esta arquitectura tiene algunas ventajas cuando se aproximan funciones mediante redes neuronales. En la red ENN los pesos de la matriz principal varián con cada patrón, por lo que se comete un error menor en la aproximación. Las redes de neuronas ENN reúnen la información en los pesos de su red auxiliar, se propone un método para obtener información de la red a través de dichos pesos en formas de reglas y asignando un factor de certeza de dichas reglas. La red ENN obtiene un error cuadrático medio menor que el error teórico de una aproximación matemática por ejemplo mediante polinomios de Taylor. Se muestra como una red ENN, entrenada a partir un conjunto de patrones obtenido de una función de variables reales, sus pesos asociados tienen unas relaciones similares a las que se veri_can con las variables independientes con dicha función de variables reales. Las redes de neuronas ENN aproximan polinomios, se extrae conocimiento de un conjunto de datos de forma similar a la regresión estadística, resolviendo de forma más adecuada el problema de multicolionalidad en caso de existir. Las relaciones a partir de los pesos asociados de la matriz de la red auxiliar se obtienen similares a los coeficientes de una regresión para el mismo conjunto numérico. Una red ENN entrenada a partir de un conjunto de datos de una función boolena extrae el conocimiento a partir de los pesos asociados, y la influencia de las variables de la regla lógica de la función booleana, queda reejada en esos pesos asociados a la red auxiliar de la red ENN. Se plantea una red de base radial (RBF) para la clasificación y predicción en problemas forestales y agrícolas, obteniendo mejores resultados que con el modelo de regresión y otros métodos. Los resultados con una red RBF mejoran al método de regresión si existe colinealidad entre los datos que se dispone y no son muy numerosos. También se detecta que variables tienen más importancia en virtud de la variable pronóstico. Obteniendo el error cuadrático medio con redes RBF menor que con otros métodos, en particular que con el modelo de regresión. Abstract A series of algorithms is proposed in this study aiming at the goal of producing information about data groups with a neural network. These algorithms are studied with Enheced Neural Networks (ENN), owing to the fact that this structure shows sever advantages when the functions are approximated by neural networks. Main matrix weights in th ENN vary on each pattern; so, a smaller error is produced when approximating. The neural network ENN joins the weight information contained in their auxiliary network. Thus, a method to obtain information on the network through those weights is proposed by means of rules adding a certainty factor. The net ENN obtains a mean squared error smaller than the theorical one emerging from a mathematical aproximation such as, for example, by means of Taylor's polynomials. This study also shows how in a neural network ENN trained from a set of patterns obtained through a function of real variables, its associated weights have relationships similar to those ones tested by means of the independent variables connected with such functions of real variables. The neural network ENN approximates polynomials through it information about a set of data may be obtained in a similar way than through statistical regression, solving in this way possible problems of multicollinearity in a more suitable way. Relationships emerging from the associated weights in the auxiliary network matrix obtained are similar to the coeficients corresponding to a regression for the same numerical set. A net ENN trained from a boolean function data set obtains its information from its associated weights. The inuence of the variables of the boolean function logical rule are reected on those weights associated to the net auxiliar of the ENN. A radial basis neural networks (RBF) for the classification and prediction of forest and agricultural problems is proposed. This scheme obtains better results than the ones obtained by means of regression and other methods. The outputs with a net RBF better the regression method if the collineality with the available data and their amount is not very large. Detection of which variables are more important basing on the forecast variable can also be achieved, obtaining a mean squared error smaller that the ones obtained through other methods, in special the one produced by the regression pattern.

Integration of multiple instructive cues by neural crest stem cells reveals cell-intrinsic biases in relative growth factor responsiveness

Growth factors can influence lineage determination of neural crest stem cells (NCSCs) in an instructive manner, in vitro. Because NCSCs are likely exposed to multiple signals in vivo, these findings raise the question of how stem cells would integrate such combined influences. Bone morphogenetic protein 2 (BMP2) promotes neuronal differentiation and glial growth factor 2 (GGF2) promotes glial differentiation; if NCSCs are exposed to saturating concentrations of both factors, BMP2 appears dominant. By contrast, if the cells are exposed to saturating concentrations of both BMP2 and transforming growth factor β1 (which promotes smooth muscle differentiation), the two factors appear codominant. Sequential addition experiments indicate that NCSCs require 48–96 hrs in GGF2 before they commit to a glial fate, whereas the cells commit to a smooth muscle fate within 24 hr in transforming growth factor β1. The delayed response to GGF2 does not reflect a lack of functional receptors; however, because the growth factor induces rapid mitogen-activated protein kinase phosphorylation in naive cells. Furthermore, GGF2 can attenuate induction of the neurogenic transcription factor mammalian achaete-scute homolog 1, by low doses of BMP2. This short-term antineurogenic influence of GGF2 is not sufficient for glial lineage commitment, however. These data imply that NCSCs exhibit cell-intrinsic biases in the timing and relative dosage sensitivity of their responses to instructive factors that influence the outcome of lineage decisions in the presence of multiple factors. The relative delay in glial lineage commitment, moreover, apparently reflects successive short-term and longer-term actions of GGF2. Such a delay may help to explain why glia normally differentiate after neurons, in vivo.

Statistical significance of protein structure prediction by threading

In this study, we estimate the statistical significance of structure prediction by threading. We introduce a single parameter ɛ that serves as a universal measure determining the probability that the best alignment is indeed a native-like analog. Parameter ɛ takes into account both length and composition of the query sequence and the number of decoys in threading simulation. It can be computed directly from the query sequence and potential of interactions, eliminating the need for sequence reshuffling and realignment. Although our theoretical analysis is general, here we compare its predictions with the results of gapless threading. Finally we estimate the number of decoys from which the native structure can be found by existing potentials of interactions. We discuss how this analysis can be extended to determine the optimal gap penalties for any sequence-structure alignment (threading) method, thus optimizing it to maximum possible performance.

Simple neural networks for the amplification and utilization of small changes in neuron firing rates

I describe physiologically plausible “voter-coincidence” neural networks such that secondary “coincidence” neurons fire on the simultaneous receipt of sufficiently large sets of input pulses from primary sets of neurons. The networks operate such that the firing rate of the secondary, output neurons increases (or decreases) sharply when the mean firing rate of primary neurons increases (or decreases) to a much smaller degree. In certain sensory systems, signals that are generally smaller than the noise levels of individual primary detectors, are manifest in very small increases in the firing rates of sets of afferent neurons. For such systems, this kind of network can act to generate relatively large changes in the firing rate of secondary “coincidence” neurons. These differential amplification systems can be cascaded to generate sharp, “yes–no” spike signals that can direct behavioral responses.

Endothelin-B receptor is expressed by neural crest cells in the avian embryo.

Disruptions of the genes encoding endothelin 3 (EDN3) and its receptor endothelin-B receptor (EDNRB) in the mouse result in defects of two neural crest (NC)-derived lineages, the melanocytes, and the enteric nervous system. To assess the mechanisms through which the EDN3/EDNRB signaling pathway can selectively act on these NC derivatives, we have studied the spatiotemporal expression pattern of the EDNRB gene in the avian embryo, a model in which NC development has been extensively studied. For this purpose, we have cloned the quail homologue of the mammalian EDNRB cDNA. EDNRB transcripts are present in NC cells before and during their emigration from the neural tube at all levels of the neuraxis. At later developmental stages, the receptor remains abundantly expressed in the peripheral nervous system including the enteric nervous system. In a previous study, we have shown that EDN3 enhances dramatically the proliferation of NC cells when they are at the pluripotent stage. We propose that the selective effect of EDN3 or EDNRB gene inactivation is due to the fact that both melanocytes and enteric nervous system precursors have to colonize large embryonic areas (skin and bowel) from a relatively small population of precursors that have to expand considerably in number. It is therefore understandable that a deficit in one of the growth-promoting pathways of NC cells has more deleterious effects on long-range migrating cells than on the NC derivatives which develop close to the neural primordium like the sensory and sympathetic ganglia.

3D Hand Pose Estimation with Neural Networks

We propose the design of a real-time system to recognize and interprethand gestures. The acquisition devices are low cost 3D sensors. 3D hand pose will be segmented, characterized and track using growing neural gas (GNG) structure. The capacity of the system to obtain information with a high degree of freedom allows the encoding of many gestures and a very accurate motion capture. The use of hand pose models combined with motion information provide with GNG permits to deal with the problem of the hand motion representation. A natural interface applied to a virtual mirrorwriting system and to a system to estimate hand pose will be designed to demonstrate the validity of the system.

Designing Regularizers and Architectures for Recurrent Neural Networks

Cette thèse contribue a la recherche vers l'intelligence artificielle en utilisant des méthodes connexionnistes. Les réseaux de neurones récurrents sont un ensemble de modèles séquentiels de plus en plus populaires capable en principe d'apprendre des algorithmes arbitraires. Ces modèles effectuent un apprentissage en profondeur, un type d'apprentissage machine. Sa généralité et son succès empirique en font un sujet intéressant pour la recherche et un outil prometteur pour la création de l'intelligence artificielle plus générale. Le premier chapitre de cette thèse donne un bref aperçu des sujets de fonds: l'intelligence artificielle, l'apprentissage machine, l'apprentissage en profondeur et les réseaux de neurones récurrents. Les trois chapitres suivants couvrent ces sujets de manière de plus en plus spécifiques. Enfin, nous présentons quelques contributions apportées aux réseaux de neurones récurrents. Le chapitre \ref{arxiv1} présente nos travaux de régularisation des réseaux de neurones récurrents. La régularisation vise à améliorer la capacité de généralisation du modèle, et joue un role clé dans la performance de plusieurs applications des réseaux de neurones récurrents, en particulier en reconnaissance vocale. Notre approche donne l'état de l'art sur TIMIT, un benchmark standard pour cette tâche. Le chapitre \ref{cpgp} présente une seconde ligne de travail, toujours en cours, qui explore une nouvelle architecture pour les réseaux de neurones récurrents. Les réseaux de neurones récurrents maintiennent un état caché qui représente leurs observations antérieures. L'idée de ce travail est de coder certaines dynamiques abstraites dans l'état caché, donnant au réseau une manière naturelle d'encoder des tendances cohérentes de l'état de son environnement. Notre travail est fondé sur un modèle existant; nous décrivons ce travail et nos contributions avec notamment une expérience préliminaire.

Designing Regularizers and Architectures for Recurrent Neural Networks

Cette thèse contribue a la recherche vers l'intelligence artificielle en utilisant des méthodes connexionnistes. Les réseaux de neurones récurrents sont un ensemble de modèles séquentiels de plus en plus populaires capable en principe d'apprendre des algorithmes arbitraires. Ces modèles effectuent un apprentissage en profondeur, un type d'apprentissage machine. Sa généralité et son succès empirique en font un sujet intéressant pour la recherche et un outil prometteur pour la création de l'intelligence artificielle plus générale. Le premier chapitre de cette thèse donne un bref aperçu des sujets de fonds: l'intelligence artificielle, l'apprentissage machine, l'apprentissage en profondeur et les réseaux de neurones récurrents. Les trois chapitres suivants couvrent ces sujets de manière de plus en plus spécifiques. Enfin, nous présentons quelques contributions apportées aux réseaux de neurones récurrents. Le chapitre \ref{arxiv1} présente nos travaux de régularisation des réseaux de neurones récurrents. La régularisation vise à améliorer la capacité de généralisation du modèle, et joue un role clé dans la performance de plusieurs applications des réseaux de neurones récurrents, en particulier en reconnaissance vocale. Notre approche donne l'état de l'art sur TIMIT, un benchmark standard pour cette tâche. Le chapitre \ref{cpgp} présente une seconde ligne de travail, toujours en cours, qui explore une nouvelle architecture pour les réseaux de neurones récurrents. Les réseaux de neurones récurrents maintiennent un état caché qui représente leurs observations antérieures. L'idée de ce travail est de coder certaines dynamiques abstraites dans l'état caché, donnant au réseau une manière naturelle d'encoder des tendances cohérentes de l'état de son environnement. Notre travail est fondé sur un modèle existant; nous décrivons ce travail et nos contributions avec notamment une expérience préliminaire.