Working Memory Networks for Learning Temporal Order, with Application to 3-D Visual Object Recognition

Working memory neural networks are characterized which encode the invariant temporal order of sequential events. Inputs to the networks, called Sustained Temporal Order REcurrent (STORE) models, may be presented at widely differing speeds, durations, and interstimulus intervals. The STORE temporal order code is designed to enable all emergent groupings of sequential events to be stably learned and remembered in real time, even as new events perturb the system. Such a competence is needed in neural architectures which self-organize learned codes for variable-rate speech perception, sensory-motor planning, or 3-D visual object recognition. Using such a working memory, a self-organizing architecture for invariant 3-D visual object recognition is described. The new model is based on the model of Seibert and Waxman (1990a), which builds a 3-D representation of an object from a temporally ordered sequence of its 2-D aspect graphs. The new model, called an ARTSTORE model, consists of the following cascade of processing modules: Invariant Preprocessor --> ART 2 --> STORE Model --> ART 2 --> Outstar Network.

Fuzzy ARTMAP: A Neural Network Architecture for Incremental Supervised Learning of Analog Multidimensional Maps

A new neural network architecture is introduced for incremental supervised learning of recognition categories and multidimensional maps in response to arbitrary sequences of analog or binary input vectors. The architecture, called Fuzzy ARTMAP, achieves a synthesis of fuzzy logic and Adaptive Resonance Theory (ART) neural networks by exploiting a close formal similarity between the computations of fuzzy subsethood and ART category choice, resonance, and learning. Fuzzy ARTMAP also realizes a new Minimax Learning Rule that conjointly minimizes predictive error and maximizes code compression, or generalization. This is achieved by a match tracking process that increases the ART vigilance parameter by the minimum amount needed to correct a predictive error. As a result, the system automatically learns a minimal number of recognition categories, or "hidden units", to met accuracy criteria. Category proliferation is prevented by normalizing input vectors at a preprocessing stage. A normalization procedure called complement coding leads to a symmetric theory in which the MIN operator (Λ) and the MAX operator (v) of fuzzy logic play complementary roles. Complement coding uses on-cells and off-cells to represent the input pattern, and preserves individual feature amplitudes while normalizing the total on-cell/off-cell vector. Learning is stable because all adaptive weights can only decrease in time. Decreasing weights correspond to increasing sizes of category "boxes". Smaller vigilance values lead to larger category boxes. Improved prediction is achieved by training the system several times using different orderings of the input set. This voting strategy can also be used to assign probability estimates to competing predictions given small, noisy, or incomplete training sets. Four classes of simulations illustrate Fuzzy ARTMAP performance as compared to benchmark back propagation and genetic algorithm systems. These simulations include (i) finding points inside vs. outside a circle; (ii) learning to tell two spirals apart; (iii) incremental approximation of a piecewise continuous function; and (iv) a letter recognition database. The Fuzzy ARTMAP system is also compared to Salzberg's NGE system and to Simpson's FMMC system.

Cortical Dynamics of Visual Motion Perception: Short-Range and Long-Range Apparent Motion

This article describes further evidence for a new neural network theory of biological motion perception. The theory clarifies why parallel streams Vl --> V2, Vl --> MT, and Vl --> V2 --> MT exist for static form and motion form processing among the areas Vl, V2, and MT of visual cortex. The theory suggests that the static form system (Static BCS) generates emergent boundary segmentations whose outputs are insensitive to direction-ofcontrast and insensitive to direction-of-motion, whereas the motion form system (Motion BCS) generates emergent boundary segmentations whose outputs are insensitive to directionof-contrast but sensitive to direction-of-motion. The theory is used to explain classical and recent data about short-range and long-range apparent motion percepts that have not yet been explained by alternative models. These data include beta motion; split motion; gamma motion and reverse-contrast gamma motion; delta motion; visual inertia; the transition from group motion to element motion in response to a Ternus display as the interstimulus interval (ISI) decreases; group motion in response to a reverse-contrast Ternus display even at short ISIs; speed-up of motion velocity as interflash distance increases or flash duration decreases; dependence of the transition from element motion to group motion on stimulus duration and size; various classical dependencies between flash duration, spatial separation, ISI, and motion threshold known as Korte's Laws; dependence of motion strength on stimulus orientation and spatial frequency; short-range and long-range form-color interactions; and binocular interactions of flashes to different eyes.

A Neural Network of Adaptively Timed Reinforcement Learning and Hippocampal Dynamics

A neural model is described of how adaptively timed reinforcement learning occurs. The adaptive timing circuit is suggested to exist in the hippocampus, and to involve convergence of dentate granule cells on CA3 pyramidal cells, and NMDA receptors. This circuit forms part of a model neural system for the coordinated control of recognition learning, reinforcement learning, and motor learning, whose properties clarify how an animal can learn to acquire a delayed reward. Behavioral and neural data are summarized in support of each processing stage of the system. The relevant anatomical sites are in thalamus, neocortex, hippocampus, hypothalamus, amygdala, and cerebellum. Cerebellar influences on motor learning are distinguished from hippocampal influences on adaptive timing of reinforcement learning. The model simulates how damage to the hippocampal formation disrupts adaptive timing, eliminates attentional blocking, and causes symptoms of medial temporal amnesia. It suggests how normal acquisition of subcortical emotional conditioning can occur after cortical ablation, even though extinction of emotional conditioning is retarded by cortical ablation. The model simulates how increasing the duration of an unconditioned stimulus increases the amplitude of emotional conditioning, but does not change adaptive timing; and how an increase in the intensity of a conditioned stimulus "speeds up the clock", but an increase in the intensity of an unconditioned stimulus does not. Computer simulations of the model fit parametric conditioning data, including a Weber law property and an inverted U property. Both primary and secondary adaptively timed conditioning are simulated, as are data concerning conditioning using multiple interstimulus intervals (ISIs), gradually or abruptly changing ISis, partial reinforcement, and multiple stimuli that lead to time-averaging of responses. Neurobiologically testable predictions are made to facilitate further tests of the model.

Working Memories for Storage and Recall of Arbitrary Temporal Sequences

A working memory model is described that is capable of storing and recalling arbitrary temporal sequences of events, including repeated items. These memories encode the invariant temporal order of sequential events that may be presented at widely differing speeds, durations, and interstimulus intervals. This temporal order code is designed to enable all possible groupings of sequential events to be stably learned and remembered in real time, even as new events perturb the system.

Comparative Performance Measures of Fuzzy ARTMAP, Learned Vector Quantization, and Back Propagation for Handwritten Character Recognition

This article compares the performance of Fuzzy ARTMAP with that of Learned Vector Quantization and Back Propagation on a handwritten character recognition task. Training with Fuzzy ARTMAP to a fixed criterion used many fewer epochs. Voting with Fuzzy ARTMAP yielded the highest recognition rates.

Adaptive Resonance Associative Map: A Hierarchical ART System for Fast Stable Associative Learning

This paper introduces a new class of predictive ART architectures, called Adaptive Resonance Associative Map (ARAM) which performs rapid, yet stable heteroassociative learning in real time environment. ARAM can be visualized as two ART modules sharing a single recognition code layer. The unit for recruiting a recognition code is a pattern pair. Code stabilization is ensured by restricting coding to states where resonances are reached in both modules. Simulation results have shown that ARAM is capable of self-stabilizing association of arbitrary pattern pairs of arbitrary complexity appearing in arbitrary sequence by fast learning in real time environment. Due to the symmetrical network structure, associative recall can be performed in both directions.

Neural Representations for Sensory-Motor Control I: Head-Centered 3-D Target Positions from Opponent Eye Commands

This article describes how corollary discharges from outflow eye movement commands can be transformed by two stages of opponent neural processing into a head-centered representation of 3-D target position. This representation implicitly defines a cyclopean coordinate system whose variables approximate the binocular vergence and spherical horizontal and vertical angles with respect to the observer's head. Various psychophysical data concerning binocular distance perception and reaching behavior are clarified by this representation. The representation provides a foundation for learning head-centered and body-centered invariant representations of both foveated and non-foveated 3-D target positions. It also enables a solution to be developed of the classical motor equivalence problem, whereby many different joint configurations of a redundant manipulator can all be used to realize a desired trajectory in 3-D space.

All-optical OFDM and distributed Raman amplification: Challenges to enable high capacities and extend reach

We review recent advances in all-optical OFDM technologies and discuss the performance of a field trial of a 2 Tbit/s Coherent WDM over 124 km with distributed Raman amplification. The results indicate that careful optimisation of the Raman pumps is essential. We also consider how all-optical OFDM systems perform favourably against energy consumption when compared with alternative coherent detection schemes. We argue that, in an energy constrained high-capacity transmission system, direct detected all-optical OFDM with 'ideal' Raman amplification is an attractive candidate for metro area datacentre interconnects with ~100 km fibre spans, with an overall energy requirement at least three times lower than coherent detection techniques.

Generation and characterisation of biologically active milk-derived protein and peptide fractions

In recent years, extensive research has been carried out on the health benefits of milk proteins and peptides. Biologically active peptides are defined as specific protein fragments which have a positive impact on the physiological functions of the body; such peptides are produced naturally in vivo, but can also be generated by physical and/or chemical processes, enzymatic hydrolysis and/or microbial fermentation. The aims of this thesis were to investigate not only the traditional methods used for the generation of bioactive peptides, but also novel processes such as heat treatment, and the role of indigenous milk proteases, e.g., in mastitic milk, in the production of such peptides. In addition, colostrum was characterised as a source of bioactive proteins and peptides. Firstly, a comprehensive study was carried out on the composition and physical properties of colostrum throughout the early-lactation period. Marked differences in the physico-chemical properties of colostrum compared with milk were observed. Various fractions of colostrum were also tested for their effect on the secretion of pro- and anti-inflammatory cytokines from a macrophage cell line and bone marrow dendritic cells, as well as insulin secretion from a pancreatic beta cell line. A significant reduction in the secretion of the pro-inflammatory cytokines, TNF-α, IL-6, IL-1β and IL-12, a significant increase in the secretion of the anti-inflammatory cytokine, IL-10, as well as a significant increase in insulin secretion were observed for various colostrum fractions. Another study examined the early proteomic changes in the milk of 8 cows in response to infusion with the endotoxin lipopolysaccharide (LPS) at quarter level in a model mastitic system; marked differences in the protein and peptide profile of milk from LPS challenged cows were observed, and a pH 4.6-soluble fraction of this milk was found to cause a substantial induction in the secretion of IL-10 from a murine macrophage cell line. Heat-induced hydrolysis of sodium caseinate was investigated from the dual viewpoints of protein breakdown and peptide formation, and, a peptide fraction produced in this manner was found to cause a significant increase in the secretion of the anti-inflammatory cytokine, IL-10, from a murine macrophage cell line. The effects of sodium caseinate hydrolysed by chymosin on the gut-derived satiety hormone glucagon-like peptide-1 (GLP-1) were investigated; the resulting casein-derived peptides displayed good in vitro and in vivo secretion of GLP-1. Overall, the studies described in this thesis expand on current knowledge and provide good evidence for the use of novel methods for the isolation, generation and characterisation of bioactive proteins and/or peptides.

Photonic integrated circuit for the manipulation of coherent optical combs

Photonic integration has become an important research topic in research for applications in the telecommunications industry. Current optical internet infrastructure has reached capacity with current generation dense wavelength division multiplexing (DWDM) systems fully occupying the low absorption region of optical fibre from 1530 nm to 1625 nm (the C and L bands). This is both due to an increase in the number of users worldwide and existing users demanding more bandwidth. Therefore, current research is focussed on using the available telecommunication spectrum more efficiently. To this end, coherent communication systems are being developed. Advanced coherent modulation schemes can be quite complex in terms of the number and array of devices required for implementation. In order to make these systems viable both logistically and commercially, photonic integration is required. In traditional DWDM systems, arrayed waveguide gratings (AWG) are used to both multiplex and demultiplex the multi-wavelength signal involved. AWGs are used widely as they allow filtering of the many DWDM wavelengths simultaneously. However, when moving to coherent telecommunication systems such as coherent optical frequency division multiplexing (OFDM) smaller FSR ranges are required from the AWG. This increases the size of the device which is counter to the miniaturisation which integration is trying to achieve. Much work was done with active filters during the 1980s. This involved using a laser device (usually below threshold) to allow selective wavelength filtering of input signals. By using more complicated cavity geometry devices such as distributed feedback (DFB) and sampled grating distributed Bragg gratings (SG-DBR) narrowband filtering is achievable with high suppression (>30 dB) of spurious wavelengths. The active nature of the devices also means that, through carrier injection, the index can be altered resulting in tunability of the filter. Used above threshold, active filters become useful in filtering coherent combs. Through injection locking, the coherence of the filtered wavelengths with the original comb source is retained. This gives active filters potential application in coherent communication system as demultiplexers. This work will focus on the use of slotted Fabry-Pérot (SFP) semiconductor lasers as active filters. Experiments were carried out to ensure that SFP lasers were useful as tunable active filters. In all experiments in this work the SFP lasers were operated above threshold and so injection locking was the mechanic by which the filters operated. Performance of the lasers under injection locking was examined using both single wavelength and coherent comb injection. In another experiment two discrete SFP lasers were used simultaneously to demultiplex a two-line coherent comb. The relative coherence of the comb lines was retained after demultiplexing. After showing that SFP lasers could be used to successfully demultiplex coherent combs a photonic integrated circuit was designed and fabricated. This involved monolithic integration of a MMI power splitter with an array of single facet SFP lasers. This device was tested much in the same way as the discrete devices. The integrated device was used to successfully demultiplex a two line coherent comb signal whilst retaining the relative coherence between the filtered comb lines. A series of modelling systems were then employed in order to understand the resonance characteristics of the fabricated devices, and to understand their performance under injection locking. Using this information, alterations to the SFP laser designs were made which were theoretically shown to provide improved performance and suitability for use in filtering coherent comb signals.

Surveillance during pregnancy: methods and response rates from a hospital based pilot study of the Pregnancy Risk Assessment Monitoring System in Ireland

Background: Many European countries including Ireland lack high quality, on-going, population based estimates of maternal behaviours and experiences during pregnancy. PRAMS is a CDC surveillance program which was established in the United States in 1987 to generate high quality, population based data to reduce infant mortality rates and improve maternal and infant health. PRAMS is the only on-going population based surveillance system of maternal behaviours and experiences that occur before, during and after pregnancy worldwide.Methods: The objective of this study was to adapt, test and evaluate a modified CDC PRAMS methodology in Ireland. The birth certificate file which is the standard approach to sampling for PRAMS in the United States was not available for the PRAMS Ireland study. Consequently, delivery record books for the period between 3 and 5 months before the study start date at a large urban obstetric hospital [8,900 births per year] were used to randomly sample 124 women. Name, address, maternal age, infant sex, gestational age at delivery, delivery method, APGAR score and birth weight were manually extracted from records. Stillbirths and early neonatal deaths were excluded using APGAR scores and hospital records. Women were sent a letter of invitation to participate including option to opt out, followed by a modified PRAMS survey, a reminder letter and a final survey.Results: The response rate for the pilot was 67%. Two per cent of women refused the survey, 7% opted out of the study and 24% did not respond. Survey items were at least 88% complete for all 82 respondents. Prevalence estimates of socially undesirable behaviours such as alcohol consumption during pregnancy were high [>50%] and comparable with international estimates.Conclusion: PRAMS is a feasible and valid method of collecting information on maternal experiences and behaviours during pregnancy in Ireland. PRAMS may offer a potential solution to data deficits in maternal health behaviour indicators in Ireland with further work. This study is important to researchers in Europe and elsewhere who may be interested in new ways of tailoring an established CDC methodology to their unique settings to resolve data deficits in maternal health.

Power system parameters forecasting using Hilbert-Huang transform and machine learning

A novel hybrid data-driven approach is developed for forecasting power system parameters with the goal of increasing the efficiency of short-term forecasting studies for non-stationary time-series. The proposed approach is based on mode decomposition and a feature analysis of initial retrospective data using the Hilbert-Huang transform and machine learning algorithms. The random forests and gradient boosting trees learning techniques were examined. The decision tree techniques were used to rank the importance of variables employed in the forecasting models. The Mean Decrease Gini index is employed as an impurity function. The resulting hybrid forecasting models employ the radial basis function neural network and support vector regression. A part from introduction and references the paper is organized as follows. The second section presents the background and the review of several approaches for short-term forecasting of power system parameters. In the third section a hybrid machine learningbased algorithm using Hilbert-Huang transform is developed for short-term forecasting of power system parameters. Fourth section describes the decision tree learning algorithms used for the issue of variables importance. Finally in section six the experimental results in the following electric power problems are presented: active power flow forecasting, electricity price forecasting and for the wind speed and direction forecasting.

A distributed architecture for the monitoring and analysis of time series data

It is estimated that the quantity of digital data being transferred, processed or stored at any one time currently stands at 4.4 zettabytes (4.4 × 2 70 bytes) and this figure is expected to have grown by a factor of 10 to 44 zettabytes by 2020. Exploiting this data is, and will remain, a significant challenge. At present there is the capacity to store 33% of digital data in existence at any one time; by 2020 this capacity is expected to fall to 15%. These statistics suggest that, in the era of Big Data, the identification of important, exploitable data will need to be done in a timely manner. Systems for the monitoring and analysis of data, e.g. stock markets, smart grids and sensor networks, can be made up of massive numbers of individual components. These components can be geographically distributed yet may interact with one another via continuous data streams, which in turn may affect the state of the sender or receiver. This introduces a dynamic causality, which further complicates the overall system by introducing a temporal constraint that is difficult to accommodate. Practical approaches to realising the system described above have led to a multiplicity of analysis techniques, each of which concentrates on specific characteristics of the system being analysed and treats these characteristics as the dominant component affecting the results being sought. The multiplicity of analysis techniques introduces another layer of heterogeneity, that is heterogeneity of approach, partitioning the field to the extent that results from one domain are difficult to exploit in another. The question is asked can a generic solution for the monitoring and analysis of data that: accommodates temporal constraints; bridges the gap between expert knowledge and raw data; and enables data to be effectively interpreted and exploited in a transparent manner, be identified? The approach proposed in this dissertation acquires, analyses and processes data in a manner that is free of the constraints of any particular analysis technique, while at the same time facilitating these techniques where appropriate. Constraints are applied by defining a workflow based on the production, interpretation and consumption of data. This supports the application of different analysis techniques on the same raw data without the danger of incorporating hidden bias that may exist. To illustrate and to realise this approach a software platform has been created that allows for the transparent analysis of data, combining analysis techniques with a maintainable record of provenance so that independent third party analysis can be applied to verify any derived conclusions. In order to demonstrate these concepts, a complex real world example involving the near real-time capturing and analysis of neurophysiological data from a neonatal intensive care unit (NICU) was chosen. A system was engineered to gather raw data, analyse that data using different analysis techniques, uncover information, incorporate that information into the system and curate the evolution of the discovered knowledge. The application domain was chosen for three reasons: firstly because it is complex and no comprehensive solution exists; secondly, it requires tight interaction with domain experts, thus requiring the handling of subjective knowledge and inference; and thirdly, given the dearth of neurophysiologists, there is a real world need to provide a solution for this domain