Using ancillary statistics in on-line learning algorithms

Neural networks are usually curved statistical models. They do not have finite dimensional sufficient statistics, so on-line learning on the model itself inevitably loses information. In this paper we propose a new scheme for training curved models, inspired by the ideas of ancillary statistics and adaptive critics. At each point estimate an auxiliary flat model (exponential family) is built to locally accommodate both the usual statistic (tangent to the model) and an ancillary statistic (normal to the model). The auxiliary model plays a role in determining credit assignment analogous to that played by an adaptive critic in solving temporal problems. The method is illustrated with the Cauchy model and the algorithm is proved to be asymptotically efficient.

The learning dynamics of a universal approximator

The learning properties of a universal approximator, a normalized committee machine with adjustable biases, are studied for on-line back-propagation learning. Within a statistical mechanics framework, numerical studies show that this model has features which do not exist in previously studied two-layer network models without adjustable biases, e.g., attractive suboptimal symmetric phases even for realizable cases and noiseless data.

Regression with input-dependent noise: A Gaussian process treatment

Gaussian processes provide natural non-parametric prior distributions over regression functions. In this paper we consider regression problems where there is noise on the output, and the variance of the noise depends on the inputs. If we assume that the noise is a smooth function of the inputs, then it is natural to model the noise variance using a second Gaussian process, in addition to the Gaussian process governing the noise-free output value. We show that prior uncertainty about the parameters controlling both processes can be handled and that the posterior distribution of the noise rate can be sampled from using Markov chain Monte Carlo methods. Our results on a synthetic data set give a posterior noise variance that well-approximates the true variance.

Regression with input-dependent noise: A Bayesian treatment

In most treatments of the regression problem it is assumed that the distribution of target data can be described by a deterministic function of the inputs, together with additive Gaussian noise having constant variance. The use of maximum likelihood to train such models then corresponds to the minimization of a sum-of-squares error function. In many applications a more realistic model would allow the noise variance itself to depend on the input variables. However, the use of maximum likelihood to train such models would give highly biased results. In this paper we show how a Bayesian treatment can allow for an input-dependent variance while overcoming the bias of maximum likelihood.

Adding constrained discontinuities to Gaussian process models of wind fields

Gaussian Processes provide good prior models for spatial data, but can be too smooth. In many physical situations there are discontinuities along bounding surfaces, for example fronts in near-surface wind fields. We describe a modelling method for such a constrained discontinuity and demonstrate how to infer the model parameters in wind fields with MCMC sampling.

Sparse representation for Gaussian process models

We develop an approach for a sparse representation for Gaussian Process (GP) models in order to overcome the limitations of GPs caused by large data sets. The method is based on a combination of a Bayesian online algorithm together with a sequential construction of a relevant subsample of the data which fully specifies the prediction of the model. Experimental results on toy examples and large real-world datasets indicate the efficiency of the approach.

Error-correcting codes on a Bethe-like lattice

We analyse Gallager codes by employing a simple mean-field approximation that distorts the model geometry and preserves important interactions between sites. The method naturally recovers the probability propagation decoding algorithm as a minimization of a proper free-energy. We find a thermodynamical phase transition that coincides with information theoretical upper-bounds and explain the practical code performance in terms of the free-energy landscape.

The TAP approach to intensive and extensive connectivity systems

The Thouless-Anderson-Palmer (TAP) approach was originally developed for analysing the Sherrington-Kirkpatrick model in the study of spin glass models and has been employed since then mainly in the context of extensively connected systems whereby each dynamical variable interacts weakly with the others. Recently, we extended this method for handling general intensively connected systems where each variable has only O(1) connections characterised by strong couplings. However, the new formulation looks quite different with respect to existing analyses and it is only natural to question whether it actually reproduces known results for systems of extensive connectivity. In this chapter, we apply our formulation of the TAP approach to an extensively connected system, the Hopfield associative memory model, showing that it produces identical results to those obtained by the conventional formulation.

Biochemical correlates of cognition: exploring the relationships between blood, brain and behaviour

This multi-modal investigation aimed to refine analytic tools including proton magnetic resonance spectroscopy (1H-MRS) and fatty acid gas chromatography-mass spectrometry (GC-MS) analysis, for use with adult and paediatric populations, to investigate potential biochemical underpinnings of cognition (Chapter 1). Essential fatty acids (EFAs) are vital for the normal development and function of neural cells. There is increasing evidence of behavioural impairments arising from dietary deprivation of EFAs and their long-chain fatty acid metabolites (Chapter 2). Paediatric liver disease was used as a deficiency model to examine the relationships between EFA status and cognitive outcomes. Age-appropriate Wechsler assessments measured Full-scale IQ (FSIQ) and Information Processing Speed (IPS) in clinical and healthy cohorts; GC-MS quantified surrogate markers of EFA status in erythrocyte membranes; and 1H-MRS quantified neurometabolite markers of neuronal viability and function in cortical tissue (Chapter 3). Post-transplant children with early-onset liver disease demonstrated specific deficits in IPS compared to age-matched acute liver failure transplant patients and sibling controls, suggesting that the time-course of the illness is a key factor (Chapter 4). No signs of EFA deficiency were observed in the clinical cohort, suggesting that EFA metabolism was not significantly impacted by liver disease. A strong, negative correlation was observed between omega-6 fatty acids and FSIQ, independent of disease diagnosis (Chapter 5). In a study of healthy adults, effect sizes for the relationship between 1H-MRS- detectable neurometabolites and cognition fell within the range of previous work, but were not statistically significant. Based on these findings, recommendations are made emphasising the need for hypothesis-driven enquiry and greater subtlety of data analysis (Chapter 6). Consistency of metabolite values between paediatric clinical cohorts and controls indicate normal neurodevelopment, but the lack of normative, age-matched data makes it difficult to assess the true strength of liver disease-associated metabolite changes (Chapter 7). Converging methods offer a challenging but promising and novel approach to exploring brain-behaviour relationships from micro- to macroscopic levels of analysis (Chapter 8).

Group diversity and group identification:the moderating role of diversity beliefs

Research on diversity in teams and organizations has revealed ambiguous results regarding the effects of group composition on workgroup performance. The categorization—elaboration model (van Knippenberg et al., 2004) accounts for this variety and proposes two different underlying processes. On the one hand diversity may bring about intergroup bias which leads to less group identification, which in turn is followed by more conflict and decreased workgroup performance. On the other hand, the information processing approach proposes positive effects of diversity because of a more elaborate processing of information brought about by a wider pool and variety of perspectives in more diverse groups. We propose that the former process is contingent on individual team members' beliefs that diversity is good or bad for achieving the team's aims. We predict that the relationship between subjective diversity and identification is more positive in ethnically diverse project teams when group members hold beliefs that are pro-diversity. Results of two longitudinal studies involving postgraduate students working in project teams confirm this hypothesis. Analyses further reveal that group identification is positively related to students' desire to stay in their groups and to their information elaboration. Finally, we found evidence for the expected moderated mediation model with indirect effects of subjective diversity on elaboration and the desire to stay, mediated through group identification, moderated by diversity beliefs.

Disordered connectivity in the autistic brain:challenges for the 'new psychophysiology'

In 2002, we published a paper [Brock, J., Brown, C., Boucher, J., Rippon, G., 2002. The temporal binding deficit hypothesis of autism. Development and Psychopathology 142, 209-224] highlighting the parallels between the psychological model of 'central coherence' in information processing [Frith, U., 1989. Autism: Explaining the Enigma. Blackwell, Oxford] and the neuroscience model of neural integration or 'temporal binding'. We proposed that autism is associated with abnormalities of information integration that is caused by a reduction in the connectivity between specialised local neural networks in the brain and possible overconnectivity within the isolated individual neural assemblies. The current paper updates this model, providing a summary of theoretical and empirical advances in research implicating disordered connectivity in autism. This is in the context of changes in the approach to the core psychological deficits in autism, of greater emphasis on 'interactive specialisation' and the resultant stress on early and/or low-level deficits and their cascading effects on the developing brain [Johnson, M.H., Halit, H., Grice, S.J., Karmiloff-Smith, A., 2002. Neuroimaging of typical and atypical development: a perspective from multiple levels of analysis. Development and Psychopathology 14, 521-536].We also highlight recent developments in the measurement and modelling of connectivity, particularly in the emerging ability to track the temporal dynamics of the brain using electroencephalography (EEG) and magnetoencephalography (MEG) and to investigate the signal characteristics of this activity. This advance could be particularly pertinent in testing an emerging model of effective connectivity based on the balance between excitatory and inhibitory cortical activity [Rubenstein, J.L., Merzenich M.M., 2003. Model of autism: increased ratio of excitation/inhibition in key neural systems. Genes, Brain and Behavior 2, 255-267; Brown, C., Gruber, T., Rippon, G., Brock, J., Boucher, J., 2005. Gamma abnormalities during perception of illusory figures in autism. Cortex 41, 364-376]. Finally, we note that the consequence of this convergence of research developments not only enables a greater understanding of autism but also has implications for prevention and remediation. © 2006.

All-optical nonlinear signal processing at a RZ receiver

We propose a novel all-optical signal processor for use at a return-to-zero receiver utilising loop mirror intensity filtering and nonlinear pulse broadening in normal dispersion fibre. The device offers reamplification and cleaning up of the optical signals, and phase margin improvement. The efficiency of the technique is demonstrated by application to 40 Gbit/s data transmission.

Enhancing creativity in a general work environment : the role of problem-solving demand

Prior research suggests management can employ cognitively demanding job attributes to promote employee creativity. However, it is not clear what specific type of cognitive demand is particularly important for creativity, what processes underpin the relationship between demanding job conditions and creativity and what factors lead to employee perceptions of demanding job attributes. This research sets out to address the aforementioned issues by examining: (i) problem-solving demand (PDS), a specific type of cognitive demand, and the processes that link PSD to creativity, and (ii) antecedents to PSD. Based on social cognitive theory, PSD was hypothesized to be positively related to creativity through the motivational mechanism of creative self-efficacy. However, the relationship between PSD and creative self-efficacy was hypothesized to be contingent on levels of intrinsic motivation. Social information processing perspective and the job crafting model were used to identify antecedents of PSD. Consequently, two social-contextual factors (supervisor developmental feedback and job autonomy) and one individual factor (proactive personality) were hypothesized to be precursors to PSD perceptions. The theorized model was tested with data obtained from a sample of 270 employees and their supervisors from 3 organisations in the People’s Republic of China. Regression results revealed that PSD was positively related to creativity but this relationship was partially mediated by creative self-efficacy. Additionally, intrinsic motivation moderated the relationship between PSD and creative self-efficacy such that the relationship was stronger for individuals high rather than low in intrinsic motivation. The findings represent a productive first step in identifying a specific cognitive demand that is conducive to employee creativity. In addition, the findings contribute to the literature by identifying a psychological mechanism that may link cognitively demanding job attributes and creativity.

Adaptive electrical signal post-processing in optical communications systems

Improving bit error rates in optical communication systems is a difficult and important problem. The error correction must take place at high speed and be extremely accurate. We show the feasibility of using hardware implementable machine learning techniques. This may enable some error correction at the speed required.

Hand somatosensory subcortical and cortical sources assessed by functional source separation:an EEG study

We propose a novel electroencephalographic application of a recently developed cerebral source extraction method (Functional Source Separation, FSS), which starts from extracranial signals and adds a functional constraint to the cost function of a basic independent component analysis model without requiring solutions to be independent. Five ad-hoc functional constraints were used to extract the activity reflecting the temporal sequence of sensory information processing along the somatosensory pathway in response to the separate left and right median nerve galvanic stimulation. Constraints required only the maximization of the responsiveness at specific latencies following sensory stimulation, without taking into account that any frequency or spatial information. After source extraction, the reliability of identified FS was assessed based on the position of single dipoles fitted on its retroprojected signals and on a discrepancy measure. The FS positions were consistent with previously reported data (two early subcortical sources localized in the brain stem and thalamus, the three later sources in cortical areas), leaving negligible residual activity at the corresponding latencies. The high-frequency component of the oscillatory activity (HFO) of the extracted component was analyzed. The integrity of the low amplitude HFOs was preserved for each FS. On the basis of our data, we suggest that FSS can be an effective tool to investigate the HFO behavior of the different neuronal pools, recruited at successive times after median nerve galvanic stimulation. As FSs are reconstructed along the entire experimental session, directional and dynamic HFO synchronization phenomena can be studied.