Description and training of neural network dynamics

Attractor properties of a popular discrete-time neural network model are illustrated through numerical simulations. The most complex dynamics is found to occur within particular ranges of parameters controlling the symmetry and magnitude of the weight matrix. A small network model is observed to produce fixed points, limit cycles, mode-locking, the Ruelle-Takens route to chaos, and the period-doubling route to chaos. Training algorithms for tuning this dynamical behaviour are discussed. Training can be an easy or difficult task, depending whether the problem requires the use of temporal information distributed over long time intervals. Such problems require training algorithms which can handle hidden nodes. The most prominent of these algorithms, back propagation through time, solves the temporal credit assignment problem in a way which can work only if the relevant information is distributed locally in time. The Moving Targets algorithm works for the more general case, but is computationally intensive, and prone to local minima.

Water network dynamics at the critical moment of a peptide's ß-turn formation:a molecular dynamics study

All-atom molecular dynamics simulations for a single molecule of Leu-Enkephalin in aqueous solution have been used to study the role of the water network during the formation of ß-turns. We give a detailed account of the intramolecular hydrogen bonding, the water-peptide hydrogen bonding, and the orientation and residence times of water molecules focusing on the short critical periods of transition to the stable ß-turns. These studies suggest that, when intramolecular hydrogen bonding between the first and fourth residue of the ß-turn is not present, the disruption of the water network and the establishment of water bridges constitute decisive factors in the formation and stability of the ß-turn. Finally, we provide possible explanations and mechanisms for the formations of different kinds of ß-turns.

Neuronal network dynamics during epileptogenesis in the medial temporal lobe

Epilepsy is one of the most common neurological disorders, a large fraction of which is resistant to pharmacotherapy. In this light, understanding the mechanisms of epilepsy and its intractable forms in particular could create new targets for pharmacotherapeutic intervention. The current project explores the dynamic changes in neuronal network function in the chronic temporal lobe epilepsy (TLE) in rat and human brain in vitro. I focused on the process of establishment of epilepsy (epileptogenesis) in the temporal lobe. Rhythmic behaviour of the hippocampal neuronal networks in healthy animals was explored using spontaneous oscillations in the gamma frequency band (SγO). The use of an improved brain slice preparation technique resulted in the natural occurence (in the absence of pharmacological stimulation) of rhythmic activity, which was then pharmacologically characterised and compared to other models of gamma oscillations (KA- and CCh-induced oscillations) using local field potential recording technique. The results showed that SγO differed from pharmacologically driven models, suggesting higher physiological relevance of SγO. Network activity was also explored in the medial entorhinal cortex (mEC), where spontaneous slow wave oscillations (SWO) were detected. To investigate the course of chronic TLE establishment, a refined Li-pilocarpine-based model of epilepsy (RISE) was developed. The model significantly reduced animal mortality and demonstrated reduced intensity, yet high morbidy with almost 70% mean success rate of developing spontaneous recurrent seizures. We used SγO to characterize changes in the hippocampal neuronal networks throughout the epileptogenesis. The results showed that the network remained largely intact, demonstrating the subtle nature of the RISE model. Despite this, a reduction in network activity was detected during the so-called latent (no seizure) period, which was hypothesized to occur due to network fragmentation and an abnormal function of kainate receptors (KAr). We therefore explored the function of KAr by challenging SγO with kainic acid (KA). The results demonstrated a remarkable decrease in KAr response during the latent period, suggesting KAr dysfunction or altered expression, which will be further investigated using a variety of electrophysiological and immunocytochemical methods. The entorhinal cortex, together with the hippocampus, is known to play an important role in the TLE. Considering this, we investigated neuronal network function of the mEC during epileptogenesis using SWO. The results demonstrated a striking difference in AMPAr function, with possible receptor upregulation or abnormal composition in the early development of epilepsy. Alterations in receptor function inevitably lead to changes in the network function, which may play an important role in the development of epilepsy. Preliminary investigations were made using slices of human brain tissue taken following surgery for intratctable epilepsy. Initial results showed that oscillogenesis could be induced in human brain slices and that such network activity was pharmacologically similar to that observed in rodent brain. Overall, our findings suggest that excitatory glutamatergic transmission is heavily involved in the process of epileptogenesis. Together with other types of receptors, KAr and AMPAr contribute to epilepsy establishment and may be the key to uncovering its mechanism.

Can we observe collective neuronal activity from macroscopic aggregate signals?

The fundamental problem faced by noninvasive neuroimaging techniques such as EEG/MEG1 is to elucidate functionally important aspects of the microscopic neuronal network dynamics from macroscopic aggregate measurements. Due to the mixing of the activities of large neuronal populations in the observed macroscopic aggregate, recovering the underlying network that generates the signal in the absence of any additional information represents a considerable challenge. Recent MEG studies have shown that macroscopic measurements contain sufficient information to allow the differentiation between patterns of activity, which are likely to represent different stimulus-specific collective modes in the underlying network (Hadjipapas, A., Adjamian, P., Swettenham, J.B., Holliday, I.E., Barnes, G.R., 2007. Stimuli of varying spatial scale induce gamma activity with distinct temporal characteristics in human visual cortex. NeuroImage 35, 518–530). The next question arising in this context is whether aspects of collective network activity can be recovered from a macroscopic aggregate signal. We propose that this issue is most appropriately addressed if MEG/EEG signals are to be viewed as macroscopic aggregates arising from networks of coupled systems as opposed to aggregates across a mass of largely independent neural systems. We show that collective modes arising in a network of simulated coupled systems can be indeed recovered from the macroscopic aggregate. Moreover, we show that nonlinear state space methods yield a good approximation of the number of effective degrees of freedom in the network. Importantly, information about hidden variables, which do not directly contribute to the aggregate signal, can also be recovered. Finally, this theoretical framework can be applied to experimental MEG/EEG data in the future, enabling the inference of state dependent changes in the degree of local synchrony in the underlying network.

Human brain slices for epilepsy research:pitfalls, solutions and future challenges

Increasingly, neuroscientists are taking the opportunity to use live human tissue obtained from elective neurosurgical procedures for electrophysiological studies in vitro. Access to this valuable resource permits unique studies into the network dynamics that contribute to the generation of pathological electrical activity in the human epileptic brain. Whilst this approach has provided insights into the mechanistic features of electrophysiological patterns associated with human epilepsy, it is not without technical and methodological challenges. This review outlines the main difficulties associated with working with epileptic human brain slices from the point of collection, through the stages of preparation, storage and recording. Moreover, it outlines the limitations, in terms of the nature of epileptic activity that can be observed in such tissue, in particular, the rarity of spontaneous ictal discharges, we discuss manipulations that can be utilised to induce such activity. In addition to discussing conventional electrophysiological techniques that are routinely employed in epileptic human brain slices, we review how imaging and multielectrode array recordings could provide novel insights into the network dynamics of human epileptogenesis. Acute studies in human brain slices are ultimately limited by the lifetime of the tissue so overcoming this issue provides increased opportunity for information gain. We review the literature with respect to organotypic culture techniques that may hold the key to prolonging the viability of this material. A combination of long-term culture techniques, viral transduction approaches and electrophysiology in human brain slices promotes the possibility of large scale monitoring and manipulation of neuronal activity in epileptic microcircuits.

A low mortality, high morbidity Reduced Intensity Status Epilepticus (RISE) model of epilepsy and epileptogenesis in the rat

Animal models of acquired epilepsies aim to provide researchers with tools for use in understanding the processes underlying the acquisition, development and establishment of the disorder. Typically, following a systemic or local insult, vulnerable brain regions undergo a process leading to the development, over time, of spontaneous recurrent seizures. Many such models make use of a period of intense seizure activity or status epilepticus, and this may be associated with high mortality and/or global damage to large areas of the brain. These undesirable elements have driven improvements in the design of chronic epilepsy models, for example the lithium-pilocarpine epileptogenesis model. Here, we present an optimised model of chronic epilepsy that reduces mortality to 1% whilst retaining features of high epileptogenicity and development of spontaneous seizures. Using local field potential recordings from hippocampus in vitro as a probe, we show that the model does not result in significant loss of neuronal network function in area CA3 and, instead, subtle alterations in network dynamics appear during a process of epileptogenesis, which eventually leads to a chronic seizure state. The model’s features of very low mortality and high morbidity in the absence of global neuronal damage offer the chance to explore the processes underlying epileptogenesis in detail, in a population of animals not defined by their resistance to seizures, whilst acknowledging and being driven by the 3Rs (Replacement, Refinement and Reduction of animal use in scientific procedures) principles.

Quantifying short-term dynamics of Parkinson's disease using self-reported symptom data from an internet social network

Background: Parkinson’s disease (PD) is an incurable neurological disease with approximately 0.3% prevalence. The hallmark symptom is gradual movement deterioration. Current scientific consensus about disease progression holds that symptoms will worsen smoothly over time unless treated. Accurate information about symptom dynamics is of critical importance to patients, caregivers, and the scientific community for the design of new treatments, clinical decision making, and individual disease management. Long-term studies characterize the typical time course of the disease as an early linear progression gradually reaching a plateau in later stages. However, symptom dynamics over durations of days to weeks remains unquantified. Currently, there is a scarcity of objective clinical information about symptom dynamics at intervals shorter than 3 months stretching over several years, but Internet-based patient self-report platforms may change this. Objective: To assess the clinical value of online self-reported PD symptom data recorded by users of the health-focused Internet social research platform PatientsLikeMe (PLM), in which patients quantify their symptoms on a regular basis on a subset of the Unified Parkinson’s Disease Ratings Scale (UPDRS). By analyzing this data, we aim for a scientific window on the nature of symptom dynamics for assessment intervals shorter than 3 months over durations of several years. Methods: Online self-reported data was validated against the gold standard Parkinson’s Disease Data and Organizing Center (PD-DOC) database, containing clinical symptom data at intervals greater than 3 months. The data were compared visually using quantile-quantile plots, and numerically using the Kolmogorov-Smirnov test. By using a simple piecewise linear trend estimation algorithm, the PLM data was smoothed to separate random fluctuations from continuous symptom dynamics. Subtracting the trends from the original data revealed random fluctuations in symptom severity. The average magnitude of fluctuations versus time since diagnosis was modeled by using a gamma generalized linear model. Results: Distributions of ages at diagnosis and UPDRS in the PLM and PD-DOC databases were broadly consistent. The PLM patients were systematically younger than the PD-DOC patients and showed increased symptom severity in the PD off state. The average fluctuation in symptoms (UPDRS Parts I and II) was 2.6 points at the time of diagnosis, rising to 5.9 points 16 years after diagnosis. This fluctuation exceeds the estimated minimal and moderate clinically important differences, respectively. Not all patients conformed to the current clinical picture of gradual, smooth changes: many patients had regimes where symptom severity varied in an unpredictable manner, or underwent large rapid changes in an otherwise more stable progression. Conclusions: This information about short-term PD symptom dynamics contributes new scientific understanding about the disease progression, currently very costly to obtain without self-administered Internet-based reporting. This understanding should have implications for the optimization of clinical trials into new treatments and for the choice of treatment decision timescales.

Random walks in local dynamics of network losses

We suggest a model for data losses in a single node (memory buffer) of a packet-switched network (like the Internet) which reduces to one-dimensional discrete random walks with unusual boundary conditions. By construction, the model has critical behavior with a sharp transition from exponentially small to finite losses with increasing data arrival rate. We show that for a finite-capacity buffer at the critical point the loss rate exhibits strong fluctuations and non-Markovian power-law correlations in time, in spite of the Markovian character of the data arrival process.

Network evolution and the spatiotemporal dynamics of knowledge sourcing

Knowledge accessing from external organisations is important to firms, especially entrepreneurial ones which often cannot generate internally all the knowledge necessary for innovation. There is, however, a lack of evidence concerning the association between the evolution of firms and the evolution of their networks. The aim of this paper is to begin to fill this gap by undertaking an exploratory analysis of the relationship between the vintage of firms and their knowledge sourcing networks. Drawing on an analysis of firms in the UK, the paper finds some evidence of a U-shaped relationship existing between firm age and the frequency of accessing knowledge from certain sources. Emerging entrepreneurial firms tend to be highly active with regard to accessing knowledge for a range of sources and geographic locations, with the rate of networking dropping somewhat during the period of peak firm growth. For instance, it is found that firms tend to less frequently access knowledge sources such as universities and research institutes in their own region during a stage of peak turnover growth. Overall, the results suggest a complex relationship between the lifecycle of the firm and its networking patterns. It is concluded that policymakers need to become more aware that network formation and utilisation by firms is likely to vary dependent upon their lifecycle position.

The learning dynamics of a universal approximator

The learning properties of a universal approximator, a normalized committee machine with adjustable biases, are studied for on-line back-propagation learning. Within a statistical mechanics framework, numerical studies show that this model has features which do not exist in previously studied two-layer network models without adjustable biases, e.g., attractive suboptimal symmetric phases even for realizable cases and noiseless data.

Dynamics of on-line learning in radial basis function networks

On-line learning is examined for the radial basis function network, an important and practical type of neural network. The evolution of generalization error is calculated within a framework which allows the phenomena of the learning process, such as the specialization of the hidden units, to be analyzed. The distinct stages of training are elucidated, and the role of the learning rate described. The three most important stages of training, the symmetric phase, the symmetry-breaking phase, and the convergence phase, are analyzed in detail; the convergence phase analysis allows derivation of maximal and optimal learning rates. As well as finding the evolution of the mean system parameters, the variances of these parameters are derived and shown to be typically small. Finally, the analytic results are strongly confirmed by simulations.

Bidirectional transport between the trans-Golgi network and the endosomal system

The exchange of proteins and lipids between the trans-Golgi network (TGN) and the endosomal system requires multiple cellular machines, whose activities are coordinated in space and time to generate pleomorphic, tubulo-vesicular carriers that deliver their content to their target compartments. These machines and their associated protein networks are recruited and/or activated on specific membrane domains where they select proteins and lipids into carriers, contribute to deform/elongate and partition membrane domains using the mechanical forces generated by actin polymerization or movement along microtubules. The coordinated action of these protein networks contributes to regulate the dynamic state of multiple receptors recycling between the cell surface, endosomes and the TGN, to maintain cell homeostasis as exemplified by the biogenesis of lysosomes and related organelles, and to establish/maintain cell polarity. The dynamic assembly and disassembly of these protein networks mediating the exchange of membrane domains between the TGN and endosomes regulates cell-cell signalling and thus the development of multi-cellular organisms. Somatic mutations in single network components lead to changes in transport dynamics that may contribute to pathological modifications underlying several human diseases such as mental retardation.

The spatial distribution and temporal dynamics of brain regions activated during the perception of object and non-object patterns

Both animal and human studies suggest that the efficiency with which we are able to grasp objects is attributable to a repertoire of motor signals derived directly from vision. This is in general agreement with the long-held belief that the automatic generation of motor signals by the perception of objects is based on the actions they afford. In this study, we used magnetoencephalography (MEG) to determine the spatial distribution and temporal dynamics of brain regions activated during passive viewing of object and non-object targets that varied in the extent to which they afforded a grasping action. Synthetic Aperture Magnetometry (SAM) was used to localize task-related oscillatory power changes within specific frequency bands, and the time course of activity within given regions-of-interest was determined by calculating time-frequency plots using a Morlet wavelet transform. Both single subject and group-averaged data on the spatial distribution of brain activity are presented. We show that: (i) significant reductions in 10-25 Hz activity within extrastriate cortex, occipito-temporal cortex, sensori-motor cortex and cerebellum were evident with passive viewing of both objects and non-objects; and (ii) reductions in oscillatory activity within the posterior part of the superior parietal cortex (area Ba7) were only evident with the perception of objects. Assuming that focal reductions in low-frequency oscillations (< 30 Hz) reflect areas of heightened neural activity, we conclude that: (i) activity within a network of brain areas, including the sensori-motor cortex, is not critically dependent on stimulus type and may reflect general changes in visual attention; and (ii) the posterior part of the superior parietal cortex, area Ba7, is activated preferentially by objects and may play a role in computations related to grasping. © 2006 Elsevier Inc. All rights reserved.

A molecular dynamics model of the atomic structure of dysprosium alumino-phosphate glass

Molecular dynamics (MD) has been used to identify the relative distribution of dysprosium in the phosphate glass DyAl0.30P3.05O9.62. The MD model has been compared directly with experimental data obtained from neutron diffraction to enable a detailed comparison beyond the total structure factor level. The MD simulation gives Dy ... Dy correlations at 3.80(5) and 6.40(5) angstrom with relative coordination numbers of 0.8(1) and 7.3(5), thus providing evidence of minority rare-earth clustering within these glasses. The nearest neighbour Dy-O peak occurs at 2.30 angstrom with each Dy atom having on average 5.8 nearest neighbour oxygen atoms. The MD simulation is consistent with the phosphate network model based on interlinked PO4 tetrahedra where the addition of network modifiers Dy3+ depolymerizes the phosphate network through the breakage of P-(O)-P bonds whilst leaving the tetrahedral units intact. The role of aluminium within the network has been taken into explicit account, and A1 is found to be predominantly (78 tetrahedrally coordinated. In fact all four A1 bonds are found to be to P (via an oxygen atom) with negligible amounts of Al-O-Dy bonds present. This provides an important insight into the role of Al additives in improving the mechanical properties of these glasses.