Improving neural network training solutions using regularisation

This paper describes the application of regularisation to the training of feedforward neural networks, as a means of improving the quality of solutions obtained. The basic principles of regularisation theory are outlined for both linear and nonlinear training and then extended to cover a new hybrid training algorithm for feedforward neural networks recently proposed by the authors. The concept of functional regularisation is also introduced and discussed in relation to MLP and RBF networks. The tendency for the hybrid training algorithm and many linear optimisation strategies to generate large magnitude weight solutions when applied to ill-conditioned neural paradigms is illustrated graphically and reasoned analytically. While such weight solutions do not generally result in poor fits, it is argued that they could be subject to numerical instability and are therefore undesirable. Using an illustrative example it is shown that, as well as being beneficial from a generalisation perspective, regularisation also provides a means for controlling the magnitude of solutions. (C) 2001 Elsevier Science B.V. All rights reserved.

System oriented neural networks - Problem formulation, methodology and application

A novel methodology is proposed for the development of neural network models for complex engineering systems exhibiting nonlinearity. This method performs neural network modeling by first establishing some fundamental nonlinear functions from a priori engineering knowledge, which are then constructed and coded into appropriate chromosome representations. Given a suitable fitness function, using evolutionary approaches such as genetic algorithms, a population of chromosomes evolves for a certain number of generations to finally produce a neural network model best fitting the system data. The objective is to improve the transparency of the neural networks, i.e. to produce physically meaningful

Microscopical evaluation of neural connectivity between paired stages of Eudiplozoon nipponicum (Monogenea : Diplozoidae)

Diplozoidae monogeneans are fish-gill ectoparasites comprising 2 individuals fused in so-called permanent copula. This unique situation occurs when 2 larvae (diporpae) make contact on the host gill, such that their union triggers maturation into an individual adult worm. The present study examined paired stages of Eudiplozoon nipponicum microscopically to ascertain whether somatic fusion involves neural connectivity between these 2 heterogenic larvae. Neuronal pathways were demonstrated in whole-mount preparations of the worm, using indirect immunocytochemical techniques interfaced with confocal scanning laser microscopy for peptidergic and serotoninergic innervations and enzyme cytochemical methodology and light microscopy for cholinergic components. Elements of the central nervous systems of paired worms are connected by commissures the region of fusion so that the 2 systems are in structural continuity. Interindividual connections were most apparent between corresponding ventral nerve cords. All 3 classes of neuronal mediators were identified throughout both central and peripheral connections of the 2 nervous systems. The anatomical complexity and apparent plasticity of the diplozoon nervous system suggest that it has a pivotal role not only in motility, feeding, and reproductive behaviors but also in the events of larval pairing and somatic fusion.

The sites of neural adaptation induced by resistance training in humans

Although it has long been supposed that resistance training causes adaptive changes in the CNS, the sites and nature of these adaptations have not previously been identified. In order to determine whether the neural adaptations to resistance training occur to a greater extent at cortical or subcortical sites in the CNS, we compared the effects of resistance training on the electromyographic (EMG) responses to transcranial magnetic (TMS) and electrical (TES) stimulation. Motor evoked potentials (MEPs) were recorded from the first dorsal interosseous muscle of 16 individuals before and after 4 weeks of resistance training for the index finger abductors (n=8), or training involving finger abduction-adduction without external resistance (n=8). TMS was delivered at rest at intensities from 5% below the passive threshold to the maximal output of the stimulator. TMS and TES were also delivered at the active threshold intensity while the participants exerted torques ranging from 5 to 60% of their maximum voluntary contraction (MVC) torque. The average latency of MEPs elicited by TES was significantly shorter than that of TMS MEPs (TES latency=21.5+/-1.4 ms; TMS latency=23.4+/-1.4 ms; P

Neural compensation for compliant loads during rhythmic movement

An experiment was performed to characterise the movement kinematics and the electromyogram (EMG) during rhythmic voluntary flexion and extension of the wrist against different compliant (elastic-viscous-inertial) loads. Three levels of each type of load, and an unloaded condition, were employed. The movements were paced at a frequency of I Hz by an auditory metronome, and visual feedback of wrist displacement in relation to a target amplitude of 100degrees was provided. Electro-myographic recordings were obtained from flexor carpi radialis (FCR) and extensor carpi radialis brevis (ECR). The movement profiles generated in the ten experimental conditions were indistinguishable, indicating that the CNS was able to compensate completely for the imposed changes in the task dynamics. When the level of viscous load was elevated, this compensation took the form of an increase in the rate of initial rise of the flexor and the extensor EMG burst. In response to increases in inertial load, the flexor and extensor EMG bursts commenced and terminated earlier in the movement cycle, and tended to be of greater duration. When the movements were performed in opposition to an elastic load, both the onset and offset of EMG activity occurred later than in the unloaded condition. There was also a net reduction in extensor burst duration with increases in elastic load, and an increase in the rate of initial rise of the extensor burst. Less pronounced alterations in the rate of initial rise of the flexor EMG burst were also observed. In all instances, increases in the magnitude of the external load led to elevations in the overall level of muscle activation. These data reveal that the elements of the central command that are modified in response to the imposition of a compliant load are contingent, not only upon the magnitude, but also upon the character of the load.

Neural adaptations to resistance training - Implications for movement control

It has long been believed that resistance training is accompanied by changes within the nervous system that play an important role in the development of strength. Many elements of the nervous system exhibit the potential for adaptation in response to resistance training, including supraspinal centres, descending neural tracts, spinal circuitry and the motor end plate connections between motoneurons and muscle fibres. Yet the specific sites of adaptation along the neuraxis have seldom been identified experimentally, and much of the evidence for neural adaptations following resistance training remains indirect. As a consequence of this current lack of knowledge, there exists uncertainty regarding the manner in which resistance training impacts upon the control and execution of functional movements. We aim to demonstrate that resistance training is likely to cause adaptations to many neural elements that are involved in the control of movement, and is therefore likely to affect movement execution during a wide range of tasks.