Dynamics of on-line gradient descent learning for multilayer neural networks

We consider the problem of on-line gradient descent learning for general two-layer neural networks. An analytic solution is presented and used to investigate the role of the learning rate in controlling the evolution and convergence of the learning process.

On-line learning in multilayer neural networks

We present an analytic solution to the problem of on-line gradient-descent learning for two-layer neural networks with an arbitrary number of hidden units in both teacher and student networks. The technique, demonstrated here for the case of adaptive input-to-hidden weights, becomes exact as the dimensionality of the input space increases.

Adaptive back-propagation in on-line learning of multilayer networks

An adaptive back-propagation algorithm is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, both numerical studies and a rigorous analysis show that the adaptive back-propagation method results in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent.

Learning with noise and regularizers in multilayer neural networks

We study the effect of two types of noise, data noise and model noise, in an on-line gradient-descent learning scenario for general two-layer student network with an arbitrary number of hidden units. Training examples are randomly drawn input vectors labeled by a two-layer teacher network with an arbitrary number of hidden units. Data is then corrupted by Gaussian noise affecting either the output or the model itself. We examine the effect of both types of noise on the evolution of order parameters and the generalization error in various phases of the learning process.

Finite-size effects in on-line learning of multilayer neural networks

We complement recent advances in thermodynamic limit analyses of mean on-line gradient descent learning dynamics in multi-layer networks by calculating fluctuations possessed by finite dimensional systems. Fluctuations from the mean dynamics are largest at the onset of specialisation as student hidden unit weight vectors begin to imitate specific teacher vectors, increasing with the degree of symmetry of the initial conditions. In light of this, we include a term to stimulate asymmetry in the learning process, which typically also leads to a significant decrease in training time.

Learning with regularizers in multilayer neural networks

We study the effect of regularization in an on-line gradient-descent learning scenario for a general two-layer student network with an arbitrary number of hidden units. Training examples are randomly drawn input vectors labelled by a two-layer teacher network with an arbitrary number of hidden units which may be corrupted by Gaussian output noise. We examine the effect of weight decay regularization on the dynamical evolution of the order parameters and generalization error in various phases of the learning process, in both noiseless and noisy scenarios.

Globally optimal parameters for on-line learning in multilayer neural networks

We present a framework for calculating globally optimal parameters, within a given time frame, for on-line learning in multilayer neural networks. We demonstrate the capability of this method by computing optimal learning rates in typical learning scenarios. A similar treatment allows one to determine the relevance of related training algorithms based on modifications to the basic gradient descent rule as well as to compare different training methods.

Globally optimal learning rates in multilayer neural networks

A method for calculating the globally optimal learning rate in on-line gradient-descent training of multilayer neural networks is presented. The method is based on a variational approach which maximizes the decrease in generalization error over a given time frame. We demonstrate the method by computing optimal learning rates in typical learning scenarios. The method can also be employed when different learning rates are allowed for different parameter vectors as well as to determine the relevance of related training algorithms based on modifications to the basic gradient descent rule.

Enterprise networking:multilayer switching and applications

Primarily targeted toward the network or MIS manager who wants to stay abreast of the latest networking technology, Enterprise Networking: Multilayer Switching and Applications offers up to date information relevant for the design of modern corporate networks and for the evaluation of new networking equipment. The book describes the architectures and standards of switching across the various protocol layers and will also address issues such as multicast quality of service, high-availability and network policies that are requirements of modern switched networks.

Exploitation of multilayer coatings for infrared surface plasmon resonance fiber sensors

We demonstrate surface plasmon resonance (SPR) fiber devices based upon ultraviolet inscription of a grating-type structure into both single-layered and multilayered thin films deposited on the flat side of a lapped D-shaped fiber. The single-layered devices were fabricated from germanium, while the multilayered ones comprised layers of germanium, silica, and silver. Some of the devices operated in air with high coupling efficiency in excess of 40 dB and an estimated index sensitivity of Delta lambda/Delta n = 90 mn from 1 to 1.15 index range, while others provided an index sensitivity of Delta lambda/Delta n = 6790 mn for refractive indices from 1.33 to 1.37. (C) 2009 Optical Society of America

The application of fibre Bragg grating networks as strain sensors and as phased array antenna true time delay elements

The fabrication of in-fibre Bragg gratings, and the application of arrays of such gratings as strain sensors and as true time delay elements for the control of phased array antennas is reported. Chirped period Bragg gratings were produced using the fibre deformation fabrication technique, with chirps of between 2.9nm and 17.3nm achieved. Arrays of 5mm and 2mm long uniform period Bragg gratings were fabricated using the inscription method, for use as true time delay elements,dissimilar wavefronts and their spectral characteristics recorded. The uniform period grating arrays were used to create minimum time delays of 9.09ps, 19.02ps and 31ps; making them suitable for controlling phased array antennas operating at RF frequencies of up to 3GHz, with 10° phase resolution. Four 4mm long chirped gratings were produced using the dissimilar wavefronts fabrication method, having chirps of 7nm, 12nm, 20nm and 30nm, and were used to create time delays of between 0.3ps and 59ps. Hence they are suitable for controlling phased array antennas at RF frequencies of up to 48GHz. The application of in fibre Bragg gratings as strain sensors within smart structure materials was investigated, with their sensitivity to applied strain and compression measured for both embedded and surface mounted uniform period and fibre Fabry-Perot filter gratings. A fibre Bragg grating sensor demultiplexing scheme based on a liquid crystal filled Fabry-Perot etalon tuneable transmission filter was proposed, successfully constructed and fully characterised. Three characteristics of the LCFP etalon were found to pose operational limitations to its application in a Bragg grating sensor system; most significantly, the resonance peak wavelength was highly (-2,77nm/°C) temperature dependent. Several methods for minimising this temperature sensitivity were investigated, but enjoyed only limited success. It was therefore concluded that this type (E7 filled) of LCFP etalon is unsuitable for use as a Bragg grating sensor demultiplexing element.

Computational capabilities of multilayer committee machines

We obtained an analytical expression for the computational complexity of many layered committee machines with a finite number of hidden layers (L < 8) using the generalization complexity measure introduced by Franco et al (2006) IEEE Trans. Neural Netw. 17 578. Although our result is valid in the large-size limit and for an overlap synaptic matrix that is ultrametric, it provides a useful tool for inferring the appropriate architecture a network must have to reproduce an arbitrary realizable Boolean function.

Effect of the base station antenna beam tilting on energy consumption in cellular networks

The objective of this paper is to combine the antenna downtilt selection with the cell size selection in order to reduce the overall radio frequency (RF) transmission power in the homogeneous High-Speed Packet Downlink (HSDPA) cellular radio access network (RAN). The analysis is based on the concept of small cells deployment. The energy consumption ratio (ECR) and the energy reduction gain (ERG) of the cellular RAN are calculated for different antenna tilts when the cell size is being reduced for a given user density and service area. The results have shown that a suitable antenna tilt and the RF power setting can achieve an overall energy reduction of up to 82.56%. Equally, our results demonstrate that a small cell deployment can considerably reduce the overall energy consumption of a cellular network.

Large extraordinary Hall effect in [Pt/Co]5/Ru/[Co/Pt]5 multilayer

This Brief Report presents giant extraordinary Hall effect (EHE) in the Ru-mediated antiferromagnetically coupled [Pt/Co]5/Ru/[Co/Pt]5 multilayers (MLs) compared with those MLs without the Ru spacer. The enhancement of the EHE is attributed to the strong Ru/Co interface scattering. Through the variation in the Pt layer thickness and the temperature, we determine the relation between the Hall voltage and the longitudinal resistivity. It is found that the conventional scaling analysis has difficulties in consistently interpreting our data.

Numerical modelling of multilayer semiconductor devices

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT