Adaptive back-propagation in on-line learning of multilayer networks

An adaptive back-propagation algorithm is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, both numerical studies and a rigorous analysis show that the adaptive back-propagation method results in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent.

On-line learning with adaptive back-propagation in two-layer networks

An adaptive back-propagation algorithm parameterized by an inverse temperature 1/T is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, we analyse these learning algorithms in both the symmetric and the convergence phase for finite learning rates in the case of uncorrelated teachers of similar but arbitrary length T. These analyses show that adaptive back-propagation results generally in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent.

A constructive learning algorithm based on back-propagation

There are been a resurgence of interest in the neural networks field in recent years, provoked in part by the discovery of the properties of multi-layer networks. This interest has in turn raised questions about the possibility of making neural network behaviour more adaptive by automating some of the processes involved. Prior to these particular questions, the process of determining the parameters and network architecture required to solve a given problem had been a time consuming activity. A number of researchers have attempted to address these issues by automating these processes, concentrating in particular on the dynamic selection of an appropriate network architecture.The work presented here specifically explores the area of automatic architecture selection; it focuses upon the design and implementation of a dynamic algorithm based on the Back-Propagation learning algorithm. The algorithm constructs a single hidden layer as the learning process proceeds using individual pattern error as the basis of unit insertion. This algorithm is applied to several problems of differing type and complexity and is found to produce near minimal architectures that are shown to have a high level of generalisation ability.

Compensation of intra-channel nonlinear fibre impairments using simplified digital back-propagation algorithm

We investigate a digital back-propagation simplification method to enable computationally-efficient digital nonlinearity compensation for a coherently-detected 112 Gb/s polarization multiplexed quadrature phase shifted keying transmission over a 1,600 km link (20x80km) with no inline compensation. Through numerical simulation, we report up to 80% reduction in required back-propagation steps to perform nonlinear compensation, in comparison to the standard back-propagation algorithm. This method takes into account the correlation between adjacent symbols at a given instant using a weighted-average approach, and optimization of the position of nonlinear compensator stage to enable practical digital back-propagation.

Impact of longitudinal power budget in coherent transmission systems employing digital back-propagation

We report the impact of longitudinal signal power profile on the transmission performance of coherently-detected 112 Gb/s m-ary polarization multiplexed quadrature amplitude modulation system after compensation of deterministic nonlinear fibre impairments. Performance improvements up to 0.6 dB (Q(eff)) are reported for a non-uniform transmission link power profile. Further investigation reveals that the evolution of the transmission performance with power profile management is fully consistent with the parametric amplification of the amplified spontaneous emission by the signal through four-wave mixing. In particular, for a non-dispersion managed system, a single-step increment of 4 dB in the amplifier gain, with respect to a uniform gain profile, at similar to 2/3(rd) of the total reach considerably improves the transmission performance for all the formats studied. In contrary a negative-step profile, emulating a failure (gain decrease or loss increase), significantly degrades the bit-error rate.

Polarization multiplexed 16QAM transmission employing modified digital back-propagation

We experimentally demonstrate performance enhancements enabled by weighted digital back propagation method for 28 Gbaud PM-16QAM transmission systems, over a 250 km ultra-large area fibre, using only one back-propagation step for the entire link, enabling up to 3 dB improvement in power tolerance with respect to linear compensation only. We observe that this is roughly the same improvement that can be obtained with the conventional, computationally heavy, non-weighted digital back propagation compensation with one step per span. As a further benchmark, we analyze performance improvement as a function of number of steps, and show that the performance improvement saturates at approximately 20 steps per span, at which a 5 dB improvement in power tolerance is obtained with respect to linear compensation only. Furthermore, we show that coarse-step self-phase modulation compensation is inefficient in wavelength division multiplexed transmission.

Impact of signal-ASE four-wave mixing on the effectiveness of digital back-propagation in 112 Gb/s PM-QPSK systems

Limitations in the performance of coherent transmission systems employing digital back-propagation due to four-wave mixing impairments are reported for the first time. A significant performance constraint is identified, originating from four-wave mixing between signals and amplified spontaneous emission noise which induces a linear increase in the standard deviation of the received field with signal power, and linear dependence on transmission distance.

Digital back-propagation for spectrally efficient WDM 112 Gbit/s PM m-ary QAM transmission

We report the performance of coherently-detected nine-channel WDM transmission over high dispersion fibers, using polarization multiplexed m-ary quadrature amplitude modulation (m = 4, 16, 64, 256) at 112 Gbit/s. Compensation of fiber nonlinearities via digital back-propagation enables up to 10 dB improvement in maximum transmittable power and similar to 8 dB Q(eff) improvement which translates to a nine-fold enhancement in transmission reach for PM-256QAM, where the largest improvements are associated with higher-order modulation formats. We further demonstrate that even under strong nonlinear distortion the transmission reach only reduces by a factor of similar to 2.5 for a 2 unit increase in capacity (log(2)m) when full band DBP is employed, in proportion to the required back-to-back OSNR.

Nonlinearity compensation via spectral inversion and digital back-propagation:a practical approach

We report performance enhancements enabled by pre-dispersed spectral inversion equivalent to that of ideal back-propagation, with further x2 increase in reach from multi-channel compensation, with spectral inversion employed upto 400km (from mid-link) with <1dB penalties. © 2012 OSA.

Impact of longitudinal power budget in coherent transmission systems employing digital back-propagation

We report the impact of longitudinal signal power profile on the transmission performance of coherently-detected 112 Gb/s m-ary polarization multiplexed quadrature amplitude modulation system after compensation of deterministic nonlinear fibre impairments. Performance improvements up to 0.6 dB (Q(eff)) are reported for a non-uniform transmission link power profile. Further investigation reveals that the evolution of the transmission performance with power profile management is fully consistent with the parametric amplification of the amplified spontaneous emission by the signal through four-wave mixing. In particular, for a non-dispersion managed system, a single-step increment of 4 dB in the amplifier gain, with respect to a uniform gain profile, at similar to 2/3(rd) of the total reach considerably improves the transmission performance for all the formats studied. In contrary a negative-step profile, emulating a failure (gain decrease or loss increase), significantly degrades the bit-error rate.

Polarization multiplexed 16QAM transmission employing modified digital back-propagation

We experimentally demonstrate performance enhancements enabled by weighted digital back propagation method for 28 Gbaud PM-16QAM transmission systems, over a 250 km ultra-large area fibre, using only one back-propagation step for the entire link, enabling up to 3 dB improvement in power tolerance with respect to linear compensation only. We observe that this is roughly the same improvement that can be obtained with the conventional, computationally heavy, non-weighted digital back propagation compensation with one step per span. As a further benchmark, we analyze performance improvement as a function of number of steps, and show that the performance improvement saturates at approximately 20 steps per span, at which a 5 dB improvement in power tolerance is obtained with respect to linear compensation only. Furthermore, we show that coarse-step self-phase modulation compensation is inefficient in wavelength division multiplexed transmission.

Unrepeatered Nyquist PDM-16QAM transmission over 364 km using Raman amplification and multi-channel digital back-propagation

Transmission of a net 467-Gb/s PDM-16QAM Nyquist-spaced superchannel is reported with an intra-superchannel net spectral efficiency (SE) of 6.6 (b/s)/Hz, over 364-km SMF-28 ULL ultra-low loss optical fiber, enabled by bi-directional second-order Raman amplification and digital nonlinearity compensation. Multi-channel digital back-propagation (MC-DBP) was applied to compensate for nonlinear interference; an improvement of 2 dB in Q2 factor was achieved when 70-GHz DBP bandwidth was applied, allowing an increase in span length of 37 km.

Modelos de propensão ao consumo baseados em rede neuronais, o caso particular do crédito pessoal

Dissertação apresentada como requisito parcial para obtenção do grau de Mestre em Estatística e Gestão de Informação

Visualization of learning in multilayer perceptron networks using principal component analysis

This paper is concerned with the use of scientific visualization methods for the analysis of feedforward neural networks (NNs). Inevitably, the kinds of data associated with the design and implementation of neural networks are of very high dimensionality, presenting a major challenge for visualization. A method is described using the well-known statistical technique of principal component analysis (PCA). This is found to be an effective and useful method of visualizing the learning trajectories of many learning algorithms such as back-propagation and can also be used to provide insight into the learning process and the nature of the error surface.

Artificial neural networks for automatic modelling of the pectus excavatum corrective prosthesis

Pectus excavatum is the most common deformity of the thorax and usually comprises Computed Tomography (CT) examination for pre-operative diagnosis. Aiming at the elimination of the high amounts of CT radiation exposure, this work presents a new methodology for the replacement of CT by a laser scanner (radiation-free) in the treatment of pectus excavatum using personally modeled prosthesis. The complete elimination of CT involves the determination of ribs external outline, at the maximum sternum depression point for prosthesis placement, based on chest wall skin surface information, acquired by a laser scanner. The developed solution resorts to artificial neural networks trained with data vectors from 165 patients. Scaled Conjugate Gradient, Levenberg-Marquardt, Resilient Back propagation and One Step Secant gradient learning algorithms were used. The training procedure was performed using the soft tissue thicknesses, determined using image processing techniques that automatically segment the skin and rib cage. The developed solution was then used to determine the ribs outline in data from 20 patient scanners. Tests revealed that ribs position can be estimated with an average error of about 6.82±5.7 mm for the left and right side of the patient. Such an error range is well below current prosthesis manual modeling (11.7±4.01 mm) even without CT imagiology, indicating a considerable step forward towards CT replacement by a 3D scanner for prosthesis personalization.