Adaptive back-propagation in on-line learning of multilayer networks

An adaptive back-propagation algorithm is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, both numerical studies and a rigorous analysis show that the adaptive back-propagation method results in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent.

On-line learning with adaptive back-propagation in two-layer networks

An adaptive back-propagation algorithm parameterized by an inverse temperature 1/T is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, we analyse these learning algorithms in both the symmetric and the convergence phase for finite learning rates in the case of uncorrelated teachers of similar but arbitrary length T. These analyses show that adaptive back-propagation results generally in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent.

Belief propagation vs. TAP for decoding corrupted messages

We employ two different methods, based on belief propagation and TAP,for decoding corrupted messages encoded by employing Sourlas's method, where the code word comprises products of K bits selected randomly from the original message. We show that the equations obtained by the two approaches are similar and provide the same solution as the one obtained by the replica approach in some cases K=2. However, we also show that for K>=3 and unbiased messages the iterative solution is sensitive to the initial conditions and is likely to provide erroneous solutions; and that it is generally beneficial to use Nishimori's temperature, especially in the case of biased messages.

Depth propagation and surface construction in 3-D vision

In stereo vision, regions with ambiguous or unspecified disparity can acquire perceived depth from unambiguous regions. This has been called stereo capture, depth interpolation or surface completion. We studied some striking induced depth effects suggesting that depth interpolation and surface completion are distinct stages of visual processing. An inducing texture (2-D Gaussian noise) had sinusoidal modulation of disparity, creating a smooth horizontal corrugation. The central region of this surface was replaced by various test patterns whose perceived corrugation was measured. When the test image was horizontal 1-D noise, shown to one eye or to both eyes without disparity, it appeared corrugated in much the same way as the disparity-modulated (DM) flanking regions. But when the test image was 2-D noise, or vertical 1-D noise, little or no depth was induced. This suggests that horizontal orientation was a key factor. For a horizontal sine-wave luminance grating, strong depth was induced, but for a square-wave grating, depth was induced only when its edges were aligned with the peaks and troughs of the DM flanking surface. These and related results suggest that disparity (or local depth) propagates along horizontal 1-D features, and then a 3-D surface is constructed from the depth samples acquired. The shape of the constructed surface can be different from the inducer, and so surface construction appears to operate on the results of a more local depth propagation process.

Depth propagation and surface construction in 3-D vision

Motion discontinuities can signal object boundaries where few or no other cues, such as luminance, colour, or texture, are available. Hence, motion-defined contours are an ecologically important counterpart to luminance contours. We developed a novel motion-defined Gabor stimulus to investigate the nature of neural operators analysing visual motion fields in order to draw parallels with known luminance operators. Luminance-defined Gabors have been successfully used to discern the spatial-extent and spatial-frequency specificity of possible visual contour detectors. We now extend these studies into the motion domain. We define a stimulus using limited-lifetime moving dots whose velocity is described over 2-D space by a Gabor pattern surrounded by randomly moving dots. Participants were asked to determine whether the orientation of the Gabor pattern (and hence of the motion contours) was vertical or horizontal in a 2AFC task, and the proportion of correct responses was recorded. We found that with practice participants became highly proficient at this task, able in certain cases to reach 90% accuracy with only 12 limited-lifetime dots. However, for both practised and novice participants we found that the ability to detect a single boundary saturates with the size of the Gaussian envelope of the Gabor at approximately 5 deg full-width at half-height. At this optimal size we then varied spatial frequency and found the optimum was at the lowest measured spatial frequency (0.1 cycle deg-1 ) and then steadily decreased with higher spatial frequencies, suggesting that motion contour detectors may be specifically tuned to a single, isolated edge.

Coherent propagation and energy transfer in low-dimension nonlinear arrays

We present a theory of coherent propagation and energy or power transfer in a low-dimension array of coupled nonlinear waveguides. It is demonstrated that in the array with nonequal cores (e.g., with the central core) stable steady-state coherent multicore propagation is possible only in the nonlinear regime, with a power-controlled phase matching. The developed theory of energy or power transfer in nonlinear discrete systems is rather generic and has a range of potential applications including both high-power fiber lasers and ultrahigh-capacity optical communication systems. © 2012 American Physical Society.

Experimental observation of autosoliton propagation in a dispersion-managed system guided by nonlinear optical loop mirrors

Observation of autosoliton propagation in a dispersion-managed optical transmission system controlled by in-line nonlinear fiber loop switches is reported for what is believed to be the first time. The system is based on a strong dispersion map with large amplifier spacing. Operation at transmission rates of 10 and 40 Gbits/s is demonstrated. ©2004 Optical Society of America.

Stable propagation of solitons with increased energy through the combined action of dispersion management and periodic saturable absorption

It is shown, through numerical simulations, that by using a combination of dispersion management and periodic saturable absorption it is possible to transmit solitonlike pulses with greatly increased energy near to the zero net dispersion wavelength. This system is shown to support the stable propagation of solitons over transoceanic distances for a wide range of input powers.

1220km propagation of 40Gbit/s single channel RZ data over dispersion managed standard (non-dispersion shifted) fibre

Error free propagation of a single polarisation optical time division multiplexed 40 Gbit/s dispersion managed pulsed data stream over dispersion (non-shifted) fibre. This distance is twice the previous record at this data rate.

Picosecond soliton propagation using nonlinear optical loop mirrors as intensity filters

The authors show that by inserting nonlinear optical loop mirrors into an optical fibre transmission line, 1.5 ps solitons may be transmitted over at least 750 km, with amplifiers spaced at 15 km intervals.

Role of plasma in femtosecond laser pulse propagation

This paper describes physics of nonlinear ultra-short laser pulse propagation affected by plasma created by the pulse itself. Major applications are also discussed. Nonlinear propagation of the femtosecond laser pulses in gaseous and solid transparent dielectric media is a fundamental physical phenomenon in a wide range of important applications such as laser lidars, laser micro-machining (ablation) and microfabrication etc. These applications require very high intensity of the laser field, typically 1013–1015 TW/cm2. Such high intensity leads to significant ionisation and creation of electron-ion or electron-hole plasma. The presence of plasma results into significant multiphoton and plasma absorption and plasma defocusing. Consequently, the propagation effects appear extremely complex and result from competitive counteraction of the above listed effects and Kerr effect, diffraction and dispersion. The theoretical models used for consistent description of laser-plasma interaction during femtosecond laser pulse propagation are derived and discussed. It turns out that the strongly nonlinear effects such self-focusing followed by the pulse splitting are essential. These phenomena feature extremely complex dynamics of both the electromagnetic field and plasma density with different spatio-temporal structures evolving at the same time. Some numerical approaches capable to handle all these complications are also discussed. ©2006 American Institute of Physics

Soliton propagation through a disordered system:statistics of the transmission delay

We have studied the soliton propagation through a segment containing random pointlike scatterers. In the limit of small concentration of scatterers when the mean distance between the scatterers is larger than the soliton width, a method has been developed for obtaining the statistical characteristics of the soliton transmission through the segment. The method is applicable for any classical particle traversing through a disordered segment with the given velocity transformation after each act of scattering. In the case of weak scattering and relatively short disordered segment the transmission time delay of a fast soliton is mostly determined by the shifts of the soliton center after each act of scattering. For sufficiently long segments the main contribution to the delay is due to the shifts of the amplitude and velocity of a fast soliton after each scatterer. Corresponding crossover lengths for both cases of light and heavy solitons have been obtained. We have also calculated the exact probability density function of the soliton transmission time delay for a sufficiently long segment. In the case of weak identical scatterers the latter is a universal function which depends on a sole parameter—the mean number of scatterers in a segment.

Inference by belief propagation in composite systems

We devise a message passing algorithm for probabilistic inference in composite systems, consisting of a large number of variables, that exhibit weak random interactions among all variables and strong interactions with a small subset of randomly chosen variables; the relative strength of the two interactions is controlled by a free parameter. We examine the performance of the algorithm numerically on a number of systems of this type for varying mixing parameter values.

40Gbit/s single channel dispersion managed pulse propagation in standard fibre over 509km

Error free propagation of a single polarisation optical time division multiplexed 40Gbit/s dispersion managed pulse data stream over 509km has been achieved in standard (non-dispersion shifted) fibre. Dispersion compensating fibre was used after each amplifier to reduce the high local dispersion of the standard fibre. © IEE 1999.