Bayesian invariant measurements of generalisation for discrete distributions

Neural network learning rules can be viewed as statistical estimators. They should be studied in Bayesian framework even if they are not Bayesian estimators. Generalisation should be measured by the divergence between the true distribution and the estimated distribution. Information divergences are invariant measurements of the divergence between two distributions. The posterior average information divergence is used to measure the generalisation ability of a network. The optimal estimators for multinomial distributions with Dirichlet priors are studied in detail. This confirms that the definition is compatible with intuition. The results also show that many commonly used methods can be put under this unified framework, by assume special priors and special divergences.

Bayesian invariant measurements of generalisation for continuous distributions

A family of measurements of generalisation is proposed for estimators of continuous distributions. In particular, they apply to neural network learning rules associated with continuous neural networks. The optimal estimators (learning rules) in this sense are Bayesian decision methods with information divergence as loss function. The Bayesian framework guarantees internal coherence of such measurements, while the information geometric loss function guarantees invariance. The theoretical solution for the optimal estimator is derived by a variational method. It is applied to the family of Gaussian distributions and the implications are discussed. This is one in a series of technical reports on this topic; it generalises the results of ¸iteZhu95:prob.discrete to continuous distributions and serve as a concrete example of a larger picture ¸iteZhu95:generalisation.

Bayesian invariant measurements of generalisation

The problem of evaluating different learning rules and other statistical estimators is analysed. A new general theory of statistical inference is developed by combining Bayesian decision theory with information geometry. It is coherent and invariant. For each sample a unique ideal estimate exists and is given by an average over the posterior. An optimal estimate within a model is given by a projection of the ideal estimate. The ideal estimate is a sufficient statistic of the posterior, so practical learning rules are functions of the ideal estimator. If the sole purpose of learning is to extract information from the data, the learning rule must also approximate the ideal estimator. This framework is applicable to both Bayesian and non-Bayesian methods, with arbitrary statistical models, and to supervised, unsupervised and reinforcement learning schemes.

Preparation, characterisation and entrapment of a non-glycosidic threitol ceramide into liposomes for presentation to invariant natural killer T cells

Dendritic cells (DCs) are able to present glycolipids to invariant natural killer T (iNKT) cells in vivo. Very few compounds have been found to stimulate iNKT cells, and of these, the best characterised is the glycolipid a-galactosylceramide, which stimulates the production of large quantities of interferon-gamma (IFN-?) and interleukin-4 (IL-4). However, aGalCer leads to overstimulation of iNKT cells. It has been demonstrated that the aGalCer analogue, threitol ceramide (ThrCer 2), successfully activates iNKT cells and overcomes the problematic iNKT cell activation-induced anergy. In this study, ThrCer 2 has been inserted into the bilayers of liposomes composed of a neutral lipid, 1,2-distearoyl-sn-glycero-3-phosphocholine (DSPC), or dimethyldioctadecylammonium bromide (DDA), a cationic lipid. Incorporation efficiencies of ThrCer within the liposomes was 96% for DSPC liposomes and 80% for DDA liposomes, with the vesicle size (large multilamellar vs. small unilamellar vesicles) making no significant difference. Langmuir-Blodgett studies suggest that both DSPC and DDA stack within the monolayer co-operatively with the ThrCer molecules with no condensing effect. In terms of cellular responses, IFN-? secretion was higher for cells treated with small DDA liposomes compared with the other liposome formulations, suggesting that ThrCer encapsulation in this liposome formulation resulted in a higher uptake by DCs.

Raman amplification and performance altering effects

This thesis presents an experimental investigation into several applications of the Raman scattering effect in communication optical fibres as well as how some of these applications can be modified to enhance the resulting performance. The majority of the work contained within is based on laboratory results using many commercially available components. The results can be divided into and presented in three main parts: Firstly, a novel application of a known effect is used to broaden Raman pump light in order to achieve a more continuous distribution of gain with respect to wavelength. Multiple experimental results are presented, all based around the prior spreading of the pump spectrum before being used in the desired transmission fibre. Gathered results show that a notable improvement can be obtained from applying such a technique along with the scope for further optimisation work. Secondly, an investigation into the interaction between the well known effect of Four Wave Mixing (FWM) and Raman scattering is provided. The work provides an introduction to the effect as well commenting on previous literature regarding the effect and its mitigation. In response to existing research experimental results are provided detailing some limitations of proposed schemes along with concepts of how further alleviation from the deleterious effects maybe obtained. Lastly, the distributed nature of the Raman gain process is explored. A novel technique on how a near constant distribution of gain can be employed is implemented practically. The application of distributed gain is then applied to the generation of optical pulses with special mathematical properties within a laboratory setting and finally the effect of pump noise upon distributed gain techniques is acknowledged.

A developmental dissociation of view-dependent and view-invariant object recognition in adolescence

Spatial generalization skills in school children aged 8-16 were studied with regard to unfamiliar objects that had been previously learned in a cross-modal priming and learning paradigm. We observed a developmental dissociation with younger children recognizing objects only from previously learnt perspectives whereas older children generalized acquired object knowledge to new viewpoints as well. Haptic and - to a lesser extent - visual priming improved spatial generalization in all but the youngest children. The data supports the idea of dissociable, view-dependent and view-invariant object representations with different developmental trajectories that are subject to modulatory effects of priming. Late-developing areas in the parietal or the prefrontal cortex may account for the retarded onset of view-invariant object recognition. © 2006 Elsevier B.V. All rights reserved.

Contrast- and illumination-invariant object recognition from active sensation

It has been suggested that the deleterious effect of contrast reversal on visual recognition is unique to faces, not objects. Here we show from priming, supervised category learning, and generalization that there is no such thing as general invariance of recognition of non-face objects against contrast reversal and, likewise, changes in direction of illumination. However, when recognition varies with rendering conditions, invariance may be restored, and effects of continuous learning may be reduced, by providing prior object knowledge from active sensation. Our findings suggest that the degree of contrast invariance achieved reflects functional characteristics of object representations learned in a task-dependent fashion.

Phase dependent forward three wave amplification in photorefractive BSO

The results of three wave mixing experiments in photorefractive Bi12SiO20 are presented. The results confirm theoretical predictions that amplification by three wave mixing is strongly dependent on the phase relationship between the three beams at the input to the medium.

Amplification of temporally modulated signal beams via two-wave mixing in Bi12SiO20

A theoretical analysis of two-wave mixing in a BSO crystal is developed in the undepleted-pump approximation for a modulated signal beam. It is shown that, for a modulation of high enough frequency, significant ac amplification is possible at three distinct values of pump-beam detuning. A signal beam that is amplitude modulated by a square wave is analyzed by means of the theory, and experimental results are presented in confirmation of the analysis. Finally, it is shown that in the presence of absorption the optimum detunings for dc and ac amplification are different.

Proposal and design of airy-based rocket pulses for invariant propagation in lossy dispersive media

A novel kind of Airy-based pulse with an invariant propagation in lossy dispersive media is proposed. The basic principle is based on an optical energy trade-off between different parts of the pulse caused by the chromatic dispersion, which is used to compensate the attenuation losses of the propagation medium. Although the ideal concept of the proposed pulses implies infinite pulse energy, the numerical simulations show that practical finite energy pulses can be designed to obtain a partially invariant propagation over a finite distance of propagation.

Band-limited airy pulses for invariant propagation in single mode fibers

By spectral analysis, and using joint time-frequency representations, we present the theoretical basis to design invariant band-limited Airy pulses with an arbitrary degree of robustness, and an arbitrary range of single mode fiber chromatic dispersion. The numerically simulated examples confirm the theoretically predicted pulse partial invariance in the propagation of the pulse in the fiber.

Cross-orientation masking is speed invariant between ocular pathways but speed dependent within them

In human (D. H. Baker, T. S. Meese, & R. J. Summers, 2007b) and in cat (B. Li, M. R. Peterson, J. K. Thompson, T. Duong, & R. D. Freeman, 2005; F. Sengpiel & V. Vorobyov, 2005) there are at least two routes to cross-orientation suppression (XOS): a broadband, non-adaptable, monocular (within-eye) pathway and a more narrowband, adaptable interocular (between the eyes) pathway. We further characterized these two routes psychophysically by measuring the weight of suppression across spatio-temporal frequency for cross-oriented pairs of superimposed flickering Gabor patches. Masking functions were normalized to unmasked detection thresholds and fitted by a two-stage model of contrast gain control (T. S. Meese, M. A. Georgeson, & D. H. Baker, 2006) that was developed to accommodate XOS. The weight of monocular suppression was a power function of the scalar quantity ‘speed’ (temporal-frequency/spatial-frequency). This weight can be expressed as the ratio of non-oriented magno- and parvo-like mechanisms, permitting a fast-acting, early locus, as befits the urgency for action associated with high retinal speeds. In contrast, dichoptic-masking functions superimposed. Overall, this (i) provides further evidence for dissociation between the two forms of XOS in humans, and (ii) indicates that the monocular and interocular varieties of XOS are space/time scale-dependent and scale-invariant, respectively. This suggests an image-processing role for interocular XOS that is tailored to natural image statistics—very different from that of the scale-dependent (speed-dependent) monocular variety.

A practical phase sensitive amplification scheme for two channel phase regeneration

A "black-box" phase sensitive amplifier is presented achieving simultaneous suppression of deterministic phase distortion on two independent 42.66 Gbit/s DPSK modulated signal wavelengths.

Impact of Raman amplification on a 2 Tb/s coherent WDM system

The impact of hybrid erbium-doped fiber amplifier (EDFA)/Raman amplification on a spectrally efficient coherent-wavelength-division-multiplexed (CoWDM) optical communication system is experimentally studied and modeled. Simulations suggested that 23-dB Raman gain over an unrepeatered span of 124 km single-mode fiber would allow a decrease of the mean input power of ~6 dB for a fixed bit-error rate (BER). Experimentally we demonstrated 1.2-dB Q-factor improvement for a 2-Tb/s seven-band CoWDM with backward Raman amplification. The system delivered an optical signal-to-noise ratio of 35 dB at the output of the receiver preamplifier providing a worst-case BER of 2 × 10 -6 over 49 subcarriers at 42.8 Gbaud, leaving a system margin (in terms of Q -factor) of ~4 dB from the forward-error correction threshold.