Bayesian invariant measurements of generalisation for discrete distributions

Neural network learning rules can be viewed as statistical estimators. They should be studied in Bayesian framework even if they are not Bayesian estimators. Generalisation should be measured by the divergence between the true distribution and the estimated distribution. Information divergences are invariant measurements of the divergence between two distributions. The posterior average information divergence is used to measure the generalisation ability of a network. The optimal estimators for multinomial distributions with Dirichlet priors are studied in detail. This confirms that the definition is compatible with intuition. The results also show that many commonly used methods can be put under this unified framework, by assume special priors and special divergences.

Bayesian invariant measurements of generalisation for continuous distributions

A family of measurements of generalisation is proposed for estimators of continuous distributions. In particular, they apply to neural network learning rules associated with continuous neural networks. The optimal estimators (learning rules) in this sense are Bayesian decision methods with information divergence as loss function. The Bayesian framework guarantees internal coherence of such measurements, while the information geometric loss function guarantees invariance. The theoretical solution for the optimal estimator is derived by a variational method. It is applied to the family of Gaussian distributions and the implications are discussed. This is one in a series of technical reports on this topic; it generalises the results of ¸iteZhu95:prob.discrete to continuous distributions and serve as a concrete example of a larger picture ¸iteZhu95:generalisation.

Bayesian invariant measurements of generalisation

The problem of evaluating different learning rules and other statistical estimators is analysed. A new general theory of statistical inference is developed by combining Bayesian decision theory with information geometry. It is coherent and invariant. For each sample a unique ideal estimate exists and is given by an average over the posterior. An optimal estimate within a model is given by a projection of the ideal estimate. The ideal estimate is a sufficient statistic of the posterior, so practical learning rules are functions of the ideal estimator. If the sole purpose of learning is to extract information from the data, the learning rule must also approximate the ideal estimator. This framework is applicable to both Bayesian and non-Bayesian methods, with arbitrary statistical models, and to supervised, unsupervised and reinforcement learning schemes.

Preparation, characterisation and entrapment of a non-glycosidic threitol ceramide into liposomes for presentation to invariant natural killer T cells

Dendritic cells (DCs) are able to present glycolipids to invariant natural killer T (iNKT) cells in vivo. Very few compounds have been found to stimulate iNKT cells, and of these, the best characterised is the glycolipid a-galactosylceramide, which stimulates the production of large quantities of interferon-gamma (IFN-?) and interleukin-4 (IL-4). However, aGalCer leads to overstimulation of iNKT cells. It has been demonstrated that the aGalCer analogue, threitol ceramide (ThrCer 2), successfully activates iNKT cells and overcomes the problematic iNKT cell activation-induced anergy. In this study, ThrCer 2 has been inserted into the bilayers of liposomes composed of a neutral lipid, 1,2-distearoyl-sn-glycero-3-phosphocholine (DSPC), or dimethyldioctadecylammonium bromide (DDA), a cationic lipid. Incorporation efficiencies of ThrCer within the liposomes was 96% for DSPC liposomes and 80% for DDA liposomes, with the vesicle size (large multilamellar vs. small unilamellar vesicles) making no significant difference. Langmuir-Blodgett studies suggest that both DSPC and DDA stack within the monolayer co-operatively with the ThrCer molecules with no condensing effect. In terms of cellular responses, IFN-? secretion was higher for cells treated with small DDA liposomes compared with the other liposome formulations, suggesting that ThrCer encapsulation in this liposome formulation resulted in a higher uptake by DCs.

A developmental dissociation of view-dependent and view-invariant object recognition in adolescence

Spatial generalization skills in school children aged 8-16 were studied with regard to unfamiliar objects that had been previously learned in a cross-modal priming and learning paradigm. We observed a developmental dissociation with younger children recognizing objects only from previously learnt perspectives whereas older children generalized acquired object knowledge to new viewpoints as well. Haptic and - to a lesser extent - visual priming improved spatial generalization in all but the youngest children. The data supports the idea of dissociable, view-dependent and view-invariant object representations with different developmental trajectories that are subject to modulatory effects of priming. Late-developing areas in the parietal or the prefrontal cortex may account for the retarded onset of view-invariant object recognition. © 2006 Elsevier B.V. All rights reserved.

Contrast- and illumination-invariant object recognition from active sensation

It has been suggested that the deleterious effect of contrast reversal on visual recognition is unique to faces, not objects. Here we show from priming, supervised category learning, and generalization that there is no such thing as general invariance of recognition of non-face objects against contrast reversal and, likewise, changes in direction of illumination. However, when recognition varies with rendering conditions, invariance may be restored, and effects of continuous learning may be reduced, by providing prior object knowledge from active sensation. Our findings suggest that the degree of contrast invariance achieved reflects functional characteristics of object representations learned in a task-dependent fashion.

Proposal and design of airy-based rocket pulses for invariant propagation in lossy dispersive media

A novel kind of Airy-based pulse with an invariant propagation in lossy dispersive media is proposed. The basic principle is based on an optical energy trade-off between different parts of the pulse caused by the chromatic dispersion, which is used to compensate the attenuation losses of the propagation medium. Although the ideal concept of the proposed pulses implies infinite pulse energy, the numerical simulations show that practical finite energy pulses can be designed to obtain a partially invariant propagation over a finite distance of propagation.

Band-limited airy pulses for invariant propagation in single mode fibers

By spectral analysis, and using joint time-frequency representations, we present the theoretical basis to design invariant band-limited Airy pulses with an arbitrary degree of robustness, and an arbitrary range of single mode fiber chromatic dispersion. The numerically simulated examples confirm the theoretically predicted pulse partial invariance in the propagation of the pulse in the fiber.

Cross-orientation masking is speed invariant between ocular pathways but speed dependent within them

In human (D. H. Baker, T. S. Meese, & R. J. Summers, 2007b) and in cat (B. Li, M. R. Peterson, J. K. Thompson, T. Duong, & R. D. Freeman, 2005; F. Sengpiel & V. Vorobyov, 2005) there are at least two routes to cross-orientation suppression (XOS): a broadband, non-adaptable, monocular (within-eye) pathway and a more narrowband, adaptable interocular (between the eyes) pathway. We further characterized these two routes psychophysically by measuring the weight of suppression across spatio-temporal frequency for cross-oriented pairs of superimposed flickering Gabor patches. Masking functions were normalized to unmasked detection thresholds and fitted by a two-stage model of contrast gain control (T. S. Meese, M. A. Georgeson, & D. H. Baker, 2006) that was developed to accommodate XOS. The weight of monocular suppression was a power function of the scalar quantity ‘speed’ (temporal-frequency/spatial-frequency). This weight can be expressed as the ratio of non-oriented magno- and parvo-like mechanisms, permitting a fast-acting, early locus, as befits the urgency for action associated with high retinal speeds. In contrast, dichoptic-masking functions superimposed. Overall, this (i) provides further evidence for dissociation between the two forms of XOS in humans, and (ii) indicates that the monocular and interocular varieties of XOS are space/time scale-dependent and scale-invariant, respectively. This suggests an image-processing role for interocular XOS that is tailored to natural image statistics—very different from that of the scale-dependent (speed-dependent) monocular variety.

Rotation invariant iris feature extraction using Gaussian Markov random fields with non-separable wavelet

Rotation invariance is important for an iris recognition system since changes of head orientation and binocular vergence may cause eye rotation. The conventional methods of iris recognition cannot achieve true rotation invariance. They only achieve approximate rotation invariance by rotating the feature vector before matching or unwrapping the iris ring at different initial angles. In these methods, the complexity of the method is increased, and when the rotation scale is beyond the certain scope, the error rates of these methods may substantially increase. In order to solve this problem, a new rotation invariant approach for iris feature extraction based on the non-separable wavelet is proposed in this paper. Firstly, a bank of non-separable orthogonal wavelet filters is used to capture characteristics of the iris. Secondly, a method of Markov random fields is used to capture rotation invariant iris feature. Finally, two-class kernel Fisher classifiers are adopted for classification. Experimental results on public iris databases show that the proposed approach has a low error rate and achieves true rotation invariance. © 2010.

Proposal and design of airy-based rocket pulses for invariant propagation in lossy dispersive media

A novel kind of Airy-based pulse with an invariant propagation in lossy dispersive media is proposed. The basic principle is based on an optical energy trade-off between different parts of the pulse caused by the chromatic dispersion, which is used to compensate the attenuation losses of the propagation medium. Although the ideal concept of the proposed pulses implies infinite pulse energy, the numerical simulations show that practical finite energy pulses can be designed to obtain a partially invariant propagation over a finite distance of propagation.