Playing in continuous spaces: Some analysis and extension of population-based incremental learning

As an alternative to traditional evolutionary algorithms (EAs), population-based incremental learning (PBIL) maintains a probabilistic model of the best individual(s). Originally, PBIL was applied in binary search spaces. Recently, some work has been done to extend it to continuous spaces. In this paper, we review two such extensions of PBIL. An improved version of the PBIL based on Gaussian model is proposed that combines two main features: a new updating rule that takes into account all the individuals and their fitness values and a self-adaptive learning rate parameter. Furthermore, a new continuous PBIL employing a histogram probabilistic model is proposed. Some experiments results are presented that highlight the features of the new algorithms.

Ornstein-Uhlenbeck operator in convex domains of Banach spaces

Studiamo l'operatore di Ornstein-Uhlenbeck e il semigruppo di Ornstein-Uhlenbeck in un sottoinsieme aperto convesso $\Omega$ di uno spazio di Banach separabile $X$ dotato di una misura Gaussiana centrata non degnere $\gamma$. In particolare dimostriamo la disuguaglianza di Sobolev logaritmica e la disuguaglianza di Poincaré, e grazie a queste disuguaglianze deduciamo le proprietà spettrali dell'operatore di Ornstein-Uhlenbeck. Inoltre studiamo l'equazione ellittica $\lambdau+L^{\Omega}u=f$ in $\Omega$, dove $L^\Omega$ è l'operatore di Ornstein-Uhlenbeck. Dimostriamo che per $\lambda>0$ e $f\in L^2(\Omega,\gamma)$ la soluzione debole $u$ appartiene allo spazio di Sobolev $W^{2,2}(\Omega,\gamma)$. Inoltre dimostriamo che $u$ soddisfa la condizione di Neumann nel senso di tracce al bordo di $\Omega$. Questo viene fatto finita approssimazione dimensionale.

Modelling of non-stationary processes using radial basis function networks

This paper reports preliminary progress on a principled approach to modelling nonstationary phenomena using neural networks. We are concerned with both parameter and model order complexity estimation. The basic methodology assumes a Bayesian foundation. However to allow the construction of pragmatic models, successive approximations have to be made to permit computational tractibility. The lowest order corresponds to the (Extended) Kalman filter approach to parameter estimation which has already been applied to neural networks. We illustrate some of the deficiencies of the existing approaches and discuss our preliminary generalisations, by considering the application to nonstationary time series.

Topographic mappings and feed-forward neural networks

This thesis is a study of the generation of topographic mappings - dimension reducing transformations of data that preserve some element of geometric structure - with feed-forward neural networks. As an alternative to established methods, a transformational variant of Sammon's method is proposed, where the projection is effected by a radial basis function neural network. This approach is related to the statistical field of multidimensional scaling, and from that the concept of a 'subjective metric' is defined, which permits the exploitation of additional prior knowledge concerning the data in the mapping process. This then enables the generation of more appropriate feature spaces for the purposes of enhanced visualisation or subsequent classification. A comparison with established methods for feature extraction is given for data taken from the 1992 Research Assessment Exercise for higher educational institutions in the United Kingdom. This is a difficult high-dimensional dataset, and illustrates well the benefit of the new topographic technique. A generalisation of the proposed model is considered for implementation of the classical multidimensional scaling (¸mds}) routine. This is related to Oja's principal subspace neural network, whose learning rule is shown to descend the error surface of the proposed ¸mds model. Some of the technical issues concerning the design and training of topographic neural networks are investigated. It is shown that neural network models can be less sensitive to entrapment in the sub-optimal global minima that badly affect the standard Sammon algorithm, and tend to exhibit good generalisation as a result of implicit weight decay in the training process. It is further argued that for ideal structure retention, the network transformation should be perfectly smooth for all inter-data directions in input space. Finally, there is a critique of optimisation techniques for topographic mappings, and a new training algorithm is proposed. A convergence proof is given, and the method is shown to produce lower-error mappings more rapidly than previous algorithms.