Modelling of non-stationary processes using radial basis function networks

This paper reports preliminary progress on a principled approach to modelling nonstationary phenomena using neural networks. We are concerned with both parameter and model order complexity estimation. The basic methodology assumes a Bayesian foundation. However to allow the construction of pragmatic models, successive approximations have to be made to permit computational tractibility. The lowest order corresponds to the (Extended) Kalman filter approach to parameter estimation which has already been applied to neural networks. We illustrate some of the deficiencies of the existing approaches and discuss our preliminary generalisations, by considering the application to nonstationary time series.

Dynamics of on-line learning in radial basis function networks

On-line learning is examined for the radial basis function network, an important and practical type of neural network. The evolution of generalization error is calculated within a framework which allows the phenomena of the learning process, such as the specialization of the hidden units, to be analyzed. The distinct stages of training are elucidated, and the role of the learning rate described. The three most important stages of training, the symmetric phase, the symmetry-breaking phase, and the convergence phase, are analyzed in detail; the convergence phase analysis allows derivation of maximal and optimal learning rates. As well as finding the evolution of the mean system parameters, the variances of these parameters are derived and shown to be typically small. Finally, the analytic results are strongly confirmed by simulations.

On-line learning in radial basis functions networks

An analytic investigation of the average case learning and generalization properties of Radial Basis Function Networks (RBFs) is presented, utilising on-line gradient descent as the learning rule. The analytic method employed allows both the calculation of generalization error and the examination of the internal dynamics of the network. The generalization error and internal dynamics are then used to examine the role of the learning rate and the specialization of the hidden units, which gives insight into decreasing the time required for training. The realizable and over-realizable cases are studied in detail; the phase of learning in which the hidden units are unspecialized (symmetric phase) and the phase in which asymptotic convergence occurs are analyzed, and their typical properties found. Finally, simulations are performed which strongly confirm the analytic results.

Monthly fluctuations in radial growth of individual lobes of the lichen Parmelia conspersa (Erhr. ex Ach.) Ach.

The radial growth (RG) of 120 lobes from 35 thalli of the foliose lichen Parmelia conspersa (Ehrh. ex Ach.) Ach. was studied monthly over 22 months in south Gwynedd, Wales, UK. Autocorrelation analysis of each lobe identified three patterns of fluctuation: 1) random fluctuations (58% of lobes), 2) a cyclic pattern of growth (23% of lobes), and 3) fluctuating growth interrupted by longer periods of very low or zero growth (19% of lobes). In 80% of thalli, two or three patterns of fluctuation were present within the same thallus. Growth fluctuations were correlated with climatic variables in 31% of lobes, most commonly with either total rainfall or number of rain days per month. Lobes correlated with climate were not associated with a particular type of growth fluctuation. RG of a lobe was positively correlated with the degree of bifurcation of the lobe tip. It is hypothesised that lobes of P. conspersa exhibit a cyclic pattern of growth due in part to lobe division. The effects of climate, periods of zero growth, and microvariations in the environment of a lobe are superimposed on this cyclic pattern resulting in the random growth of many lobes. Random growth fluctuations may contribute to the maintenance of thallus symmetry in P. conspersa.