Modelling of non-stationary processes using radial basis function networks

This paper reports preliminary progress on a principled approach to modelling nonstationary phenomena using neural networks. We are concerned with both parameter and model order complexity estimation. The basic methodology assumes a Bayesian foundation. However to allow the construction of pragmatic models, successive approximations have to be made to permit computational tractibility. The lowest order corresponds to the (Extended) Kalman filter approach to parameter estimation which has already been applied to neural networks. We illustrate some of the deficiencies of the existing approaches and discuss our preliminary generalisations, by considering the application to nonstationary time series.

Dynamics of on-line learning in radial basis function networks

On-line learning is examined for the radial basis function network, an important and practical type of neural network. The evolution of generalization error is calculated within a framework which allows the phenomena of the learning process, such as the specialization of the hidden units, to be analyzed. The distinct stages of training are elucidated, and the role of the learning rate described. The three most important stages of training, the symmetric phase, the symmetry-breaking phase, and the convergence phase, are analyzed in detail; the convergence phase analysis allows derivation of maximal and optimal learning rates. As well as finding the evolution of the mean system parameters, the variances of these parameters are derived and shown to be typically small. Finally, the analytic results are strongly confirmed by simulations.

On-line learning in radial basis functions networks

An analytic investigation of the average case learning and generalization properties of Radial Basis Function Networks (RBFs) is presented, utilising on-line gradient descent as the learning rule. The analytic method employed allows both the calculation of generalization error and the examination of the internal dynamics of the network. The generalization error and internal dynamics are then used to examine the role of the learning rate and the specialization of the hidden units, which gives insight into decreasing the time required for training. The realizable and over-realizable cases are studied in detail; the phase of learning in which the hidden units are unspecialized (symmetric phase) and the phase in which asymptotic convergence occurs are analyzed, and their typical properties found. Finally, simulations are performed which strongly confirm the analytic results.

A basis function approach to Bayesian inference in diffusion processes

In this paper, we present a framework for Bayesian inference in continuous-time diffusion processes. The new method is directly related to the recently proposed variational Gaussian Process approximation (VGPA) approach to Bayesian smoothing of partially observed diffusions. By adopting a basis function expansion (BF-VGPA), both the time-dependent control parameters of the approximate GP process and its moment equations are projected onto a lower-dimensional subspace. This allows us both to reduce the computational complexity and to eliminate the time discretisation used in the previous algorithm. The new algorithm is tested on an Ornstein-Uhlenbeck process. Our preliminary results show that BF-VGPA algorithm provides a reasonably accurate state estimation using a small number of basis functions.

Synthesis of sulforaphane and inhibition of cytochrome P450 enzymes as a basis for antigenotoxicity

Many dietary factors have been associated with a decreased risk of developing cancer. One potential mechanism by which these factors, chemopreventors, protect against cancer may be via alteration of carcinogen metabolism. The broccoli constituent sulforaphane (1-isothiocyanate-4-methylsulinylbutane) (CH3-S0-(CH2)4-NCS) has been isolated as a potential inducer of phase II detoxification enzymes and also protects rodents against 9,10-dimethyl-1,2-benz[aJanthracene-induced mammary tumours. The ability of sulforaphane to also modulate phase I activation enzymes (cytochrome P450) (CYP450) was studied here. Sulforaphane was synthesised with an overall yield of 15%, essentially via 1-methylsulfinylphthalimidobutane, which was oxidised to the sulfoxide moiety. Deprotective removal of phthalimide yielded the amine, which was converted into sulforaphane by reaction with N,N'-thionocarbonyldiimidazole. Purity (95 %) was checked by 1H-NMR,13C-NMR and infrared and mass spectrometry.Sulforaphane was a competitive inhibitor of CYP2E1 in acetone-induced Sprague-Dawley rat microsomes (Ki 37.9 ± 4.5μM), as measured by the p-nitrophenol hydroxylase assay. Ethoxyresorufin deethylase activity (EROD), a measurement of CYP1A activity, was also inhibited by sulforaphane (100μM) but was not competitive, and a preincubation time-dependence was observed. In view of these results, the capacity of sulforaphane to inhibit N-nitrosodimethylamine (NDMA)-induced genotoxicity (CYP2E1-mediated) was studied using mouse liver activation systems. Sulforaphane (>0.8μM) inhibited the mutagenicity of NDMA (4.4 mg/plate) in Salmonella typhimurium strain TA100 after pre-incubation for 45 min with acetone-induced liver 9000 g supernatants from Balb/c mice. Unscheduled DNA synthesis induced by NDMA (33μ5 M) in mouse hepatocytes was also reduced by sulforaphane in a concentration-dependent manner (0.064-20μM). Sulforaphane was not genotoxic itself in any of these systems and cytotoxic only at high concentrations (>0.5 mM and > 40μM respectively). The ability of sulforaphane to modulate the orthologous human enzymes was studied using a human epithelial liver cell line (THLE) expressing individual human CYP450 isoenzymes. Using the Comet assay (a measurement of DNA strand breakage under alkaline conditions), NDMA (0.01-1μg/ml) and IQ (0.1-10μg/ml) were used to produce strand breaks in T5-2E1 cells (expressing human CYP2E1) and T5-1A2 cells (expressing human CYP1A2) respectively, however no response was observed in T5-neo cells (without CYP450 cDNA transfection). Sulforaphane inhibited both NDMA and IQ-induced DNA strand breakage in a concentration-dependent manner (0.1-10μM).The inhibition of metabolic activation as a basis for the antigenotoxic action of sulforaphane in these systems (bacteria, rodent hepatocytes and human cells) is further supported by the lack of this chemopreventor to influence NaN3 mutagenicity in S. typhimurium and H202-induced DNA strand breakage in T5-neo cells. These findings suggest that inhibition of CYP2E1 and CYP1A by sulforaphane may contribute to its chemoprotective potential.

A new variational radial basis function approximation for inference in multivariate diffusions

In this paper we present a radial basis function based extension to a recently proposed variational algorithm for approximate inference for diffusion processes. Inference, for state and in particular (hyper-) parameters, in diffusion processes is a challenging and crucial task. We show that the new radial basis function approximation based algorithm converges to the original algorithm and has beneficial characteristics when estimating (hyper-)parameters. We validate our new approach on a nonlinear double well potential dynamical system.

Approximating differentiable relationships between delay embedded dynamical systems with radical basis functions

This thesis is about the study of relationships between experimental dynamical systems. The basic approach is to fit radial basis function maps between time delay embeddings of manifolds. We have shown that under certain conditions these maps are generically diffeomorphisms, and can be analysed to determine whether or not the manifolds in question are diffeomorphically related to each other. If not, a study of the distribution of errors may provide information about the lack of equivalence between the two. The method has applications wherever two or more sensors are used to measure a single system, or where a single sensor can respond on more than one time scale: their respective time series can be tested to determine whether or not they are coupled, and to what degree. One application which we have explored is the determination of a minimum embedding dimension for dynamical system reconstruction. In this special case the diffeomorphism in question is closely related to the predictor for the time series itself. Linear transformations of delay embedded manifolds can also be shown to have nonlinear inverses under the right conditions, and we have used radial basis functions to approximate these inverse maps in a variety of contexts. This method is particularly useful when the linear transformation corresponds to the delay embedding of a finite impulse response filtered time series. One application of fitting an inverse to this linear map is the detection of periodic orbits in chaotic attractors, using suitably tuned filters. This method has also been used to separate signals with known bandwidths from deterministic noise, by tuning a filter to stop the signal and then recovering the chaos with the nonlinear inverse. The method may have applications to the cancellation of noise generated by mechanical or electrical systems. In the course of this research a sophisticated piece of software has been developed. The program allows the construction of a hierarchy of delay embeddings from scalar and multi-valued time series. The embedded objects can be analysed graphically, and radial basis function maps can be fitted between them asynchronously, in parallel, on a multi-processor machine. In addition to a graphical user interface, the program can be driven by a batch mode command language, incorporating the concept of parallel and sequential instruction groups and enabling complex sequences of experiments to be performed in parallel in a resource-efficient manner.