Dynamical local models for segmentation and prediction of financial time series

In the analysis and prediction of many real-world time series, the assumption of stationarity is not valid. A special form of non-stationarity, where the underlying generator switches between (approximately) stationary regimes, seems particularly appropriate for financial markets. We introduce a new model which combines a dynamic switching (controlled by a hidden Markov model) and a non-linear dynamical system. We show how to train this hybrid model in a maximum likelihood approach and evaluate its performance on both synthetic and financial data.

Non-rigid object tracking in complex scenes

A recently proposed colour based tracking algorithm has been established to track objects in real circumstances [Zivkovic, Z., Krose, B. 2004. An EM-like algorithm for color-histogram-based object tracking. In: Proc, IEEE Conf. on Computer Vision and Pattern Recognition, pp. 798-803]. To improve the performance of this technique in complex scenes, in this paper we propose a new algorithm for optimally adapting the ellipse outlining the objects of interest. This paper presents a Lagrangian based method to integrate a regularising component into the covariance matrix to be computed. Technically, we intend to reduce the residuals between the estimated probability distribution and the expected one. We argue that, by doing this, the shape of the ellipse can be properly adapted in the tracking stage. Experimental results show that the proposed method has favourable performance in shape adaption and object localisation.

Hierarchical GTM: Constructing localized non-linear projection manifolds in a principled way

It has been argued that a single two-dimensional visualization plot may not be sufficient to capture all of the interesting aspects of complex data sets, and therefore a hierarchical visualization system is desirable. In this paper we extend an existing locally linear hierarchical visualization system PhiVis ¸iteBishop98a in several directions: bf(1) We allow for em non-linear projection manifolds. The basic building block is the Generative Topographic Mapping. bf(2) We introduce a general formulation of hierarchical probabilistic models consisting of local probabilistic models organized in a hierarchical tree. General training equations are derived, regardless of the position of the model in the tree. bf(3) Using tools from differential geometry we derive expressions for local directional curvatures of the projection manifold. Like PhiVis, our system is statistically principled and is built interactively in a top-down fashion using the EM algorithm. It enables the user to interactively highlight those data in the parent visualization plot which are captured by a child model. We also incorporate into our system a hierarchical, locally selective representation of magnification factors and directional curvatures of the projection manifolds. Such information is important for further refinement of the hierarchical visualization plot, as well as for controlling the amount of regularization imposed on the local models. We demonstrate the principle of the approach on a toy data set and apply our system to two more complex 12- and 19-dimensional data sets.

Hierarchical GTM: constructing localized non-linear projection manifolds in a principled way

It has been argued that a single two-dimensional visualization plot may not be sufficient to capture all of the interesting aspects of complex data sets, and therefore a hierarchical visualization system is desirable. In this paper we extend an existing locally linear hierarchical visualization system PhiVis ¸iteBishop98a in several directions: bf(1) We allow for em non-linear projection manifolds. The basic building block is the Generative Topographic Mapping (GTM). bf(2) We introduce a general formulation of hierarchical probabilistic models consisting of local probabilistic models organized in a hierarchical tree. General training equations are derived, regardless of the position of the model in the tree. bf(3) Using tools from differential geometry we derive expressions for local directional curvatures of the projection manifold. Like PhiVis, our system is statistically principled and is built interactively in a top-down fashion using the EM algorithm. It enables the user to interactively highlight those data in the ancestor visualization plots which are captured by a child model. We also incorporate into our system a hierarchical, locally selective representation of magnification factors and directional curvatures of the projection manifolds. Such information is important for further refinement of the hierarchical visualization plot, as well as for controlling the amount of regularization imposed on the local models. We demonstrate the principle of the approach on a toy data set and apply our system to two more complex 12- and 18-dimensional data sets.

Constructing localized non-linear projection manifolds in a principled way:hierarchical generative topographic mapping

It has been argued that a single two-dimensional visualization plot may not be sufficient to capture all of the interesting aspects of complex data sets, and therefore a hierarchical visualization system is desirable. In this paper we extend an existing locally linear hierarchical visualization system PhiVis (Bishop98a) in several directions: 1. We allow for em non-linear projection manifolds. The basic building block is the Generative Topographic Mapping. 2. We introduce a general formulation of hierarchical probabilistic models consisting of local probabilistic models organized in a hierarchical tree. General training equations are derived, regardless of the position of the model in the tree. 3. Using tools from differential geometry we derive expressions for local directionalcurvatures of the projection manifold. Like PhiVis, our system is statistically principled and is built interactively in a top-down fashion using the EM algorithm. It enables the user to interactively highlight those data in the parent visualization plot which are captured by a child model.We also incorporate into our system a hierarchical, locally selective representation of magnification factors and directional curvatures of the projection manifolds. Such information is important for further refinement of the hierarchical visualization plot, as well as for controlling the amount of regularization imposed on the local models. We demonstrate the principle of the approach on a toy data set andapply our system to two more complex 12- and 19-dimensional data sets.

Hidden state models for time series

Amongst all the objectives in the study of time series, uncovering the dynamic law of its generation is probably the most important. When the underlying dynamics are not available, time series modelling consists of developing a model which best explains a sequence of observations. In this thesis, we consider hidden space models for analysing and describing time series. We first provide an introduction to the principal concepts of hidden state models and draw an analogy between hidden Markov models and state space models. Central ideas such as hidden state inference or parameter estimation are reviewed in detail. A key part of multivariate time series analysis is identifying the delay between different variables. We present a novel approach for time delay estimating in a non-stationary environment. The technique makes use of hidden Markov models and we demonstrate its application for estimating a crucial parameter in the oil industry. We then focus on hybrid models that we call dynamical local models. These models combine and generalise hidden Markov models and state space models. Probabilistic inference is unfortunately computationally intractable and we show how to make use of variational techniques for approximating the posterior distribution over the hidden state variables. Experimental simulations on synthetic and real-world data demonstrate the application of dynamical local models for segmenting a time series into regimes and providing predictive distributions.

Guiding local regression using visualisation

Solving many scientific problems requires effective regression and/or classification models for large high-dimensional datasets. Experts from these problem domains (e.g. biologists, chemists, financial analysts) have insights into the domain which can be helpful in developing powerful models but they need a modelling framework that helps them to use these insights. Data visualisation is an effective technique for presenting data and requiring feedback from the experts. A single global regression model can rarely capture the full behavioural variability of a huge multi-dimensional dataset. Instead, local regression models, each focused on a separate area of input space, often work better since the behaviour of different areas may vary. Classical local models such as Mixture of Experts segment the input space automatically, which is not always effective and it also lacks involvement of the domain experts to guide a meaningful segmentation of the input space. In this paper we addresses this issue by allowing domain experts to interactively segment the input space using data visualisation. The segmentation output obtained is then further used to develop effective local regression models.

Guiding local regression using visualisation

Solving many scientific problems requires effective regression and/or classification models for large high-dimensional datasets. Experts from these problem domains (e.g. biologists, chemists, financial analysts) have insights into the domain which can be helpful in developing powerful models but they need a modelling framework that helps them to use these insights. Data visualisation is an effective technique for presenting data and requiring feedback from the experts. A single global regression model can rarely capture the full behavioural variability of a huge multi-dimensional dataset. Instead, local regression models, each focused on a separate area of input space, often work better since the behaviour of different areas may vary. Classical local models such as Mixture of Experts segment the input space automatically, which is not always effective and it also lacks involvement of the domain experts to guide a meaningful segmentation of the input space. In this paper we addresses this issue by allowing domain experts to interactively segment the input space using data visualisation. The segmentation output obtained is then further used to develop effective local regression models.

The impact of ageing and gender on visual mental imagery processes:a study of performance on tasks from the Complete Visual Mental Imagery Battery (CVMIB)

In this study we aim to evaluate the impact of ageing and gender on different visual mental imagery processes. Two hundred and fifty-one participants (130 women and 121 men; age range = 18–77 years) were given an extensive neuropsychological battery including tasks probing the generation, maintenance, inspection, and transformation of visual mental images (Complete Visual Mental Imagery Battery, CVMIB). Our results show that all mental imagery processes with the exception of the maintenance are affected by ageing, suggesting that other deficits, such as working memory deficits, could account for this effect. However, the analysis of the transformation process, investigated in terms of mental rotation and mental folding skills, shows a steeper decline in mental rotation, suggesting that age could affect rigid transformations of objects and spare non-rigid transformations. Our study also adds to previous ones in showing gender differences favoring men across the lifespan in the transformation process, and, interestingly, it shows a steeper decline in men than in women in inspecting mental images, which could partially account for the mixed results about the effect of ageing on this specific process. We also discuss the possibility to introduce the CVMIB in clinical assessment in the context of theoretical models of mental imagery.

Bayesian inference for wind field retrieval

In many problems in spatial statistics it is necessary to infer a global problem solution by combining local models. A principled approach to this problem is to develop a global probabilistic model for the relationships between local variables and to use this as the prior in a Bayesian inference procedure. We show how a Gaussian process with hyper-parameters estimated from Numerical Weather Prediction Models yields meteorologically convincing wind fields. We use neural networks to make local estimates of wind vector probabilities. The resulting inference problem cannot be solved analytically, but Markov Chain Monte Carlo methods allow us to retrieve accurate wind fields.

Bayesian inference for wind field retrieval

In many problems in spatial statistics it is necessary to infer a global problem solution by combining local models. A principled approach to this problem is to develop a global probabilistic model for the relationships between local variables and to use this as the prior in a Bayesian inference procedure. We show how a Gaussian process with hyper-parameters estimated from Numerical Weather Prediction Models yields meteorologically convincing wind fields. We use neural networks to make local estimates of wind vector probabilities. The resulting inference problem cannot be solved analytically, but Markov Chain Monte Carlo methods allow us to retrieve accurate wind fields.

Spatial coherence does not affect contrast discrimination for multiple Gabor stimuli

Gestalt grouping rules imply a process or mechanism for grouping together local features of an object into a perceptual whole. Several psychophysical experiments have been interpreted as evidence for constrained interactions between nearby spatial filter elements and this has led to the hypothesis that element linking might be mediated by these interactions. A common tacit assumption is that these interactions result in response modulation which disturbs a local contrast code. We addressed this possibility by performing contrast discrimination experiments using two-dimensional arrays of multiple Gabor patches arranged either (i) vertically, (ii) in circles (coherent conditions), or (iii) randomly (incoherent condition), as well as for a single Gabor patch. In each condition, contrast increments were applied to either the entire test stimulus (experiment 1) or a single patch whose position was cued (experiment 2). In experiment 3, the texture stimuli were reduced to a single contour by displaying only the central vertical strip. Performance was better for the multiple-patch conditions than for the single-patch condition, but whether the multiple-patch stimulus was coherent or not had no systematic effect on the results in any of the experiments. We conclude that constrained local interactions do not interfere with a local contrast code for our suprathreshold stimuli, suggesting that, in general, this is not the way in which element linking is achieved. The possibility that interactions are involved in enhancing the detectability of contour elements at threshold remains unchallenged by our experiments.

Probabilistic methods for drug discovery

This thesis introduces a flexible visual data exploration framework which combines advanced projection algorithms from the machine learning domain with visual representation techniques developed in the information visualisation domain to help a user to explore and understand effectively large multi-dimensional datasets. The advantage of such a framework to other techniques currently available to the domain experts is that the user is directly involved in the data mining process and advanced machine learning algorithms are employed for better projection. A hierarchical visualisation model guided by a domain expert allows them to obtain an informed segmentation of the input space. Two other components of this thesis exploit properties of these principled probabilistic projection algorithms to develop a guided mixture of local experts algorithm which provides robust prediction and a model to estimate feature saliency simultaneously with the training of a projection algorithm.Local models are useful since a single global model cannot capture the full variability of a heterogeneous data space such as the chemical space. Probabilistic hierarchical visualisation techniques provide an effective soft segmentation of an input space by a visualisation hierarchy whose leaf nodes represent different regions of the input space. We use this soft segmentation to develop a guided mixture of local experts (GME) algorithm which is appropriate for the heterogeneous datasets found in chemoinformatics problems. Moreover, in this approach the domain experts are more involved in the model development process which is suitable for an intuition and domain knowledge driven task such as drug discovery. We also derive a generative topographic mapping (GTM) based data visualisation approach which estimates feature saliency simultaneously with the training of a visualisation model.

Folate metabolism in the female weanling rat

The metabolism of a mixture of [2-14C] and [3',5',7,9-3H] folic acid was studied in female weanling rats. Intact folates and folate catabolites were excreted in the urine. Folate polyglutamates were found in the tissues. Rats treated with the oestrogen diethylstilbestrol and 17 -ethynyloestradiol exhibited marked changes in the metabolic handling of folic acid and folate catabolism was greatly increased compared to controls. Allopurinol treatment gave greater label retention in the gut, with a substantial increase in catabolism compared to normals. A dose response relationship was illustrated between allopurinol dose and folate catabolism. After lead acetate dosing there was little radioactivity in the urine and tissues over 72h and more radioactivity was retained in the faeces compared to normals. Excretion of intact folates was depressed, especially 5MeTHF and 10CHOTHF. A tenfold increase in both lead and folic acid dosage resulted in an even further decrease of radioactivity in the tissues and urine over 72h. Excretion in the faeces was further elevated. Ferrous sulphate administration resulted in increased catabolism. The retention of radioactivity in the liver, kidney and gut was greatly reduced. A new method of folate analysis; Sephadex LH-20 was introduced. In vitro superoxide anion formation was illustrated using an allopurinol/xanthine oxidase system. Histological studies were employed to qualitatively and quantitatively illustrate the oxidative status in livers and brains of allopurinol and ferrous sulphate dosed rats. Increased dose related formazan deposition was observed when livers of pretreated animals were incubated with nitroblue tetrazolium. Formazan deposition was reduced in pretreated animals also treated with the anti-oxidants vitamin E, mannitol or 4-hydroxy-methyl-4,6-ditertiary-butylphenol. A possible route of folate catabolism is scission by a non-enzymic oxidation involving active oxygen species.

Mechanisms of drying of particulate slurries

The literature on the evaporation of pure liquid drops and the drying of drops of solutions and slurries has been reviewed with particular reference to spray drying. A 0.1-0.2 mm glass filament-thermocouple was constructed and used to study simultaneously, heat and mass transfer from a single suspended drop placed in a humidity and temperature controlled, 28 mm OD vertical wind tunnel. Heat conduction through the filament was minimised eg at 100¦C it accounted for only 9.3% of the total heat transferred to a drop. Evaporation of single water drops was also studied in a 101 mm OD vertical wind tunnel. The Nusselt number was found to be a function of the Reynolds, Prandtl and Transfer number over an air temperature range of 17¦C to 107¦C. The proposed correlation is: Nu = 2+(-12.96B+0.76)Re¦-5Pr0-33 Experimental drying studies were carried out on single suspended 1 to 2.5 mm diameter drops of aqueous sodium sulphate decahydrate, sodium chloride, potassium sulphate, copper sulphate and sodium acetate solutions and slurries at temperatures of 20¦C to 124¦C. Dried crusts were examined by Scanning Electron Microscopy. The drying history of any material depended upon the nature of the crust formed. Sodium acetate formed a non-rigid skin prior to the formation of a rigid crust. A modified receding evaporation interface model was proposed for the drying of solutions and slurries. This covered both the constant rate period prior to crust formation and the subsequent falling rate period. The model was solved numerically for the variation in core temperature, drop weight and crust thickness. Good agreement was obtained between model predictions and experimental results for materials forming rigid crusts i.e. sodium sulphate decahydrate, sodium chloride, potassium sulphate and copper sulphate. However, the drying histories of drops of 10-20% weight initial concentration sodium acetate were unpredictable since formation of a non-rigid skin deviated from the model assumption of a rigid outer surface. At higher initial concentrations (40% weight) where a rigid crust was formed for sodium acetate, good agreement was obtained between experimental results and model predictions. Single suspended drop studies are concluded to provide a valuable insight into the drying mechanisms of specific solutions and slurries.