The n-tuple classifier: too good to ignore

The n-tuple pattern recognition method has been tested using a selection of 11 large data sets from the European Community StatLog project, so that the results could be compared with those reported for the 23 other algorithms the project tested. The results indicate that this ultra-fast memory-based method is a viable competitor with the others, which include optimisation-based neural network algorithms, even though the theory of memory-based neural computing is less highly developed in terms of statistical theory.

Neural networks: A principled perspective

Introductory accounts of artificial neural networks often rely for motivation on analogies with models of information processing in biological networks. One limitation of such an approach is that it offers little guidance on how to find optimal algorithms, or how to verify the correct performance of neural network systems. A central goal of this paper is to draw attention to a quite different viewpoint in which neural networks are seen as algorithms for statistical pattern recognition based on a principled, i.e. theoretically well-founded, framework. We illustrate the concept of a principled viewpoint by considering a specific issue concerned with the interpretation of the outputs of a trained network. Finally, we discuss the relevance of such an approach to the issue of the validation and verification of neural network systems.

Theoretical foundations of neural networks

Neural networks have often been motivated by superficial analogy with biological nervous systems. Recently, however, it has become widely recognised that the effective application of neural networks requires instead a deeper understanding of the theoretical foundations of these models. Insight into neural networks comes from a number of fields including statistical pattern recognition, computational learning theory, statistics, information geometry and statistical mechanics. As an illustration of the importance of understanding the theoretical basis for neural network models, we consider their application to the solution of multi-valued inverse problems. We show how a naive application of the standard least-squares approach can lead to very poor results, and how an appreciation of the underlying statistical goals of the modelling process allows the development of a more general and more powerful formalism which can tackle the problem of multi-modality.

Bayesian methods for neural networks

Bayesian techniques have been developed over many years in a range of different fields, but have only recently been applied to the problem of learning in neural networks. As well as providing a consistent framework for statistical pattern recognition, the Bayesian approach offers a number of practical advantages including a potential solution to the problem of over-fitting. This chapter aims to provide an introductory overview of the application of Bayesian methods to neural networks. It assumes the reader is familiar with standard feed-forward network models and how to train them using conventional techniques.

Bayesian methods for neural networks

Bayesian techniques have been developed over many years in a range of different fields, but have only recently been applied to the problem of learning in neural networks. As well as providing a consistent framework for statistical pattern recognition, the Bayesian approach offers a number of practical advantages including a potential solution to the problem of over-fitting. This chapter aims to provide an introductory overview of the application of Bayesian methods to neural networks. It assumes the reader is familiar with standard feed-forward network models and how to train them using conventional techniques.

Practical assessment of neural network applications

This paper reports the initial results of a joint research project carried out by Aston University and Lloyd's Register to develop a practical method of assessing neural network applications. A set of assessment guidelines for neural network applications were developed and tested on two applications. These case studies showed that it is practical to assess neural networks in a statistical pattern recognition framework. However there is need for more standardisation in neural network technology and a wider takeup of good development practice amongst the neural network community.

Developments of the generative topographic mapping

The generative topographic mapping (GTM) model was introduced by Bishop et al. (1998, Neural Comput. 10(1), 215-234) as a probabilistic re- formulation of the self-organizing map (SOM). It offers a number of advantages compared with the standard SOM, and has already been used in a variety of applications. In this paper we report on several extensions of the GTM, including an incremental version of the EM algorithm for estimating the model parameters, the use of local subspace models, extensions to mixed discrete and continuous data, semi-linear models which permit the use of high-dimensional manifolds whilst avoiding computational intractability, Bayesian inference applied to hyper-parameters, and an alternative framework for the GTM based on Gaussian processes. All of these developments directly exploit the probabilistic structure of the GTM, thereby allowing the underlying modelling assumptions to be made explicit. They also highlight the advantages of adopting a consistent probabilistic framework for the formulation of pattern recognition algorithms.

Evaluating the effectiveness of Bayesian feature selection

A practical Bayesian approach for inference in neural network models has been available for ten years, and yet it is not used frequently in medical applications. In this chapter we show how both regularisation and feature selection can bring significant benefits in diagnostic tasks through two case studies: heart arrhythmia classification based on ECG data and the prognosis of lupus. In the first of these, the number of variables was reduced by two thirds without significantly affecting performance, while in the second, only the Bayesian models had an acceptable accuracy. In both tasks, neural networks outperformed other pattern recognition approaches.

Structured codebooks for SCS watermarking

Digital watermarking aims at embedding information in digital data. The watermark is usually required to be imperceptible, unremovable and to have a high information content. Unfortunately, these three requirements are contradicting. For example, having a more robust watermark makes it either more perceptible or/and less informative. For Gaussian data and additive white Gaussian noise, an optimal but also impractical scheme has already be devised. Since then, many practical schemes have tried to approach the theoretical limits. This paper investigate improvements to current state-of-the-art embedding schemes.

Neural networks: A principled perspective

Introductory accounts of artificial neural networks often rely for motivation on analogies with models of information processing in biological networks. One limitation of such an approach is that it offers little guidance on how to find optimal algorithms, or how to verify the correct performance of neural network systems. A central goal of this paper is to draw attention to a quite different viewpoint in which neural networks are seen as algorithms for statistical pattern recognition based on a principled, i.e. theoretically well-founded, framework. We illustrate the concept of a principled viewpoint by considering a specific issue concerned with the interpretation of the outputs of a trained network. Finally, we discuss the relevance of such an approach to the issue of the validation and verification of neural network systems.

Non-rigid object tracking in complex scenes

A recently proposed colour based tracking algorithm has been established to track objects in real circumstances [Zivkovic, Z., Krose, B. 2004. An EM-like algorithm for color-histogram-based object tracking. In: Proc, IEEE Conf. on Computer Vision and Pattern Recognition, pp. 798-803]. To improve the performance of this technique in complex scenes, in this paper we propose a new algorithm for optimally adapting the ellipse outlining the objects of interest. This paper presents a Lagrangian based method to integrate a regularising component into the covariance matrix to be computed. Technically, we intend to reduce the residuals between the estimated probability distribution and the expected one. We argue that, by doing this, the shape of the ellipse can be properly adapted in the tracking stage. Experimental results show that the proposed method has favourable performance in shape adaption and object localisation.

The innate immune system and the clearance of apoptotic cells

Apoptosis, programmed cell death, is used by multicellular organisms to remove cells that are in excess, damaged or diseased. Activation of the apoptosis programme generates "eat me" signals on the surface of the apoptotic cell that mediate recognition and clearance by the innate immune system. CD14, a pattern recognition receptor expressed on macrophages, is widely known for its ability to recognise the pathogen-associated molecular pattern lipopolysaccharide (LPS) and promote inflammation. However, CD14 has also been shown to mediate binding and removal of apoptotic cells in a process that is anti-inflammatory suggesting CD14 is capable of producing two distinct, ligand-dependent macrophage responses. Whilst the molecular basis for this dichotomy has yet to be defined it is clear that CD14 defines a point of interest on the macrophage surface where we may study ligand-specific responses of macrophages. Our work seeks to define the molecular mechanisms underlying the involvement of CD14 in the non-inflammatory clearance of apoptotic cells. Here we used three different differentiation strategies to generate macrophages from the monocytic cell line THP-1. The resultant macrophage models were characterised to assess the expression and function of CD14 within each model system. Whilst each macrophage model shows increased levels of surface CD14 expression, our results demonstrate significant differences in the various models’ abilities to respond to LPS and clear apoptotic cells in a CD14-dependent manner. TLR4 levels correlated positively with LPS responsiveness but not CD14-dependent apoptotic cell clearance or anti-inflammatory responses to apoptotic cells. These observations suggest CD14-dependent apoptotic cell clearance is not dependent on TLR4. Taken together our data support the notion that the CD14 ligand-dependent responses to LPS and apoptotic cells derive from changes at the macrophage surface. The nature and composition of the CD14-co-receptor complex for LPS and apoptotic cell binding and responses is the subject of further study.

CD14 and innate immune clearance of apoptotic cells

Rapid elimination of cells undergoing programmed cell death (apoptosis) is vital to maintain tissue homeostasis. The phagocytic removal of apoptotic cells (AC) is mediated by innate immune molecules, professional phagocytes and amateur phagocytes that recognise "eat me" signals on the surface of the AC. CD14, a pattern recognition receptor expressed on macrophages, is widely known for its ability to recognise the pathogen-associated molecular pattern lipopolysaccharide (LPS) and promote inflammation. CD14 also mediates the binding and removal of AC, a process that is considered to be anti-inflammatory therefore suggesting CD14 is capable of producing two distinct ligand-dependent responses. Our work seeks to define the molecular mechanisms underlying the involvement of CD14 in the non-inflammatory clearance of AC. Here we describe three different differentiation strategies used to generate macrophages from the monocytic cell line THP-1. Whilst CD14 expression was increased in each macrophage model we demonstrate significant differences in the various macrophage models' abilities to respond to LPS and clear AC. We show that CD14 expression correlates with CD14-dependent AC clearance and anti-inflammatory responses to AC. However LPS responsiveness correlates, as expected, with TLR4 but not CD14 expression. These observations suggest CD14-dependent AC clearance is not dependent on TLR4. Taken together our data support the notion that CD14 ligand-dependent responses to LPS and AC are derived from changes at the macrophage surface. The nature and composition of the CD14-co-receptor complex for LPS and AC binding and consequent responses is the subject of further study.

Mechanisms of altered cortical excitability in photosensitive epilepsy

Despite the multiplicity of approaches and techniques so far applied for identifying the pathophysiological mechanisms of photosensitive epilepsy, a generally agreed explanation of the phenomenon is still lacking. The present thesis reports on three interlinked original experimental studies conducted to explore the neurophysiological correlates and the phatophysiological mechanism of photosensitive epilepsy. In the first study I assessed the role of the habituation of the Visual Evoked Response test as a possible biomarker of epileptic visual sensitivity. The two subsequent studies were designed to address specific research questions emerging from the results of the first study. The findings of the three intertwined studies performed provide experimental evidence that photosensitivity is associated with changes in a number of electrophysiological measures suggestive of altered balance between excitatory and inhibitory cortical processes. Although a strong clinical association does exist between specific epileptic syndromes and visual sensitivity, results from this research indicate that photosensitivity trait seems to be the expression of specific pathophysiological mechanisms quite distinct from the “epileptic” phenotype. The habituation of Pattern Reversal Visual Evoked Potential (PR-VEP) appears as a reliable candidate endo-phenotype of visual sensitivity. Interpreting the findings of this study in the context of the broader literature on visual habituation we can hypothesise the existence of a shared neurophysiological background between photosensitive epilepsy and migraine. Future studies to elucidate the relationship between the proposed indices of cortical excitability and specific polymorphisms of excitatroy and inhibitory neurotransmission will need to be conducted to assess their potential role as biomarkers of photosensitivity.

A study of cluster analysis techniques and their applications

This thesis seeks to describe the development of an inexpensive and efficient clustering technique for multivariate data analysis. The technique starts from a multivariate data matrix and ends with graphical representation of the data and pattern recognition discriminant function. The technique also results in distances frequency distribution that might be useful in detecting clustering in the data or for the estimation of parameters useful in the discrimination between the different populations in the data. The technique can also be used in feature selection. The technique is essentially for the discovery of data structure by revealing the component parts of the data. lhe thesis offers three distinct contributions for cluster analysis and pattern recognition techniques. The first contribution is the introduction of transformation function in the technique of nonlinear mapping. The second contribution is the us~ of distances frequency distribution instead of distances time-sequence in nonlinear mapping, The third contribution is the formulation of a new generalised and normalised error function together with its optimal step size formula for gradient method minimisation. The thesis consists of five chapters. The first chapter is the introduction. The second chapter describes multidimensional scaling as an origin of nonlinear mapping technique. The third chapter describes the first developing step in the technique of nonlinear mapping that is the introduction of "transformation function". The fourth chapter describes the second developing step of the nonlinear mapping technique. This is the use of distances frequency distribution instead of distances time-sequence. The chapter also includes the new generalised and normalised error function formulation. Finally, the fifth chapter, the conclusion, evaluates all developments and proposes a new program. for cluster analysis and pattern recognition by integrating all the new features.