Neural networks for time-varying data

This paper reviews some basic issues and methods involved in using neural networks to respond in a desired fashion to a temporally-varying environment. Some popular network models and training methods are introduced. A speech recognition example is then used to illustrate the central difficulty of temporal data processing: learning to notice and remember relevant contextual information. Feedforward network methods are applicable to cases where this problem is not severe. The application of these methods are explained and applications are discussed in the areas of pure mathematics, chemical and physical systems, and economic systems. A more powerful but less practical algorithm for temporal problems, the moving targets algorithm, is sketched and discussed. For completeness, a few remarks are made on reinforcement learning.

Neural networks

An overview of neural networks, covering multilayer perceptrons, radial basis functions, constructive algorithms, Kohonen and K-means unupervised algorithms, RAMnets, first and second order training methods, and Bayesian regularisation methods.

Globally optimal parameters for on-line learning in multilayer neural networks

We present a framework for calculating globally optimal parameters, within a given time frame, for on-line learning in multilayer neural networks. We demonstrate the capability of this method by computing optimal learning rates in typical learning scenarios. A similar treatment allows one to determine the relevance of related training algorithms based on modifications to the basic gradient descent rule as well as to compare different training methods.

Discourse analysis for translation and translator training: status, needs, methods

Editorial

The sensitivity of mapping methods to reference data quality:training supervised image classifications with imperfect reference data

The accuracy of a map is dependent on the reference dataset used in its construction. Classification analyses used in thematic mapping can, for example, be sensitive to a range of sampling and data quality concerns. With particular focus on the latter, the effects of reference data quality on land cover classifications from airborne thematic mapper data are explored. Variations in sampling intensity and effort are highlighted in a dataset that is widely used in mapping and modelling studies; these may need accounting for in analyses. The quality of the labelling in the reference dataset was also a key variable influencing mapping accuracy. Accuracy varied with the amount and nature of mislabelled training cases with the nature of the effects varying between classifiers. The largest impacts on accuracy occurred when mislabelling involved confusion between similar classes. Accuracy was also typically negatively related to the magnitude of mislabelled cases and the support vector machine (SVM), which has been claimed to be relatively insensitive to training data error, was the most sensitive of the set of classifiers investigated, with overall classification accuracy declining by 8% (significant at 95% level of confidence) with the use of a training set containing 20% mislabelled cases.

Bayesian training of mixture density networks

Mixture Density Networks (MDNs) are a well-established method for modelling the conditional probability density which is useful for complex multi-valued functions where regression methods (such as MLPs) fail. In this paper we extend earlier research of a regularisation method for a special case of MDNs to the general case using evidence based regularisation and we show how the Hessian of the MDN error function can be evaluated using R-propagation. The method is tested on two data sets and compared with early stopping.

Training Bayesian networks for image segmentation

We are concerned with the problem of image segmentation in which each pixel is assigned to one of a predefined finite number of classes. In Bayesian image analysis, this requires fusing together local predictions for the class labels with a prior model of segmentations. Markov Random Fields (MRFs) have been used to incorporate some of this prior knowledge, but this not entirely satisfactory as inference in MRFs is NP-hard. The multiscale quadtree model of Bouman and Shapiro (1994) is an attractive alternative, as this is a tree-structured belief network in which inference can be carried out in linear time (Pearl 1988). It is an hierarchical model where the bottom-level nodes are pixels, and higher levels correspond to downsampled versions of the image. The conditional-probability tables (CPTs) in the belief network encode the knowledge of how the levels interact. In this paper we discuss two methods of learning the CPTs given training data, using (a) maximum likelihood and the EM algorithm and (b) emphconditional maximum likelihood (CML). Segmentations obtained using networks trained by CML show a statistically-significant improvement in performance on synthetic images. We also demonstrate the methods on a real-world outdoor-scene segmentation task.

The training experience of primary care pharmacists

Objectives — To map the tasks, activities and training provision for primary care pharmacists (PCPs) and to identify perceived future training needs. Methods — Survey undertaken in 1998/1999 using a pre-piloted, postal, self-completion questionnaire to two samples of PCPs. Setting — PCPs in (a) the West Midlands and (b) England (outside West Midlands). Key findings — The response rate was 66 per cent. A majority (68 per cent) had worked in the role for less than two years. Eighty per cent had some form of continuing education or training for the role although only 50 per cent had a formal qualification. Over two-thirds had contributed to the funding of their training, with one-third providing all funding. Seventy-four per cent of PCPs agreed that pharmacists should go through a procedure to ensure competence (accreditation) before being allowed to work for a general medical practice or primary care group. Views on the need for formal education/training prior to work differed: 82 per cent of those with formal qualifications, but only 46 per cent of those without, considered that this should be a requirement. There was general agreement that training/education had met training needs. Views on future training closely reflected previous training experiences, with a focus upon pharmaceutical roles rather than upon generic skill development and the acquisition of management skills. Conclusions — The study provides a snapshot in time of the experience of pioneer PCPs and the training available to them. PCPs will need further training or updating if they are to provide the wider roles required by the developing needs of the National Health Service. Consideration should be given to formal recognition of the training of PCPs in order to assure competence. The expectation that pharmacists should fund their own training is likely to be a barrier to uptake of training and uncertainties over funding will militate against consistency of training.

Delivering phonological and phonics training within whole-class teaching

Background: Early, intensive phonological awareness and phonics training is widely held to be beneficial for children with poor phonological awareness. However, most studies have delivered this training separately from children's normal whole-class reading lessons. Aims: We examined whether integrating this training into whole class, mixed-ability reading lessons could impact on children with poor phonological awareness, whilst also benefiting normally developing readers. Sample: Teachers delivered the training within a broad reading programme to whole classes of children from Reception to the end of Year 1 (N=251). A comparison group of children received standard teaching methods (N=213). Method: Children's literacy was assessed at the beginning of Reception, and then at the end of each year until 1 year post-intervention. Results: The strategy significantly impacted on reading performance for normally developing readers and those with poor phonological awareness, vastly reducing the incidence of reading difficulties from 20% in comparison schools to 5% in intervention schools. Conclusions: Phonological and phonics training is highly effective for children with poor phonological awareness, even when incorporated into whole-class teaching.

Probabilistic methods for drug discovery

This thesis introduces a flexible visual data exploration framework which combines advanced projection algorithms from the machine learning domain with visual representation techniques developed in the information visualisation domain to help a user to explore and understand effectively large multi-dimensional datasets. The advantage of such a framework to other techniques currently available to the domain experts is that the user is directly involved in the data mining process and advanced machine learning algorithms are employed for better projection. A hierarchical visualisation model guided by a domain expert allows them to obtain an informed segmentation of the input space. Two other components of this thesis exploit properties of these principled probabilistic projection algorithms to develop a guided mixture of local experts algorithm which provides robust prediction and a model to estimate feature saliency simultaneously with the training of a projection algorithm.Local models are useful since a single global model cannot capture the full variability of a heterogeneous data space such as the chemical space. Probabilistic hierarchical visualisation techniques provide an effective soft segmentation of an input space by a visualisation hierarchy whose leaf nodes represent different regions of the input space. We use this soft segmentation to develop a guided mixture of local experts (GME) algorithm which is appropriate for the heterogeneous datasets found in chemoinformatics problems. Moreover, in this approach the domain experts are more involved in the model development process which is suitable for an intuition and domain knowledge driven task such as drug discovery. We also derive a generative topographic mapping (GTM) based data visualisation approach which estimates feature saliency simultaneously with the training of a visualisation model.