Dynamics of learning with restricted training sets

The dynamics of supervised learning in layered neural networks were studied in the regime where the size of the training set is proportional to the number of inputs. The evolution of macroscopic observables, including the two relevant performance measures can be predicted by using the dynamical replica theory. Three approximation schemes aimed at eliminating the need to solve a functional saddle-point equation at each time step have been derived.

Noise, regularizers, and unrealizable scenarios in online learning from restricted training sets

We study the dynamics of on-line learning in multilayer neural networks where training examples are sampled with repetition and where the number of examples scales with the number of network weights. The analysis is carried out using the dynamical replica method aimed at obtaining a closed set of coupled equations for a set of macroscopic variables from which both training and generalization errors can be calculated. We focus on scenarios whereby training examples are corrupted by additive Gaussian output noise and regularizers are introduced to improve the network performance. The dependence of the dynamics on the noise level, with and without regularizers, is examined, as well as that of the asymptotic values obtained for both training and generalization errors. We also demonstrate the ability of the method to approximate the learning dynamics in structurally unrealizable scenarios. The theoretical results show good agreement with those obtained by computer simulations.

VaxiJen:a server for prediction of protective antigens, tumour antigens and subunit vaccines

Background - Vaccine development in the post-genomic era often begins with the in silico screening of genome information, with the most probable protective antigens being predicted rather than requiring causative microorganisms to be grown. Despite the obvious advantages of this approach – such as speed and cost efficiency – its success remains dependent on the accuracy of antigen prediction. Most approaches use sequence alignment to identify antigens. This is problematic for several reasons. Some proteins lack obvious sequence similarity, although they may share similar structures and biological properties. The antigenicity of a sequence may be encoded in a subtle and recondite manner not amendable to direct identification by sequence alignment. The discovery of truly novel antigens will be frustrated by their lack of similarity to antigens of known provenance. To overcome the limitations of alignment-dependent methods, we propose a new alignment-free approach for antigen prediction, which is based on auto cross covariance (ACC) transformation of protein sequences into uniform vectors of principal amino acid properties. Results - Bacterial, viral and tumour protein datasets were used to derive models for prediction of whole protein antigenicity. Every set consisted of 100 known antigens and 100 non-antigens. The derived models were tested by internal leave-one-out cross-validation and external validation using test sets. An additional five training sets for each class of antigens were used to test the stability of the discrimination between antigens and non-antigens. The models performed well in both validations showing prediction accuracy of 70% to 89%. The models were implemented in a server, which we call VaxiJen. Conclusion - VaxiJen is the first server for alignment-independent prediction of protective antigens. It was developed to allow antigen classification solely based on the physicochemical properties of proteins without recourse to sequence alignment. The server can be used on its own or in combination with alignment-based prediction methods.

Efficient use of training data in the n-tuple recognition method

A simple technique is presented for improving the robustness of the n-tuple recognition method against inauspicious choices of architectural parameters, guarding against the saturation problem, and improving the utilisation of small data sets. Experiments are reported which confirm that the method significantly improves performance and reduces saturation in character recognition problems.

Bayesian training of mixture density networks

Mixture Density Networks (MDNs) are a well-established method for modelling the conditional probability density which is useful for complex multi-valued functions where regression methods (such as MLPs) fail. In this paper we extend earlier research of a regularisation method for a special case of MDNs to the general case using evidence based regularisation and we show how the Hessian of the MDN error function can be evaluated using R-propagation. The method is tested on two data sets and compared with early stopping.

A study in Britain and France of the development of cross-cultural in-house sales training using distance learning.

The thesis is concerned with cross-cultural distance learning in two countries: Great Britain and France. Taking the example of in-house sales training, it argues that it is possible to develop courses for use in two or more countries of differing culture and language. Two courses were developed by the researcher. Both were essentially print-based distance-learning courses designed to help salespeople achieve a better understanding of their customers. One used a quantitative, the other qualitative approach. One considered the concept of the return on investment and the other, for which a video support was also developed, considered the analysis of a customer's needs. Part 1 of the thesis considers differences in the training context between France and Britain followed by a review of the learning process with reference to distance learning. Part 2 looks at the choice of training medium course design and evaluation and sets out the methodology adopted, including problems encountered in this type of fieldwork. Part 3 analyses the data and draws conclusions from the findings, before offering a series of guidelines for those concerned with the development of cross-cultural in-house training courses. The results of the field tests on the two courses were analysed in relation to the socio-cultural, educational and experiential background of the learners as well as their preferred learning styles. The thesis argues that it is possible to develop effective in-house sales training courses to be used in two cultures and identifies key considerations which need to be taken into account when carrying out this type of work.