On-line learning of mutually orthogonal subspaces for face recognition by image sets.

We address the problem of face recognition by matching image sets. Each set of face images is represented by a subspace (or linear manifold) and recognition is carried out by subspace-to-subspace matching. In this paper, 1) a new discriminative method that maximises orthogonality between subspaces is proposed. The method improves the discrimination power of the subspace angle based face recognition method by maximizing the angles between different classes. 2) We propose a method for on-line updating the discriminative subspaces as a mechanism for continuously improving recognition accuracy. 3) A further enhancement called locally orthogonal subspace method is presented to maximise the orthogonality between competing classes. Experiments using 700 face image sets have shown that the proposed method outperforms relevant prior art and effectively boosts its accuracy by online learning. It is shown that the method for online learning delivers the same solution as the batch computation at far lower computational cost and the locally orthogonal method exhibits improved accuracy. We also demonstrate the merit of the proposed face recognition method on portal scenarios of multiple biometric grand challenge.

Comparison of on-line learning algorithms for RBF models in greenhouse environmental control problems

The problem with the adequacy of radial basis function neural networks to model the inside air temperature as a function of the outside air temperature and solar radiation, and the inside relative humidity in an hydroponic greenhouse is addressed.

Designing on-line learning systems using participation

The review of literature pertaining to systems analysis and design and the design of systems for on-line teaching and learning has identified some "gaps" and shown the need for participation in educational system design. This paper presents research which was conducted to develop an approach for the design of educational systems involving the participation of student and academics in the design of educational on-line learning systems.

Evaluation of the fuzzy ARTMAP neural network using off-line and on-line learning strategies

This paper describes an experimental study of the Fuzzy ARTMAP (FAM) neural network as an autonomous learning system for pattern classification tasks. A benchmark database of radar signals from ionosphere has been employed for the system to classify arbitrary sequences of pattern into distinct categories. A number of simulations have been conducted systematically to evaluate the applicability and usefulness of FAM in this context. First, we identify the 'optimum' parameter settings of FAM for the problem at hand, and investigate the effects of different training schemes and learning rules on classification results, using an off-line learning methodology. We then examine a voting strategy to improve classification accuracy by combining results from multiple FAM classifiers. In addition to off-line learning, we evaluate the prospect of using FAM as an autonomously learning pattern classification system for on-line, non-stationary environments. The performance of FAM is comparable with other reported results, but with the added advantage of on-line and incremental learning.

On-line learning of a fuzzy controller for a precise vehicle cruise control system

Usually, vehicle applications require the use of artificial intelligent techniques to implement control methods, due to noise provided by sensors or the impossibility of full knowledge about dynamics of the vehicle (engine state, wheel pressure or occupiers weight). This work presents a method to on-line evolve a fuzzy controller for commanding vehicles? pedals at low speeds; in this scenario, the slightest alteration in the vehicle or road conditions can vary controller?s behavior in a non predictable way. The proposal adapts singletons positions in real time, and trapezoids used to codify the input variables are modified according with historical data. Experimentation in both simulated and real vehicles are provided to show how fast and precise the method is, even compared with a human driver or using different vehicles.

On-line learning in multilayer neural networks

We present an analytic solution to the problem of on-line gradient-descent learning for two-layer neural networks with an arbitrary number of hidden units in both teacher and student networks. The technique, demonstrated here for the case of adaptive input-to-hidden weights, becomes exact as the dimensionality of the input space increases.

Adaptive back-propagation in on-line learning of multilayer networks

An adaptive back-propagation algorithm is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, both numerical studies and a rigorous analysis show that the adaptive back-propagation method results in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent.

Using ancillary statistics in on-line learning algorithms

Neural networks are usually curved statistical models. They do not have finite dimensional sufficient statistics, so on-line learning on the model itself inevitably loses information. In this paper we propose a new scheme for training curved models, inspired by the ideas of ancillary statistics and adaptive critics. At each point estimate an auxiliary flat model (exponential family) is built to locally accommodate both the usual statistic (tangent to the model) and an ancillary statistic (normal to the model). The auxiliary model plays a role in determining credit assignment analogous to that played by an adaptive critic in solving temporal problems. The method is illustrated with the Cauchy model and the algorithm is proved to be asymptotically efficient.

Finite-size effects in on-line learning of multilayer neural networks

We complement recent advances in thermodynamic limit analyses of mean on-line gradient descent learning dynamics in multi-layer networks by calculating fluctuations possessed by finite dimensional systems. Fluctuations from the mean dynamics are largest at the onset of specialisation as student hidden unit weight vectors begin to imitate specific teacher vectors, increasing with the degree of symmetry of the initial conditions. In light of this, we include a term to stimulate asymmetry in the learning process, which typically also leads to a significant decrease in training time.

Globally optimal parameters for on-line learning in multilayer neural networks

We present a framework for calculating globally optimal parameters, within a given time frame, for on-line learning in multilayer neural networks. We demonstrate the capability of this method by computing optimal learning rates in typical learning scenarios. A similar treatment allows one to determine the relevance of related training algorithms based on modifications to the basic gradient descent rule as well as to compare different training methods.

Globally optimal on-line learning rules for multi-layer neural networks

We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This rule maximizes the total reduction in generalization error over the whole learning process. A simple example demonstrates that the locally optimal rule, which maximizes the rate of decrease in generalization error, may perform poorly in comparison.