On-line learning in radial basis functions networks

An analytic investigation of the average case learning and generalization properties of Radial Basis Function Networks (RBFs) is presented, utilising on-line gradient descent as the learning rule. The analytic method employed allows both the calculation of generalization error and the examination of the internal dynamics of the network. The generalization error and internal dynamics are then used to examine the role of the learning rate and the specialization of the hidden units, which gives insight into decreasing the time required for training. The realizable and over-realizable cases are studied in detail; the phase of learning in which the hidden units are unspecialized (symmetric phase) and the phase in which asymptotic convergence occurs are analyzed, and their typical properties found. Finally, simulations are performed which strongly confirm the analytic results.

Incorporating curvature information into on-line learning

We analyse the dynamics of a number of second order on-line learning algorithms training multi-layer neural networks, using the methods of statistical mechanics. We first consider on-line Newton's method, which is known to provide optimal asymptotic performance. We determine the asymptotic generalization error decay for a soft committee machine, which is shown to compare favourably with the result for standard gradient descent. Matrix momentum provides a practical approximation to this method by allowing an efficient inversion of the Hessian. We consider an idealized matrix momentum algorithm which requires access to the Hessian and find close correspondence with the dynamics of on-line Newton's method. In practice, the Hessian will not be known on-line and we therefore consider matrix momentum using a single example approximation to the Hessian. In this case good asymptotic performance may still be achieved, but the algorithm is now sensitive to parameter choice because of noise in the Hessian estimate. On-line Newton's method is not appropriate during the transient learning phase, since a suboptimal unstable fixed point of the gradient descent dynamics becomes stable for this algorithm. A principled alternative is to use Amari's natural gradient learning algorithm and we show how this method provides a significant reduction in learning time when compared to gradient descent, while retaining the asymptotic performance of on-line Newton's method.

Globally optimal on-line learning rules

We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.

On-line learning of unrealizable tasks

The dynamics of on-line learning is investigated for structurally unrealizable tasks in the context of two-layer neural networks with an arbitrary number of hidden neurons. Within a statistical mechanics framework, a closed set of differential equations describing the learning dynamics can be derived, for the general case of unrealizable isotropic tasks. In the asymptotic regime one can solve the dynamics analytically in the limit of large number of hidden neurons, providing an analytical expression for the residual generalization error, the optimal and critical asymptotic training parameters, and the corresponding prefactor of the generalization error decay.

On-line learning in neural networks

On-line learning is one of the most powerful and commonly used techniques for training large layered networks and has been used successfully in many real-world applications. Traditional analytical methods have been recently complemented by ones from statistical physics and Bayesian statistics. This powerful combination of analytical methods provides more insight and deeper understanding of existing algorithms and leads to novel and principled proposals for their improvement. This book presents a coherent picture of the state-of-the-art in the theoretical analysis of on-line learning. An introduction relates the subject to other developments in neural networks and explains the overall picture. Surveys by leading experts in the field combine new and established material and enable non-experts to learn more about the techniques and methods used. This book, the first in the area, provides a comprehensive view of the subject and will be welcomed by mathematicians, scientists and engineers, whether in industry or academia.

The theory of on-line learning: a statistical physics approach

In this paper we review recent theoretical approaches for analysing the dynamics of on-line learning in multilayer neural networks using methods adopted from statistical physics. The analysis is based on monitoring a set of macroscopic variables from which the generalisation error can be calculated. A closed set of dynamical equations for the macroscopic variables is derived analytically and solved numerically. The theoretical framework is then employed for defining optimal learning parameters and for analysing the incorporation of second order information into the learning process using natural gradient descent and matrix-momentum based methods. We will also briefly explain an extension of the original framework for analysing the case where training examples are sampled with repetition.

A Bayesian approach to on-line learning

Online learning is discussed from the viewpoint of Bayesian statistical inference. By replacing the true posterior distribution with a simpler parametric distribution, one can define an online algorithm by a repetition of two steps: An update of the approximate posterior, when a new example arrives, and an optimal projection into the parametric family. Choosing this family to be Gaussian, we show that the algorithm achieves asymptotic efficiency. An application to learning in single layer neural networks is given.

Mapping teaching, learning and assessment in the MPharm in UK schools of pharmacy

Aim To undertake a national study of teaching, learning and assessment in UK schools of pharmacy. Design Triangulation of course documentation, 24 semi-structured interviews undertaken with 29 representatives from the schools and a survey of all final year students (n=1,847) in the 15 schools within the UK during 2003–04. Subjects and setting All established UK pharmacy schools and final year MPharm students. Outcome measures Data were combined and analysed under the topics of curriculum, teaching and learning, assessment, multi-professional teaching and learning, placement education and research projects. Results Professional accreditation was the main driver for curriculum design but links to preregistration training were poor. Curricula were consistent but offered little student choice. On average half the curriculum was science-based. Staff supported the science content but students less so. Courses were didactic but schools were experimenting with new methods of learning. Examinations were the principal form of assessment but the contribution of practice to the final degree ranged considerably (21–63%). Most students considered the assessment load to be about right but with too much emphasis upon knowledge. Assessment of professional competence was focused upon dispensing and pharmacy law. All schools undertook placement teaching in hospitals but there was little in community/primary care. There was little inter-professional education. Resources and logistics were the major limiters. Conclusions There is a need for an integrated review of the accreditation process for the MPharm and preregistration training and redefinition of professional competence at an undergraduate level.

On-line learning in a changing environment

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT

A comparison of the effects of social constructivist and traditional approaches to teaching on students' attitude and achievement in high school chemistry

Current views of the nature of knowledge and of learning suggest that instructional approaches in science education pay closer attention to how students learn rather than on teaching. This study examined the use of approaches to teaching science based on two contrasting perspectives in learning, social constructivist and traditional, and the effects they have on students' attitudes and achievement. Four categories of attitudes were measured using the Upper Secondary Attitude Questionnaire: Attitude towards school, towards the importance of science, towards science as a career, and towards science as a subject in school. Achievement was measured by average class grades and also with a researcher/teacher constructed 30-item test that involved three sub-scales of items based on knowledge, and applications involving near-transfer and far-transfer of concepts. The sample consisted of 202 students in nine intact classrooms in chemistry at a large high school in Miami, Florida, and involved two teachers. Results were analyzed using a two-way analysis of covariance (ANCOVA) with a pretest in attitude as the covariate for attitudes and prior achievement as the covariate for achievement. A comparison of the adjusted mean scores was made between the two groups and between females and males. ^ With constructivist-based teaching, students showed more favorable attitude towards science as a subject, obtained significantly higher scores in class achievement, total achievement and achievement on the knowledge sub-scale of the knowledge and application test. Students in the traditional group showed more favorable attitude towards school. Females showed significantly more positive attitude towards the importance of science and obtained significantly higher scores in class achievement. No significant interaction effects were obtained for method of instruction by gender. ^ This study lends some support to the view that constructivist-based approaches to teaching science is a viable alternative to traditional modes of teaching. It is suggested that in science education, more consideration be given to those aspects of classroom teaching that foster closer coordination between social influences and individual learning. ^

Revisiting Entrepreneurship Education Literature: Implications for Learning and Teaching Entrepreneurship

Courses and programs about entrepreneurship show so much variation that it is hard to identify typical teaching strategies. Although diversity is good, consistency is needed because the value of entrepreneurship education has not been established. A literature review on teaching and learning in entrepreneurship was conducted; three challenges were identified.

Issues in On-Line Legal Translation Teaching

A description is given of the structure of, theoretical background of, and experiences gained from, a course in legal English-Spanish translation taught at the University of Alicante.

Enhancing student learning with case-based teaching and audience response systems in an interdisciplinary Food Science course

A growing body of research in higher education suggests that teachers should move away from traditional lecturing towards more active and student-focus education approaches. Several classroom techniques are available to engage students and achieve more effective teaching and better learning experiences. The purpose of this paper is to share an example of how two of them – case-based teaching, and the use of response technologies – were implemented into a graduate-level food science course. The paper focuses in particular on teaching sensory science and sensometrics, including several concrete examples used during the course, and discussing in each case some of the observed outcomes. Overall, it was observed that the particular initiatives were effective in engaging student participation and promoting a more active way of learning. Case-base teaching provided students with the opportunity to apply their knowledge and their analytical skills to complex, real-life scenarios relevant to the subject matter. The use of audience response systems further facilitated class discussion, and was extremely well received by the students, providing a more enjoyable classroom experience.

Learning Process in Teaching and Bio-pedagogical Knowledge Building

Bio-pedagogy is built on praxis, i.e. the interrelationship between reflection and innovative action where these two merge in the construction of senses to generate knowledge. Then, the following question arises: How is teaching understood? How can practice be renovated from the action-reflection-action in a recurring manner and in life itself? A way to search for those answers is the systematization of experiences –a modality of qualitative research. It promotes the transformation of a common practice, based on knowledge building by holistic approaches to the educational process complexity. The systematization of bio-pedagogical experiences involves self-organization, joy, uncertainty and passion; it respects freedom and autonomy, and generates relational spaces, which promote creative processes in learning.