Incorporating curvature information into on-line learning

We analyse the dynamics of a number of second order on-line learning algorithms training multi-layer neural networks, using the methods of statistical mechanics. We first consider on-line Newton's method, which is known to provide optimal asymptotic performance. We determine the asymptotic generalization error decay for a soft committee machine, which is shown to compare favourably with the result for standard gradient descent. Matrix momentum provides a practical approximation to this method by allowing an efficient inversion of the Hessian. We consider an idealized matrix momentum algorithm which requires access to the Hessian and find close correspondence with the dynamics of on-line Newton's method. In practice, the Hessian will not be known on-line and we therefore consider matrix momentum using a single example approximation to the Hessian. In this case good asymptotic performance may still be achieved, but the algorithm is now sensitive to parameter choice because of noise in the Hessian estimate. On-line Newton's method is not appropriate during the transient learning phase, since a suboptimal unstable fixed point of the gradient descent dynamics becomes stable for this algorithm. A principled alternative is to use Amari's natural gradient learning algorithm and we show how this method provides a significant reduction in learning time when compared to gradient descent, while retaining the asymptotic performance of on-line Newton's method.

Training Bayesian networks for image segmentation

We are concerned with the problem of image segmentation in which each pixel is assigned to one of a predefined finite number of classes. In Bayesian image analysis, this requires fusing together local predictions for the class labels with a prior model of segmentations. Markov Random Fields (MRFs) have been used to incorporate some of this prior knowledge, but this not entirely satisfactory as inference in MRFs is NP-hard. The multiscale quadtree model of Bouman and Shapiro (1994) is an attractive alternative, as this is a tree-structured belief network in which inference can be carried out in linear time (Pearl 1988). It is an hierarchical model where the bottom-level nodes are pixels, and higher levels correspond to downsampled versions of the image. The conditional-probability tables (CPTs) in the belief network encode the knowledge of how the levels interact. In this paper we discuss two methods of learning the CPTs given training data, using (a) maximum likelihood and the EM algorithm and (b) emphconditional maximum likelihood (CML). Segmentations obtained using networks trained by CML show a statistically-significant improvement in performance on synthetic images. We also demonstrate the methods on a real-world outdoor-scene segmentation task.

On-line learning of unrealizable tasks

The dynamics of on-line learning is investigated for structurally unrealizable tasks in the context of two-layer neural networks with an arbitrary number of hidden neurons. Within a statistical mechanics framework, a closed set of differential equations describing the learning dynamics can be derived, for the general case of unrealizable isotropic tasks. In the asymptotic regime one can solve the dynamics analytically in the limit of large number of hidden neurons, providing an analytical expression for the residual generalization error, the optimal and critical asymptotic training parameters, and the corresponding prefactor of the generalization error decay.

On-line learning in neural networks

On-line learning is one of the most powerful and commonly used techniques for training large layered networks and has been used successfully in many real-world applications. Traditional analytical methods have been recently complemented by ones from statistical physics and Bayesian statistics. This powerful combination of analytical methods provides more insight and deeper understanding of existing algorithms and leads to novel and principled proposals for their improvement. This book presents a coherent picture of the state-of-the-art in the theoretical analysis of on-line learning. An introduction relates the subject to other developments in neural networks and explains the overall picture. Surveys by leading experts in the field combine new and established material and enable non-experts to learn more about the techniques and methods used. This book, the first in the area, provides a comprehensive view of the subject and will be welcomed by mathematicians, scientists and engineers, whether in industry or academia.

The theory of on-line learning: a statistical physics approach

In this paper we review recent theoretical approaches for analysing the dynamics of on-line learning in multilayer neural networks using methods adopted from statistical physics. The analysis is based on monitoring a set of macroscopic variables from which the generalisation error can be calculated. A closed set of dynamical equations for the macroscopic variables is derived analytically and solved numerically. The theoretical framework is then employed for defining optimal learning parameters and for analysing the incorporation of second order information into the learning process using natural gradient descent and matrix-momentum based methods. We will also briefly explain an extension of the original framework for analysing the case where training examples are sampled with repetition.

Challenges for expatriate managers:an exploration of cross-cultural management, adjustment, and training issues faced by Anglo-American managers in Hong Kong

This field work study furthers understanding about expatriate management, in particular, the nature of cross-cultural management in Hong Kong involving Anglo-American expatriate and Chinese host national managers, the important features of adjustment for expatriates living and working there, and the type of training which will assist them to adjust and to work successfully in this Asian environment. Qualitative and quantitative data on each issue was gathered during in-depth interviews in Hong Kong, using structured interview schedules, with 39 expatriate and 31 host national managers drawn from a cross-section of functional areas and organizations. Despite the adoption of Western technology and the influence of Western business practices, micro-level management in Hong Kong retains a cultural specificity which is consistent with the norms and values of Chinese culture. There are differences in how expatriates and host nationals define their social roles, and Hong Kong's recent colonial history appears to influence cross-cultural interpersonal interactions. The inability of the spouse and/or family to adapt to Hong Kong is identified as a major reason for expatriate assignments to fail, though the causes have less to do with living away from family and friends, than with Hong Kong's highly urbanized environment and the heavy demands of work. Culture shock is not identified as a major problem, but in Hong Kong micro-level social factors require greater adjustment than macro-level societal factors. The adjustment of expatriate managers is facilitated by a strong orientation towards career development and hard work, possession of technical/professional expertise, and a willingness to engage in a process of continuous 'active learning' with respect to the host national society and culture. A four-part model of manager training suitable for Hong Kong is derived from the study data. It consists of a pre-departure briefing, post-arrival cross-cultural training, language training in basic Cantonese and in how to communicate more effectively in English with non-native speakers, and the assignment of a mentor to newly arrived expatriate managers.

Mapping teaching, learning and assessment in the MPharm in UK schools of pharmacy

Aim To undertake a national study of teaching, learning and assessment in UK schools of pharmacy. Design Triangulation of course documentation, 24 semi-structured interviews undertaken with 29 representatives from the schools and a survey of all final year students (n=1,847) in the 15 schools within the UK during 2003–04. Subjects and setting All established UK pharmacy schools and final year MPharm students. Outcome measures Data were combined and analysed under the topics of curriculum, teaching and learning, assessment, multi-professional teaching and learning, placement education and research projects. Results Professional accreditation was the main driver for curriculum design but links to preregistration training were poor. Curricula were consistent but offered little student choice. On average half the curriculum was science-based. Staff supported the science content but students less so. Courses were didactic but schools were experimenting with new methods of learning. Examinations were the principal form of assessment but the contribution of practice to the final degree ranged considerably (21–63%). Most students considered the assessment load to be about right but with too much emphasis upon knowledge. Assessment of professional competence was focused upon dispensing and pharmacy law. All schools undertook placement teaching in hospitals but there was little in community/primary care. There was little inter-professional education. Resources and logistics were the major limiters. Conclusions There is a need for an integrated review of the accreditation process for the MPharm and preregistration training and redefinition of professional competence at an undergraduate level.

Mirror-image relations in category learning

The discrimination of patterns that are mirror-symmetric counterparts of each other is difficult and requires substantial training. We explored whether mirror-image discrimination during expertise acquisition is based on associative learning strategies or involves a representational shift towards configural pattern descriptions that permit resolution of symmetry relations. Subjects were trained to discriminate between sets of unfamiliar grey-level patterns in two conditions, which either required the separation of mirror images or not. Both groups were subsequently tested in a 4-class category-learning task employing the same set of stimuli. The results show that subjects who had successfully learned to discriminate between mirror-symmetric counterparts were distinctly faster in the categorization task, indicating a transfer of conceptual knowledge between the two tasks. Additional computer simulations suggest that the development of such symmetry concepts involves the construction of configural, protoholistic descriptions, in which positions of pattern parts are encoded relative to a spatial frame of reference.

The development and evaluation of distance learning materials for administrative personnel in a multi-national company

There has been substantial research into the role of distance learning in education. Despite the rise in the popularity and practice of this form of learning in business, there has not been a parallel increase in the amount of research carried out in this field. An extensive investigation was conducted into the entire distance learning system of a multi-national company with particular emphasis on the design, implementation and evaluation of the materials. In addition, the performance and attitudes of trainees were examined. The results of a comparative study indicated that trainees using distance learning had significantly higher test scores than trainees using conventional face-to-face training. The influence of the previous distance learning experience, educational background and selected study environment of trainees was investigated. Trainees with previous experience of distance learning were more likely to complete the course and with significantly higher test scores than trainees with no previous experience. The more advanced the educational background of trainees, the greater the likelihood of their completing the course, although there was no significant difference in the test scores achieved. Trainees preferred to use the materials at home and those opting to study in this environment scored significantly higher than those studying in the office, the study room at work or in a combination of environments. The influence of learning styles (Kolb, 1976) was tested. The results indicated that the convergers had the greatest completion rates and scored significantly higher than trainees with the assimilator, accommodator and diverger learning styles. The attitudes of the trainees, supervisors and trainers were examined using questionnaire, interview and discussion techniques. The findings highlighted the potential problems of lack of awareness and low motivation which could prove to be major obstacles to the success of distance learning in business.

Learning for life:educational provision at Cadbury, 1831-1981

Cadbury showed concern for the welfare of its labour force in a variety of ways and not least in the provision of educational and educative-recreational facilities. The firm regarded the education of employees as having a positive effect on the efficiency of the business at the same time as being of benefit to the individual, the local community and the nation. The life-long education of people was seen as essential for personal fulfilament, social improvement, economic competitiveness and the proper functioning of democratic procedures. The educational system built up at Cadbury, and the philosophy on which it was founded, acquired both a domestic and international reputation. Its main components were the day continuation education of juniors; the Bournville Works Evening Institute; vocational and non-vocational scholarships; emphasis on the primary importance of general education as a basis for life, work and technical training; stress on equality of educational opportunity for females; and leisure and sporting amenities which the firm felt to be educative in the sense that they contributed to personal psychological and physical development and social skills. The system was primarily shaped and constructed in the first three decades of the twentieth century and went into decline and eventual demise in the 1960's and 1970's as a result of economic pressures, social changes, enhanced state arrangements for education, shifts in Cadbury management thinking and the merger with Schweppes in 1969.

Intercultural training for international placements

Despite its increasing popularity, much intercultural training is not developed with the same level of rigour as training in other areas. Further, research on intercultural training has brought inconsistent results about the effectiveness of such training. This PhD thesis develops a rigorous model of intercultural training and applies it to the preparation of British students going on work/study placements in France and Germany. It investigates the reasons for inconsistent training success by looking at the cognitive learning processes in intercultural training, relating them to training goals, and by examining the short- and long-term transfer of intercultural training into real-life encounters with people from other cultures. Two cognitive trainings based on critical incidents were designed for online delivery. The training content relied on cultural practice dimensions from the GWBE study (House, Hanges, Javidan, Dorfman & Gupta, 2004). Of the two trainings, the 'singlemode training' aimed to develop declarative knowledge, which is necessary to analyse and understand other cultures. The 'concurrent training' aimed to develop declarative and procedural knowledge, which is needed to develop skills for dealing with difficult situations in a culturally appropriate way. Participants (N-48) were randomly assigned to one of the two training conditions. Declarative learning appeared as a process of steady knowledge increase, while procedural learning involved cognitive re-categorisation rather than knowledge increase. In a negotiation role play with host-country nationals directly after the online training, participants of the concurrent training exhibited a more initiative negotiation style than participants of the single-mode training. Comparing cultural adjustment and performance of training participants during their time abroad with an untrained control group, participants of the concurrent training showed the qualitatively best development in adjustment and performance. Besides intercultural training, multicultural personality traits were assessed and proved to be a powerful predictor of adjustment and, indirectly, of performance abroad.

An evaluation of pharmacy education and pre-registration training

This thesis covers two major aspects of pharmacy education; undergraduate education and pre-registration training. A cohort of pharmacy graduates were surveyed over a period of four years, on issues related to undergraduate education, pre-registration training and continuing education. These graduates were the first-ever to sit the pre-registration examination. In addition, the opinions of pre-registration tutors were obtained on pre-registration training, during the year that competence-based assessment was introduced. It was concluded that although the undergraduate course provided a broad base of knowledge suitable for graduates in all branches of pharmacy, several issues were identified which would require attention in future developments of the course. These were: 1. the strong support for the expansion of clinical, social and practice-based teaching. 2. the strong support to retain the scientific content to the same extent as in the three-year course. 3. a greater use of problem-based learning methods. The graduates supported the provision of a pre-registration continuing education course to help prepare for the examination and in areas inadequately covered in the undergraduate course. There was also support for the introduction of some form of split branch training. There was no strong evidence to suggest that the training had been an application of undergraduate education. In general, competence-based training was well regarded by tutors as an appropriate and effective method of skill assessment. However, community tutors felt it was difficult to carry out effectively due to day-to-day time constraints. The assistant tutors in hospital pharmacy were found to have a very important role in provision of training, and should be adequately trained and supported. The study recommends the introduction of uniform training and a quality assurance mechanism for all tutors and assistants undertaking this role.

Evaluation of the quality and promotion of open learning materials

In the UK, Open Learning has been used in industrial training for at least the last decade. Trainers and Open Learning practitioners have been concerned about the quality of the products and services being delivered. The argument put forward in this thesis is that there is ambiguity amongst industrialists over the meanings of `Open Learning' and `Quality in Open Learning'. For clarity, a new definition of Open Learning is proposed which challenges the traditional learner-centred approach favoured by educationalists. It introduces the concept that there are benefits afforded to the trainer/employer/teacher as well as to the learner. This enables a focussed view of what quality in Open Learning really means. Having discussed these issues, a new quantitative method of evaluating Open Learning is proposed. This is based upon an assessment of the degree of compliance with which products meet Parts 1 & 2 of the Open Learning Code of Practice. The vehicle for these research studies has been a commercial contract commissioned by the Training Agency for the Engineering Industry Training Board (EITB) to examine the quality of Open Learning products supplied to the engineering industry. A major part of this research has been the application of the evaluation technique to a range of 67 Open Learning products (in eight subject areas). The findings were that good quality products can be found right across the price range - so can average and poor quality ones. The study also shows quite convincingly that there are good quality products to be found at less than 50. Finally the majority (24 out of 34) of the good quality products were text based.

Doctoral work in translation studies as an interdisciplinary mutual learning process:how a translator, teacher educator and linguistic typologist worked together

The focus of this paper is on the doctoral research training experienced by one of the authors and the ways in which the diverse linguistic and disciplinary perspectives of her two supervisors (co-authors of this paper) mediated the completion of her study. The doctoral candidate is a professional translator/interpreter and translation teacher. The paper describes why and how she identified her research area and then focused on the major research questions in collaboration with her two supervisors, who brought their differing perspectives from the field of linguistics to this translation research, even though they are not translators by profession or disciplinary background and do not speak Korean. In addition, the discussion considers the focus, purpose and theoretical orientation of the research itself (which addressed questions of readability in translated English-Korean texts through detailed analysis of a corpus and implications for professional translator training) as well as the supervisory and conceptual processes and practices involved. The authors contend that doctoral research of this kind can be seen as a mutual learning process and that inter-disciplinary research can make a contribution not only to the development of rigorous research in the field of translation studies but also to the other disciplinary fields involved.

A bridge too far? Breaching the research/learning and teaching nexus in a previously research-led university

The relationship between research and learning and teaching represents what has been described as ‘amongst the most intellectually tangled, managerially complex and politically contentious issues in mass higher education’ (Scott, 2005, p 53). Despite this, arguments that in order to achieve high quality scholarly outcomes, university teachers need to adopt an approach to teaching similar to that of research (i.e. founded upon academic rigour and evidence), has long been discussed in the literature (see for example, Elton, 2005 & Healey, 2000). However, the practicalities of promoting an empirical and evidence-based approach to teaching within a research-led institution makes dealing with the research/learning and teaching nexus a somewhat challenging proposition. Drawing upon the findings of a mixed methodological study, this paper critically analyses the pedagogical, organisational and practical issues encountered by academics and support staff working within a newly established Centre for Learning Innovation and Professional Practice. Comprising an eclectic group of staff drawn from across the five Schools in the University, the Centre is dedicated to enhancing student learning through the development of evidence based teaching practice. Based upon the premise that the promotion of research-led teaching will act to bring teaching and research together, and in doing so enhance students learning experiences (Simmons & Elen 2007), the paper critically analyses the challenges encountered by staff responsible for developing and introducing a new learning & teaching focused organisational strategy (by reflecting on the previous 12 months work). In doing so it makes a significant contribution to current academic theory and debate in the areas of pedagogic practice and organisational management. Focusing specifically on the impact of the new policy on various aspects of university life including, pedagogic practice, student support, staff training, and organisational management, the paper critically addresses the cultural and attitudinal challenges of change management (Kotter, 1996) within a ‘grey-brick’ university. It concludes by arguing that the move towards becoming a more learning-focused university has started to develop an awareness of the positive impact the change initiative is having on the student experience and wider institution; whilst also drawing attention to the organisational challenges ahead.