Noise, regularizers, and unrealizable scenarios in online learning from restricted training sets

We study the dynamics of on-line learning in multilayer neural networks where training examples are sampled with repetition and where the number of examples scales with the number of network weights. The analysis is carried out using the dynamical replica method aimed at obtaining a closed set of coupled equations for a set of macroscopic variables from which both training and generalization errors can be calculated. We focus on scenarios whereby training examples are corrupted by additive Gaussian output noise and regularizers are introduced to improve the network performance. The dependence of the dynamics on the noise level, with and without regularizers, is examined, as well as that of the asymptotic values obtained for both training and generalization errors. We also demonstrate the ability of the method to approximate the learning dynamics in structurally unrealizable scenarios. The theoretical results show good agreement with those obtained by computer simulations.

Mirror-image relations in category learning

The discrimination of patterns that are mirror-symmetric counterparts of each other is difficult and requires substantial training. We explored whether mirror-image discrimination during expertise acquisition is based on associative learning strategies or involves a representational shift towards configural pattern descriptions that permit resolution of symmetry relations. Subjects were trained to discriminate between sets of unfamiliar grey-level patterns in two conditions, which either required the separation of mirror images or not. Both groups were subsequently tested in a 4-class category-learning task employing the same set of stimuli. The results show that subjects who had successfully learned to discriminate between mirror-symmetric counterparts were distinctly faster in the categorization task, indicating a transfer of conceptual knowledge between the two tasks. Additional computer simulations suggest that the development of such symmetry concepts involves the construction of configural, protoholistic descriptions, in which positions of pattern parts are encoded relative to a spatial frame of reference.

An investigation of the factor-analytic approach to the determination of abilities involved in psychomotor learning

This research began with an attempt to solve a practical problem, namely, the prediction of the rate at which an operator will learn a task. From a review of the literature, communications with researchers in this area and the study of psychomotor learning in factories it was concluded that a more fundamental approach was required which included the development of a task taxonomy. This latter objective had been researched for over twenty years by E. A. Fleishman and his approach was adopted. Three studies were carried out to develop and extend Fleishman's approach to the industrial area. However, the results of these studies were not in accord with FIeishman's conclusions and suggested that a critical re-assessment was required of the arguments, methods and procedures used by Fleishman and his co-workers. It was concluded that Fleishman's findings were to some extent an artifact of the approximate methods and procedures which he used in the original factor analyses and that using the more modern computerised factor analytic methods a reliable ability taxonomy could be developed to describe the abilities involved in the learning of psychomotor tasks. The implications for a changing-task or changing-subject model were drawn and it was concluded that a changing task and subject model needs to be developed.

Building belief in organizational learning: the role of organizational learning efficacy

There appears to be a missing dimension in OL literature to embrace the collective experience of emotion, both within groups and communities and also across the organization as a whole. The concept of OL efficacy- as a stimulus offering energy and direction for learning - remains unexplored. This research involved engaging with a company we have called ‘Electroco’ in depth to create a rich and nuanced representation of OL and members’ perceptions of OL over an extended time-frame (five years). We drew upon grounded theory research methodology (Locke, 2001), to elicit feedback from the organization, which was then used to inform future research plans and/ or refine emerging ideas. The concept of OL efficacy gradually emerged as a factor to be considered when exploring the relationship between individual learning and OL. . Bearing in mind Bandura’s (1982) conceptualization of self-efficacy (linked with mastery, modelling, verbal persuasion and emotional arousal), we developed a coding strategy encompassing these four factors as conceptualized at the organizational level. We added a fifth factor: ‘control of OL.’ We focused on feelings across the organization and the extent of consensus or otherwise around these five attributes. The construct has potential significance for how people are managed in many ways. Not only is OL efficacy is difficult for competitors to copy (arising as it does from the collective experience of working within a specific context); the self-efficacy concept suggests that success can be engineered with ‘small wins’ to reinforce mastery perceptions. Leaders can signal the importance of interaction with the external context, and encourage reflection on the strategies adopted by competitors or benchmark organizations (modelling). The theory also underlines the key role managers may play in persuading others about their organization’s propensity to learn (by focusing on success stories, for example). Research is set to continue within other sectors, including the high-performance financial service sector as well as the health-care technology sector.

More than a cognitive experience:unfamiliarity, invalidation, and emotion in organizational learning

Literature on organizational learning (OL) lacks an integrative framework that captures the emotions involved as OL proceeds. Drawing on personal construct theory, we suggest that organizations learn where their members reconstrue meaning around questions of strategic significance for the organization. In this 5-year study of an electronics company, we explore the way in which emotions change as members perceive progress or a lack of progress around strategic themes. Our framework also takes into account whether OL involves experiences that are familiar or unfamiliar and the implications for emotions. We detected similar patterns of emotion arising over time for three different themes in our data, thereby adding to OL perspectives that are predominantly cognitive in orientation. © The Author(s) 2013.

The influence of reference groups in the learning situation

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT

Personal knowledge development in online learning:a model for measuring externalisation, combination and internalisation

This paper presents a model for measuring personal knowledge development in online learning environments. It is based on Nonaka‘s SECI model of organisational knowledge creation. It is argued that Socialisation is not a relevant mode in the context of online learning and was therefore not covered in the measurement instrument. Therefore, the remaining three of SECI‘s knowledge conversion modes, namely Externalisation, Combination, and Internalisation were used and a measurement instrument was created which also examines the interrelationships between the three modes. Data was collected using an online survey, in which online learners report on their experiences of personal knowledge development in online learning environments. In other words, the instrument measures the magnitude of online learners‘ Externalisation and combination activities as well as their level of internalisation, which is the outcome of their personal knowledge development in online learning.

Good practice guide in learning and teaching

This is the second edition of our Aston Business School (ABS) Good Practice Guide and the enthusiasm of the contributors appears undiminished. I am again reminded that I work with a group of very committed, dedicated and professional colleagues. Once again this publication is produced to celebrate and promote good teaching across the School and to offer encouragement to those imaginative and innovative staff who continue to wish to challenge students to learn to maximum effect. It is hoped that others will pick up some good ideas from the articles contained in this volume. Contributors to this Guide were not chosen because they are the best teachers in the School, although they are undoubtedly all amongst my colleagues who are exponents of enthusiastic and inspiring approaches to learning. The Quality Unit approached these individuals because they declared on their Annual Module Reflection Forms that they were doing something interesting and worthwhile which they thought others might find useful. Amongst those reading the Guide I am sure that there are many other individuals who are trying to operate similar examples of good practice in their teaching, learning and assessment methods. I hope that this publication will provoke these people into providing comments and articles of their own and that these will form the basis of next year’s Guide. It may also provoke some people to try these methods in their own teaching. The themes of the articles this year can be divided into two groups. The first theme is the quest to help students to help themselves to learn via student-run tutorials, surprise tests and mock examinations linked with individual tutorials. The second theme is making learning come to life in exciting practical ways by, for example, hands-on workshops and simulations, story telling, rhetorical questioning and discussion groups. A common theme is one of enthusiasm, reflection and commitment on behalf of the lecturers concerned. None of the approaches discussed in this publication are low effort activities on the part of the facilitator, but this effort is regarded as worthwhile as a means of creating greater student engagement. As Biggs (2003)[1] says, in his similarly inspiring way, students learn more the less passive they are in their learning. (Ref). The articles in this publication bear witness of this and much more. Since last year Aston Business School has launched its Research Centre in Higher Education Learning and Management (HELM) which is another initiative to promote excellent learning and teaching. Even before this institution has become fully operational, at least one of the articles in this publication has seen the light of day in the research arena and at least two others are ripe for dissemination to a wider audience via journal publication. More news of our successes in this activity will appear in next year’s edition. May I thank the contributors for taking time out of their busy schedules to write the articles this summer, and to Julie Green who runs the ABS Quality Unit, for putting our diverse approaches into a coherent and publishable form and for chasing us when we have needed it! I would also like to thank Ann Morton and her colleagues in the Centre for Staff Development who have supported this publication. During the last year the Centre has further stimulated the learning and teaching life of the School (and the wider University) via their Learning and Teaching Week and sponsorship of Teaching Quality Enhancement Fund (TQEF) projects. Pedagogic excellence is in better health at Aston than ever before – long may this be because this is what life in HE should be about.

Finite-size effects in on-line learning of multilayer neural networks

We complement recent advances in thermodynamic limit analyses of mean on-line gradient descent learning dynamics in multi-layer networks by calculating fluctuations possessed by finite dimensional systems. Fluctuations from the mean dynamics are largest at the onset of specialisation as student hidden unit weight vectors begin to imitate specific teacher vectors, increasing with the degree of symmetry of the initial conditions. In light of this, we include a term to stimulate asymmetry in the learning process, which typically also leads to a significant decrease in training time.

Globally optimal parameters for on-line learning in multilayer neural networks

We present a framework for calculating globally optimal parameters, within a given time frame, for on-line learning in multilayer neural networks. We demonstrate the capability of this method by computing optimal learning rates in typical learning scenarios. A similar treatment allows one to determine the relevance of related training algorithms based on modifications to the basic gradient descent rule as well as to compare different training methods.

Noisy fitness evaluation in genetic algorithms and the dynamics of learning

A theoretical model is presented which describes selection in a genetic algorithm (GA) under a stochastic fitness measure and correctly accounts for finite population effects. Although this model describes a number of selection schemes, we only consider Boltzmann selection in detail here as results for this form of selection are particularly transparent when fitness is corrupted by additive Gaussian noise. Finite population effects are shown to be of fundamental importance in this case, as the noise has no effect in the infinite population limit. In the limit of weak selection we show how the effects of any Gaussian noise can be removed by increasing the population size appropriately. The theory is tested on two closely related problems: the one-max problem corrupted by Gaussian noise and generalization in a perceptron with binary weights. The averaged dynamics can be accurately modelled for both problems using a formalism which describes the dynamics of the GA using methods from statistical mechanics. The second problem is a simple example of a learning problem and by considering this problem we show how the accurate characterization of noise in the fitness evaluation may be relevant in machine learning. The training error (negative fitness) is the number of misclassified training examples in a batch and can be considered as a noisy version of the generalization error if an independent batch is used for each evaluation. The noise is due to the finite batch size and in the limit of large problem size and weak selection we show how the effect of this noise can be removed by increasing the population size. This allows the optimal batch size to be determined, which minimizes computation time as well as the total number of training examples required.

Globally optimal learning rates in multilayer neural networks

A method for calculating the globally optimal learning rate in on-line gradient-descent training of multilayer neural networks is presented. The method is based on a variational approach which maximizes the decrease in generalization error over a given time frame. We demonstrate the method by computing optimal learning rates in typical learning scenarios. The method can also be employed when different learning rates are allowed for different parameter vectors as well as to determine the relevance of related training algorithms based on modifications to the basic gradient descent rule.