Good practice guide in learning and teaching

Once again this publication is produced to celebrate and promote good teaching and learning support and to offer encouragement to those imaginative and innovative staff who continue to wish to challenge students to learn to maximum effect. It is hoped that others will pick up some good ideas from the articles contained in this volume. We have again changed our approach for this 2007/08 edition (our fifth) of the Aston Business School Good Practice Guide. As before, some contributions were selected from those identifying interesting best practice on their Annual Module Reflection Forms in 2006/2007. Brookes? contribution this year is directly from her annual reflection. Other contributors received HELM (Research Centre in Higher Education Learning and Management) small research grants in 2006/2007. Part of the conditions were for them to write an article for this publication. We have also been less tight on the length of the articles this year. Some contributions are, therefore, on the way to being journal articles. HELM will be working with these authors to help develop these for publication. Looking back over the last five years it is brilliant to see how many different people have contributed over the years and, therefore, how much innovative learning and teaching work has been taking place in ABS over this time. In the first edition we were just pleased for people to write a few pages on their teaching. Now things have changed dramatically. The majority of the articles are grounded in empirical research (some funded by HELM small research grants) and Palmer?s article was produced as part of the University?s Postgraduate Certificate in Learning and Teaching. Most encouraging of all, four of this year?s articles have since been developed further and submitted to refereed journals. We await news of publication as we go to press. It is not surprising that how to manage large groups still remains a central theme of the articles, ABS has a large and still growing student body. Essex and Simpson have looked at trying to encourage students to attend taught sessions, on the basis that there is a strong correlation between attendance and higher performance. Their findings are forming the platform of a further study currently being carried out in the Undergraduate Programme. A number of the other articles concentrate on trying to encourage students to engage with study in an innovative way. This is particularly obvious in Shaw?s work. Everyone who has been around campus lately has had evidence that the students on Duncan?s modules have clearly been inspired. I found myself, for example, playing golf in the student dining room as part of this initiative! The articles by Jarzabkowski & Guilietti and Ho involved much larger surveys. This is another first for the Good Practice Guide and marks the first step on what will clearly be larger research efforts for these authors in this area. We look forward to the journal publications which will result from this work. The last articles are the result of HELM?s hosting of the national conference of the Higher Education Academy?s Business, Management, Accounting and Finance (BMAF) Subject Centre Conference in May 2007. Belal and Foster have written about their impressions of the Conference and Andrews has included the paper she gave. The papers on employability and widening participation are the centre of HELM?s current work. In the second volume we mentioned the launch of the School?s Research Centre in Higher Education Learning and Management (HELM). Since then HELM has stimulated a lot of activity across the School (and University) particularly linking research and teaching. A list of the HELM seminars for 2007/2008 is listed as Appendix 1 of this publication. Further details can be obtained from Catherine Foster (c.s.foster@aston.ac.uk), who coordinates the HELM seminars. We have also been working on a list of target journals to guide ABS staff who wish to publish in this area. These are included as Appendix 2 of this publication. May I thank the contributors for taking time out of their busy schedules to write the articles and to Julie Green, the Quality Manager, for putting the varying diverse approaches into a coherent and publishable form and for agreeing to fund the printing of this volume.

Good practice guide in learning and teaching

Once again this publication is produced to celebrate and promote good teaching and learning support and to offer encouragement to those imaginative and innovative staff who continue to wish to challenge students to learn to maximum effect. It is hoped that others will pick up some good ideas from the articles contained in this volume. We have again changed our approach for this 2006/07 edition (our fourth) of the Aston Business School Good Practice Guide. As before, some contributions were selected from those identifying interesting best practice on their Annual Module reflection forms in 2005/2006. Other contributors received HELM (Research Centre in Higher Education Learning and Management) small research grants in 2005/2006. Part of the conditions were for them to write an article for this publication. We have also been less tight on the length of the articles this year. Some contributions are, therefore, on the way to being journal articles. HELM will be working with these authors to help develop these for publication. The themes covered in this year?s articles are all central to the issues faced by those providing HE teaching and learning opportunities in the 21st Century. Specifically this is providing support and feedback to students in large classes, embracing new uses of technology to encourage active learning and addressing cultural issues in a diverse student population. Michael Grojean and Yves Guillaume used Blackboard™ to give a more interactive learning experience and improve feedback to students. It would be easy for other staff to adopt this approach. Patrick Tissington and Qin Zhou (HELM small research grant holders) were keen to improve the efficiency of student support, as does Roger McDermott. Celine Chew shares her action learning project, completed as part of the Aston University PG Certificate in Teaching and Learning. Her use of Blackboard™ puts emphasis on the learner having to do something to help them meet the learning outcomes. This is what learning should be like, but many of our students seem used to a more passive learning experience, so much needs to be done on changing expectations and cultures about learning. Regina Herzfeldt also looks at cultures. She was awarded a HELM small research grant and carried out some significant new research on cultural diversity in ABS and what it means for developing teaching methods. Her results fit in with what many of us are experiencing in practice. Gina leaves us with some challenges for the future. Her paper certainly needs to be published. This volume finishes with Stuart Cooper and Matt Davies reflecting on how to keep students busy in lectures and Pavel Albores working with students on podcasting. Pavel?s work, which was the result of another HELM small research grant, will also be prepared for publication as a journal article. The students learnt more from this work that any formal lecture and Pavel will be using the approach again this year. Some staff have been awarded HELM small research grants in 2006/07 and these will be published in the next Good Practice Guide. In the second volume we mentioned the launch of the School?s Research Centre in Higher Education Learning and Management (HELM). Since then HELM has stimulated a lot of activity across the School (and University) particularly linking research and teaching. A list of the HELM seminars for 2006/2007 is listed as Appendix 1 of this publication. Further details can be obtained from Catherine Foster (c.s.foster@aston.ac.uk), who coordinates the HELM seminars. For 2006 and 2005 HELM listed, 20 refereed journal articles, 7 book chapters, 1 published conference papers, 20 conference presentations, two official reports, nine working papers and £71,535 of grant money produced in this research area across the School. I hope that this shows that reflection on learning is alive and well in ABS. We have also been working on a list of target journals to guide ABS staff who wish to publish in this area. These are included as Appendix 2 of this publication. May I thank the contributors for taking time out of their busy schedules to write the articles and to Julie Green, the Quality Manager, for putting the varying diverse approaches into a coherent and publishable form and for agreeing to fund the printing of this volume.

Good practice guide in learning and teaching

Once again this publication is produced to celebrate and promote good teaching and learning support and to offer encouragement to those imaginative and innovative staff who continue to wish to challenge students to learn to maximum effect. It is hoped that others will pick up some good ideas from the articles contained in this volume. We had changed our editorial approach in drawing together the articles for this 2005/6 edition (our third) of the ABS Good Practice Guide. Firstly we have expanded our contributors beyond ABS academics. This year?s articles have also been written by staff from other areas of the University, a PhD student, a post-doctoral researcher and staff working in learning support. We see this as an acknowledgement that the learning environment involves a range of people in the process of student support. We have also expanded the maximum length of the articles from two to five pages, in order to allow greater reflection on the issues. The themes of the papers cluster around issues relating to diversity (widening participation and internationalisation of the student body), imaginative use of new technology (electronic reading on BlackboardTM ) and reflective practitioners, (reflection on rigour and relevance; on how best to train students in research ethics, relevance in the curriculum and the creativity of the teaching process) Discussion of efforts to train the HE teachers of the future looks forward to the next academic year when the Higher Education Academy?s professional standards will be introduced across the sector. In the last volume we mentioned the launch of the School?s Research Centre in Higher Education Learning and Management (HELM). Since then HELM has stimulated a lot of activity across the School (and University) particularly linking research and teaching. A list of the HELM seminars is listed as an appendix to this publication. Further details can be obtained from Catherine Foster (c.s.foster@aston.ac.uk) who coordinates the HELM seminars. HELM has also won its first independent grant from the EU Leonardo programme to look at the effect of business education on employment. In its annual report to the ABS Research Committee HELM listed for 2004 and 2005, 11 refereed journal articles, 4 book chapters, 3 published conference papers, 18 conference papers, one official reports and £72,500 of grant money produced in this research area across the School. I hope that this shows that reflection on learning is live and well in ABS. May I thank the contributors for taking time out of their busy schedules to write the articles and to Julie Green, the Quality Manager, for putting our diverse approaches into a coherent and publishable form.

Adaptive back-propagation in on-line learning of multilayer networks

An adaptive back-propagation algorithm is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, both numerical studies and a rigorous analysis show that the adaptive back-propagation method results in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent.

Using ancillary statistics in on-line learning algorithms

Neural networks are usually curved statistical models. They do not have finite dimensional sufficient statistics, so on-line learning on the model itself inevitably loses information. In this paper we propose a new scheme for training curved models, inspired by the ideas of ancillary statistics and adaptive critics. At each point estimate an auxiliary flat model (exponential family) is built to locally accommodate both the usual statistic (tangent to the model) and an ancillary statistic (normal to the model). The auxiliary model plays a role in determining credit assignment analogous to that played by an adaptive critic in solving temporal problems. The method is illustrated with the Cauchy model and the algorithm is proved to be asymptotically efficient.

Globally optimal parameters for on-line learning in multilayer neural networks

We present a framework for calculating globally optimal parameters, within a given time frame, for on-line learning in multilayer neural networks. We demonstrate the capability of this method by computing optimal learning rates in typical learning scenarios. A similar treatment allows one to determine the relevance of related training algorithms based on modifications to the basic gradient descent rule as well as to compare different training methods.

Globally optimal on-line learning rules for multi-layer neural networks

We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This rule maximizes the total reduction in generalization error over the whole learning process. A simple example demonstrates that the locally optimal rule, which maximizes the rate of decrease in generalization error, may perform poorly in comparison.

Dynamics of on-line learning in radial basis function networks

On-line learning is examined for the radial basis function network, an important and practical type of neural network. The evolution of generalization error is calculated within a framework which allows the phenomena of the learning process, such as the specialization of the hidden units, to be analyzed. The distinct stages of training are elucidated, and the role of the learning rate described. The three most important stages of training, the symmetric phase, the symmetry-breaking phase, and the convergence phase, are analyzed in detail; the convergence phase analysis allows derivation of maximal and optimal learning rates. As well as finding the evolution of the mean system parameters, the variances of these parameters are derived and shown to be typically small. Finally, the analytic results are strongly confirmed by simulations.

On-line learning with adaptive back-propagation in two-layer networks

An adaptive back-propagation algorithm parameterized by an inverse temperature 1/T is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, we analyse these learning algorithms in both the symmetric and the convergence phase for finite learning rates in the case of uncorrelated teachers of similar but arbitrary length T. These analyses show that adaptive back-propagation results generally in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent.

On-line learning in radial basis functions networks

An analytic investigation of the average case learning and generalization properties of Radial Basis Function Networks (RBFs) is presented, utilising on-line gradient descent as the learning rule. The analytic method employed allows both the calculation of generalization error and the examination of the internal dynamics of the network. The generalization error and internal dynamics are then used to examine the role of the learning rate and the specialization of the hidden units, which gives insight into decreasing the time required for training. The realizable and over-realizable cases are studied in detail; the phase of learning in which the hidden units are unspecialized (symmetric phase) and the phase in which asymptotic convergence occurs are analyzed, and their typical properties found. Finally, simulations are performed which strongly confirm the analytic results.

Incorporating curvature information into on-line learning

We analyse the dynamics of a number of second order on-line learning algorithms training multi-layer neural networks, using the methods of statistical mechanics. We first consider on-line Newton's method, which is known to provide optimal asymptotic performance. We determine the asymptotic generalization error decay for a soft committee machine, which is shown to compare favourably with the result for standard gradient descent. Matrix momentum provides a practical approximation to this method by allowing an efficient inversion of the Hessian. We consider an idealized matrix momentum algorithm which requires access to the Hessian and find close correspondence with the dynamics of on-line Newton's method. In practice, the Hessian will not be known on-line and we therefore consider matrix momentum using a single example approximation to the Hessian. In this case good asymptotic performance may still be achieved, but the algorithm is now sensitive to parameter choice because of noise in the Hessian estimate. On-line Newton's method is not appropriate during the transient learning phase, since a suboptimal unstable fixed point of the gradient descent dynamics becomes stable for this algorithm. A principled alternative is to use Amari's natural gradient learning algorithm and we show how this method provides a significant reduction in learning time when compared to gradient descent, while retaining the asymptotic performance of on-line Newton's method.

Globally optimal on-line learning rules

We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.

On-line learning of unrealizable tasks

The dynamics of on-line learning is investigated for structurally unrealizable tasks in the context of two-layer neural networks with an arbitrary number of hidden neurons. Within a statistical mechanics framework, a closed set of differential equations describing the learning dynamics can be derived, for the general case of unrealizable isotropic tasks. In the asymptotic regime one can solve the dynamics analytically in the limit of large number of hidden neurons, providing an analytical expression for the residual generalization error, the optimal and critical asymptotic training parameters, and the corresponding prefactor of the generalization error decay.

On-line learning in neural networks

On-line learning is one of the most powerful and commonly used techniques for training large layered networks and has been used successfully in many real-world applications. Traditional analytical methods have been recently complemented by ones from statistical physics and Bayesian statistics. This powerful combination of analytical methods provides more insight and deeper understanding of existing algorithms and leads to novel and principled proposals for their improvement. This book presents a coherent picture of the state-of-the-art in the theoretical analysis of on-line learning. An introduction relates the subject to other developments in neural networks and explains the overall picture. Surveys by leading experts in the field combine new and established material and enable non-experts to learn more about the techniques and methods used. This book, the first in the area, provides a comprehensive view of the subject and will be welcomed by mathematicians, scientists and engineers, whether in industry or academia.