On-line learning in neural networks

On-line learning is one of the most powerful and commonly used techniques for training large layered networks and has been used successfully in many real-world applications. Traditional analytical methods have been recently complemented by ones from statistical physics and Bayesian statistics. This powerful combination of analytical methods provides more insight and deeper understanding of existing algorithms and leads to novel and principled proposals for their improvement. This book presents a coherent picture of the state-of-the-art in the theoretical analysis of on-line learning. An introduction relates the subject to other developments in neural networks and explains the overall picture. Surveys by leading experts in the field combine new and established material and enable non-experts to learn more about the techniques and methods used. This book, the first in the area, provides a comprehensive view of the subject and will be welcomed by mathematicians, scientists and engineers, whether in industry or academia.

Noise, regularizers, and unrealizable scenarios in online learning from restricted training sets

We study the dynamics of on-line learning in multilayer neural networks where training examples are sampled with repetition and where the number of examples scales with the number of network weights. The analysis is carried out using the dynamical replica method aimed at obtaining a closed set of coupled equations for a set of macroscopic variables from which both training and generalization errors can be calculated. We focus on scenarios whereby training examples are corrupted by additive Gaussian output noise and regularizers are introduced to improve the network performance. The dependence of the dynamics on the noise level, with and without regularizers, is examined, as well as that of the asymptotic values obtained for both training and generalization errors. We also demonstrate the ability of the method to approximate the learning dynamics in structurally unrealizable scenarios. The theoretical results show good agreement with those obtained by computer simulations.

The theory of on-line learning: a statistical physics approach

In this paper we review recent theoretical approaches for analysing the dynamics of on-line learning in multilayer neural networks using methods adopted from statistical physics. The analysis is based on monitoring a set of macroscopic variables from which the generalisation error can be calculated. A closed set of dynamical equations for the macroscopic variables is derived analytically and solved numerically. The theoretical framework is then employed for defining optimal learning parameters and for analysing the incorporation of second order information into the learning process using natural gradient descent and matrix-momentum based methods. We will also briefly explain an extension of the original framework for analysing the case where training examples are sampled with repetition.

Organisational learning - a critical systems thinking discipline

Original Paper European Journal of Information Systems (2001) 10, 135–146; doi:10.1057/palgrave.ejis.3000394 Organisational learning—a critical systems thinking discipline P Panagiotidis1,3 and J S Edwards2,4 1Deloitte and Touche, Athens, Greece 2Aston Business School, Aston University, Aston Triangle, Birmingham, B4 7ET, UK Correspondence: Dr J S Edwards, Aston Business School, Aston University, Aston Triangle, Birmingham, B4 7ET, UK. E-mail: j.s.edwards@aston.ac.uk 3Petros Panagiotidis is Manager responsible for the Process and Systems Integrity Services of Deloitte and Touche in Athens, Greece. He has a BSc in Business Administration and an MSc in Management Information Systems from Western International University, Phoenix, Arizona, USA; an MSc in Business Systems Analysis and Design from City University, London, UK; and a PhD degree from Aston University, Birmingham, UK. His doctorate was in Business Systems Analysis and Design. His principal interests now are in the ERP/DSS field, where he serves as project leader and project risk managment leader in the implementation of SAP and JD Edwards/Cognos in various major clients in the telecommunications and manufacturing sectors. In addition, he is responsible for the development and application of knowledge management systems and activity-based costing systems. 4John S Edwards is Senior Lecturer in Operational Research and Systems at Aston Business School, Birmingham, UK. He holds MA and PhD degrees (in mathematics and operational research respectively) from Cambridge University. His principal research interests are in knowledge management and decision support, especially methods and processes for system development. He has written more than 30 research papers on these topics, and two books, Building Knowledge-based Systems and Decision Making with Computers, both published by Pitman. Current research work includes the effect of scale of operations on knowledge management, interfacing expert systems with simulation models, process modelling in law and legal services, and a study of the use of artifical intelligence techniques in management accounting. Top of pageAbstract This paper deals with the application of critical systems thinking in the domain of organisational learning and knowledge management. Its viewpoint is that deep organisational learning only takes place when the business systems' stakeholders reflect on their actions and thus inquire about their purpose(s) in relation to the business system and the other stakeholders they perceive to exist. This is done by reflecting both on the sources of motivation and/or deception that are contained in their purpose, and also on the sources of collective motivation and/or deception that are contained in the business system's purpose. The development of an organisational information system that captures, manages and institutionalises meaningful information—a knowledge management system—cannot be separated from organisational learning practices, since it should be the result of these very practices. Although Senge's five disciplines provide a useful starting-point in looking at organisational learning, we argue for a critical systems approach, instead of an uncritical Systems Dynamics one that concentrates only on the organisational learning practices. We proceed to outline a methodology called Business Systems Purpose Analysis (BSPA) that offers a participatory structure for team and organisational learning, upon which the stakeholders can take legitimate action that is based on the force of the better argument. In addition, the organisational learning process in BSPA leads to the development of an intrinsically motivated information organisational system that allows for the institutionalisation of the learning process itself in the form of an organisational knowledge management system. This could be a specific application, or something as wide-ranging as an Enterprise Resource Planning (ERP) implementation. Examples of the use of BSPA in two ERP implementations are presented.

Cohesion or confusion? Towards a typology for organizational learning research

The study of organizational learning is no longer in its infancy. Since Cyert and March first introduced the notion in the early 1960s, a plethora of books and journal publications have presented their own interpretations of the meaning and significance of the term. Despite such endeavours, there is little common agreement about what organizational learning represents and how future research may build cumulatively upon the many diverse ideas articulated. The intention here is by no means to address these issues, which have been comprehensively examined elsewhere. The purpose is rather to compare and contrast approaches in order to analyse similarities and dissimilarities, together with research challenges, for each approach. This is achieved by presenting a comparative framework to categorize the literature according to (a) its prescriptive/explanatory bias and (b) in line with the level of analysis, examining whether there is a focus on the organization as a whole or upon individuals and their work communities instead. The review concludes by presenting some preliminary suggestions for cross-quadrant research. © Blackwell Publishing Ltd 2006.

A study in Britain and France of the development of cross-cultural in-house sales training using distance learning.

The thesis is concerned with cross-cultural distance learning in two countries: Great Britain and France. Taking the example of in-house sales training, it argues that it is possible to develop courses for use in two or more countries of differing culture and language. Two courses were developed by the researcher. Both were essentially print-based distance-learning courses designed to help salespeople achieve a better understanding of their customers. One used a quantitative, the other qualitative approach. One considered the concept of the return on investment and the other, for which a video support was also developed, considered the analysis of a customer's needs. Part 1 of the thesis considers differences in the training context between France and Britain followed by a review of the learning process with reference to distance learning. Part 2 looks at the choice of training medium course design and evaluation and sets out the methodology adopted, including problems encountered in this type of fieldwork. Part 3 analyses the data and draws conclusions from the findings, before offering a series of guidelines for those concerned with the development of cross-cultural in-house training courses. The results of the field tests on the two courses were analysed in relation to the socio-cultural, educational and experiential background of the learners as well as their preferred learning styles. The thesis argues that it is possible to develop effective in-house sales training courses to be used in two cultures and identifies key considerations which need to be taken into account when carrying out this type of work.

Teaching and learning German language and culture in higher education in Britain:problems, challenges and didactical implications. A case study

The aim of this thesis is to explore key aspects and problems of the institutionalised teaching and learning of German language and culture in the context of German Studies in British Higher Education (HE). This investigation focuses on teaching and learning experiences in one department of German Studies in the UK, which is the micro-context of the present study, in order to provide an in-depth insight into real-life problems, strengths and weaknesses as they occur in the practice of teaching and learning German. Following Lamb (2004) and Holliday (1994), the present study acts on the assumption that each micro-context does not exist in vacuo but is always embedded in a wider socio-political and education environment, namely the macro-context, which largely determines how and what is taught. The macro-analysis of the present study surveys the socio-political developments that have recently affected the sector of modern languages and specifically the discipline of German Studies in the UK. It demonstrates the impact they have had on teaching and learning German at the undergraduate level in Britain. This context is interesting inasmuch as the situation in Britain is to a large extent a paradigmatic example of the developments in German Studies in English-speaking countries. Subsequently, the present study explores learning experiences of a group of thirty-five first year students. It focuses on their previous experiences in learning German, exposure to the target language, motivation, learning strategies and difficulties encountered, when learning German at the tertiary level. Then, on the basis of interviews with five lecturers of German, teaching experience in the context under study is explored, problems and successful teaching strategies discussed.

A speech analysis system for use in educational research

This thesis describes work undertaken in order to fulfil a need experienced in the Department of Educational Enquiry at the University of Aston in Birmingham for speech analysis facilities suitable for use in teaching and research work within the Department. The hardware and software developed during the research project provides displays of speech fundamental frequency and intensity in real time. The system is suitable for the provision of visual feedback of these parameters of a subject's speech in a learning situation, and overcomes the inadequacies of equipment currently used for this task in that it provides a clear indication of fundamental frequency contours as the subject is speaking. The thesis considers the use of such equipment in several related fields, and the approaches that have been reported to one of the major problems of speech analysis, namely pitch-period estimation. A number of different systems are described, and their suitability for the present purposes is discussed. Finally, a novel method of pitch-period estimation is developed, and a speech analysis system incorporating this method is described. Comparison is made between the results produced by this system and those produced by a conventional speech spectrograph.

Doctoral work in translation studies as an interdisciplinary mutual learning process:how a translator, teacher educator and linguistic typologist worked together

The focus of this paper is on the doctoral research training experienced by one of the authors and the ways in which the diverse linguistic and disciplinary perspectives of her two supervisors (co-authors of this paper) mediated the completion of her study. The doctoral candidate is a professional translator/interpreter and translation teacher. The paper describes why and how she identified her research area and then focused on the major research questions in collaboration with her two supervisors, who brought their differing perspectives from the field of linguistics to this translation research, even though they are not translators by profession or disciplinary background and do not speak Korean. In addition, the discussion considers the focus, purpose and theoretical orientation of the research itself (which addressed questions of readability in translated English-Korean texts through detailed analysis of a corpus and implications for professional translator training) as well as the supervisory and conceptual processes and practices involved. The authors contend that doctoral research of this kind can be seen as a mutual learning process and that inter-disciplinary research can make a contribution not only to the development of rigorous research in the field of translation studies but also to the other disciplinary fields involved.

Peer mentoring and experiential learning:the pathway to success

An analysis of the value of peer mentoring as an experiential learning approach

Volunteering for success? An Analysis of Student Perceptions of the Pedagogical and Social Outcomes of Peer Mentoring in Higher Education

Previous studies into student volunteering have shown how formally organized volunteering activities have social, economic and practical benefits for student volunteers and the recipients of their volunteerism (Egerton, 2002; Vernon & Foster, 2002); moreover student volunteering provides the means by which undergraduates are able to acquire and hone transferable skills sought by employers following graduation (Eldridge & Wilson, 2003; Norris et al, 2006). Although much is known about the benefits of student volunteering, few previous studies have focused on the pedagogical value of student mentoring from the perspectives of both student mentee and mentor. Utilising grounded theory methodology this paper provides a critical analysis of an exploratory study analysing students’ perceptions of the pedagogical and social outcomes of student mentoring. It looks at students’ perceptions of mentoring, and being mentored, in terms of the learning experience and development of knowledge and skills. In doing so the paper considers how volunteering in a mentoring capacity adds ‘value’ to students’ experiences of higher education. From a public policy perspective, the economic, educational, vocational and social outcomes of student volunteering in general, and student mentoring in particular, make this an important subject meriting investigation. In terms of employability, the role of mentoring in equipping mentors and mentees with transferable, employability competencies has not been investigated. By critiquing the mentoring experiences of undergraduates within a single institution, this paper will make an important contribution to policy debates with regards to the pedagogical and employability related outcomes of student volunteering and mentoring.

Learning task specific distributed paragraph representations using a 2-tier convolutional neural network

We introduce a type of 2-tier convolutional neural network model for learning distributed paragraph representations for a special task (e.g. paragraph or short document level sentiment analysis and text topic categorization). We decompose the paragraph semantics into 3 cascaded constitutes: word representation, sentence composition and document composition. Specifically, we learn distributed word representations by a continuous bag-of-words model from a large unstructured text corpus. Then, using these word representations as pre-trained vectors, distributed task specific sentence representations are learned from a sentence level corpus with task-specific labels by the first tier of our model. Using these sentence representations as distributed paragraph representation vectors, distributed paragraph representations are learned from a paragraph-level corpus by the second tier of our model. It is evaluated on DBpedia ontology classification dataset and Amazon review dataset. Empirical results show the effectiveness of our proposed learning model for generating distributed paragraph representations.

Exploring ant-based algorithms for gene expression data analysis

Objective: Recently, much research has been proposed using nature inspired algorithms to perform complex machine learning tasks. Ant colony optimization (ACO) is one such algorithm based on swarm intelligence and is derived from a model inspired by the collective foraging behavior of ants. Taking advantage of the ACO in traits such as self-organization and robustness, this paper investigates ant-based algorithms for gene expression data clustering and associative classification. Methods and material: An ant-based clustering (Ant-C) and an ant-based association rule mining (Ant-ARM) algorithms are proposed for gene expression data analysis. The proposed algorithms make use of the natural behavior of ants such as cooperation and adaptation to allow for a flexible robust search for a good candidate solution. Results: Ant-C has been tested on the three datasets selected from the Stanford Genomic Resource Database and achieved relatively high accuracy compared to other classical clustering methods. Ant-ARM has been tested on the acute lymphoblastic leukemia (ALL)/acute myeloid leukemia (AML) dataset and generated about 30 classification rules with high accuracy. Conclusions: Ant-C can generate optimal number of clusters without incorporating any other algorithms such as K-means or agglomerative hierarchical clustering. For associative classification, while a few of the well-known algorithms such as Apriori, FP-growth and Magnum Opus are unable to mine any association rules from the ALL/AML dataset within a reasonable period of time, Ant-ARM is able to extract associative classification rules.

Analysis of constructivist, network-based discourses:concepts, prospects, and illustrations

The authors propose a new approach to discourse analysis which is based on meta data from social networking behavior of learners who are submerged in a socially constructivist e-learning environment. It is shown that traditional data modeling techniques can be combined with social network analysis - an approach that promises to yield new insights into the largely uncharted domain of network-based discourse analysis. The chapter is treated as a non-technical introduction and is illustrated with real examples, visual representations, and empirical findings. Within the setting of a constructivist statistics course, the chapter provides an illustration of what network-based discourse analysis is about (mainly from a methodological point of view), how it is implemented in practice, and why it is relevant for researchers and educators.

Experiential learning:inspiring the business leaders of tomorrow

Purpose: The complex challenges of sustainable development and the need to embed these issues effectively into the education of future business leaders has never been more urgent. The purpose of this paper is to discuss different approaches taken by two UK signatories to the UN Principles for Responsible Management Education (PRME). Design/methodology/approach: The two approaches examined are: MSc Entrepreneurship students opting for placements with social enterprises; and MBA students undertaking workshops using "live" case studies. A content analysis of the experiences of students from their written reflective narratives is presented. This is supplemented by reflections of the facilitators and tutors. Findings: The analysis reveals that the opportunity to work with social entrepreneurs and/or "responsible" business professionals provides the business students with inspirational role models and positive social learning opportunities. Research limitations/implications: This paper suggests that experiential learning is an effective way of integrating ethics, responsibility and sustainability into the curriculum but the research draws on the experience of two schools. Further research is important to explore these findings in other contexts. Practical implications: The authors argue that direct exposure to a business culture (and/or behaviour) that is predicated upon ethical/social responsibility and sustainability is an effective means to embed these values in the curriculum. Originality/value: This paper contributes by drawing on social psychological research related to behaviour change to examine how experiential learning on traditional Business Masters programmes can provide students with the knowledge, motivation and skills to contribute positively to society, in a way that more traditional pedagogies cannot. © Emerald Group Publishing Limited.