Noise, regularizers, and unrealizable scenarios in online learning from restricted training sets

We study the dynamics of on-line learning in multilayer neural networks where training examples are sampled with repetition and where the number of examples scales with the number of network weights. The analysis is carried out using the dynamical replica method aimed at obtaining a closed set of coupled equations for a set of macroscopic variables from which both training and generalization errors can be calculated. We focus on scenarios whereby training examples are corrupted by additive Gaussian output noise and regularizers are introduced to improve the network performance. The dependence of the dynamics on the noise level, with and without regularizers, is examined, as well as that of the asymptotic values obtained for both training and generalization errors. We also demonstrate the ability of the method to approximate the learning dynamics in structurally unrealizable scenarios. The theoretical results show good agreement with those obtained by computer simulations.

The importance of power dynamics in the development of asynchronous online learning communities

This research explored how a more student-directed learning design can support the creation of togetherness and belonging in a community of distance learners in formal higher education. Postgraduate students in a New Zealand School of Education experienced two different learning tasks as part of their online distance learning studies. The tasks centered around two online asynchronous discussions each for the same period of time and with the same group of students, but following two different learning design principles. All messages were analyzed using a twostep analysis process, content analysis and social network analysis. Although the findings showed a balance of power between the tutor and the students in the first high e-moderated activity, a better pattern of group interaction and community feeling was found in the low e-moderated activity. The paper will discuss the findings in terms of the implications for learning design and the role of the tutor.

Personal knowledge development in online learning:a model for measuring externalisation, combination and internalisation

This paper presents a model for measuring personal knowledge development in online learning environments. It is based on Nonaka‘s SECI model of organisational knowledge creation. It is argued that Socialisation is not a relevant mode in the context of online learning and was therefore not covered in the measurement instrument. Therefore, the remaining three of SECI‘s knowledge conversion modes, namely Externalisation, Combination, and Internalisation were used and a measurement instrument was created which also examines the interrelationships between the three modes. Data was collected using an online survey, in which online learners report on their experiences of personal knowledge development in online learning environments. In other words, the instrument measures the magnitude of online learners‘ Externalisation and combination activities as well as their level of internalisation, which is the outcome of their personal knowledge development in online learning.

Online learning in discrete hidden Markov models

We present and analyze three different online algorithms for learning in discrete Hidden Markov Models (HMMs) and compare their performance with the Baldi-Chauvin Algorithm. Using the Kullback-Leibler divergence as a measure of the generalization error we draw learning curves in simplified situations and compare the results. The performance for learning drifting concepts of one of the presented algorithms is analyzed and compared with the Baldi-Chauvin algorithm in the same situations. A brief discussion about learning and symmetry breaking based on our results is also presented. © 2006 American Institute of Physics.

A Bayesian approach to on-line learning

Online learning is discussed from the viewpoint of Bayesian statistical inference. By replacing the true posterior distribution with a simpler parametric distribution, one can define an online algorithm by a repetition of two steps: An update of the approximate posterior, when a new example arrives, and an optimal projection into the parametric family. Choosing this family to be Gaussian, we show that the algorithm achieves asymptotic efficiency. An application to learning in single layer neural networks is given.

Parallel strategy for optimal learning in perceptrons

We developed a parallel strategy for learning optimally specific realizable rules by perceptrons, in an online learning scenario. Our result is a generalization of the Caticha–Kinouchi (CK) algorithm developed for learning a perceptron with a synaptic vector drawn from a uniform distribution over the N-dimensional sphere, so called the typical case. Our method outperforms the CK algorithm in almost all possible situations, failing only in a denumerable set of cases. The algorithm is optimal in the sense that it saturates Bayesian bounds when it succeeds.

Can diversity amongst learners improve online object tracking?

We present a novel analysis of the state of the art in object tracking with respect to diversity found in its main component, an ensemble classifier that is updated in an online manner. We employ established measures for diversity and performance from the rich literature on ensemble classification and online learning, and present a detailed evaluation of diversity and performance on benchmark sequences in order to gain an insight into how the tracking performance can be improved. © Springer-Verlag 2013.

Learning to be different:heterogeneity and efficiency in distributed smart camera networks

In this paper we study the self-organising behaviour of smart camera networks which use market-based handover of object tracking responsibilities to achieve an efficient allocation of objects to cameras. Specifically, we compare previously known homogeneous configurations, when all cameras use the same marketing strategy, with heterogeneous configurations, when each camera makes use of its own, possibly different marketing strategy. Our first contribution is to establish that such heterogeneity of marketing strategies can lead to system wide outcomes which are Pareto superior when compared to those possible in homogeneous configurations. However, since the particular configuration required to lead to Pareto efficiency in a given scenario will not be known in advance, our second contribution is to show how online learning of marketing strategies at the individual camera level can lead to high performing heterogeneous configurations from the system point of view, extending the Pareto front when compared to the homogeneous case. Our third contribution is to show that in many cases, the dynamic behaviour resulting from online learning leads to global outcomes which extend the Pareto front even when compared to static heterogeneous configurations. Our evaluation considers results obtained from an open source simulation package as well as data from a network of real cameras. © 2013 IEEE.

Bayesian regression filter and the issue of priors

We propose a Bayesian framework for regression problems, which covers areas which are usually dealt with by function approximation. An online learning algorithm is derived which solves regression problems with a Kalman filter. Its solution always improves with increasing model complexity, without the risk of over-fitting. In the infinite dimension limit it approaches the true Bayesian posterior. The issues of prior selection and over-fitting are also discussed, showing that some of the commonly held beliefs are misleading. The practical implementation is summarised. Simulations using 13 popular publicly available data sets are used to demonstrate the method and highlight important issues concerning the choice of priors.

Gaussian processes:iterative sparse approximations

In recent years there has been an increased interest in applying non-parametric methods to real-world problems. Significant research has been devoted to Gaussian processes (GPs) due to their increased flexibility when compared with parametric models. These methods use Bayesian learning, which generally leads to analytically intractable posteriors. This thesis proposes a two-step solution to construct a probabilistic approximation to the posterior. In the first step we adapt the Bayesian online learning to GPs: the final approximation to the posterior is the result of propagating the first and second moments of intermediate posteriors obtained by combining a new example with the previous approximation. The propagation of em functional forms is solved by showing the existence of a parametrisation to posterior moments that uses combinations of the kernel function at the training points, transforming the Bayesian online learning of functions into a parametric formulation. The drawback is the prohibitive quadratic scaling of the number of parameters with the size of the data, making the method inapplicable to large datasets. The second step solves the problem of the exploding parameter size and makes GPs applicable to arbitrarily large datasets. The approximation is based on a measure of distance between two GPs, the KL-divergence between GPs. This second approximation is with a constrained GP in which only a small subset of the whole training dataset is used to represent the GP. This subset is called the em Basis Vector, or BV set and the resulting GP is a sparse approximation to the true posterior. As this sparsity is based on the KL-minimisation, it is probabilistic and independent of the way the posterior approximation from the first step is obtained. We combine the sparse approximation with an extension to the Bayesian online algorithm that allows multiple iterations for each input and thus approximating a batch solution. The resulting sparse learning algorithm is a generic one: for different problems we only change the likelihood. The algorithm is applied to a variety of problems and we examine its performance both on more classical regression and classification tasks and to the data-assimilation and a simple density estimation problems.

Operations management

Operations Management, 3rd Edition provides a clear and accessible introduction to this important area of study, focusing on all key areas of operations in both manufacturing and service industries. Features: Focuses on the subject from a European perspective. Deals with the management of the creation of goods and the delivery of services to the customer. Covers the main areas of operations strategy, the design of operations system and the management of operations over time. Incorporates more strategic and international commentary. Includes a strategy link section consisting of a paragraph relating each chapter topic to operations strategy. Includes more end of chapter and quantitative exercises. Cases have been updated throughout and now include: Service including public sector, international, a mix of mini–cases and a longer case for each chapter. Accompanied by a comprehensive package of online learning support materials including: A robust testbank featuring 1500 questions, PowerPoint slides and a comprehensive instructor's manual An interactive e–Book is included with every new copy of this text, featuring a wealth of embedded media, including: Animated worked examples, simulations, virtual tours, videos, flashcards and practice quizzes.

Educating the iPod generation:undergraduate attitudes, experiences and knowledge of vodcast and podcast use

There is an increasing pressure on university staff to provide ever more information and resources to students. This study investigated student opinions on (audio) podcasts and (video) vodcasts and how well they met requirements and aided learning processes. Two experiments within the Aston University looked at student opinion on, and usage of, podcasts and vodcasts for a selection of their psychology lectures. Recordings were produced first using a hand-held camcorder, and then using the in-house media department. WebCT was used to distribute the podcasts and vodcasts, attitude questionnaires were then circulated at two time points. Overall students indicated that podcasts and vodcasts were a beneficial addition resource for learning, particularly when used in conjunction with lecturers’ slides and as a tool for revision/assessment. The online material translated into students having increased understanding of the material, which supplemented and enhanced their learning without being a substitute for traditional lectures. There is scope for the provision of portable media files to become standard practice within higher education; integrating distance and online learning with traditional approaches to improve teaching and learning.

Incremental tensor biased discriminant analysis:a new color-based visual tracking method

Most existing color-based tracking algorithms utilize the statistical color information of the object as the tracking clues, without maintaining the spatial structure within a single chromatic image. Recently, the researches on the multilinear algebra provide the possibility to hold the spatial structural relationship in a representation of the image ensembles. In this paper, a third-order color tensor is constructed to represent the object to be tracked. Considering the influence of the environment changing on the tracking, the biased discriminant analysis (BDA) is extended to the tensor biased discriminant analysis (TBDA) for distinguishing the object from the background. At the same time, an incremental scheme for the TBDA is developed for the tensor biased discriminant subspace online learning, which can be used to adapt to the appearance variant of both the object and background. The experimental results show that the proposed method can track objects precisely undergoing large pose, scale and lighting changes, as well as partial occlusion. © 2009 Elsevier B.V.

Self-organising zooms for decentralised redundancy management in visual sensor networks

When visual sensor networks are composed of cameras which can adjust the zoom factor of their own lens, one must determine the optimal zoom levels for the cameras, for a given task. This gives rise to an important trade-off between the overlap of the different cameras’ fields of view, providing redundancy, and image quality. In an object tracking task, having multiple cameras observe the same area allows for quicker recovery, when a camera fails. In contrast having narrow zooms allow for a higher pixel count on regions of interest, leading to increased tracking confidence. In this paper we propose an approach for the self-organisation of redundancy in a distributed visual sensor network, based on decentralised multi-objective online learning using only local information to approximate the global state. We explore the impact of different zoom levels on these trade-offs, when tasking omnidirectional cameras, having perfect 360-degree view, with keeping track of a varying number of moving objects. We further show how employing decentralised reinforcement learning enables zoom configurations to be achieved dynamically at runtime according to an operator’s preference for maximising either the proportion of objects tracked, confidence associated with tracking, or redundancy in expectation of camera failure. We show that explicitly taking account of the level of overlap, even based only on local knowledge, improves resilience when cameras fail. Our results illustrate the trade-off between maintaining high confidence and object coverage, and maintaining redundancy, in anticipation of future failure. Our approach provides a fully tunable decentralised method for the self-organisation of redundancy in a changing environment, according to an operator’s preferences.