Learning with noise and regularizers in multilayer neural networks

We study the effect of two types of noise, data noise and model noise, in an on-line gradient-descent learning scenario for general two-layer student network with an arbitrary number of hidden units. Training examples are randomly drawn input vectors labeled by a two-layer teacher network with an arbitrary number of hidden units. Data is then corrupted by Gaussian noise affecting either the output or the model itself. We examine the effect of both types of noise on the evolution of order parameters and the generalization error in various phases of the learning process.

Particle competition and cooperation for semi-supervised learning with label noise

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Training with noise is equivalent to Tikhonov regularization

It is well known that the addition of noise to the input data of a neural network during training can, in some circumstances, lead to significant improvements in generalization performance. Previous work has shown that such training with noise is equivalent to a form of regularization in which an extra term is added to the error function. However, the regularization term, which involves second derivatives of the error function, is not bounded below, and so can lead to difficulties if used directly in a learning algorithm based on error minimization. In this paper we show that, for the purposes of network training, the regularization term can be reduced to a positive definite form which involves only first derivatives of the network mapping. For a sum-of-squares error function, the regularization term belongs to the class of generalized Tikhonov regularizers. Direct minimization of the regularized error function provides a practical alternative to training with noise.

Learning with regularizers in multilayer neural networks

We study the effect of regularization in an on-line gradient-descent learning scenario for a general two-layer student network with an arbitrary number of hidden units. Training examples are randomly drawn input vectors labelled by a two-layer teacher network with an arbitrary number of hidden units which may be corrupted by Gaussian output noise. We examine the effect of weight decay regularization on the dynamical evolution of the order parameters and generalization error in various phases of the learning process, in both noiseless and noisy scenarios.

Constant-random practice and the adaptive process in motor learning with varying amounts of constant practice

The adaptive process in motor learning was examined in terms of effects of varying amounts of constant practice performed before random practice. Participants pressed five response keys sequentially, the last one coincident with the lighting of a final visual stimulus provided by a complex coincident timing apparatus. Different visual stimulus speeds were used during the random practice. 33 children (M age=11.6 yr.) were randomly assigned to one of three experimental groups: constant-random, constant-random 33%, and constant-random 66%. The constant-random group practiced constantly until they reached a criterion of performance stabilization three consecutive trials within 50 msec. of error. The other two groups had additional constant practice of 33 and 66%, respectively, of the number of trials needed to achieve the stabilization criterion. All three groups performed 36 trials under random practice; in the adaptation phase, they practiced at a different visual stimulus speed adopted in the stabilization phase. Global performance measures were absolute, constant, and variable errors, and movement pattern was analyzed by relative timing and overall movement time. There was no group difference in relation to global performance measures and overall movement time. However, differences between the groups were observed on movement pattern, since constant-random 66% group changed its relative timing performance in the adaptation phase.

Effect of instructed extinction on verbal and autonomic indices of Pavlovian learning with fear-relevant and fear-irrelevant conditional stimuli

We investigated the effects of conditional stimulus fear-relevance and of instructed extinction on human Pavlovian conditioning as indexed by electrodermal responses and verbal ratings of conditional stimulus unpleasantness. Half of the participants (n = 64) were trained with pictures of snakes and spiders (fear-relevant) as conditional stimuli, whereas the others were trained with pictures of flowers and mushrooms (fear-irrelevant) in a differential aversive Pavlovian conditioning procedure. Half of the participants in each group were instructed after the completion of acquisition that no more unconditional stimuli were to be presented. Extinction of differential electrodermal responses required more trials after training with fear-relevant pictures. Moreover, there was some evidence that verbal instructions did not affect extinction of second interval electrodermal responses to fear-relevant pictures. However, neither fear-relevance nor instructions affected the changes in rated conditional stimulus pleasantness. This dissociation across measures is interpreted as reflecting renewal of Pavlovian learning.

Preparing students for the virtual organisation: an evaluation of learning with virtual learning technologies

The paper explores the development of learning behaviours in a virtual management course and the factors that impacted on this development. Data suggest that most teams experienced three kinds of learning behaviours – social, operational and content learning. We propose that the need for technical expertise and team participation will vary during these different stages of learning. Addressing the characteristics of these stages, we comment on the development of a ‘completion phase’ of team development. We argue that the extent to which teams demonstrate different learning stages has a significant impact on the development of on-line learning behaviours. Discussing these results, we suggest why different teams develop distinct learning behaviours, with accordant emphasis on teaching as a moderating and co ordinating role, despite current virtual team pedagogical expectations.

Learning with tangible interfaces

Trabalho apresentado no âmbito do Mestrado em Engenharia Informática, como requisito parcial para obtenção do grau de Mestre em Engenharia Informática

Designing market-based instruments for biodiversity conservation: learning with the Australian experience and evaluating Portugal's potential

Dissertação apresentada na Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa para a obtenção do grau de Mestre em Engenharia do Ambiente, perfil Gestão de Sistemas Ambientais

Teaching and learning with social software in higher education: content management systems integrated with Web-based applications as a key factor for success

The change of paradigm imposed by the Bologna process, in which the student will be responsible for their own learning, and the presence of a new generation of students with higher technological skills, represent a huge challenge for higher education institutions. The use of new Web Social concepts in teaching process, supported by applications commonly called Web 2.0, with which these new students feel at ease, can bring benefits in terms of motivation and the frequency and quality of students' involvement in academic activities. An e-learning platform with web-based applications as a complement can significantly contribute to the development of different skills in higher education students, covering areas which are usually in deficit.

Machine learning with Lipschitz classifiers

Magdeburg, Univ., Fak. für Elektrotechnik und Informationstechnik, Diss., 2010

Capturing and Organizing Prior Student Learning with the OCW Backpack

The purpose of this paper is to present an approach for students to have non-traditional learning assessed for credit and introduce a tool that facilitates this process. The OCW Backpack system can connect self-learners with KNEXT assessment services to obtain college credit for prior learning. An ex post facto study based on historical data collected over the past two years at Kaplan University (KU) is presented to validate the portfolio assessment process. Cumulative GPA was compared for students who received experiential credit for learning derived from personal or professional experience with a matched sample of students with no experiential learning credits. The study found that students who received experiential credits perform better than the matched sample students on GPA. The findings validate the KU portfolio assessment process. Additionally, the results support the capability of the OCW Backpack to capture the critical information necessary to evaluate non-traditional learning for university credit.

Speed of convergence of recursive least squares learning with ARMA perceptions

This paper fills a gap in the existing literature on least squareslearning in linear rational expectations models by studying a setup inwhich agents learn by fitting ARMA models to a subset of the statevariables. This is a natural specification in models with privateinformation because in the presence of hidden state variables, agentshave an incentive to condition forecasts on the infinite past recordsof observables. We study a particular setting in which it sufficesfor agents to fit a first order ARMA process, which preserves thetractability of a finite dimensional parameterization, while permittingconditioning on the infinite past record. We describe how previousresults (Marcet and Sargent [1989a, 1989b] can be adapted to handlethe convergence of estimators of an ARMA process in our self--referentialenvironment. We also study ``rates'' of convergence analytically and viacomputer simulation.

Explicit/implicit training of cooperative learning with junior elementary students /

This experimental study examined the effects of cooperative learning and expliciUimpliGit instruction on student achievement and attitudes toward working in cooperative groups. Specifically, fourth- and fifth-grade students (n=48) were randomly assigned to two conditions: cooperative learning with explicit instruction and cooperative learning with implicit instruction. All participants were given initial training either explicitly or implicitly in cooperative learning procedures via 10 one-hour sessions. Following the instruction period, all students participated in completing a group project related to a famous artists unit. It was hypothesized that the explicit instruction training would enhance students' scores on the famous artists test and the group projects, as well as improve students' attitudes toward cooperative learning. Although the explicit training group did not achieve significantly higher scores on the famous artists test, significant differences were found in group project results between the explicit and implicit groups. The explicit group also exhibited more favourable and positive attitudes toward cooperative learning. The findings of this study demonstrate that combining cooperative learning with explicit instruction is an effective classroom strategy and a useful practice for presenting and learning new information, as well as working in groups with success.

Active Learning with Statistical Models

For many types of learners one can compute the statistically 'optimal' way to select data. We review how these techniques have been used with feedforward neural networks. We then show how the same principles may be used to select data for two alternative, statistically-based learning architectures: mixtures of Gaussians and locally weighted regression. While the techniques for neural networks are expensive and approximate, the techniques for mixtures of Gaussians and locally weighted regression are both efficient and accurate.