Learning from Incomplete Data

Real-world learning tasks often involve high-dimensional data sets with complex patterns of missing features. In this paper we review the problem of learning from incomplete data from two statistical perspectives---the likelihood-based and the Bayesian. The goal is two-fold: to place current neural network approaches to missing data within a statistical framework, and to describe a set of algorithms, derived from the likelihood-based framework, that handle clustering, classification, and function approximation from incomplete data in a principled and efficient manner. These algorithms are based on mixture modeling and make two distinct appeals to the Expectation-Maximization (EM) principle (Dempster, Laird, and Rubin 1977)---both for the estimation of mixture components and for coping with the missing data.

Differential brain activity during emotional versus nonemotional reversal learning

The ability to change an established stimulus–behavior association based on feedback is critical for adaptive social behaviors. This ability has been examined in reversal learning tasks, where participants first learn a stimulus–response association (e.g., select a particular object to get a reward) and then need to alter their response when reinforcement contingencies change. Although substantial evidence demonstrates that the OFC is a critical region for reversal learning, previous studies have not distinguished reversal learning for emotional associations from neutral associations. The current study examined whether OFC plays similar roles in emotional versus neutral reversal learning. The OFC showed greater activity during reversals of stimulus–outcome associations for negative outcomes than for neutral outcomes. Similar OFC activity was also observed during reversals involving positive outcomes. Furthermore, OFC activity is more inversely correlated with amygdala activity during negative reversals than during neutral reversals. Overall, our results indicate that the OFC is more activated by emotional than neutral reversal learning and that OFC's interactions with the amygdala are greater for negative than neutral reversal learning.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Perceptual learning in motion discrimination that generalizes across motion directions

When human subjects discriminate motion directions of two visual stimuli, their discrimination improves with practice. This improved performance has been found to be specific to the practiced directions and does not transfer to new motion directions. Indeed, such stimulus-specific learning has become a trademark finding in almost all perceptual learning studies and has been used to infer the loci of learning in the brain. For example, learning in motion discrimination has been inferred to occur in the visual area MT (medial temporal cortex) of primates, where neurons are selectively tuned to motion directions. However, such motion discrimination task is extremely difficult, as is typical of most perceptual learning tasks. When the difficulty is moderately reduced, learning transfers to new motion directions. This result challenges the idea of using simple visual stimuli to infer the locus of learning in low-level visual processes and suggests that higher-level processing is essential even in “simple” perceptual learning tasks.

New Techniques in Deep Representation Learning

Thesis (Ph.D.)--University of Washington, 2016-06

The importance of power dynamics in the development of asynchronous online learning communities

This research explored how a more student-directed learning design can support the creation of togetherness and belonging in a community of distance learners in formal higher education. Postgraduate students in a New Zealand School of Education experienced two different learning tasks as part of their online distance learning studies. The tasks centered around two online asynchronous discussions each for the same period of time and with the same group of students, but following two different learning design principles. All messages were analyzed using a twostep analysis process, content analysis and social network analysis. Although the findings showed a balance of power between the tutor and the students in the first high e-moderated activity, a better pattern of group interaction and community feeling was found in the low e-moderated activity. The paper will discuss the findings in terms of the implications for learning design and the role of the tutor.

Learning from Geometry

Subspaces and manifolds are two powerful models for high dimensional signals. Subspaces model linear correlation and are a good fit to signals generated by physical systems, such as frontal images of human faces and multiple sources impinging at an antenna array. Manifolds model sources that are not linearly correlated, but where signals are determined by a small number of parameters. Examples are images of human faces under different poses or expressions, and handwritten digits with varying styles. However, there will always be some degree of model mismatch between the subspace or manifold model and the true statistics of the source. This dissertation exploits subspace and manifold models as prior information in various signal processing and machine learning tasks.

A near-low-rank Gaussian mixture model measures proximity to a union of linear or affine subspaces. This simple model can effectively capture the signal distribution when each class is near a subspace. This dissertation studies how the pairwise geometry between these subspaces affects classification performance. When model mismatch is vanishingly small, the probability of misclassification is determined by the product of the sines of the principal angles between subspaces. When the model mismatch is more significant, the probability of misclassification is determined by the sum of the squares of the sines of the principal angles. Reliability of classification is derived in terms of the distribution of signal energy across principal vectors. Larger principal angles lead to smaller classification error, motivating a linear transform that optimizes principal angles. This linear transformation, termed TRAIT, also preserves some specific features in each class, being complementary to a recently developed Low Rank Transform (LRT). Moreover, when the model mismatch is more significant, TRAIT shows superior performance compared to LRT.

The manifold model enforces a constraint on the freedom of data variation. Learning features that are robust to data variation is very important, especially when the size of the training set is small. A learning machine with large numbers of parameters, e.g., deep neural network, can well describe a very complicated data distribution. However, it is also more likely to be sensitive to small perturbations of the data, and to suffer from suffer from degraded performance when generalizing to unseen (test) data.

From the perspective of complexity of function classes, such a learning machine has a huge capacity (complexity), which tends to overfit. The manifold model provides us with a way of regularizing the learning machine, so as to reduce the generalization error, therefore mitigate overfiting. Two different overfiting-preventing approaches are proposed, one from the perspective of data variation, the other from capacity/complexity control. In the first approach, the learning machine is encouraged to make decisions that vary smoothly for data points in local neighborhoods on the manifold. In the second approach, a graph adjacency matrix is derived for the manifold, and the learned features are encouraged to be aligned with the principal components of this adjacency matrix. Experimental results on benchmark datasets are demonstrated, showing an obvious advantage of the proposed approaches when the training set is small.

Stochastic optimization makes it possible to track a slowly varying subspace underlying streaming data. By approximating local neighborhoods using affine subspaces, a slowly varying manifold can be efficiently tracked as well, even with corrupted and noisy data. The more the local neighborhoods, the better the approximation, but the higher the computational complexity. A multiscale approximation scheme is proposed, where the local approximating subspaces are organized in a tree structure. Splitting and merging of the tree nodes then allows efficient control of the number of neighbourhoods. Deviation (of each datum) from the learned model is estimated, yielding a series of statistics for anomaly detection. This framework extends the classical {\em changepoint detection} technique, which only works for one dimensional signals. Simulations and experiments highlight the robustness and efficacy of the proposed approach in detecting an abrupt change in an otherwise slowly varying low-dimensional manifold.

Learning for self-regulation: Improving instructional benefits for pupils, teachers, parents, schools, and society at large

Compulsory education laws oblige primary and secondary schools to give each pupil positive encouragement in, for example, social, emotional, cognitive, creative, and ethical respects. This is a fairly smooth process for most pupils, but it is not as easy to achieve with others. A pattern of pupil, home or family, and school variables turns out to be responsible for a long-term process that may lead to a pupil’s dropping out of education. A systemic approach will do much to introduce more clarity into the diagnosis, potential reduction and possible prevention of some persistent educational problems that express themselves in related phenomena, for example low school motivation and achievement; forced underachievement of high ability pupils; concentration of bullying and violent behaviour in and around some types of classes and schools; and drop-out percentages that are relatively constant across time. Such problems have a negative effect on pupils, teachers, parents, schools, and society alike. In this address, I would therefore like to clarify some of the systemic causes and processes that we have identified between specific educational and pupil characteristics. Both theory and practice can assist in developing, implementing, and checking better learning methods and coaching procedures, particularly for pupils at risk. This development approach will take time and require co-ordination, but it will result in much better processes and outcomes than we are used to. First, I will diagnose some systemic aspects of education that do not seem to optimise the learning processes and school careers of some types of pupils in particular. Second, I will specify cognitive, social, motivational, and self-regulative aspects of learning tasks and relate corresponding learning processes to relevant instructional and wider educational contexts. I will elaborate these theoretical notions into an educational design with systemic instructional guidelines and multilevel procedures that may improve learning processes for different types of pupils. Internet-based Information and Communication Technology, or ICT, also plays a major role here. Third, I will report on concrete developments made in prototype research and trials. The development process concerns ICT-based differentiation of learning materials and procedures, and ICT-based strategies to improve pupil development and learning. Fourth, I will focus on the experience gained in primary and secondary educational practice with respect to implementation. We can learn much from such practical experience, in particular about the conditions for developing and implementing the necessary changes in and around schools. Finally, I will propose future research. As I hope to make clear, theory-based development and implementation research can join forces with systemic innovation and differentiated assessment in educational practice, to pave the way for optimal “learning for self-regulation” for pupils, teachers, parents, schools, and society at large.

Techniques for batch reinforcement learning in robotics

This thesis addresses the Batch Reinforcement Learning methods in Robotics. This sub-class of Reinforcement Learning has shown promising results and has been the focus of recent research. Three contributions are proposed that aim to extend the state-of-art methods allowing for a faster and more stable learning process, such as required for learning in Robotics. The Q-learning update-rule is widely applied, since it allows to learn without the presence of a model of the environment. However, this update-rule is transition-based and does not take advantage of the underlying episodic structure of collected batch of interactions. The Q-Batch update-rule is proposed in this thesis, to process experiencies along the trajectories collected in the interaction phase. This allows a faster propagation of obtained rewards and penalties, resulting in faster and more robust learning. Non-parametric function approximations are explored, such as Gaussian Processes. This type of approximators allows to encode prior knowledge about the latent function, in the form of kernels, providing a higher level of exibility and accuracy. The application of Gaussian Processes in Batch Reinforcement Learning presented a higher performance in learning tasks than other function approximations used in the literature. Lastly, in order to extract more information from the experiences collected by the agent, model-learning techniques are incorporated to learn the system dynamics. In this way, it is possible to augment the set of collected experiences with experiences generated through planning using the learned models. Experiments were carried out mainly in simulation, with some tests carried out in a physical robotic platform. The obtained results show that the proposed approaches are able to outperform the classical Fitted Q Iteration.

Researching University Teaching in Australia: themes and issues in academics' reflections

University teaching is a diverse enterprise which encompasses a range of disciplines, cirricula, teaching methods, learning tasks and learning approaches. Within this diversity, common themes and issues which reflect academics' understanding of effective teaching may be discerned. Drawing on written data collected from 708 practising teachers who were nominated by their Heads or Deans as exhibiting exemplary teaching practice, and from interviews conducted with 44 of these, a number of these themes and issues are identified and illustrated The findings offer insights which may stimulate further reflection on, and discussion of, the quality of teaching in higher education.

Improving management skills through business simulation

A survey was conducted among students of the Accounting and Administration undergraduate degree at ISCAP – IPP (School of Accounting and Administration of Polytechnic Institute of Porto) in order to understand their perception value of their course Business Simulation (BS). This course is provided in a business environment where students can learn by doing through the management of a company as they were in the real life, but risk-free. The learning tasks are provided in an action-oriented format to maximize the learning process. Students learn by doing a set of tasks every session and have also to produce reports and presentations during the course. BS is part of the undergraduate degree of Accounting and Administration at ISCAP – IPP since the beginning of 2003. The questionnaire we used captured the students’ perception about general and specific skills and competencies considered important for managers and accountants in the real life, about the methodology used in the course, which is totally different from the traditional form, and also about the adequacy of the course included as part of the undergraduate degree. The results showed that students’ perception is highly positive and almost all of them think they improve the skills needed for a job during the course. These results are consistent with [1] Adler and Milne’s research in which the authors found that students agree with the use of action-oriented learning tasks in order to provide them the needed attitudes, skills, and knowledge. The improvement of group skills is the most important issue for students, which can be understandable as BS is the only course from the degree in Accounting and Administration they really have to work in groups.

An unsupervised approach to feature discretization and selection

Many learning problems require handling high dimensional datasets with a relatively small number of instances. Learning algorithms are thus confronted with the curse of dimensionality, and need to address it in order to be effective. Examples of these types of data include the bag-of-words representation in text classification problems and gene expression data for tumor detection/classification. Usually, among the high number of features characterizing the instances, many may be irrelevant (or even detrimental) for the learning tasks. It is thus clear that there is a need for adequate techniques for feature representation, reduction, and selection, to improve both the classification accuracy and the memory requirements. In this paper, we propose combined unsupervised feature discretization and feature selection techniques, suitable for medium and high-dimensional datasets. The experimental results on several standard datasets, with both sparse and dense features, show the efficiency of the proposed techniques as well as improvements over previous related techniques.

Comportamento da tibiotársica na sequência de movimento de sentar para de pé em crianças prematuras

Introdução: A prematuridade constitui um fator de risco para a ocorrência de lesões ao nível do sistema nervoso central, sendo que uma idade gestacional inferior a 36 semanas potencia esse mesmo risco, nomeadamente para a paralisia cerebral (PC) do tipo diplegia espástica. A sequência de movimento de sentado para de pé (SPP), sendo uma das aprendizagens motoras que exige um controlo postural (CP) ao nível da tibiotársica, parece ser uma tarefa funcional frequentemente comprometida em crianças prematuras com e sem PC. Objetivo(s): Descrever o comportamento dos músculos da tibiotársica, tibial anterior (TA) e solear (SOL), no que diz respeito ao timing de ativação, magnitude e co-ativação muscular durante a fase I e início da fase II na sequência de movimento de SPP realizada por cinco crianças prematuras com PC do tipo diplegia espástica e cinco crianças prematuras sem diagnóstico de alteração neuromotoras, sendo as primeiras sujeitas a um programa de intervenção baseado nos princípios do conceito de Bobath – Tratamento do Neurodesenvolvimento (TND). Métodos: Foram avaliadas 10 crianças prematuras, cinco com PC e cinco sem diagnóstico de alterações neuromotoras, tendo-se recorrido à eletromiografia de superfície para registar parâmetros musculares, nomeadamente timings, magnitudes e valores de co-ativação dos músculos TA e SOL, associados à fase I e inico da fase II da sequência de movimento de SPP. Procedeu-se ao registo de imagem de modo a facilitar a avaliação dos componentes de movimento associados a esta tarefa. Estes procedimentos foram realizados num único momento, no caso das crianças sem diagnóstico de alterações neuromotoras e em dois momentos, antes e após a aplicação de um programa de intervenção segundo o Conceito de Bobath – TND no caso das crianças com PC. A estas foi ainda aplicado o Teste da Medida das Funções Motoras (TMFM–88) e a Classificação Internacional da Funcionalidade Incapacidade e Saúde – crianças e jovens (CIF-CJ). Resultados: Através da eletromiografia constatou-se que ambos os grupos apresentaram timings de ativação afastados da janela temporal considerada como ajustes posturais antecipatórios (APAs), níveis elevados de co-ativação, em alguns casos com inversão na ordem de recrutamento muscular o que foi possível modificar nas crianças com PC após o período de intervenção. Nestas, verificou-se ainda que, a sequência de movimento de SPP foi realizada com menor número de compensações e com melhor relação entre estruturas proximais e distais compatível com o aumento do score final do TMFM-88 e modificação positiva nos itens de atividade e participação da CIF-CJ. Conclusão: As crianças prematuras com e sem PC apresentaram alterações no CP da tibiotársica e níveis elevados de co-ativação muscular. Após o período de intervenção as crianças com PC apresentaram modificações positivas no timing e co-ativação muscular, com impacto funcional evidenciado no aumento do score final da TMFM-88 e modificações positivas na CIF-CJ.

Trabalho cooperativo como finalidade e estratégia de aprendizagem. Um estudo de caso em biologia humana

A aprendizagem cooperativa é concebida como uma estratégia pedagógica que privilegia uma aprendizagem personalizada e que potencia o sucesso educativo não só individual mas também coletivo. É conseguida através da cooperação de todos os membros do grupo, em que o desempenho de cada um influencia e é influenciado pelo desempenho do Outro. O grupo é concebido como uma organização social cuja eficiência implica a capacidade de construção e manutenção do grupo como um todo e de promoção do sucesso educativo de todos os elementos. É este quadro teórico que enforma a conceção, implementação e avaliação de uma intervenção pedagógica em Biologia Humana do 10º ano do Curso Tecnológico de Desporto. A estratégia pedagógica carateriza-se pela operacionalização articulada de estruturas e papéis de cooperação na abordagem da unidade didática - Transformação e Utilização de Energia - com a finalidade de promover a aprendizagem integrada de competências de cooperação e de conhecimento substantivo da área da Biologia. A avaliação da intervenção pedagógica incidiu nos objetivos de investigação - 1) Identificar o impacto da estratégia de intervenção pedagógica no desenvolvimento de competências de cooperação e 2) Identificar o impacto da estratégia de intervenção pedagógica no desenvolvimento de competências disciplinares –, efetuada a partir da análise de tarefas de aprendizagem e de um questionário de avaliação final global aplicado aos alunos. A estratégia de intervenção pedagógica é percecionada pelos alunos como tendo contribuído para a promoção do desenvolvimento não só de competências de cooperação mas também do conhecimento substantivo e do pensamento crítico.