Apprentissage des réseaux de neurones profonds et applications en traitement automatique de la langue naturelle

En apprentissage automatique, domaine qui consiste à utiliser des données pour apprendre une solution aux problèmes que nous voulons confier à la machine, le modèle des Réseaux de Neurones Artificiels (ANN) est un outil précieux. Il a été inventé voilà maintenant près de soixante ans, et pourtant, il est encore de nos jours le sujet d'une recherche active. Récemment, avec l'apprentissage profond, il a en effet permis d'améliorer l'état de l'art dans de nombreux champs d'applications comme la vision par ordinateur, le traitement de la parole et le traitement des langues naturelles. La quantité toujours grandissante de données disponibles et les améliorations du matériel informatique ont permis de faciliter l'apprentissage de modèles à haute capacité comme les ANNs profonds. Cependant, des difficultés inhérentes à l'entraînement de tels modèles, comme les minima locaux, ont encore un impact important. L'apprentissage profond vise donc à trouver des solutions, en régularisant ou en facilitant l'optimisation. Le pré-entraînnement non-supervisé, ou la technique du ``Dropout'', en sont des exemples. Les deux premiers travaux présentés dans cette thèse suivent cette ligne de recherche. Le premier étudie les problèmes de gradients diminuants/explosants dans les architectures profondes. Il montre que des choix simples, comme la fonction d'activation ou l'initialisation des poids du réseaux, ont une grande influence. Nous proposons l'initialisation normalisée pour faciliter l'apprentissage. Le second se focalise sur le choix de la fonction d'activation et présente le rectifieur, ou unité rectificatrice linéaire. Cette étude a été la première à mettre l'accent sur les fonctions d'activations linéaires par morceaux pour les réseaux de neurones profonds en apprentissage supervisé. Aujourd'hui, ce type de fonction d'activation est une composante essentielle des réseaux de neurones profonds. Les deux derniers travaux présentés se concentrent sur les applications des ANNs en traitement des langues naturelles. Le premier aborde le sujet de l'adaptation de domaine pour l'analyse de sentiment, en utilisant des Auto-Encodeurs Débruitants. Celui-ci est encore l'état de l'art de nos jours. Le second traite de l'apprentissage de données multi-relationnelles avec un modèle à base d'énergie, pouvant être utilisé pour la tâche de désambiguation de sens.

From Word Embeddings to Large Vocabulary Neural Machine Translation

Dans ce mémoire, nous examinons certaines propriétés des représentations distribuées de mots et nous proposons une technique pour élargir le vocabulaire des systèmes de traduction automatique neurale. En premier lieu, nous considérons un problème de résolution d'analogies bien connu et examinons l'effet de poids adaptés à la position, le choix de la fonction de combinaison et l'impact de l'apprentissage supervisé. Nous enchaînons en montrant que des représentations distribuées simples basées sur la traduction peuvent atteindre ou dépasser l'état de l'art sur le test de détection de synonymes TOEFL et sur le récent étalon-or SimLex-999. Finalament, motivé par d'impressionnants résultats obtenus avec des représentations distribuées issues de systèmes de traduction neurale à petit vocabulaire (30 000 mots), nous présentons une approche compatible à l'utilisation de cartes graphiques pour augmenter la taille du vocabulaire par plus d'un ordre de magnitude. Bien qu'originalement développée seulement pour obtenir les représentations distribuées, nous montrons que cette technique fonctionne plutôt bien sur des tâches de traduction, en particulier de l'anglais vers le français (WMT'14).

Hierarchical Prediction Machines and Big Data Analytics

An emerging consensus in cognitive science views the biological brain as a hierarchically-organized predictive processing system. This is a system in which higher-order regions are continuously attempting to predict the activity of lower-order regions at a variety of (increasingly abstract) spatial and temporal scales. The brain is thus revealed as a hierarchical prediction machine that is constantly engaged in the effort to predict the flow of information originating from the sensory surfaces. Such a view seems to afford a great deal of explanatory leverage when it comes to a broad swathe of seemingly disparate psychological phenomena (e.g., learning, memory, perception, action, emotion, planning, reason, imagination, and conscious experience). In the most positive case, the predictive processing story seems to provide our first glimpse at what a unified (computationally-tractable and neurobiological plausible) account of human psychology might look like. This obviously marks out one reason why such models should be the focus of current empirical and theoretical attention. Another reason, however, is rooted in the potential of such models to advance the current state-of-the-art in machine intelligence and machine learning. Interestingly, the vision of the brain as a hierarchical prediction machine is one that establishes contact with work that goes under the heading of 'deep learning'. Deep learning systems thus often attempt to make use of predictive processing schemes and (increasingly abstract) generative models as a means of supporting the analysis of large data sets. But are such computational systems sufficient (by themselves) to provide a route to general human-level analytic capabilities? I will argue that they are not and that closer attention to a broader range of forces and factors (many of which are not confined to the neural realm) may be required to understand what it is that gives human cognition its distinctive (and largely unique) flavour. The vision that emerges is one of 'homomimetic deep learning systems', systems that situate a hierarchically-organized predictive processing core within a larger nexus of developmental, behavioural, symbolic, technological and social influences. Relative to that vision, I suggest that we should see the Web as a form of 'cognitive ecology', one that is as much involved with the transformation of machine intelligence as it is with the progressive reshaping of our own cognitive capabilities.

Uptake pathways: the potential of Bayesian belief networks to assist the management, monitoring and evaluation of development-orientated research

The effectiveness of development assistance has come under renewed scrutiny in recent years. In an era of growing economic liberalisation, research organisations are increasingly being asked to account for the use of public funds by demonstrating achievements. However, in the natural resources (NR) research field, conventional economic assessment techniques have focused on quantifying the impact achieved rather understanding the process that delivered it. As a result, they provide limited guidance for planners and researchers charged with selecting and implementing future research. In response, “pathways” or logic models have attracted increased interest in recent years as a remedy to this shortcoming. However, as commonly applied these suffer from two key limitations in their ability to incorporate risk and assess variance from plan. The paper reports the results of a case study that used a Bayesian belief network approach to address these limitations and outlines its potential value as a tool to assist the planning, monitoring and evaluation of development-orientated research.

A forward constrained selection algorithm for probabilistic neural network

A new probabilistic neural network (PNN) learning algorithm based on forward constrained selection (PNN-FCS) is proposed. An incremental learning scheme is adopted such that at each step, new neurons, one for each class, are selected from the training samples arid the weights of the neurons are estimated so as to minimize the overall misclassification error rate. In this manner, only the most significant training samples are used as the neurons. It is shown by simulation that the resultant networks of PNN-FCS have good classification performance compared to other types of classifiers, but much smaller model sizes than conventional PNN.

‘Did Ronald McDonald also tend to scare you as a child?’: working to emplace consumption, commodities, and citizen-students in a large classroom setting

So-called ‘radical’ and ‘critical’ pedagogy seems to be everywhere these days on the landscapes of geographical teaching praxis and theory. Part of the remit of radical/critical pedagogy involves a de-centring of the traditional ‘banking’ method of pedagogical praxis. Yet, how do we challenge this ‘banking’ model of knowledge transmission in both a large-class setting and around the topic of commodity geographies where the banking model of information transfer still holds sway? This paper presents a theoretically and pedagogically driven argument, as well as a series of practical teaching ‘techniques’ and tools—mind-mapping and group work—designed to promote ‘deep learning’ and a progressive political potential in a first-year large-scale geography course centred around lectures on the Geographies of Consumption and Material Culture. Here students are not only asked to place themselves within and without the academic materials and other media but are urged to make intimate connections between themselves and their own consumptive acts and the commodity networks in which they are enmeshed. Thus, perhaps pedagogy needs to be emplaced firmly within the realms of research practice rather than as simply the transference of research findings.

A Cultura e a Pré-Disposição para um Ambiente de Aprendizagem Organizacional: Um Estudo da Cultura da Isabela S/A

The increase of the global competitiveness of the global alliances of the economic blocks, generates an increasing stream of information that makes with that the companies are always the seek of new developments. The climate of changes generates concern with what it must be assimilated by organization, and things that don¿t serve the definitive company. The organizational environment of the companies is taken by uncertainty. The organizational culture becomes itself very important for the organization, therefore it judges what the company can use in the relation with its environment, and what does not serve to its organizational structure. Together with the culture is necessary that itself a learning environment, leading thus culture. The work considered here studies the organizational culture of a company making relation with the organizational Learning, and helps to clarify the profile of a culture that propitiates the organizational learning. This work analyzes the Isabela¿s S/A organizational culture, making relation with the organizational learning. Of this form, look it self to describe and to identify the culture of the searched company, studying its difficulties and advantages, making relation with the deep learning. The company is characterized, the situation problem and the subject of study. In the theoretical referential was made an analysis of the culture, its formation until the types of culture, applying itself after a model of organizational learning. A model for the study of the culture is applied, where self is characterized, tipology of the culture, and applied a model for the study of the learning. The results of the considered study, answer the research problem, where they are analysed with the application of the diagnostic research of the searched company. This discloses an organization with a strong culture, however flexible to assimilate changes. It has a harmony between the values that generate the culture and that they are characterized, spreading out thus an environment that propitiates organizational learning, a culture that generates learning.

Criar o próprio jogo didático ou apenas jogar? Efeitos de diferentes estratégias de ensino na motivação e aprendizado de ciências

The use of games as educational tools is common, however the effectiveness of games with educational purposes is still poorly known. In this study we evaluated three different low-cost teaching strategies make and play your own board game, just play an educational science game and make a poster to be exposed in the school regarding: (1) science learning; (2) use of deep learning strategies (DLS); and (3) intrinsic motivation. We tested the hypothesis that, in these three parameters evaluated, scores would be higher in the group that made and play their own game, followed respectively by the group that just played a game and the group that made a poster. The research involved 214 fifth-grade students from six elementary schools in Natal/RN. A group of students made and played their own science board game (N = 68), a second group played a science game (N = 75), and a third group made a poster to be exposed at school (N = 71). Our hypothesis was partly empirically supported, since there was no significant difference in science learning and in the use of DLS between the group that made their own game and the group that just played the game; however, both groups had significantly higher scores in science learning and in use of DLS than the group that made the poster. There was no significant difference in the scores of intrinsic motivation among the three experimental groups. Our results indicate that activities related to non-digital games can provide a favorable context for learning in the school environment. We conclude that the use of games for educational purposes (both making a game and just playing a game) is an efficient and viable alternative to teach science in Brazilian public school

Head, eye and arm coordination in table tennis

The aim of this study was to determine the role of head, eye and arm movements during the execution of a table tennis forehand stroke. Three-dimensional kinematic analysis of line-of-gaze, arm and ball was used to describe visual and motor behaviour. Skilled and less skilled participants returned the ball to cued right or left target areas under three levels of temporal constraint: pre-, early- and late-cue conditions. In the pre- and early-cue conditions, both high and low skill participants tracked the ball early in flight and kept gaze stable on a location in advance of the ball before ball-bat contact. Skilled participants demonstrated an earlier onset of ball tracking and recorded higher performance accuracy than less skilled counterparts. The manipulation of cue condition showed the limits of adaptation to maintain accuracy on the target. Participants were able to accommodate the constraints imposed by the early-cue condition by using a shorter quiet eye duration, earlier quiet eye offset and reduced arm velocity at contact. In the late-cue condition, modifications to gaze, head and arm movements were not sufficient to preserve accuracy. The findings highlight the functional coupling between perception and action during time-constrained, goal-directed actions.

Model selection for discriminative restricted boltzmann machines through meta-heuristic techniques

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Structure learning of graphical models for task-oriented robot grasping

In the collective imaginaries a robot is a human like machine as any androids in science fiction. However the type of robots that you will encounter most frequently are machinery that do work that is too dangerous, boring or onerous. Most of the robots in the world are of this type. They can be found in auto, medical, manufacturing and space industries. Therefore a robot is a system that contains sensors, control systems, manipulators, power supplies and software all working together to perform a task. The development and use of such a system is an active area of research and one of the main problems is the development of interaction skills with the surrounding environment, which include the ability to grasp objects. To perform this task the robot needs to sense the environment and acquire the object informations, physical attributes that may influence a grasp. Humans can solve this grasping problem easily due to their past experiences, that is why many researchers are approaching it from a machine learning perspective finding grasp of an object using information of already known objects. But humans can select the best grasp amongst a vast repertoire not only considering the physical attributes of the object to grasp but even to obtain a certain effect. This is why in our case the study in the area of robot manipulation is focused on grasping and integrating symbolic tasks with data gained through sensors. The learning model is based on Bayesian Network to encode the statistical dependencies between the data collected by the sensors and the symbolic task. This data representation has several advantages. It allows to take into account the uncertainty of the real world, allowing to deal with sensor noise, encodes notion of causality and provides an unified network for learning. Since the network is actually implemented and based on the human expert knowledge, it is very interesting to implement an automated method to learn the structure as in the future more tasks and object features can be introduced and a complex network design based only on human expert knowledge can become unreliable. Since structure learning algorithms presents some weaknesses, the goal of this thesis is to analyze real data used in the network modeled by the human expert, implement a feasible structure learning approach and compare the results with the network designed by the expert in order to possibly enhance it.

Play!

Play is a scholarly topic with a long and convoluted history. This faculty colloquium will make a few obligatory nods to the literature, but then focus instead on how play might be related to the Bucknell educational mission. The talk will be structured more in the spirit of a “This I Believe” essay. It is meant to challenge the Bucknell community to consider play as an important component of a healthy environment. Some questions we will visit: Why as we mature do we view play as the opposite of work? How can play be used to practice skills and explore our world-view? What is the connection between play and creative habits? How can we pair play with reflection to stimulate deep learning? How might we alter our risk/reward structures to encourage productive play?

E-Learning-Webseite als facettenreiches Werkzeug in Forschung und Lehre

Im Rahmen des blended learning kann eine E-Learning-Webseite als Begleitmaterial einer Lehrveranstaltung eingesetzt werden oder Studierende zur aktiven Teilnahme an der Erstellung der Webseiteninhalte anregen. Darüber hinaus eignet sich eine solche Webseite als Plattform zur E-Learning-Forschung. Auch empirische Studien können dort eingebettet werden. Eine weitere wissenschaftliche Anwendung bietet die Analyse des Nutzerverhaltens, mit der sich aktuelle Forschungsergebnisse zum Lernen mit Hypermedien überprüfen lassen. Wir beschreiben eine solche, vielseitig einsetzbare Webseite, die eine Verknüpfung von universitärer Lehre und Forschung ermöglicht und als Anregung für ähnliche Projekte dienen kann. Erste Erfahrungen werden dabei berichtet und ausgewählte Empfehlungen für Dozierende und Forscher abgeleitet.

The Role for Virtual Patients in the Future of Medical Education

The medical education community is working-across disciplines and across the continuum-to address the current challenges facing the medical education system and to implement strategies to improve educational outcomes. Educational technology offers the promise of addressing these important challenges in ways not previously possible. The authors propose a role for virtual patients (VPs), which they define as multimedia, screen-based interactive patient scenarios. They believe VPs offer capabilities and benefits particularly well suited to addressing the challenges facing medical education. Well-designed, interactive VP-based learning activities can promote the deep learning that is needed to handle the rapid growth in medical knowledge. Clinically oriented learning from VPs can capture intrinsic motivation and promote mastery learning. VPs can also enhance trainees' application of foundational knowledge to promote the development of clinical reasoning, the foundation of medical practice. Although not the entire solution, VPs can support competency-based education. The data created by the use of VPs can serve as the basis for multi-institutional research that will enable the medical education community both to better understand the effectiveness of educational interventions and to measure progress toward an improved system of medical education.