Advances in scaling deep learning algorithms

Les algorithmes d'apprentissage profond forment un nouvel ensemble de méthodes puissantes pour l'apprentissage automatique. L'idée est de combiner des couches de facteurs latents en hierarchies. Cela requiert souvent un coût computationel plus elevé et augmente aussi le nombre de paramètres du modèle. Ainsi, l'utilisation de ces méthodes sur des problèmes à plus grande échelle demande de réduire leur coût et aussi d'améliorer leur régularisation et leur optimization. Cette thèse adresse cette question sur ces trois perspectives. Nous étudions tout d'abord le problème de réduire le coût de certains algorithmes profonds. Nous proposons deux méthodes pour entrainer des machines de Boltzmann restreintes et des auto-encodeurs débruitants sur des distributions sparses à haute dimension. Ceci est important pour l'application de ces algorithmes pour le traitement de langues naturelles. Ces deux méthodes (Dauphin et al., 2011; Dauphin and Bengio, 2013) utilisent l'échantillonage par importance pour échantilloner l'objectif de ces modèles. Nous observons que cela réduit significativement le temps d'entrainement. L'accéleration atteint 2 ordres de magnitude sur plusieurs bancs d'essai. Deuxièmement, nous introduisont un puissant régularisateur pour les méthodes profondes. Les résultats expérimentaux démontrent qu'un bon régularisateur est crucial pour obtenir de bonnes performances avec des gros réseaux (Hinton et al., 2012). Dans Rifai et al. (2011), nous proposons un nouveau régularisateur qui combine l'apprentissage non-supervisé et la propagation de tangente (Simard et al., 1992). Cette méthode exploite des principes géometriques et permit au moment de la publication d'atteindre des résultats à l'état de l'art. Finalement, nous considérons le problème d'optimiser des surfaces non-convexes à haute dimensionalité comme celle des réseaux de neurones. Tradionellement, l'abondance de minimum locaux était considéré comme la principale difficulté dans ces problèmes. Dans Dauphin et al. (2014a) nous argumentons à partir de résultats en statistique physique, de la théorie des matrices aléatoires, de la théorie des réseaux de neurones et à partir de résultats expérimentaux qu'une difficulté plus profonde provient de la prolifération de points-selle. Dans ce papier nous proposons aussi une nouvelle méthode pour l'optimisation non-convexe.

Occupational therapy and its relevance in social projects. relationships, learning and opportunities: collection of experiences of University of Alberta Mscot students in Bogotá, Colombia

Occupational therapists are equipped to promote wellbeing through occupation and to enable participation and meaningful engagement of people in their social and physical environments (WFOT, 2012). As such, the role of the occupational therapists is profoundly linked to the social, cultural and environmental characteristics of the contexts in which occupations take place. The central role that context plays in occupational performance creates an interesting dichotomy for the occupational therapist: on one hand, a profound understanding of cultural and social factors is required from the Occupational Therapy (OT) in order to develop a meaningful and successful collaboration with the person; on the other hand, the ability of the occupational therapists to recognize and explore the contextual factor of an occupation-person dyad transcends cultural and spatial barriers. As a result, occupational therapists are equipped to engage in international collaboration and practice, and as such face unique and enriching challenges. International fieldwork experiences have become a tool through which occupational therapists in training can develop the critical skills for understanding the impact of cultural and social factors on occupation. An OT student in an international fieldwork experience faces numerous challenges in leading a process that is both relevant and respectful to the characteristics of the local context: language, cultural perceptions of occupation and personhood, religious backgrounds, health care access, etc. These challenges stand out as ethical considerations that must be considered when navigating an international fieldwork experience (AOTA, 2009). For more than five years now, the Faculty of Rehabilitation Medicine (FRM) of the University of Alberta (UoFA) and the School of Medicine and Health Sciences at the Universidad del Rosario (UR), Bogota, Colombia, have sustained a productive and meaningful international collaboration. This collaboration includes a visit by Dr. Albert Cook, professor of the FRM and former dean, to the UR as the main guest speaker in the International Congress of Technologies for Disability Support (IBERDISCAP) in 2008. Furthermore, Dr. Cook was a speaker in the research seminar of the Assistive Technology Research Group of the Universidad del Rosario. Following Dr. Cook’s visit, Professors Liliana Álvarez and Adriana Ríos travelled to Edmonton and initiated collaboration with the FRM, resulting in the signing of an agreement between the FRM and the UR in 2009, agreement that has been maintained to this day. The main goal of this agreement is to increase academic and cultural cooperation between the UR and the UofA. Other activities have included the cooperation between Dr. Kim Adams (who has largely maintained interest and effort in supporting the capacity building of the UR rehabilitation programs in coordinating the provision of research placement opportunities for UR students at the UofA), an Assistive Technology course for clinicians and students led by Dr. Adams, and a research project that researched the use of basic cell phones to provide social interaction and health information access for people with disabilities in a low-income community in Colombia (led by Tim Barlott, OT, MSc, under the supervision of Dr. Adams). Since the beginning, the occupational therapy programs of the Universidad del Rosario and the University of Alberta have promoted this collaboration and have strived to engage in interactions that provide further development opportunities for students and staff. As part of this process, the international placement experience of UofA OT students was born under the leadership of: Claudia Rozo, OT program director at UR, placement and academic leadership of Elvis Castro and Angélica Monsalve, professors of the occupational therapy program at UR; and Dr. Lili Liu, OT department director at UofA, Cori Schmitz, Academic coordinator of clinical education at the UofA; and Tim Barlott and Liliana Álvarez leading the international and cross-cultural aspect of this collaboration.This publication summarizes and illustrates the process of international placement in community settings in Colombia, undertaken by occupational therapy students of the University of Alberta. It is our hope that this document can provide and document the ethical considerations of international fieldwork experience, the special characteristics of communities and the ways in which cultural and social competences are developed and help international students navigate the international setting. We also hope that this document will stimulate discussion among professional and academic communities about the importance and richness of international placement experiences and encourage staff and students to articulate their daily efforts with the global occupational therapy agenda.

Food for thought: the role of dietary flavonoids in enhancing human memory, learning and neuro-cognitive performance

Emerging evidence suggests that dietary-derived flavonoids have the potential to improve human memory and neuro-cognitive performance via their ability to protect vulnerable neurons, enhance existing neuronal function and stimulate neuronal regeneration. Long-term potentiation (LTP) is widely considered to be one of the major mechanisms underlying memory acquisition, consolidation and storage in the brain and is known to be controlled at the molecular level by the activation of a number of neuronal signalling pathways. These pathways include the phosphatidylinositol-3 kinase/protein kinase B/Akt (Akt), protein kinase C, protein kinase A, Ca-calmodulin kinase and mitogen-activated protein kinase pathways. Growing evidence suggests that flavonoids exert effects on LTP, and consequently memory and cognitive performance, through their interactions with these signalling pathways. Of particular interest is the ability of flavonoids to activate the extracellular signal-regulated kinase and the Akt signalling pathways leading to the activation of the cAMP-response element-binding protein, a transcription factor responsible for increasing the expression of a number of neurotrophins important in LTP and long-term memory. One such neurotrophin is brain-derived neurotrophic factor, which is known to be crucial in controlling synapse growth, in promoting an increase in dendritic spine density and in enhancing synaptic receptor density. The present review explores the potential of flavonoids and their metabolite forms to promote memory and learning through their interactions with neuronal signalling pathways pivotal in controlling LTP and memory in human subjects.

HOW FAR do WE GET USING MACHINE LEARNING BLACK-BOXES?

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Particle competition and cooperation for semi-supervised learning with label noise

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

A Socioeconomic Approach to the Development of E-Learning

A multitude of products, systems, approaches, views and notions characterize the field of e-learning. This article attempts to disentangle the field by using economic and sociological theories, theories of marketing management and strategy as well as practical experience gained by the author while working with leading edge suppliers of e-learning. On this basis, a distinction between knowledge creation e-learning and knowledge transfer e-learning is made. The various views are divided into four different ideal-typical paradigms, each with its own characteristics and limitations. Selecting the right paradigm to use in the development of an e-learning strategy may prove crucial to success. Implications for the development of an e-learning strategy in businesses and educational institutions are outlined.

Linking Applied Ethnobotany to Social Learning: A Participatory Tool for the Promotion of Indigenous Plants Use in Matutuíne, Southern Mozambique

In its search for pathways towards a more sustainable management of natural resources, development oriented research increasingly faces the challenge to develop new concepts and tools based on transdisciplinarity. Transdisciplinarity can, in terms of an idealized goal, be defined as a research approach that identifies and solves problems not only independently of disciplinary boundaries, but also including the knowledge and perceptions of non-scientific actors in a participatory process. In Mozambique, the Centre for Development and Environment (Berne, Switzerland), in partnership with Impacto and Helvetas (Maputo, Mozambique), has elaborated a new transdisciplinary tool to identify indigenous plants with a potential for commercialization. The tool combines methods from applied ethnobotany with participatory research in a social learning process. This approach was devised to support a development project aimed at creating alternative sources of income for rural communities of Matutuíne district, Southern Mozambique, while reducing the pressure on the natural environment. The methodology, which has been applied and tested, is innovative in that it combines important data collection through participatory research with a social learning process involving both local and external actors. This mutual learning process provides a space for complementary forms of knowledge to meet, eventually leading to the adoption of an integrated approach to natural resource management with an understanding of its ecological, socio-economic and cultural aspects; local stakeholders are included in the identification of potentials for sustainable development. Sustainable development itself, as a normative concept, can only be defined through social learning and consensus building between the local and external stakeholders.

Expanding Learning Time in Schools: Considering the Challenges of Implementation and the Potential Impact

Considering the broader context of school reform that is seeking education strategies that might deliver substantial impact, this article examines four questions related to the policy and practice of expanding learning time: (a) why do educators find the standard American school calendar insufficient to meet students’ educational needs, especially those of disadvantaged students? (b) how do educators implement a longer day and/or year, addressing concerns about both educational quality and costs? (c) what does research report about outcomes of expanding time in schools? and (d) what are the future prospects for increasing the number of expanded-time schools? The paper examines these questions by considering research, policy, and practice at the national level and, throughout, by drawing upon additional evidence from Massachusetts, one of the leading states in the expanded-time movement. In considering the latter two questions, the article explores the knowns and unknowns related to expanded learning time and offers suggestions for further research.

Learning kinds for students of technical high school : An analysis from the David Ausubel's theory

This article presents the findings of a field research, not experimental, observational, correlating, basic, of mixed data, micro sociologic, leading to a study of surveys.The object of study is to find learning kinds, and the unit of analysis were 529 high school students between 16 and 21 years old. Its purpose is to understand the impact of learning by rote, guided, self learned and meaningful learning and its achievement degree besides the learning outcomes of differentiated curriculum based on David Ausubel's thoughts, associated with different economic specialties (MINEDUC, 1998) where the population of the study is trained. To collect data, the test TADA - DO2 was used, this test has a reliability index of 0.911 according to Cronbach. From the hits it can be stated from the null hypothesis that there is a significant association (a = 0,05) between the learning kinds and the learning expected of differentiated training plan for both, male and female. It is complex to state that the training of the middle-level technicians leads to a successful employment.

Learning kinds for students of technical high school : An analysis from the David Ausubel's theory

This article presents the findings of a field research, not experimental, observational, correlating, basic, of mixed data, micro sociologic, leading to a study of surveys.The object of study is to find learning kinds, and the unit of analysis were 529 high school students between 16 and 21 years old. Its purpose is to understand the impact of learning by rote, guided, self learned and meaningful learning and its achievement degree besides the learning outcomes of differentiated curriculum based on David Ausubel's thoughts, associated with different economic specialties (MINEDUC, 1998) where the population of the study is trained. To collect data, the test TADA - DO2 was used, this test has a reliability index of 0.911 according to Cronbach. From the hits it can be stated from the null hypothesis that there is a significant association (a = 0,05) between the learning kinds and the learning expected of differentiated training plan for both, male and female. It is complex to state that the training of the middle-level technicians leads to a successful employment.

Learning kinds for students of technical high school : An analysis from the David Ausubel's theory

This article presents the findings of a field research, not experimental, observational, correlating, basic, of mixed data, micro sociologic, leading to a study of surveys.The object of study is to find learning kinds, and the unit of analysis were 529 high school students between 16 and 21 years old. Its purpose is to understand the impact of learning by rote, guided, self learned and meaningful learning and its achievement degree besides the learning outcomes of differentiated curriculum based on David Ausubel's thoughts, associated with different economic specialties (MINEDUC, 1998) where the population of the study is trained. To collect data, the test TADA - DO2 was used, this test has a reliability index of 0.911 according to Cronbach. From the hits it can be stated from the null hypothesis that there is a significant association (a = 0,05) between the learning kinds and the learning expected of differentiated training plan for both, male and female. It is complex to state that the training of the middle-level technicians leads to a successful employment.

Regularization for sparsity in statistical analysis and machine learning

Pragmatism is the leading motivation of regularization. We can understand regularization as a modification of the maximum-likelihood estimator so that a reasonable answer could be given in an unstable or ill-posed situation. To mention some typical examples, this happens when fitting parametric or non-parametric models with more parameters than data or when estimating large covariance matrices. Regularization is usually used, in addition, to improve the bias-variance tradeoff of an estimation. Then, the definition of regularization is quite general, and, although the introduction of a penalty is probably the most popular type, it is just one out of multiple forms of regularization. In this dissertation, we focus on the applications of regularization for obtaining sparse or parsimonious representations, where only a subset of the inputs is used. A particular form of regularization, L1-regularization, plays a key role for reaching sparsity. Most of the contributions presented here revolve around L1-regularization, although other forms of regularization are explored (also pursuing sparsity in some sense). In addition to present a compact review of L1-regularization and its applications in statistical and machine learning, we devise methodology for regression, supervised classification and structure induction of graphical models. Within the regression paradigm, we focus on kernel smoothing learning, proposing techniques for kernel design that are suitable for high dimensional settings and sparse regression functions. We also present an application of regularized regression techniques for modeling the response of biological neurons. Supervised classification advances deal, on the one hand, with the application of regularization for obtaining a na¨ıve Bayes classifier and, on the other hand, with a novel algorithm for brain-computer interface design that uses group regularization in an efficient manner. Finally, we present a heuristic for inducing structures of Gaussian Bayesian networks using L1-regularization as a filter. El pragmatismo es la principal motivación de la regularización. Podemos entender la regularización como una modificación del estimador de máxima verosimilitud, de tal manera que se pueda dar una respuesta cuando la configuración del problema es inestable. A modo de ejemplo, podemos mencionar el ajuste de modelos paramétricos o no paramétricos cuando hay más parámetros que casos en el conjunto de datos, o la estimación de grandes matrices de covarianzas. Se suele recurrir a la regularización, además, para mejorar el compromiso sesgo-varianza en una estimación. Por tanto, la definición de regularización es muy general y, aunque la introducción de una función de penalización es probablemente el método más popular, éste es sólo uno de entre varias posibilidades. En esta tesis se ha trabajado en aplicaciones de regularización para obtener representaciones dispersas, donde sólo se usa un subconjunto de las entradas. En particular, la regularización L1 juega un papel clave en la búsqueda de dicha dispersión. La mayor parte de las contribuciones presentadas en la tesis giran alrededor de la regularización L1, aunque también se exploran otras formas de regularización (que igualmente persiguen un modelo disperso). Además de presentar una revisión de la regularización L1 y sus aplicaciones en estadística y aprendizaje de máquina, se ha desarrollado metodología para regresión, clasificación supervisada y aprendizaje de estructura en modelos gráficos. Dentro de la regresión, se ha trabajado principalmente en métodos de regresión local, proponiendo técnicas de diseño del kernel que sean adecuadas a configuraciones de alta dimensionalidad y funciones de regresión dispersas. También se presenta una aplicación de las técnicas de regresión regularizada para modelar la respuesta de neuronas reales. Los avances en clasificación supervisada tratan, por una parte, con el uso de regularización para obtener un clasificador naive Bayes y, por otra parte, con el desarrollo de un algoritmo que usa regularización por grupos de una manera eficiente y que se ha aplicado al diseño de interfaces cerebromáquina. Finalmente, se presenta una heurística para inducir la estructura de redes Bayesianas Gaussianas usando regularización L1 a modo de filtro.

Traditional education vs modern education. What is the impact of teaching techniques' evolution on students learning process?

The main objective of this article is to focus on the analysis of teaching techniques, ranging from the use of the blackboard and chalk in old traditional classes, using slides and overhead projectors in the eighties and use of presentation software in the nineties, to the video, electronic board and network resources nowadays. Furthermore, all the aforementioned, is viewed under the different mentalities in which the teacher conditions the student using the new teaching technique, improving soft skills but maybe leading either to encouragement or disinterest, and including the lack of educational knowledge consolidation at scientific, technology and specific levels. In the same way, we study the process of adaptation required for teachers, the differences in the processes of information transfer and education towards the student, and even the existence of teachers who are not any longer appealed by their work due which has become much simpler due to new technologies and the greater ease in the development of classes due to the criteria described on the new Grade Programs adopted by the European Higher Education Area. Moreover, it is also intended to understand the evolution of students’ profiles, from the eighties to present time, in order to understand certain attitudes, behaviours, accomplishments and acknowledgements acquired over the semesters within the degree Programs. As an Educational Innovation Group, another key question also arises. What will be the learning techniques in the future?. How these evolving matters will affect both positively and negatively on the mentality, attitude, behaviour, learning, achievement of goals and satisfaction levels of all elements involved in universities’ education? Clearly, this evolution from chalk to the electronic board, the three-dimensional view of our works and their sequence, greatly facilitates the understanding and adaptation later on to the business world, but does not answer to the unknowns regarding the knowledge and the full development of achievement’s indicators in basic skills of a degree. This is the underlying question which steers the roots of the presented research.

Facilitating the creation of interactive multi-device Learning Objects using an online authoring tool

Learning Objects facilitate reuse leading to cost and time savings as well as to the enhancement of the quality of educational resources. However, teachers find it difficult to create or to find high quality Learning Objects, and the ones they find need to be customized. Teachers can overcome this problem using suitable authoring systems that enable them to create high quality Learning Objects with little effort. This paper presents an open source online e-Learning authoring tool called ViSH Editor together with four novel interactive Learning Objects that can be created with it: Flashcards, Virtual Tours, Enriched Videos and Interactive Presentations. All these Learning Objects are created as web applications, which can be accessed via mobile devices. Besides, they can be exported to SCORM including their metadata in IEEE LOM format. All of them are described in the paper including an example of each. This approach for creating Learning Objects was validated through two evaluations: a survey among authors and a formal quality evaluation of 209 Learning Objects created with the tool. The results show that ViSH Editor facilitates educators the creation of high quality Learning Objects.

Diseño del curso para operador "Coeficientes inherentes de reactividad" a plataforma e-Learning

Las centrales nucleares necesitan de personal altamente especializado y formado. Es por ello por lo que el sector de la formación especializada en centrales nucleares necesita incorporar los últimos avances en métodos formativos. Existe una gran cantidad de cursos de formación presenciales y es necesario transformar dichos cursos para utilizarlos con las nuevas tecnologías de la información. Para ello se necesitan equipos multidisciplinares, en los que se incluyen ingenieros, que deben identificar los objetivos formativos, competencias, contenidos y el control de calidad del propio curso. En este proyecto se utilizan técnicas de ingeniería del conocimiento como eje metodológico para transformar un curso de formación presencial en formación on-line a través de tecnologías de la información. En la actualidad, las nuevas tecnologías de la información y comunicación están en constante evolución. De esta forma se han sumergido en el mundo transformando la visión que teníamos de éste para dar lugar a nuevas oportunidades. Es por ello que este proyecto busca la unión entre el e-learning y el mundo empresarial. El objetivo es el diseño, en plataforma e-learning, de un curso técnico que instruya a operadores de sala de control de una central nuclear. El trabajo realizado en este proyecto ha sido, además de transformar un curso presencial en on-line, en obtener una metodología para que otros cursos se puedan transformar. Para conseguir este cometido, debemos preocuparnos tanto por el contenido de los cursos como por su gestión. Por este motivo, el proyecto comienza con definiciones básicas de terminología propia de e-learning. Continúa con la generación de una metodología que aplique la gestión de conocimiento para transformar cualquier curso presencial a esta plataforma. Definida la metodología, se aplicará para el diseño del curso específico de Coeficientes Inherentes de Reactividad. Finaliza con un estudio económico que dé viabilidad al proyecto y con la creación de un modelo económico que estime el precio para cualquier curso futuro. Abstract Nuclear power plants need highly specialized and trained personnel. Thus, nuclear power plant Specialized Training Sector requires the incorporation of the latest advances in training methods. A large array of face-to-face training courses exist and it has become necessary to transform said courses in order to apply them with the new information systems available. For this, multidisciplinary equipment is needed where the engineering workforce must identify educational objectives, competences and abilities, contents and quality control of the different courses. In this project, knowledge engineering techniques are employed as the methodological axis in order to transform a face-to-face training course into on-line training through the use of new information technologies. Nowadays, new information and communication technologies are in constant evolution. They have introduced themselves into our world, transforming our previous vision of them, leading to new opportunities. For this reason, the present Project seeks to unite the use of e-learning and the Business and Corporate world. The main objective is the design, in an e-learning platform, of a technical course that will train nuclear power plant control-room operators. The work carried out in this Project has been, in addition to the transformation of a face-to-face course into an online one, the obtainment of a methodology to employ in the future transformation of other courses. In order to achieve this mission, our interest must focus on the content as well as on the management of the various courses. Hence, the Project starts with basic definitions of e-learning terminology. Next, a methodology that applies knowledge management for the transformation of any face-to-face course into e-learning has been generated. Once this methodology is defined, it has been applied for the design process of the Inherent Coefficients of Reactivity course. Finally, an economic study has been developed in order to determine the viability of the Project and an economic model has been created to estimate the price of any given course