Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

Work less, teach better and get better feedback: why we tried innovative teaching methods

Speaker: Patrick McSweeney Organiser: Time: 15/10/2014 11:00-11:45 Location: B32/3077 Abstract Having started at Southampton in 2005 I have seen quite a few changes to the way courses are taught and studied. I will reflect on some of the interesting changes I have observed and suggest their causes. As a practical example I will talk about codestrom, a peer feedback tool for learning programming. We have found that this teaching method has improved the student experience and reduced the work load for the module team. Together we will discuss how this and other recent developments can enable other teaching innovations which benefit staff as well as students. Hopefully the new class of PhD students will be able to contribute from the point of view of having recently been undergraduate students here and else where.

Intelligent agents to improve adaptivity in a web-based learning environment

En esta tesis se propone el uso de agentes inteligentes en entornos de aprendizaje en línea con el fin de mejorar la asistencia y motivación del estudiante a través de contenidos personalizados que tienen en cuenta el estilo de aprendizaje del estudiante y su nivel de conocimiento. Los agentes propuestos se desempeñan como asistentes personales que ayudan al estudiante a llevar a cabo las actividades de aprendizaje midiendo su progreso y motivación. El entorno de agentes se construye a través de una arquitectura multiagente llamada MASPLANG diseñada para dar soporte adaptativo (presentación y navegación adaptativa) a un sistema hipermedia educativo desarrollado en la Universitat de Girona para impartir educación virtual a través del web. Un aspecto importante de esta propuesta es la habilidad de construir un modelo de estudiante híbrido que comienza con un modelo estereotípico del estudiante basado en estilos de aprendizaje y se modifica gradualmente a medida que el estudiante interactúa con el sistema (gustos subjetivos). Dentro del contexto de esta tesis, el aprendizaje se define como el proceso interno que, bajo factores de cambio resulta en la adquisición de la representación interna de un conocimiento o de una actitud. Este proceso interno no se puede medir directamente sino a través de demostraciones observables externas que constituyen el comportamiento relacionado con el objeto de conocimiento. Finalmente, este cambio es el resultado de la experiencia o entrenamiento y tiene una durabilidad que depende de factores como la motivación y el compromiso. El MASPLANG está compuesto por dos niveles de agentes: los intermediarios llamados IA (agentes de información) que están en el nivel inferior y los de Interfaz llamados PDA (agentes asistentes) que están en el nivel superior. Los agentes asistentes atienden a los estudiantes cuando trabajan con el material didáctico de un curso o una lección de aprendizaje. Esta asistencia consiste en la recolección y análisis de las acciones de los estudiantes para ofrecer contenidos personalizados y en la motivación del estudiante durante el aprendizaje mediante el ofrecimiento de contenidos de retroalimentación, ejercicios adaptados al nivel de conocimiento y mensajes, a través de interfaces de usuario animadas y atractivas. Los agentes de información se encargan del mantenimiento de los modelos pedagógico y del dominio y son los que están en completa interacción con las bases de datos del sistema (compendio de actividades del estudiante y modelo del dominio). El escenario de funcionamiento del MASPLANG está definido por el tipo de usuarios y el tipo de contenidos que ofrece. Como su entorno es un sistema hipermedia educativo, los usuarios se clasifican en profesores quienes definen y preparan los contenidos para el aprendizaje adaptativo, y los estudiantes quienes llevan a cabo las actividades de aprendizaje de forma personalizada. El perfil de aprendizaje inicial del estudiante se captura a través de la evaluación del cuestionario ILS (herramienta de diagnóstico del modelo FSLSM de estilos de aprendizaje adoptado para este estudio) que se asigna al estudiante en su primera interacción con el sistema. Este cuestionario consiste en un conjunto de preguntas de naturaleza sicológica cuyo objetivo es determinar los deseos, hábitos y reacciones del estudiante que orientarán la personalización de los contenidos y del entorno de aprendizaje. El modelo del estudiante se construye entonces teniendo en cuenta este perfil de aprendizaje y el nivel de conocimiento obtenido mediante el análisis de las acciones del estudiante en el entorno.

Space partitioning for scalable K-means

K-Means is a popular clustering algorithm which adopts an iterative refinement procedure to determine data partitions and to compute their associated centres of mass, called centroids. The straightforward implementation of the algorithm is often referred to as `brute force' since it computes a proximity measure from each data point to each centroid at every iteration of the K-Means process. Efficient implementations of the K-Means algorithm have been predominantly based on multi-dimensional binary search trees (KD-Trees). A combination of an efficient data structure and geometrical constraints allow to reduce the number of distance computations required at each iteration. In this work we present a general space partitioning approach for improving the efficiency and the scalability of the K-Means algorithm. We propose to adopt approximate hierarchical clustering methods to generate binary space partitioning trees in contrast to KD-Trees. In the experimental analysis, we have tested the performance of the proposed Binary Space Partitioning K-Means (BSP-KM) when a divisive clustering algorithm is used. We have carried out extensive experimental tests to compare the proposed approach to the one based on KD-Trees (KD-KM) in a wide range of the parameters space. BSP-KM is more scalable than KDKM, while keeping the deterministic nature of the `brute force' algorithm. In particular, the proposed space partitioning approach has shown to overcome the well-known limitation of KD-Trees in high-dimensional spaces and can also be adopted to improve the efficiency of other algorithms in which KD-Trees have been used.

Moving feedback forward: theory to practice

There is substantial research interest in tutor feedback and students’ perception and use of such feedback. This paper considers some of the major issues raised in relation to tutor feedback and student learning. We explore some of the current feedback drivers, most notably the need for feedback to move away from simply a monologue from a tutor to a student to a valuable tutor–student dialogue. In relation to moving feedback forward the notions of self regulation, dialogue and social learning are explored and then considered in relation to how such theory can translate into practice. The paper proposes a framework (GOALS) as a tool through which tutors can move theory into practice with the aim of improving student learning from feedback.

ASSET: moving forward through feedback

Providing high quality and timely feedback to students is often a challenge for many staff in higher education as it can be both time-consuming and frustratingly repetitive. From the student perspective, feedback may sometimes be considered unhelpful, confusing and inconsistent and may not always be provided within a timeframe that is considered to be ‘useful’. The ASSET project, based at the University of Reading, addresses many of these inherent challenges by encouraging the provision of feedback that supports learning, i.e. feedback that contains elements of ‘feed-forward’, is of a high quality and is delivered in a timely manner. In particular, the project exploits the pedagogic benefits of video/audio media within a Web 2.0 context to provide a new, interactive resource, ‘ASSET’, to enhance the feedback experience for both students and staff. A preliminary analysis of both our quantitative and qualitative pedagogic data demonstrate that the ASSET project has instigated change in the ways in which both staff and students think about, deliver, and engage with feedback. For example, data from our online questionnaires and focus groups with staff and students indicate a positive response to the use of video as a medium for delivering feedback to students. In particular, the academic staff engaged in piloting the ASSET resource indicated that i) using video has made them think more, and in some cases differently, about the ways in which they deliver feedback to students and ii) they now see video as an effective means of making feedback more useful and engaging for students. Moreover, the majority of academic staff involved in the project have said they will continue to use video feedback. From the student perspective, 60% of those students whose lecturers used ASSET to provide video feedback said that “receiving video feedback encouraged me to take more notice of the feedback compared with normal methods” and 80% would like their lecturer to continue to use video as a method for providing feedback. An important aim of the project was for it to complement existing University-wide initiatives on feedback and for ASSET to become a ‘model’ resource for staff and students wishing to explore video as a medium for feedback provision. An institutional approach was therefore adopted and key members of Senior Management, academics, T&L support staff, IT support and Student Representatives were embedded within the project from the start. As with all initiatives of this kind, a major issue is the future sustainability of the ASSET resource and to have had both ‘top-down’ and ‘bottom-up’ support for the project has been extremely beneficial. In association with the project team the University is currently exploring the creation of an open-source, two-tiered video supply solution and a ‘framework’ (that other HEIs can adopt and/or adapt) to support staff in using video for feedback provision. In this way students and staff will have new opportunities to explore video and to exploit the benefits of this medium for supporting learning.

PDM 2006: Proceedings of the International Workshop on Data Mining

Recently major processor manufacturers have announced a dramatic shift in their paradigm to increase computing power over the coming years. Instead of focusing on faster clock speeds and more powerful single core CPUs, the trend clearly goes towards multi core systems. This will also result in a paradigm shift for the development of algorithms for computationally expensive tasks, such as data mining applications. Obviously, work on parallel algorithms is not new per se but concentrated efforts in the many application domains are still missing. Multi-core systems, but also clusters of workstations and even large-scale distributed computing infrastructures provide new opportunities and pose new challenges for the design of parallel and distributed algorithms. Since data mining and machine learning systems rely on high performance computing systems, research on the corresponding algorithms must be on the forefront of parallel algorithm research in order to keep pushing data mining and machine learning applications to be more powerful and, especially for the former, interactive. To bring together researchers and practitioners working in this exciting field, a workshop on parallel data mining was organized as part of PKDD/ECML 2006 (Berlin, Germany). The six contributions selected for the program describe various aspects of data mining and machine learning approaches featuring low to high degrees of parallelism: The first contribution focuses the classic problem of distributed association rule mining and focuses on communication efficiency to improve the state of the art. After this a parallelization technique for speeding up decision tree construction by means of thread-level parallelism for shared memory systems is presented. The next paper discusses the design of a parallel approach for dis- tributed memory systems of the frequent subgraphs mining problem. This approach is based on a hierarchical communication topology to solve issues related to multi-domain computational envi- ronments. The forth paper describes the combined use and the customization of software packages to facilitate a top down parallelism in the tuning of Support Vector Machines (SVM) and the next contribution presents an interesting idea concerning parallel training of Conditional Random Fields (CRFs) and motivates their use in labeling sequential data. The last contribution finally focuses on very efficient feature selection. It describes a parallel algorithm for feature selection from random subsets. Selecting the papers included in this volume would not have been possible without the help of an international Program Committee that has provided detailed reviews for each paper. We would like to also thank Matthew Otey who helped with publicity for the workshop.

K-Surfer: a KNIME extension for the management and analysis of Human brain MRI FreeSurfer/FSL data

Human brain imaging techniques, such as Magnetic Resonance Imaging (MRI) or Diffusion Tensor Imaging (DTI), have been established as scientific and diagnostic tools and their adoption is growing in popularity. Statistical methods, machine learning and data mining algorithms have successfully been adopted to extract predictive and descriptive models from neuroimage data. However, the knowledge discovery process typically requires also the adoption of pre-processing, post-processing and visualisation techniques in complex data workflows. Currently, a main problem for the integrated preprocessing and mining of MRI data is the lack of comprehensive platforms able to avoid the manual invocation of preprocessing and mining tools, that yields to an error-prone and inefficient process. In this work we present K-Surfer, a novel plug-in of the Konstanz Information Miner (KNIME) workbench, that automatizes the preprocessing of brain images and leverages the mining capabilities of KNIME in an integrated way. K-Surfer supports the importing, filtering, merging and pre-processing of neuroimage data from FreeSurfer, a tool for human brain MRI feature extraction and interpretation. K-Surfer automatizes the steps for importing FreeSurfer data, reducing time costs, eliminating human errors and enabling the design of complex analytics workflow for neuroimage data by leveraging the rich functionalities available in the KNIME workbench.

Using metrics from complex networks to evaluate machine translation

Establishing metrics to assess machine translation (MT) systems automatically is now crucial owing to the widespread use of MT over the web. In this study we show that such evaluation can be done by modeling text as complex networks. Specifically, we extend our previous work by employing additional metrics of complex networks, whose results were used as input for machine learning methods and allowed MT texts of distinct qualities to be distinguished. Also shown is that the node-to-node mapping between source and target texts (English-Portuguese and Spanish-Portuguese pairs) can be improved by adding further hierarchical levels for the metrics out-degree, in-degree, hierarchical common degree, cluster coefficient, inter-ring degree, intra-ring degree and convergence ratio. The results presented here amount to a proof-of-principle that the possible capturing of a wider context with the hierarchical levels may be combined with machine learning methods to yield an approach for assessing the quality of MT systems. (C) 2010 Elsevier B.V. All rights reserved.

Flipped classroom and learning strategies

This paper seeks to answer the research question "How does the flipped classroom affect students’ learning strategies?" In e-learning research, several studies have focused on how students and teachers perceive the flipped classroom approach. In general, these studies have reported pleasing results. Nonetheless, few, if any, studies have attempted to find out the potential effects of the flipped classroom approach on how students learn. This study was based on two cases: 1) a business modelling course and 2) a research methodology course. In both cases, participating students were from information systems courses at Dalarna University in Sweden. Recorded lectures replaced regular lectures. The recorded lectures were followed by seminars that focused on the learning content of each lecture in various ways. Three weeks after the final seminar, we arranged for two focus group interviews to take place in each course, with 8 to 10 students participating in each group. We asked open questions on how the students thought they had been affected and more dedicated questions that were generated from a literature study on the effects of flipped classroom courses. These questions dealt with issues about mobility, the potential for repeating lectures, formative feedback, the role of seminars, responsibility, empowerment, lectures before seminars, and any problems encountered. Our results show that, in general, students thought differently about learning after the courses in relation to more traditional approaches, especially regarding the need to be more active. Most students enjoyed the mobility aspect and the accessibility of recorded lectures, although a few claimed it demanded a more disciplined attitude. Most students also expressed a feeling of increased activity and responsibility when participating in seminars. Some even felt empowered because they could influence seminar content. The length of and possibility to navigate in recorded lectures was also considered important. The arrangement of the seminar rooms should promote face-to-face discussions. Finally, the types of questions and tasks were found to affect the outcomes of the seminars. The overall conclusion with regard to students’ learning strategies is that to be an active, responsible, empowered, and critical student you have to be an informed student with possibilities and mandate to influence how, where and when to learn and be able to receive continuous feedback during the learning process. Flipped classroom can support such learning.

Low cost non-intrusive home energy monitoring

This thesis presents a low cost non-intrusive home energy monitor built on top of Non-Intrusive Load Monitoring (NILM) concepts and techniques. NILM solutions are already considered low cost alternatives to the big majority of existing commercial energy monitors but the goal here is to make its cost even lower by using a mini netbook as a whole in one solution. The mini netbook is installed in the homes main circuit breaker and computes power consumption by reading current and voltage from the built-in sound card. At the same time, feedback to the users is provided using the 11’’ LCD screen as well as other built-in I/O modules. Our meter is also capable of detecting changes in power and tries to find out which appliance lead to that change and it is being used as part of an eco-feedback platform that was build to study the long terms of energy eco-feedback in individuals. In this thesis the steps that were taken to come up with such a system are presented, from the basics of AC power measurements to the implementation of an event detector and classifier that was used to disaggregate the power load. In the last chapter results from some validation tests that have been performed are presented in order to validate the experiment. It is believed that such a system will not only be important as an energy monitor, but also as an open system than can be easily changed to accommodate and test new or existing nonintrusive load monitoring techniques.

Algoritmos genéticos aplicados a um comitê de LS-SVM em problemas de classificação

The pattern classification is one of the machine learning subareas that has the most outstanding. Among the various approaches to solve pattern classification problems, the Support Vector Machines (SVM) receive great emphasis, due to its ease of use and good generalization performance. The Least Squares formulation of SVM (LS-SVM) finds the solution by solving a set of linear equations instead of quadratic programming implemented in SVM. The LS-SVMs provide some free parameters that have to be correctly chosen to achieve satisfactory results in a given task. Despite the LS-SVMs having high performance, lots of tools have been developed to improve them, mainly the development of new classifying methods and the employment of ensembles, in other words, a combination of several classifiers. In this work, our proposal is to use an ensemble and a Genetic Algorithm (GA), search algorithm based on the evolution of species, to enhance the LSSVM classification. In the construction of this ensemble, we use a random selection of attributes of the original problem, which it splits the original problem into smaller ones where each classifier will act. So, we apply a genetic algorithm to find effective values of the LS-SVM parameters and also to find a weight vector, measuring the importance of each machine in the final classification. Finally, the final classification is obtained by a linear combination of the decision values of the LS-SVMs with the weight vector. We used several classification problems, taken as benchmarks to evaluate the performance of the algorithm and compared the results with other classifiers

Utilizando Pesos estáticos e dinâmicos em sistemas multi-classificadores com diferentes níveis de diversidade

Although some individual techniques of supervised Machine Learning (ML), also known as classifiers, or algorithms of classification, to supply solutions that, most of the time, are considered efficient, have experimental results gotten with the use of large sets of pattern and/or that they have a expressive amount of irrelevant data or incomplete characteristic, that show a decrease in the efficiency of the precision of these techniques. In other words, such techniques can t do an recognition of patterns of an efficient form in complex problems. With the intention to get better performance and efficiency of these ML techniques, were thought about the idea to using some types of LM algorithms work jointly, thus origin to the term Multi-Classifier System (MCS). The MCS s presents, as component, different of LM algorithms, called of base classifiers, and realized a combination of results gotten for these algorithms to reach the final result. So that the MCS has a better performance that the base classifiers, the results gotten for each base classifier must present an certain diversity, in other words, a difference between the results gotten for each classifier that compose the system. It can be said that it does not make signification to have MCS s whose base classifiers have identical answers to the sames patterns. Although the MCS s present better results that the individually systems, has always the search to improve the results gotten for this type of system. Aim at this improvement and a better consistency in the results, as well as a larger diversity of the classifiers of a MCS, comes being recently searched methodologies that present as characteristic the use of weights, or confidence values. These weights can describe the importance that certain classifier supplied when associating with each pattern to a determined class. These weights still are used, in associate with the exits of the classifiers, during the process of recognition (use) of the MCS s. Exist different ways of calculating these weights and can be divided in two categories: the static weights and the dynamic weights. The first category of weights is characterizes for not having the modification of its values during the classification process, different it occurs with the second category, where the values suffers modifications during the classification process. In this work an analysis will be made to verify if the use of the weights, statics as much as dynamics, they can increase the perfomance of the MCS s in comparison with the individually systems. Moreover, will be made an analysis in the diversity gotten for the MCS s, for this mode verify if it has some relation between the use of the weights in the MCS s with different levels of diversity

Particle Competition and Cooperation in Networks for Semi-Supervised Learning

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)