Accelerating autonomous learning by using heuristic selection of actions

This paper investigates how to make improved action selection for online policy learning in robotic scenarios using reinforcement learning (RL) algorithms. Since finding control policies using any RL algorithm can be very time consuming, we propose to combine RL algorithms with heuristic functions for selecting promising actions during the learning process. With this aim, we investigate the use of heuristics for increasing the rate of convergence of RL algorithms and contribute with a new learning algorithm, Heuristically Accelerated Q-learning (HAQL), which incorporates heuristics for action selection to the Q-Learning algorithm. Experimental results on robot navigation show that the use of even very simple heuristic functions results in significant performance enhancement of the learning rate.

Efeitos da exposição a estímulos aversivos e apetitivos incontroláveis sobre o comportamento verbal em contingências de reforço positivo

Foram analisados efeitos de diferentes histórias de incontrolabilidade por perda ou ganho de pontos sobre o desempenho posterior de participantes humanos na construção de frases. Inicialmente, os participantes podiam ganhar ou perder pontos independentemente de qualquer característica da frase construída. Posteriormente, recebiam pontos por construir frases iniciadas apenas pelo pronome "ele". Os resultados mostram que a exposição à incontrolabilidade pode dificultar condições posteriores de novas aprendizagens sob reforçamento positivo. Interessantemente, essas dificuldades foram menos acentuadas e, em certos casos, até mesmo superadas, no caso de uma história de exposição a ganhos incontroláveis de pontos. Em contrapartida, no caso de uma história de perdas incontroláveis de pontos, aprendizagens subsequentes sob reforço positivo tenderam a ser prejudicadas. Esses resultados contribuem para os estudos de incontrolabilidade e desamparo aprendido, em particular por apresentar alternativas metodológicas passíveis de aplicação a respostas verbais em humanos.

Students' collective knowledge construction in the virtual learning environment ""ToLigado - your school interactive newspaper""

Introduction. The ToLigado Project - Your School Interactive Newspaper is an interactive virtual learning environment conceived, developed, implemented and supported by researchers at the School of the Future Research Laboratory of the University of Sao Paulo, Brazil. Method. This virtual learning environment aims to motivate trans-disciplinary research among public school students and teachers in 2,931 schools equipped with Internet-access computer rooms. Within this virtual community, students produce collective multimedia research documents that are immediately published in the portal. The project also aims to increase students' autonomy for research, collaborative work and Web authorship. Main sections of the portal are presented and described. Results. Partial results of the first two years' implementation are presented and indicate a strong motivation among students to produce knowledge despite the fragile hardware and software infrastructure at the time. Discussion. In this new environment, students should be seen as 'knowledge architects' and teachers as facilitators, or 'curiosity managers'. The ToLigado portal may constitute a repository for future studies regarding student attitudes in virtual learning environments, students' behaviour as 'authors', Web authorship involving collective knowledge production, teachers' behaviour as facilitators, and virtual learning environments as digital repositories of students' knowledge construction and social capital in virtual learning communities.

Learning about knowledge management for improving environmental impact assessment in a government agency: The Western Australian experience

How does knowledge management (KM) by a government agency responsible for environmental impact assessment (EIA) potentially contribute to better environmental assessment and management practice? Staff members at government agencies in charge of the EIA process are knowledge workers who perform judgement-oriented tasks highly reliant on individual expertise, but also grounded on the agency`s knowledge accumulated over the years. Part of an agency`s knowledge can be codified and stored in an organizational memory, but is subject to decay or loss if not properly managed. The EIA agency operating in Western Australia was used as a case study. Its KM initiatives were reviewed, knowledge repositories were identified and staff surveyed to gauge the utilisation and effectiveness of such repositories in enabling them to perform EIA tasks. Key elements of KM are the preparation of substantive guidance and spatial information management. It was found that treatment of cumulative impacts on the environment is very limited and information derived from project follow-up is not properly captured and stored, thus not used to create new knowledge and to improve practice and effectiveness. Other opportunities for improving organizational learning include the use of after-action reviews. The learning about knowledge management in EIA practice gained from Western Australian experience should be of value to agencies worldwide seeking to understand where best to direct their resources for their own knowledge repositories and environmental management practice. (C) 2011 Elsevier Ltd. All rights reserved.

A new graphic format to facilitate the understanding of technological innovation models: the seesaw of competitiveness

Many models exist in the literature to explain the success of technological innovation. However, no studies have been made regarding graphic formats representing the technological innovation models and their impact, or on the understanding of these models by non-specialists in technology management. Thus, the main objective of this paper is to propose a new graphic configuration to represent the technological innovation management. Based on the literature, the innovation model is presented in the traditional format. Next, the same model is designed in the graphic format - named `the see-saw of competitiveness` - showing the interfaces among the identified factors. The two graphic formats were compared by a group of graduate students in terms of the ease in understanding the conceptual model of innovation. The statistical analysis shows that the seesaw of competitiveness is preferred.

Improving Alzheimer`s Disease Diagnosis with Machine Learning Techniques

There is not a specific test to diagnose Alzheimer`s disease (AD). Its diagnosis should be based upon clinical history, neuropsychological and laboratory tests, neuroimaging and electroencephalography (EEG). Therefore, new approaches are necessary to enable earlier and more accurate diagnosis and to follow treatment results. In this study we used a Machine Learning (ML) technique, named Support Vector Machine (SVM), to search patterns in EEG epochs to differentiate AD patients from controls. As a result, we developed a quantitative EEG (qEEG) processing method for automatic differentiation of patients with AD from normal individuals, as a complement to the diagnosis of probable dementia. We studied EEGs from 19 normal subjects (14 females/5 males, mean age 71.6 years) and 16 probable mild to moderate symptoms AD patients (14 females/2 males, mean age 73.4 years. The results obtained from analysis of EEG epochs were accuracy 79.9% and sensitivity 83.2%. The analysis considering the diagnosis of each individual patient reached 87.0% accuracy and 91.7% sensitivity.

Evolutionary model trees for handling continuous classes in machine learning

Model trees are a particular case of decision trees employed to solve regression problems. They have the advantage of presenting an interpretable output, helping the end-user to get more confidence in the prediction and providing the basis for the end-user to have new insight about the data, confirming or rejecting hypotheses previously formed. Moreover, model trees present an acceptable level of predictive performance in comparison to most techniques used for solving regression problems. Since generating the optimal model tree is an NP-Complete problem, traditional model tree induction algorithms make use of a greedy top-down divide-and-conquer strategy, which may not converge to the global optimal solution. In this paper, we propose a novel algorithm based on the use of the evolutionary algorithms paradigm as an alternate heuristic to generate model trees in order to improve the convergence to globally near-optimal solutions. We call our new approach evolutionary model tree induction (E-Motion). We test its predictive performance using public UCI data sets, and we compare the results to traditional greedy regression/model trees induction algorithms, as well as to other evolutionary approaches. Results show that our method presents a good trade-off between predictive performance and model comprehensibility, which may be crucial in many machine learning applications. (C) 2010 Elsevier Inc. All rights reserved.