Accelerating autonomous learning by using heuristic selection of actions

This paper investigates how to make improved action selection for online policy learning in robotic scenarios using reinforcement learning (RL) algorithms. Since finding control policies using any RL algorithm can be very time consuming, we propose to combine RL algorithms with heuristic functions for selecting promising actions during the learning process. With this aim, we investigate the use of heuristics for increasing the rate of convergence of RL algorithms and contribute with a new learning algorithm, Heuristically Accelerated Q-learning (HAQL), which incorporates heuristics for action selection to the Q-Learning algorithm. Experimental results on robot navigation show that the use of even very simple heuristic functions results in significant performance enhancement of the learning rate.

Cross-situational and supervised learning in the emergence of communication

Scenarios for the emergence or bootstrap of a lexicon involve the repeated interaction between at least two agents who must reach a consensus on how to name N objects using H words. Here we consider minimal models of two types of learning algorithms: cross-situational learning, in which the individuals determine the meaning of a word by looking for something in common across all observed uses of that word, and supervised operant conditioning learning, in which there is strong feedback between individuals about the intended meaning of the words. Despite the stark differences between these learning schemes, we show that they yield the same communication accuracy in the limits of large N and H, which coincides with the result of the classical occupancy problem of randomly assigning N objects to H words.

Learning processes and the neural analysis of conditioning

Classical and operant conditioning principles, such as the behavioral discrepancy-derived assumption that reinforcement always selects antecedent stimulus and response relations, have been studied at the neural level, mainly by observing the strengthening of neuronal responses or synaptic connections. A review of the literature on the neural basis of behavior provided extensive scientific data that indicate a synthesis between the two conditioning processes based mainly on stimulus control in learning tasks. The resulting analysis revealed the following aspects. Dopamine acts as a behavioral discrepancy signal in the midbrain pathway of positive reinforcement, leading toward the nucleus accumbens. Dopamine modulates both types of conditioning in the Aplysia mollusk and in mammals. In vivo and in vitro mollusk preparations show convergence of both types of conditioning in the same motor neuron. Frontal cortical neurons are involved in behavioral discrimination in reversal and extinction procedures, and these neurons preferentially deliver glutamate through conditioned stimulus or discriminative stimulus pathways. Discriminative neural responses can reliably precede operant movements and can also be common to stimuli that share complex symbolic relations. The present article discusses convergent and divergent points between conditioning paradigms at the neural level of analysis to advance our knowledge on reinforcement.

Influence of food components on lipid metabolism: scenarios and perspective on the control and prevention of dyslipidemias

Cardiovascular diseases (CVD) are the main causes of death in the Western world. Among the risk factors that are modifiable by diet, for reducing cardiovascular disease risks, the total plasma concentrations of cholesterol, triglycerides, LDL-C, and HDL-C are the most important. Dietary measures can balance these components of the lipid profile thus reducing the risk of cardiovascular diseases. The main food components that affect the lipid profile and can be modified by diet are the saturated and trans fats, unsaturated fats, cholesterol, phytosterols, plant protein, and soluble fiber. A wealth of evidence suggests that saturated and trans fats and cholesterol in the diet raise the total plasma cholesterol and LDL-C. Trans fats also reduce HDL-C, an important lipoprotein for mediating the reverse cholesterol transport. On the other hand, phytosterols, plant proteins, isoflavones, and soluble fiber are protective diet factors against cardiovascular diseases by modulating plasma lipoprotein levels. These food components at certain concentrations are able to reduce the total cholesterol, TG, and LDL-C and raise the plasma levels of HDL-C. Therefore, diet is an important tool for the prevention and control of cardiovascular diseases, and should be taken into account as a whole, i.e., not only the food components that modulate plasma concentrations of lipoproteins, but also the diet content of macro nutrients and micronutrients should be considered.

The Process of Simulation Authoring/Creation by Chemistry Teachers: A Service Learning Experience

Two case studies are presented to describe the process of public school teachers authoring and creating chemistry simulations. They are part of the Virtual Didactic Laboratory for Chemistry, a project developed by the School of the Future of the University of Sao Paulo. the documental analysis of the material produced by two groups of teachers reflects different selection process for both themes and problem-situations when creating simulations. The study demonstrates the potential for chemistry learning with an approach that takes students' everyday lives into account and is based on collaborative work among teachers and researches. Also, from the teachers' perspectives, the possibilities of interaction that a simulation offers for classroom activities are considered.

Students' collective knowledge construction in the virtual learning environment ""ToLigado - your school interactive newspaper""

Introduction. The ToLigado Project - Your School Interactive Newspaper is an interactive virtual learning environment conceived, developed, implemented and supported by researchers at the School of the Future Research Laboratory of the University of Sao Paulo, Brazil. Method. This virtual learning environment aims to motivate trans-disciplinary research among public school students and teachers in 2,931 schools equipped with Internet-access computer rooms. Within this virtual community, students produce collective multimedia research documents that are immediately published in the portal. The project also aims to increase students' autonomy for research, collaborative work and Web authorship. Main sections of the portal are presented and described. Results. Partial results of the first two years' implementation are presented and indicate a strong motivation among students to produce knowledge despite the fragile hardware and software infrastructure at the time. Discussion. In this new environment, students should be seen as 'knowledge architects' and teachers as facilitators, or 'curiosity managers'. The ToLigado portal may constitute a repository for future studies regarding student attitudes in virtual learning environments, students' behaviour as 'authors', Web authorship involving collective knowledge production, teachers' behaviour as facilitators, and virtual learning environments as digital repositories of students' knowledge construction and social capital in virtual learning communities.

Cooperation, interaction and learning in local production systems: Evidence in Brazilian firms

In a local production system (LPS), besides external economies, the interaction, cooperation, and learning are indicated by the literature as complementary ways of enhancing the LPS's competitiveness and gains. In Brazil, the greater part of LPSs, mostly composed by small enterprises, displays incipient relationships and low levels of interaction and cooperation among their actors. The size of the participating enterprises itself for specificities that engender organizational constraints, which, in turn, can have a considerable impact on their relationships and learning dynamics. For that reason, it is the purpose of this article to present an analysis of interaction, cooperation, and learning relationships among several types of actors pertaining to an LPS in the farming equipment and machinery sector, bearing in mind the specificities of small enterprises. To this end, the fieldwork carried out in this study aimed at: (i) investigating external and internal knowledge sources conducive to learning and (ii) identifying and analyzing motivating and inhibiting factors related to specificities of small enterprises in order to bring the LPS members closer together and increase their cooperation and interaction. Empirical evidence shows that internal aspects of the enterprises, related to management and infrastructure, can have a strong bearing on their joint actions, interaction and learning processes.

The effectiveness of educational practice in diabetic foot: a view from Brazil

Background: The aim of the present study was to evaluate the prevention and self-inspection behavior of diabetic subjects with foot at ulcer risk, no previous episode, who participated in the routine visits and standardized education provided by the service and who received prescribed footwear. This evaluation was carried out using a questionnaire scoring from 0-10 (high scores reflect worse practice compliance). Results: 60 patients were studied (30 of each sex); mean age was 62 years, mean duration of the disease was 17 years. As for compliance, 90% showed a total score <= 5, only 8.7% regularly wore the footwear supplied; self foot inspection 65%, 28,3% with additional familiar inspection; creaming 77%; proper washing and drying 88%; proper cutting of toe nails 83%; no cuticle cutting 83%; routine shoe inspection 77%; no use of pumice stones or similar abrasive 70%; no barefoot walking 95%. Conclusion: the planned and multidisciplinary educational approach enabled high compliance of the ulcer prevention care needed in diabetic patients at risk for complications. In contrast, compliance observed for the use of footwear provided was extremely low, demonstrating that the issue of its acceptability should be further and carefully addressed. In countries of such vast dimensions as Brazil multidisciplinary educational approaches can and should be performed by the services providing care for patients with foot at risk for complications according to the reality of local scenarios. Furthermore, every educational program should assess the learning, results obtained and efficacy in the target population by use of an adequate evaluation system.

Evaluation of gastrointestinal motility in awake rats: a learning exercise for undergraduate biomedical students

Souza MA, Souza MH, Palheta RC Jr, Cruz PR, Medeiros BA, Rola FH, Magalhaes PJ, Troncon LE, Santos AA. Evaluation of gastrointestinal motility in awake rats: a learning exercise for undergraduate biomedical students. Adv Physiol Educ 33: 343-348, 2009; doi: 10.1152/advan.90176.2008.-Current medical curricula devote scarce time for practical activities on digestive physiology, despite frequent misconceptions about dyspepsia and dysmotility phenomena. Thus, we designed a hands-on activity followed by a small-group discussion on gut motility. Male awake rats were randomly submitted to insulin, control, or hypertonic protocols. Insulin and control rats were gavage fed with 5% glucose solution, whereas hypertonic-fed rats were gavage fed with 50% glucose solution. Insulin treatment was performed 30 min before a meal. All meals (1.5 ml) contained an equal mass of phenol red dye. After 10, 15, or 20 min of meal gavage, rats were euthanized. Each subset consisted of six to eight rats. Dye recovery in the stomach and proximal, middle, and distal small intestine was measured by spectrophotometry, a safe and reliable method that can be performed by minimally trained students. In a separate group of rats, we used the same protocols except that the test meal contained (99m)Tc as a marker. Compared with control, the hypertonic meal delayed gastric emptying and gastrointestinal transit, whereas insulinic hypoglycemia accelerated them. The session helped engage our undergraduate students in observing and analyzing gut motor behavior. In conclusion, the fractional dye retention test can be used as a teaching tool to strengthen the understanding of basic physiopathological features of gastrointestinal motility.

Assessment of Preclinical Learning on Oral Surgery Using Three Instructional Strategies

The purpose of this investigation was to evaluate three learning methods for teaching basic oral surgical skills Thirty predoctoral dental students without any surgical knowledge or previous surgical experience were divided Into three groups (n=10 each) according to instructional strategy Group 1, active learning Group 2, text reading only, and Group 3, text reading and video demonstration After instruction, the apprentices were allowed to practice incision dissection and suture maneuvers in a bench learning model During the students' performance, a structured practice evaluation test to account for correct or incorrect maneuvers was applied by trained observers Evaluation tests were repeated after thirty and sixty days Data from resulting scores between groups and periods were considered for statistical analysis (ANOVA and Tukey Kramer) with a significant level of a=0 05 Results showed that the active learning group presented the significantly best learning outcomes related to immediate assimilation of surgical procedures compared to other groups All groups results were similar after sixty days of the first practice Assessment tests were fundamental to evaluate teaching strategies and allowed theoretical and proficiency learning feedbacks Repetition and interactive practice promoted retention of knowledge on basic oral surgical skills

E-Learning Used in a Training Course on Atraumatic Restorative Treatment (ART) for Brazilian Dentists

The purpose of this study was to assess the benefits of using e-learning resources in a dental training course on Atraumatic Restorative Treatment (ART). This e-course was given in a DVD format, which presented the ART technique and philosophy. The participants were twenty-four dentists from the Brazilian public health system. Prior to receiving the DVD, the dentists answered a questionnaire regarding their personal data, previous knowledge about ART, and general interest in training courses. The dentists also participated in an assessment process consisting of a test applied before and after the course. A single researcher corrected the tests, and intraexaminer reproducibility was calculated (kappa=0.89). Paired t-tests were carried out to compare the means between the assessments, showing a significant improvement in the performance of the subjects on the test taken after the course (p<0.05). A linear regression model was used with the difference between the means as the outcome. A greater improvement on the test results was observed among female dentists (p=0.034), dentists working for a shorter period of time in the public health system (p=0.042), and dentists who used the ART technique only for urgent and/or temporary treatment (p=0.010). In conclusion, e-learning has the potential of improving the knowledge that dentists working in the public health system have about ART, especially those with less clinical experience and less knowledge about the subject.

A Paradigm Shift in Predoctoral Dental Curricula in Brazil: Evaluating the Process of Change

In 2002, the Brazilian Ministry of Education approved the official curricular guidelines for undergraduate courses in Brazil to be adopted by the nation's 188 dental schools. In 2005-06, the Brazilian Dental Education Association (BDEA) promoted workshops in forty-eight of the schools to verify the degree of transformation of the curriculum based on these guidelines. Among the areas analyzed were course philosophy (variables were v1: knowledge production based on the needs of the Brazilian Public Health System [BPHS]; v2: health determinants; and v3: postgraduate studies and permanent education); pedagogical skills (v4: curricular structure; v5: changes in pedagogic and didactic skills; and v6: course program orientation); and dental practice scenarios (v7: diversity of the scenarios for training/learning; v8: academic health care centers opened to the BPHS; and v9: participation of students in health care delivery for the population). The subjects consisted of faculty members (n=711), students (n=228), and employees (n=14). The results showed an incipient degree of curriculum transformation. The degree of innovation was statistically different depending on the type of university (public or private) for variables I, 2, 4, 5, 6, and 7. Private schools reported a higher level of innovation than public institutions. Resistance to transforming the dental curriculum according to the official guidelines may be linked to an ideological conception that supports the private practice model, continues to have faculty members direct all classroom activities, and prevents students from developing an understanding of professional practice as targeted towards the oral health needs of all segments of society.

Testing Evolutionary and Dispersion Scenarios for the Settlement of the New World

Background: Discussion surrounding the settlement of the New World has recently gained momentum with advances in molecular biology, archaeology and bioanthropology. Recent evidence from these diverse fields is found to support different colonization scenarios. The currently available genetic evidence suggests a ""single migration'' model, in which both early and later Native American groups derive from one expansion event into the continent. In contrast, the pronounced anatomical differences between early and late Native American populations have led others to propose more complex scenarios, involving separate colonization events of the New World and a distinct origin for these groups. Methodology/Principal Findings: Using large samples of Early American crania, we: 1) calculated the rate of morphological differentiation between Early and Late American samples under three different time divergence assumptions, and compared our findings to the predicted morphological differentiation under neutral conditions in each case; and 2) further tested three dispersal scenarios for the colonization of the New World by comparing the morphological distances among early and late Amerindians, East Asians, Australo-Melanesians and early modern humans from Asia to geographical distances associated with each dispersion model. Results indicate that the assumption of a last shared common ancestor outside the continent better explains the observed morphological differences between early and late American groups. This result is corroborated by our finding that a model comprising two Asian waves of migration coming through Bering into the Americas fits the cranial anatomical evidence best, especially when the effects of diversifying selection to climate are taken into account. Conclusions: We conclude that the morphological diversity documented through time in the New World is best accounted for by a model postulating two waves of human expansion into the continent originating in East Asia and entering through Beringia.

Bayesian online algorithms for learning in discrete Hidden Markov Models

We propose and analyze two different Bayesian online algorithms for learning in discrete Hidden Markov Models and compare their performance with the already known Baldi-Chauvin Algorithm. Using the Kullback-Leibler divergence as a measure of generalization we draw learning curves in simplified situations for these algorithms and compare their performances.

Physical approximations for the nonlinear evolution of perturbations in inhomogeneous dark energy scenarios

The abundance and distribution of collapsed objects such as galaxy clusters will become an important tool to investigate the nature of dark energy and dark matter. Number counts of very massive objects are sensitive not only to the equation of state of dark energy, which parametrizes the smooth component of its pressure, but also to the sound speed of dark energy, which determines the amount of pressure in inhomogeneous and collapsed structures. Since the evolution of these structures must be followed well into the nonlinear regime, and a fully relativistic framework for this regime does not exist yet, we compare two approximate schemes: the widely used spherical collapse model and the pseudo-Newtonian approach. We show that both approximation schemes convey identical equations for the density contrast, when the pressure perturbation of dark energy is parametrized in terms of an effective sound speed. We also make a comparison of these approximate approaches to general relativity in the linearized regime, which lends some support to the approximations.