Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

A teaching/learning support tool for introductory programming courses

In This work we present a Web-based tool developed with the aim of reinforcing teaching and learning of introductory programming courses. This tool provides support for teaching and learning. From the teacher's perspective the system introduces important gains with respect to the classical teaching methodology. It reinforces lecture and laboratory sessions, makes it possible to give personalized attention to the student, assesses the degree of participation of the students and most importantly, performs a continuous assessment of the student's progress. From the student's perspective it provides a learning framework, consisting in a help environment and a correction environment, which facilitates their personal work. With this tool students are more motivated to do programming

Aversive Stimuli and Functional Networks of Fear Learning in PTSD : 734

Background: One characteristic of post traumatic stress disorder is an inability to adapt to a safe environment i.e. to change behavior when predictions of adverse outcomes are not met. Recent studies have also indicated that PTSD patients have altered pain processing, with hyperactivation of the putamen and insula to aversive stimuli (Geuze et al, 2007). The present study examined neuronal responses to aversive and predicted aversive events. Methods: Twenty-four trauma exposed non-PTSD controls and nineteen subjects with PTSD underwent fMRI imaging during a partial reinforcement fear conditioning paradigm, with a mild electric shock as the unconditioned stimuli (UCS). Three conditions were analyzed: actual presentations of the UCS, events when a UCS was expected, but omitted (CS+), and events when the UCS was neither expected nor delivered (CS-). Results: The UCS evoked significant alterations in the pain matrix consisting of the brainstem, the midbrain, the thalamus, the insula, the anterior and middle cingulate and the contralateral somatosensory cortex. PTSD subjects displayed bilaterally elevated putamen activity to the electric shock, as compared to controls. In trials when USC was expected, but omitted, significant activations were observed in the brainstem, the midbrain, the anterior insula and the anterior cingulate. PTSD subjects displayed similar activations, but also elevated activations in the amygdala and the posterior insula. Conclusions: These results indicate altered fear and safety learning in PTSD, and neuronal activations are further explored in terms of functional connectivity using psychophysiological interaction analyses.

Spatial learning by rats across visually disconnected environments

Experiments were designed to examine some properties of spatial representations in rats. Adult subjects were trained to escape through a hole at a fixed position in a large circular arena (see Schenk 1989). The experiments were conducted in the dark, with a limited number of controlled visual light cues in order to assess the minimal cue requirement for place learning. Three identical light cues (shape, height and distance from the table) were used. Depending on the condition, they were either permanently on, or alternatively on or off, depending on the position of the rat in the field. Two questions were asked: a) how many identical visual cues were necessary for spatial discrimination in the dark, and b) could rats integrate the relative positions of separate cues, under conditions in which the rat was never allowed to perceive all three cues simultaneously. The results suggest that rats are able to achieve a place discrimination task even if the three cues necessary for efficient orientation can never be seen simultaneously. A dissociation between the discrimination of the spatial position of the goal and the capacity to reach it by a direct path suggests that a reduced number of cues might require prolonged locomotion to allow an accurate orientation in the environment.

Curriculum sequencing for an e-learning system based on learning styles

This work shows the use of adaptation techniques involved in an e-learning system that considers students' learning styles and students' knowledge states. The mentioned e-learning system is built on a multiagent framework designed to examine opportunities to improve the teaching and to motivate the students to learn what they want in a user-friendly and assisted environment

Searching for and positioning of contextualized learning objects

Learning object economies are marketplaces for the sharing and reuse of learning objects (LO). There are many motivations for stimulating the development of the LO economy. The main reason is the possibility of providing the right content, at the right time, to the right learner according to adequate quality standards in the context of a lifelong learning process; in fact, this is also the main objective of education. However, some barriers to the development of a LO economy, such as the granularity and editability of LO, must be overcome. Furthermore, some enablers, such as learning design generation and standards usage, must be promoted in order to enhance LO economy. For this article, we introduced the integration of distributed learning object repositories (DLOR) as sources of LO that could be placed in adaptive learning designs to assist teachers’ design work. Two main issues presented as a result: how to access distributed LO, and where to place the LO in the learning design. To address these issues, we introduced two processes: LORSE, a distributed LO searching process, and LOOK, a micro context-based positioning process, respectively. Using these processes, the teachers were able to reuse LO from different sources to semi-automatically generate an adaptive learning design without leaving their virtual environment. A layered evaluation yielded good results for the process of placing learning objects from controlled learning object repositories into a learning design, and permitting educators to define different open issues that must be covered when they use uncontrolled learning object repositories for this purpose. We verified the satisfaction users had with our solution

Discovering the campus together: a mobile and computer-based learning experience

One of the most relevant difficulties faced by first-year undergraduate students is to settle into the educational environment of universities. This paper presents a case study that proposes a computer-assisted collaborative experience designed to help students in their transition from high school to university. This is done by facilitating their first contact with the campus and its services, the university community, methodologies and activities. The experience combines individual and collaborative activities, conducted in and out of the classroom, structured following the Jigsaw Collaborative Learning Flow Pattern. A specific environment including portable technologies with network and computer applications has been developed to support and facilitate the orchestration of a flow of learning activities into a single integrated learning setting. The result is a Computer-Supported Collaborative Blended Learning scenario, which has been evaluated with first-year university students of the degrees of Software and Audiovisual Engineering within the subject Introduction to Information and Communications Technologies. The findings reveal that the scenario improves significantly students’ interest in their studies and their understanding about the campus and services provided. The environment is also an innovative approach to successfully support the heterogeneous activities conducted by both teachers and students during the scenario. This paper introduces the goals and context of the case study, describes how the technology was employed to conduct the learning scenario, the evaluation methods and the main results of the experience.

Three essays in control, entrepreneurship and innovation

One of the key emphases of these three essays is to provide practical managerial insight. However, good practical insight, can only be created by grounding it firmly on theoretical and empirical research. Practical experience-based understanding without theoretical grounding remains tacit and cannot be easily disseminated. Theoretical understanding without links to real life remains sterile. My studies aim to increase the understanding of how radical innovation could be generated at large established firms and how it can have an impact on business performance as most businesses pursue innovation with one prime objective: value creation. My studies focus on large established firms with sales revenue exceeding USD $ 1 billion. Usually large established firms cannot rely on informal ways of management, as these firms tend to be multinational businesses operating with subsidiaries, offices, or production facilities in more than one country. I. Internal and External Determinants of Corporate Venture Capital Investment The goal of this chapter is to focus on CVC as one of the mechanisms available for established firms to source new ideas that can be exploited. We explore the internal and external determinants under which established firms engage in CVC to source new knowledge through investment in startups. We attempt to make scholars and managers aware of the forces that influence CVC activity by providing findings and insights to facilitate the strategic management of CVC. There are research opportunities to further understand the CVC phenomenon. Why do companies engage in CVC? What motivates them to continue "playing the game" and keep their active CVC investment status. The study examines CVC investment activity, and the importance of understanding the influential factors that make a firm decide to engage in CVC. The main question is: How do established firms' CVC programs adapt to changing internal conditions and external environments. Adaptation typically involves learning from exploratory endeavors, which enable companies to transform the ways they compete (Guth & Ginsberg, 1990). Our study extends the current stream of research on CVC. It aims to contribute to the literature by providing an extensive comparison of internal and external determinants leading to CVC investment activity. To our knowledge, this is the first study to examine the influence of internal and external determinants on CVC activity throughout specific expansion and contraction periods determined by structural breaks occurring between 1985 to 2008. Our econometric analysis indicates a strong and significant positive association between CVC activity and R&D, cash flow availability and environmental financial market conditions, as well as a significant negative association between sales growth and the decision to engage into CVC. The analysis of this study reveals that CVC investment is highly volatile, as demonstrated by dramatic fluctuations in CVC investment activity over the past decades. When analyzing the overall cyclical CVC period from 1985 to 2008 the results of our study suggest that CVC activity has a pattern influenced by financial factors such as the level of R&D, free cash flow, lack of sales growth, and external conditions of the economy, with the NASDAQ price index as the most significant variable influencing CVC during this period. II. Contribution of CVC and its Interaction with R&D to Value Creation The second essay takes into account the demands of corporate executives and shareholders regarding business performance and value creation justifications for investments in innovation. Billions of dollars are invested in CVC and R&D. However there is little evidence that CVC and its interaction with R&D create value. Firms operating in dynamic business sectors seek to innovate to create the value demanded by changing market conditions, consumer preferences, and competitive offerings. Consequently, firms operating in such business sectors put a premium on finding new, sustainable and competitive value propositions. CVC and R&D can help them in this challenge. Dushnitsky and Lenox (2006) presented evidence that CVC investment is associated with value creation. However, studies have shown that the most innovative firms do not necessarily benefit from innovation. For instance Oyon (2007) indicated that between 1995 and 2005 the most innovative automotive companies did not obtain adequate rewards for shareholders. The interaction between CVC and R&D has generated much debate in the CVC literature. Some researchers see them as substitutes suggesting that firms have to choose between CVC and R&D (Hellmann, 2002), while others expect them to be complementary (Chesbrough & Tucci, 2004). This study explores the interaction that CVC and R&D have on value creation. This essay examines the impact of CVC and R&D on value creation over sixteen years across six business sectors and different geographical regions. Our findings suggest that the effect of CVC and its interaction with R&D on value creation is positive and significant. In dynamic business sectors technologies rapidly relinquish obsolete, consequently firms operating in such business sectors need to continuously develop new sources of value creation (Eisenhardt & Martin, 2000; Qualls, Olshavsky, & Michaels, 1981). We conclude that in order to impact value creation, firms operating in business sectors such as Engineering & Business Services, and Information Communication & Technology ought to consider CVC as a vital element of their innovation strategy. Moreover, regarding the CVC and R&D interaction effect, our findings suggest that R&D and CVC are complementary to value creation hence firms in certain business sectors can be better off supporting both R&D and CVC simultaneously to increase the probability of generating value creation. III. MCS and Organizational Structures for Radical Innovation Incremental innovation is necessary for continuous improvement but it does not provide a sustainable permanent source of competitiveness (Cooper, 2003). On the other hand, radical innovation pursuing new technologies and new market frontiers can generate new platforms for growth providing firms with competitive advantages and high economic margin rents (Duchesneau et al., 1979; Markides & Geroski, 2005; O'Connor & DeMartino, 2006; Utterback, 1994). Interestingly, not all companies distinguish between incremental and radical innovation, and more importantly firms that manage innovation through a one-sizefits- all process can almost guarantee a sub-optimization of certain systems and resources (Davila et al., 2006). Moreover, we conducted research on the utilization of MCS along with radical innovation and flexible organizational structures as these have been associated with firm growth (Cooper, 2003; Davila & Foster, 2005, 2007; Markides & Geroski, 2005; O'Connor & DeMartino, 2006). Davila et al. (2009) identified research opportunities for innovation management and provided a list of pending issues: How do companies manage the process of radical and incremental innovation? What are the performance measures companies use to manage radical ideas and how do they select them? The fundamental objective of this paper is to address the following research question: What are the processes, MCS, and organizational structures for generating radical innovation? Moreover, in recent years, research on innovation management has been conducted mainly at either the firm level (Birkinshaw, Hamel, & Mol, 2008a) or at the project level examining appropriate management techniques associated with high levels of uncertainty (Burgelman & Sayles, 1988; Dougherty & Heller, 1994; Jelinek & Schoonhoven, 1993; Kanter, North, Bernstein, & Williamson, 1990; Leifer et al., 2000). Therefore, we embarked on a novel process-related research framework to observe the process stages, MCS, and organizational structures that can generate radical innovation. This article is based on a case study at Alcan Engineered Products, a division of a multinational company provider of lightweight material solutions. Our observations suggest that incremental and radical innovation should be managed through different processes, MCS and organizational structures that ought to be activated and adapted contingent to the type of innovation that is being pursued (i.e. incremental or radical innovation). More importantly, we conclude that radical can be generated in a systematic way through enablers such as processes, MCS, and organizational structures. This is in line with the findings of Jelinek and Schoonhoven (1993) and Davila et al. (2006; 2007) who show that innovative firms have institutionalized mechanisms, arguing that radical innovation cannot occur in an organic environment where flexibility and consensus are the main managerial mechanisms. They rather argue that radical innovation requires a clear organizational structure and formal MCS.

Learning of courtship components in Drosophila mercatorum (Paterson & Wheller) (Diptera, Drosophilidae)

In Drosophila, courtship is an elaborate sequence of behavioural patterns that enables the flies to identify conspecific mates from those of closely related species. This is important because drosophilids usually gather in feeding sites, where males of various species court females vigorously. We investigated the effects of previous experience on D. mercatorum courtship, by testing if virgin males learn to improve their courtship by observing other flies (social learning), or by adjusting their pre-existent behaviour based on previous experiences (facilitation). Behaviours recorded in a controlled environment were courtship latency, courtship (orientation, tapping and wing vibration), mating and other behaviours not related to sexual activities. This study demonstrated that males of D. mercatorum were capable of improving their mating ability based on prior experiences, but they had no social learning on the development of courtship.

The Relationship between Teachers and Students in the Classroom: Communicative Language Teaching Approach and Cooperative Learning Strategy to Improve Learning

The aim of this research is to to investigate how a supportive relationship between teachers and students in the classroom can improve the learning process. By having a good relationship with students, teachers can offer to students chances to be motivated and feel engaged in the learning process. Students will be engaged actively in the learning instead of being passive learners. I wish to investigate how using communicative approach and cooperative learning strategies while teaching do affect and improve students’ learning performance. To achieve these goals qualitative data collection was used as the primary method. The results show that teachers and students value a supportive and caring relationship between them and that interaction is essential to the teacher-student relationship. This sense of caring and supporting from teachers motivates students to become a more interested learner. Students benefit and are motivated when their teachers create a safe and trustful environment. And also the methods and strategies teachers uses, makes students feel engaged and stimulated to participate in the learning process. The students have in their mind that a positive relationship with their teachers positively impacts their interest and motivation in school which contributes to the enhancement of the learning process.

Learning and adaptation in physical heterogeneous teams of robots

In this paper we present a novel approach to assigning roles to robots in a team of physical heterogeneous robots. Its members compete for these roles and get rewards for them. The rewards are used to determine each agent’s preferences and which agents are better adapted to the environment. These aspects are included in the decision making process. Agent interactions are modelled using the concept of an ecosystem in which each robot is a species, resulting in emergent behaviour of the whole set of agents. One of the most important features of this approach is its high adaptability. Unlike some other learning techniques, this approach does not need to start a whole exploitation process when the environment changes. All this is exemplified by means of experiments run on a simulator. In addition, the algorithm developed was applied as applied to several teams of robots in order to analyse the impact of heterogeneity in these systems

Student experiences in a course blended learning

Fascination is a project for design students, developed jointly by the Technical University of Catalonia (UPC) and the University of Technology Thonburi (KMUTT), which works with two groups of students, one group of participants in Spain and another group in Thailand where, hands-on activities, a range of technologies are used to prepare students for the lessons, through learning activities and content. This research paper presents the test of both a general model and a tool for measuring the participants’ experiences in a course that uses a blended learning methodology, with the aim of collecting empirical evidence to justify the effort of applying this methodology, based on the participants’ satisfaction. The procedure used in the conceptualization of the survey, the generation of topics, the collection of data, and the validation of the scale of various items are described here. The information, provided by the 26 people surveyed about the course and the virtual environment that was used, was analyzed to measure their perceptions and explore possible relations. Finally the conclusions of the research and the future work are presented.

Understanding informal payments in health care: motivation of health workers in Tanzania

BACKGROUND: There is growing evidence that informal payments for health care are fairly common in many low- and middle-income countries. Informal payments are reported to have a negative consequence on equity and quality of care; it has been suggested, however, that they may contribute to health worker motivation and retention. Given the significance of motivation and retention issues in human resources for health, a better understanding of the relationships between the two phenomena is needed. This study attempts to assess whether and in what ways informal payments occur in Kibaha, Tanzania. Moreover, it aims to assess how informal earnings might help boost health worker motivation and retention. METHODS: Nine focus groups were conducted in three health facilities of different levels in the health system. In total, 64 health workers participated in the focus group discussions (81% female, 19% male) and where possible, focus groups were divided by cadre. All data were processed and analysed by means of the NVivo software package. RESULTS: The use of informal payments in the study area was confirmed by this study. Furthermore, a negative relationship between informal payments and job satisfaction and better motivation is suggested. Participants mentioned that they felt enslaved by patients as a result of being bribed and this resulted in loss of self-esteem. Furthermore, fear of detection was a main demotivating factor. These factors seem to counterbalance the positive effect of financial incentives. Moreover, informal payments were not found to be related to retention of health workers in the public health system. Other factors such as job security seemed to be more relevant for retention. CONCLUSION: This study suggests that the practice of informal payments contributes to the general demotivation of health workers and negatively affects access to health care services and quality of the health system. Policy action is needed that not only provides better financial incentives for individuals but also tackles an environment in which corruption is endemic.

Dynamic properties of human brain structure: learning-related changes in cortical areas and associated fiber connections.

Recent findings in neuroscience suggest that adult brain structure changes in response to environmental alterations and skill learning. Whereas much is known about structural changes after intensive practice for several months, little is known about the effects of single practice sessions on macroscopic brain structure and about progressive (dynamic) morphological alterations relative to improved task proficiency during learning for several weeks. Using T1-weighted and diffusion tensor imaging in humans, we demonstrate significant gray matter volume increases in frontal and parietal brain areas following only two sessions of practice in a complex whole-body balancing task. Gray matter volume increase in the prefrontal cortex correlated positively with subject's performance improvements during a 6 week learning period. Furthermore, we found that microstructural changes of fractional anisotropy in corresponding white matter regions followed the same temporal dynamic in relation to task performance. The results make clear how marginal alterations in our ever changing environment affect adult brain structure and elucidate the interrelated reorganization in cortical areas and associated fiber connections in correlation with improvements in task performance.