Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Adaptive intelligent agent approach to guide the web navigation on the PLAN-G distance learning platform

Hypermedia systems based on the Web for open distance education are becoming increasinglypopular as tools for user-driven access learning information. Adaptive hypermedia is a new direction in research within the area of user-adaptive systems, to increase its functionality by making it personalized [Eklu 961. This paper sketches a general agents architecture to include navigationaladaptability and user-friendly processes which would guide and accompany the student during hislher learning on the PLAN-G hypermedia system (New Generation Telematics Platform to Support Open and Distance Learning), with the aid of computer networks and specifically WWW technology [Marz 98-1] [Marz 98-2]. The PLAN-G actual prototype is successfully used with some informatics courses (the current version has no agents yet). The propased multi-agent system, contains two different types of adaptive autonomous software agents: Personal Digital Agents {Interface), to interacl directly with the student when necessary; and Information Agents (Intermediaries), to filtrate and discover information to learn and to adapt navigation space to a specific student

Electronic voting system for computer supported collaborative learning

The EVS4CSCL project starts in the context of a Computer Supported Collaborative Learning environment (CSCL). Previous UOC projects created a CSCL generic platform (CLPL) to facilitate the development of CSCL applications. A discussion forum (DF) was the first application developed over the framework. This discussion forum was different from other products on the marketplace because of its focus on the learning process. The DF carried out the specification and elaboration phases from the discussion learning process but there was a lack in the consensus phase. The consensus phase in a learning environment is not something to be achieved but tested. Common tests are done by Electronic Voting System (EVS) tools, but consensus test is not an assessment test. We are not evaluating our students by their answers but by their discussion activity. Our educational EVS would be used as a discussion catalyst proposing a discussion about the results after an initial query or it would be used after a discussion period in order to manifest how the discussion changed the students mind (consensus). It should be also used by the teacher as a quick way to know where the student needs some reinforcement. That is important in a distance-learning environment where there is no direct contact between the teacher and the student and it is difficult to detect the learning lacks. In an educational environment, assessment it is a must and the EVS will provide direct assessment by peer usefulness evaluation, teacher marks on every query created and indirect assessment from statistics regarding the user activity.

An empirical study of machine learning techniques applied to image watermarking

A study of how the machine learning technique, known as gentleboost, could improve different digital watermarking methods such as LSB, DWT, DCT2 and Histogram shifting.

Repeating with the right hemisphere: reduced interactions between phonological and lexical-semantic systems in crossed aphasia?

Knowledge on the patterns of repetition amongst individuals who develop language deficits in association with right hemisphere lesions (crossed aphasia) is very limited. Available data indicate that repetition in some crossed aphasics experiencing phonological processing deficits is not heavily influenced by lexical-semantic variables (lexicality, imageability, and frequency) as is regularly reported in phonologically-impaired cases with left hemisphere damage. Moreover, in view of the fact that crossed aphasia is rare, information on the role of right cortical areas and white matter tracts underpinning language repetition deficits is scarce. In this study, repetition performance was assessed in two patients with crossed conduction aphasia and striatal/capsular vascular lesions encompassing the right arcuate fasciculus (AF) and inferior frontal-occipital fasciculus (IFOF), the temporal stem and the white matter underneath the supramarginal gyrus. Both patients showed lexicality effects repeating better words than non-words, but manipulation of other lexical-semantic variables exerted less influence on repetition performance. Imageability and frequency effects, production of meaning-based paraphrases during sentence repetition, or better performance on repeating novel sentences than overlearned clichés were hardly ever observed in these two patients. In one patient, diffusion tensor imaging disclosed damage to the right long direct segment of the AF and IFOF with relative sparing of the anterior indirect and posterior segments of the AF, together with fully developed left perisylvian white matter pathways. These findings suggest that striatal/capsular lesions extending into the right AF and IFOF in some individuals with right hemisphere language dominance are associated with atypical repetition patterns which might reflect reduced interactions between phonological and lexical-semantic processes.

Embedding Emotional Context in Recommender Systems

Emotions are crucial for user's decision making in recommendation processes. We first introduce ambient recommender systems, which arise from the analysis of new trends on the exploitation of the emotional context in the next generation of recommender systems. We then explain some results of these new trends in real-world applications through the smart prediction assistant (SPA) platform in an intelligent learning guide with more than three million users. While most approaches to recommending have focused on algorithm performance. SPA makes recommendations to users on the basis of emotional information acquired in an incremental way. This article provides a cross-disciplinary perspective to achieve this goal in such recommender systems through a SPA platform. The methodology applied in SPA is the result of a bunch of technology transfer projects for large real-world rccommender systems

Curriculum sequencing for an e-learning system based on learning styles

This work shows the use of adaptation techniques involved in an e-learning system that considers students' learning styles and students' knowledge states. The mentioned e-learning system is built on a multiagent framework designed to examine opportunities to improve the teaching and to motivate the students to learn what they want in a user-friendly and assisted environment

Computational Representation of Collaborative Learning Flow Patterns Using IMS Learning Design

The identification and integration of reusable and customizable CSCL (Computer Supported Collaborative Learning) may benefit from the capture of best practices in collaborative learning structuring. The authors have proposed CLFPs (Collaborative Learning Flow Patterns) as a way of collecting these best practices. To facilitate the process of CLFPs by software systems, the paper proposes to specify these patterns using IMS Learning Design (IMS-LD). Thus, teachers without technical knowledge can particularize and integrate CSCL tools. Nevertheless, the support of IMS-LD for describing collaborative learning activities has some deficiencies: the collaborative tools that can be defined in these activities are limited. Thus, this paper proposes and discusses an extension to IMS-LD that enables to specify several characteristics of the use of tools that mediate collaboration. In order to obtain a Unit of Learning based on a CLFP, a three stage process is also proposed. A CLFP-based Unit of Learning example is used to illustrate the process and the need of the proposed extension.

CLFP intrinsic constraints-based group management of blended learning situations

When applying a Collaborative Learning Flow Pattern (CLFP) to structure sequences of activities in real contexts, one of the tasks is to organize groups of students according to the constraints imposed by the pattern. Sometimes,unexpected events occurring at runtime force this pre-defined distribution to be changed. In such situations, an adjustment of the group structures to be adapted to the new context is needed. If the collaborative pattern is complex, this group redefinitionmight be difficult and time consuming to be carried out in real time. In this context, technology can help on notifying the teacher which incompatibilitiesbetween the actual context and the constraints imposed by the pattern. This chapter presents a flexible solution for supporting teachers in the group organization profiting from the intrinsic constraints defined by a CLFPs codified in IMS Learning Design. A prototype of a web-based tool for the TAPPS and Jigsaw CLFPs and the preliminary results of a controlled user study are alsopresented as a first step towards flexible technological systems to support grouping tasks in this context.

Clinical Learning Environment and Supervision. Development and Validation of the CLES Evaluation Scale

The purpose of the study is: (1) to describe how nursing students' experienced their clinical learning environment and the supervision given by staff nurses working in hospital settings; and (2) to develop and test an evaluation scale of Clinical Learning Environment and Supervision (CLES). The study has been carried out in different phases. The pilot study (n=163) explored the association between the characteristics of a ward and its evaluation as a learning environment by students. The second version of research instrument (which was developed by the results of this pilot study) were tested by an expert panel (n=9 nurse teachers) and test-retest group formed by student nurses (n=38). After this evaluative phase, the CLES was formed as the basic research instrument for this study and it was tested with the Finnish main sample (n=416). In this phase, a concurrent validity instrument (Dunn & Burnett 1995) was used to confirm the validation process of CLES. The international comparative study was made by comparing the Finnish main sample with a British sample (n=142). The international comparative study was necessary for two reasons. In the instrument developing process, there is a need to test the new instrument in some other nursing culture. Other reason for comparative international study is the reflecting the impact of open employment markets in the European Union (EU) on the need to evaluate and to integrate EU health care educational systems. The results showed that the individualised supervision system is the most used supervision model and the supervisory relationship with personal mentor is the most meaningful single element of supervision evaluated by nursing students. The ward atmosphere and the management style of ward manager are the most important environmental factors of the clinical ward. The study integrates two theoretical elements - learning environment and supervision - in developing a preliminary theoretical model. The comparative international study showed that, Finnish students were more satisfied and evaluated their clinical placements and supervision with higher scores than students in the United Kingdom (UK). The difference between groups was statistical highly significant (p= 0.000). In the UK, clinical placements were longer but students met their nurse teachers less frequently than students in Finland. Arrangements for supervision were similar. This research process has produced the evaluation scale (CLES), which can be used in research and quality assessments of clinical learning environment and supervision in Finland and in the UK. CLES consists of 27 items and it is sub-divided into five sub-dimensions. Cronbach's alpha coefficient varied from high 0.94 to marginal 0.73. CLES is a compact evaluation scale and user-friendliness makes it suitable for continuing evaluation.

An Automatic Approach for Learning and Tuning Gaussian Interval Type-2 Fuzzy Membership Functions Applied to Lung CAD Classification System

The potential of type-2 fuzzy sets for managing high levels of uncertainty in the subjective knowledge of experts or of numerical information has focused on control and pattern classification systems in recent years. One of the main challenges in designing a type-2 fuzzy logic system is how to estimate the parameters of type-2 fuzzy membership function (T2MF) and the Footprint of Uncertainty (FOU) from imperfect and noisy datasets. This paper presents an automatic approach for learning and tuning Gaussian interval type-2 membership functions (IT2MFs) with application to multi-dimensional pattern classification problems. T2MFs and their FOUs are tuned according to the uncertainties in the training dataset by a combination of genetic algorithm (GA) and crossvalidation techniques. In our GA-based approach, the structure of the chromosome has fewer genes than other GA methods and chromosome initialization is more precise. The proposed approach addresses the application of the interval type-2 fuzzy logic system (IT2FLS) for the problem of nodule classification in a lung Computer Aided Detection (CAD) system. The designed IT2FLS is compared with its type-1 fuzzy logic system (T1FLS) counterpart. The results demonstrate that the IT2FLS outperforms the T1FLS by more than 30% in terms of classification accuracy.

Learning and adaptation in physical heterogeneous teams of robots

In this paper we present a novel approach to assigning roles to robots in a team of physical heterogeneous robots. Its members compete for these roles and get rewards for them. The rewards are used to determine each agent’s preferences and which agents are better adapted to the environment. These aspects are included in the decision making process. Agent interactions are modelled using the concept of an ecosystem in which each robot is a species, resulting in emergent behaviour of the whole set of agents. One of the most important features of this approach is its high adaptability. Unlike some other learning techniques, this approach does not need to start a whole exploitation process when the environment changes. All this is exemplified by means of experiments run on a simulator. In addition, the algorithm developed was applied as applied to several teams of robots in order to analyse the impact of heterogeneity in these systems

Active learning methods for remote sensing image classification

In this paper, we propose two active learning algorithms for semiautomatic definition of training samples in remote sensing image classification. Based on predefined heuristics, the classifier ranks the unlabeled pixels and automatically chooses those that are considered the most valuable for its improvement. Once the pixels have been selected, the analyst labels them manually and the process is iterated. Starting with a small and nonoptimal training set, the model itself builds the optimal set of samples which minimizes the classification error. We have applied the proposed algorithms to a variety of remote sensing data, including very high resolution and hyperspectral images, using support vector machines. Experimental results confirm the consistency of the methods. The required number of training samples can be reduced to 10% using the methods proposed, reaching the same level of accuracy as larger data sets. A comparison with a state-of-the-art active learning method, margin sampling, is provided, highlighting advantages of the methods proposed. The effect of spatial resolution and separability of the classes on the quality of the selection of pixels is also discussed.

Analysis, modelling and classification of geospatial data using machine learning

The research considers the problem of spatial data classification using machine learning algorithms: probabilistic neural networks (PNN) and support vector machines (SVM). As a benchmark model simple k-nearest neighbor algorithm is considered. PNN is a neural network reformulation of well known nonparametric principles of probability density modeling using kernel density estimator and Bayesian optimal or maximum a posteriori decision rules. PNN is well suited to problems where not only predictions but also quantification of accuracy and integration of prior information are necessary. An important property of PNN is that they can be easily used in decision support systems dealing with problems of automatic classification. Support vector machine is an implementation of the principles of statistical learning theory for the classification tasks. Recently they were successfully applied for different environmental topics: classification of soil types and hydro-geological units, optimization of monitoring networks, susceptibility mapping of natural hazards. In the present paper both simulated and real data case studies (low and high dimensional) are considered. The main attention is paid to the detection and learning of spatial patterns by the algorithms applied.