Experience generalization for multi-agent reinforcement learning

On-line learning methods have been applied successfully in multi-agent systems to achieve coordination among agents. Learning in multi-agent systems implies in a non-stationary scenario perceived by the agents, since the behavior of other agents may change as they simultaneously learn how to improve their actions. Non-stationary scenarios can be modeled as Markov Games, which can be solved using the Minimax-Q algorithm a combination of Q-learning (a Reinforcement Learning (RL) algorithm which directly learns an optimal control policy) and the Minimax algorithm. However, finding optimal control policies using any RL algorithm (Q-learning and Minimax-Q included) can be very time consuming. Trying to improve the learning time of Q-learning, we considered the QS-algorithm. in which a single experience can update more than a single action value by using a spreading function. In this paper, we contribute a Minimax-QS algorithm which combines the Minimax-Q algorithm and the QS-algorithm. We conduct a series of empirical evaluation of the algorithm in a simplified simulator of the soccer domain. We show that even using a very simple domain-dependent spreading function, the performance of the learning algorithm can be improved.

Artificial immune systems for classification of petroleum well drilling operations

This paper presents two approaches of Artificial Immune System for Pattern Recognition (CLONALG and Parallel AIRS2) to classify automatically the well drilling operation stages. The classification is carried out through the analysis of some mud-logging parameters. In order to validate the performance of AIS techniques, the results were compared with others classification methods: neural network, support vector machine and lazy learning.

A multimedia environment for supporting the teaching of robotics systems

This paper aims at describing an educational system for teaching and learning robotic systems. Multimedia resources were used to construct a virtual laboratory where users are able to use functionalities of a virtual robotic arm, by moving and clicking the mouse without caring about the detailed internal robot operation. Moreover through the multimedia system the user can interact with a real robot arm. The engineering students are the target public of the developed system. With its contents and interactive capabilities, it has been used as a support to the traditional face-to-face classes on the subject of robotics.. In the paper it is first introduced the metaphor of Virtual Laboratory used in the system. Next, it is described the Graphical and Multimedia Environment approach: an interactive graphic user interface with a 3D environment for simulation. Design and implementation issues of the real-time interactive multimedia learning system, which supports the W3C SMIL standard for presenting the real-time multimedia teaching material, are described. Finally, some preliminary conclusions and possible future works from this research are presented.

Power-law distribution in a learning process: competition, learning and natural selection

In the present work, we propose a model for the statistical distribution of people versus number of steps acquired by them in a learning process, based on competition, learning and natural selection. We consider that learning ability is normally distributed. We found that the number of people versus step acquired by them in a learning process is given through a power law. As competition, learning and selection is also at the core of all economical and social systems, we consider that power-law scaling is a quantitative description of this process in social systems. This gives an alternative thinking in holistic properties of complex systems. (C) 2004 Elsevier B.V. All rights reserved.

An easy-learning and easy-teaching tool for indoor thermal analysis - ArcTech

ArcTech is a software being developed, applied and improved with the aim of becoming an efficient sensitization tool to support the teaching-learning process of Architecture courses. The application deals initially with the thermal comfort of buildings. The output generated by the software shows if a student is able to produce a pleasant environment, in terms of thermal sensation along a 24-hours period. Although one can find the very same characteristics in fully-developed commercial software, the reason to create ArcTech is related to the flexibility of the system to be adapted by the instructor and also to the need of simple tools for the evaluation of specific topics along the courses. The first part of ArcTech is dedicated to data management and that was developed using the visual programming language Delphi 7 and Firebird as the database management system. The second part contains the parameters that can be changed by the system administrator and those related to project visualization. The interface of the system, in which the student will learn how to implement and to evaluate the project alternatives, was built using Macromedia Flash. The software was applied to undergraduate students revealing its easy-learning and easy-teaching interface.

Abductive reasoning, information, and mechanical systems

We investigate, from a philosophical perspective, the relation between abductive reasoning and information in the context of biological systems. Emphasis is given to the organizational role played by abductive reasoning in practical activities of embodied embedded agency that involve meaningful information. From this perspective, meaningful information is provisionally characterized as a selforganizing process of pattern generation that constrains coherent action. We argue that this process can be considered as a part of evolutionarily developed learning abilities of organisms in order to help with their survival. We investigate the case of inorganic mechanical systems (like robots), which deal only with stable forms of habits, rather than with evolving learning abilities. Some difficulties are considered concerning the hypothesis that mechanical systems may operate with meaningful information, present in abductive reasoning. Finally, an example of hypotheses creation in the domain of medical sciences is presented in order to illustrate the complexity of abduction in practical reasoning concerning human activities. © 2007 Springer-Verlag Berlin Heidelberg.

A video-based biometric authentication for e-learning web applications

In the last years there was an exponential growth in the offering of Web-enabled distance courses and in the number of enrolments in corporate and higher education using this modality. However, the lack of efficient mechanisms that assures user authentication in this sort of environment, in the system login as well as throughout his session, has been pointed out as a serious deficiency. Some studies have been led about possible biometric applications for web authentication. However, password based authentication still prevails. With the popularization of biometric enabled devices and resultant fall of prices for the collection of biometric traits, biometrics is reconsidered as a secure remote authentication form for web applications. In this work, the face recognition accuracy, captured on-line by a webcam in Internet environment, is investigated, simulating the natural interaction of a person in the context of a distance course environment. Partial results show that this technique can be successfully applied to confirm the presence of users throughout the course attendance in an educational distance course. An efficient client/server architecture is also proposed. © 2009 Springer Berlin Heidelberg.

Teaching and learning data structures supported by computers: An experiment using CADILAG tool

This paper aims to present the use of a learning object (CADILAG), developed to facilitate understanding data structure operations by using visual presentations and animations. The CADILAG allows visualizing the behavior of algorithms usually discussed during Computer Science and Information System courses. For each data structure it is possible visualizing its content and its operation dynamically. Its use was evaluated an the results are presented. © 2012 AISTI.

Information Visualization to support decision making in the context of distance learning: A study with VLE TelEduc

Digital data sets constitute rich sources of information, which can be extracted and evaluated applying computational tools, for example, those ones for Information Visualization. Web-based applications, such as social network environments, forums and virtual environments for Distance Learning, are good examples for such sources. The great amount of data has direct impact on processing and analysis tasks. This paper presents the computational tool Mapper, defined and implemented to use visual representations - maps, graphics and diagrams - for supporting the decision making process by analyzing data stored in Virtual Learning Environment TelEduc-Unesp. © 2012 IEEE.

An efficient implementation of tabu search in feeder reconfiguration of distribution systems

This paper presents an efficient tabu search algorithm (TSA) to solve the problem of feeder reconfiguration of distribution systems. The main characteristics that make the proposed TSA particularly efficient are a) the way in which the neighborhood of the current solution was defined; b) the way in which the objective function value was estimated; and c) the reduction of the neighborhood using heuristic criteria. Four electrical systems, described in detail in the specialized literature, were used to test the proposed TSA. The result demonstrate that it is computationally very fast and finds the best solutions known in the specialized literature. © 2012 IEEE.

An approach to implement data fusion techniques in wireless sensor networks using genetic machine learning algorithms

Wireless Sensor Networks (WSNs) can be used to monitor hazardous and inaccessible areas. In these situations, the power supply (e.g. battery) of each node cannot be easily replaced. One solution to deal with the limited capacity of current power supplies is to deploy a large number of sensor nodes, since the lifetime and dependability of the network will increase through cooperation among nodes. Applications on WSN may also have other concerns, such as meeting temporal deadlines on message transmissions and maximizing the quality of information. Data fusion is a well-known technique that can be useful for the enhancement of data quality and for the maximization of WSN lifetime. In this paper, we propose an approach that allows the implementation of parallel data fusion techniques in IEEE 802.15.4 networks. One of the main advantages of the proposed approach is that it enables a trade-off between different user-defined metrics through the use of a genetic machine learning algorithm. Simulations and field experiments performed in different communication scenarios highlight significant improvements when compared with, for instance, the Gur Game approach or the implementation of conventional periodic communication techniques over IEEE 802.15.4 networks. © 2013 Elsevier B.V. All rights reserved.

Applying machine learning based on multiscale classifiers to detect remote phenology patterns in Cerrado savanna trees

Plant phenology is one of the most reliable indicators of species responses to global climate change, motivating the development of new technologies for phenological monitoring. Digital cameras or near remote systems have been efficiently applied as multi-channel imaging sensors, where leaf color information is extracted from the RGB (Red, Green, and Blue) color channels, and the changes in green levels are used to infer leafing patterns of plant species. In this scenario, texture information is a great ally for image analysis that has been little used in phenology studies. We monitored leaf-changing patterns of Cerrado savanna vegetation by taking daily digital images. We extract RGB channels from the digital images and correlate them with phenological changes. Additionally, we benefit from the inclusion of textural metrics for quantifying spatial heterogeneity. Our first goals are: (1) to test if color change information is able to characterize the phenological pattern of a group of species; (2) to test if the temporal variation in image texture is useful to distinguish plant species; and (3) to test if individuals from the same species may be automatically identified using digital images. In this paper, we present a machine learning approach based on multiscale classifiers to detect phenological patterns in the digital images. Our results indicate that: (1) extreme hours (morning and afternoon) are the best for identifying plant species; (2) different plant species present a different behavior with respect to the color change information; and (3) texture variation along temporal images is promising information for capturing phenological patterns. Based on those results, we suggest that individuals from the same species and functional group might be identified using digital images, and introduce a new tool to help phenology experts in the identification of new individuals from the same species in the image and their location on the ground. © 2013 Elsevier B.V. All rights reserved.

The interlanguage development in the spanish learning as foreign language in Teletandem under the View of the Complexity Theory

This article presents, under the perspective of Complexity Theory, the characteristics of the learning process of Spanish as a foreign language in Teletandem. Data were collected from two pairs of Portuguese-Spanish interagents, who were engaged in a systematic and regular interaction, based on the tandem principles. It was found that the learning experience is developed with the peculiarities that arise from the context, agents, members and their nuances, which revealed the presence of a shallow space between the systems of native and foreign languages.