Cost dependent strategy for electricity markets bidding based on adaptive reinforcement learning

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is a multi-agent electricity market simulator to model market players and simulate their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. MASCEM provides several dynamic strategies for agents’ behavior. This paper presents a method that aims to provide market players with strategic bidding capabilities, allowing them to obtain the higher possible gains out of the market. This method uses a reinforcement learning algorithm to learn from experience how to choose the best from a set of possible bids. These bids are defined accordingly to the cost function that each producer presents.

PCMAT, an adaptive learning platform

This paper is about PCMAT, an adaptive learning platform for Mathematics in Basic Education schools. Based on a constructivist approach, PCMAT aims at verifying how techniques from adaptive hypermedia systems can improve e-learning based systems. To achieve this goal, PCMAT includes a Pedagogical Model that contains a set of adaptation rules that influence the student-platform interaction. PCMAT was subject to a preliminary testing with students aged between 12 and 14 years old on the subject of direct proportionality. The results from this preliminary test are quite promising as they seem to demonstrate the validity of our proposal.

Adaptation model for PCMAT – Mathematics collaborative learning platform

The aim of this paper is to present an adaptation model for an Adaptive Educational Hypermedia System, PCMAT. The adaptation of the application is based on progressive self-assessment (exercises, tasks, and so on) and applies the constructivist learning theory and the learning styles theory. Our objective is the creation of a better, more adequate adaptation model that takes into account the complexities of different users.

Content adaptation for an adaptive hypermedia system

The aim of this paper is presenting the recommendation module of the Mathematics Collaborative Learning Platform (PCMAT). PCMAT is an Adaptive Educational Hypermedia System (AEHS), with a constructivist approach, which presents contents and activities adapted to the characteristics and learning style of students of mathematics in basic schools. The recommendation module is responsible for choosing different learning resources for the platform, based on the user's characteristics and performance. Since the main purpose of an adaptive system is to provide the user with content and interface adaptation, the recommendation module is integral to PCMAT’s adaptation model.

An adaptive educational system for higher education

The main objective of an Adaptive System is to adequate its relation with the user (content presentation, navigation, interface, etc.) according to a predefined but updatable model of the user that reflects his objectives, preferences, knowledge and competences [Brusilovsky, 2001], [De Bra, 2004]. For Educational Adaptive Systems, the emphasis is placed on the student knowledge in the domain application and learning style, to allow him to reach the learning objectives proposed for his training [Chepegin, 2004]. In Educational AHS, the User Model (UM), or Student Model, has increased relevance: when the student reaches the objectives of the course, the system must be able to readapt, for example, to his knowledge [Brusilovsky, 2001]. Learning Styles are understood as something that intent to define models of how given person learns. Generally it is understood that each person has a Learning Style different and preferred with the objective of achieving better results. Some case studies have proposed that teachers should assess the learning styles of their students and adapt their classroom and methods to best fit each student's learning style [Kolb, 2005], [Martins, 2008]. The learning process must take into consideration the individual cognitive and emotional parts of the student. In summary each Student is unique so the Student personal progress must be monitored and teaching shoul not be not generalized and repetitive [Jonassen, 1991], [Martins, 2008]. The aim of this paper is to present an Educational Adaptive Hypermedia Tool based on Progressive Assessment.

User modeling In adaptive hypermedia educational systems

This document is a survey in the research area of User Modeling (UM) for the specific field of Adaptive Learning. The aims of this document are: To define what it is a User Model; To present existing and well known User Models; To analyze the existent standards related with UM; To compare existing systems. In the scientific area of User Modeling (UM), numerous research and developed systems already seem to promise good results, but some experimentation and implementation are still necessary to conclude about the utility of the UM. That is, the experimentation and implementation of these systems are still very scarce to determine the utility of some of the referred applications. At present, the Student Modeling research goes in the direction to make possible reuse a student model in different systems. The standards are more and more relevant for this effect, allowing systems communicate and to share data, components and structures, at syntax and semantic level, even if most of them still only allow syntax integration.

Service offloading in adaptive real-time systems

Smartphones and other internet enabled devices are now common on our everyday life, thus unsurprisingly a current trend is to adapt desktop PC applications to execute on them. However, since most of these applications have quality of service (QoS) requirements, their execution on resource-constrained mobile devices presents several challenges. One solution to support more stringent applications is to offload some of the applications’ services to surrogate devices nearby. Therefore, in this paper, we propose an adaptable offloading mechanism which takes into account the QoS requirements of the application being executed (particularly its real-time requirements), whilst allowing offloading services to several surrogate nodes. We also present how the proposed computing model can be implemented in an Android environment

Coordinated transportation of a large object by a team of three robots

Dynamical systems theory in this work is used as a theoretical language and tool to design a distributed control architecture for a team of three robots that must transport a large object and simultaneously avoid collisions with either static or dynamic obstacles. The robots have no prior knowledge of the environment. The dynamics of behavior is defined over a state space of behavior variables, heading direction and path velocity. Task constraints are modeled as attractors (i.e. asymptotic stable states) of the behavioral dynamics. For each robot, these attractors are combined into a vector field that governs the behavior. By design the parameters are tuned so that the behavioral variables are always very close to the corresponding attractors. Thus the behavior of each robot is controlled by a time series of asymptotical stable states. Computer simulations support the validity of the dynamical model architecture.

Coordinated transportation of a large object by a team of two robots

In this paper dynamical systems theory is used as a theoretical language and tool to design a distributed control architecture for a team of two robots that must transport a large object and simultaneously avoid collisions with obstacles (either static or dynamic). This work extends the previous work with two robots (see [1] and [5]). However here we demonstrate that it’s possible to simplify the architecture presented in [1] and [5] and reach an equally stable global behavior. The robots have no prior knowledge of the environment. The dynamics of behavior is defined over a state space of behavior variables, heading direction and path velocity. Task constrains are modeled as attractors (i.e. asymptotic stable states) of a behavioral dynamics. For each robot, these attractors are combined into a vector field that governs the behavior. By design the parameters are tuned so that the behavioral variables are always very close to the corresponding attractors. Thus the behavior of each robot is controlled by a time series of asymptotic stable states. Computer simulations support the validity of the dynamical model architecture.

Using attractor dynamics to generate decentralized motion control of two mobile robots transporting a long object in coordination

Dynamical systems theory is used here as a theoretical language and tool to design a distributed control architecture for a team of two mobile robots that must transport a long object and simultaneously avoid obstacles. In this approach the level of modeling is at the level of behaviors. A “dynamics” of behavior is defined over a state space of behavioral variables (heading direction and path velocity). The environment is also modeled in these terms by representing task constraints as attractors (i.e. asymptotically stable states) or reppelers (i.e. unstable states) of behavioral dynamics. For each robot attractors and repellers are combined into a vector field that governs the behavior. The resulting dynamical systems that generate the behavior of the robots may be nonlinear. By design the systems are tuned so that the behavioral variables are always very close to one attractor. Thus the behavior of each robot is controled by a time series of asymptotically stable states. Computer simulations support the validity of our dynamic model architectures.

Adaptive tackling of the swinging problem for a 2 DOF crane – payload system

The control of a crane carrying its payload by an elastic string corresponds to a task in which precise, indirect control of a subsystem dynamically coupled to a directly controllable subsystem is needed. This task is interesting since the coupled degree of freedom has little damping and it is apt to keep swinging accordingly. The traditional approaches apply the input shaping technology to assist the human operator responsible for the manipulation task. In the present paper a novel adaptive approach applying fixed point transformations based iterations having local basin of attraction is proposed to simultaneously tackle the problems originating from the imprecise dynamic model available for the system to be controlled and the swinging problem, too. The most important phenomenological properties of this approach are also discussed. The control considers the 4th time-derivative of the trajectory of the payload. The operation of the proposed control is illustrated via simulation results.

On the robustness of the slotine-Li and the FPT/SVD-based adaptive controllers

A comparative study concerning the robustness of a novel, Fixed Point Transformations/Singular Value Decomposition (FPT/SVD)-based adaptive controller and the Slotine-Li (S&L) approach is given by numerical simulations using a three degree of freedom paradigm of typical Classical Mechanical systems, the cart + double pendulum. The effects of the imprecision of the available dynamical model, presence of dynamic friction at the axles of the drives, and the existence of external disturbance forces unknown and not modeled by the controller are considered. While the Slotine-Li approach tries to identify the parameters of the formally precise, available analytical model of the controlled system with the implicit assumption that the generalized forces are precisely known, the novel one makes do with a very rough, affine form and a formally more precise approximate model of that system, and uses temporal observations of its desired vs. realized responses. Furthermore, it does not assume the lack of unknown perturbations caused either by internal friction and/or external disturbances. Its another advantage is that it needs the execution of the SVD as a relatively time-consuming operation on a grid of a rough system-model only one time, before the commencement of the control cycle within which it works only with simple computations. The simulation examples exemplify the superiority of the FPT/SVD-based control that otherwise has the deficiency that it can get out of the region of its convergence. Therefore its design and use needs preliminary simulation investigations. However, the simulations also exemplify that its convergence can be guaranteed for various practical purposes.

Adaptive learning in agents behaviour: A framework for electricity markets simulation

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM (Multi-Agent System for Competitive Electricity Markets) is a multi-agent electricity market simulator that models market players and simulates their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. This paper presents a methodology to provide decision support to electricity market negotiating players. This model allows integrating different strategic approaches for electricity market negotiations, and choosing the most appropriate one at each time, for each different negotiation context. This methodology is integrated in ALBidS (Adaptive Learning strategic Bidding System) – a multiagent system that provides decision support to MASCEM's negotiating agents so that they can properly achieve their goals. ALBidS uses artificial intelligence methodologies and data analysis algorithms to provide effective adaptive learning capabilities to such negotiating entities. The main contribution is provided by a methodology that combines several distinct strategies to build actions proposals, so that the best can be chosen at each time, depending on the context and simulation circumstances. The choosing process includes reinforcement learning algorithms, a mechanism for negotiating contexts analysis, a mechanism for the management of the efficiency/effectiveness balance of the system, and a mechanism for competitor players' profiles definition.

An adaptive IoT management infrastructure for eco transport networks

The complexity associated with fast growing of B2B and the lack of a (complete) suite of open standards makes difficulty to maintain the underlying collaborative processes. Aligned to this challenge, this paper aims to be a contribution to an open architecture of logistics and transport processes management system. A model of an open integrated system is being defined as an open computational responsibility from the embedded systems (on-board) as well as a reference implementation (prototype) of a host system to validate the proposed open interfaces. Embedded subsystem can, natively, be prepared to cooperate with other on-board units and with IT-systems in an infrastructure commonly referred to as a center information system or back-office. In interaction with a central system the proposal is to adopt an open framework for cooperation where the embedded unit or the unit placed somewhere (land/sea) interacts in response to a set of implemented capabilities.

Vision based context categorization for all-terrain robot

Dissertation presented at the Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa to obtain the Master degree in Electrical and Computer Engineering.