A Negotiation Style Recommender Based on Computational Ecology in Open Negotiation Environments

The system described herein represents the first example of a recommender system in digital ecosystems where agents negotiate services on behalf of small companies. The small companies compete not only with price or quality, but with a wider service-by-service composition by subcontracting with other companies. The final result of these offerings depends on negotiations at the scale of millions of small companies. This scale requires new platforms for supporting digital business ecosystems, as well as related services like open-id, trust management, monitors and recommenders. This is done in the Open Negotiation Environment (ONE), which is an open-source platform that allows agents, on behalf of small companies, to negotiate and use the ecosystem services, and enables the development of new agent technologies. The methods and tools of cyber engineering are necessary to build up Open Negotiation Environments that are stable, a basic condition for predictable business and reliable business environments. Aiming to build stable digital business ecosystems by means of improved collective intelligence, we introduce a model of negotiation style dynamics from the point of view of computational ecology. This model inspires an ecosystem monitor as well as a novel negotiation style recommender. The ecosystem monitor provides hints to the negotiation style recommender to achieve greater stability of an open negotiation environment in a digital business ecosystem. The greater stability provides the small companies with higher predictability, and therefore better business results. The negotiation style recommender is implemented with a simulated annealing algorithm at a constant temperature, and its impact is shown by applying it to a real case of an open negotiation environment populated by Italian companies

Enhancing fault management performance of two-step QoS routing algorithms in GMPLS networks

In this paper a novel methodology aimed at minimizing the probability of network failure and the failure impact (in terms of QoS degradation) while optimizing the resource consumption is introduced. A detailed study of MPLS recovery techniques and their GMPLS extensions are also presented. In this scenario, some features for reducing the failure impact and offering minimum failure probabilities at the same time are also analyzed. Novel two-step routing algorithms using this methodology are proposed. Results show that these methods offer high protection levels with optimal resource consumption

Enhancing MPLS QoS routing algorithms by using the network protection degree paradigm

IP based networks still do not have the required degree of reliability required by new multimedia services, achieving such reliability will be crucial in the success or failure of the new Internet generation. Most of existing schemes for QoS routing do not take into consideration parameters concerning the quality of the protection, such as packet loss or restoration time. In this paper, we define a new paradigm to develop new protection strategies for building reliable MPLS networks, based on what we have called the network protection degree (NPD). This NPD consists of an a priori evaluation, the failure sensibility degree (FSD), which provides the failure probability and an a posteriori evaluation, the failure impact degree (FID), to determine the impact on the network in case of failure. Having mathematical formulated these components, we point out the most relevant components. Experimental results demonstrate the benefits of the utilization of the NPD, when used to enhance some current QoS routing algorithms to offer a certain degree of protection

Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Multicast algorithms evaluation using an adaptative routing in ATM networks

This paper proposes a multicast implementation based on adaptive routing with anticipated calculation. Three different cost measures for a point-to-multipoint connection: bandwidth cost, connection establishment cost and switching cost can be considered. The application of the method based on pre-evaluated routing tables makes possible the reduction of bandwidth cost and connection establishment cost individually

An ATM distributed simulator for network management research

Due to the high cost of a large ATM network working up to full strength to apply our ideas about network management, i.e., dynamic virtual path (VP) management and fault restoration, we developed a distributed simulation platform for performing our experiments. This platform also had to be capable of other sorts of tests, such as connection admission control (CAC) algorithms, routing algorithms, and accounting and charging methods. The platform was posed as a very simple, event-oriented and scalable simulation. The main goal was the simulation of a working ATM backbone network with a potentially large number of nodes (hundreds). As research into control algorithms and low-level, or rather cell-level methods, was beyond the scope of this study, the simulation took place at a connection level, i.e., there was no real traffic of cells. The simulated network behaved like a real network accepting and rejecting SNMP ones, or experimental tools using the API node

Improving Clustering Algorithms for Image Segmentation using Contour and Region Information

In image segmentation, clustering algorithms are very popular because they are intuitive and, some of them, easy to implement. For instance, the k-means is one of the most used in the literature, and many authors successfully compare their new proposal with the results achieved by the k-means. However, it is well known that clustering image segmentation has many problems. For instance, the number of regions of the image has to be known a priori, as well as different initial seed placement (initial clusters) could produce different segmentation results. Most of these algorithms could be slightly improved by considering the coordinates of the image as features in the clustering process (to take spatial region information into account). In this paper we propose a significant improvement of clustering algorithms for image segmentation. The method is qualitatively and quantitative evaluated over a set of synthetic and real images, and compared with classical clustering approaches. Results demonstrate the validity of this new approach

Relaxation and derelaxation of pure and hydrogenated amorphous silicon during thermal annealing experiments

The structural relaxation of pure amorphous silicon (a-Si) and hydrogenated amorphous silicon (a-Si:H) materials, that occurs during thermal annealing experiments, has been analyzed by Raman spectroscopy and differential scanning calorimetry. Unlike a-Si, the heat evolved from a-Si:H cannot be explained by relaxation of the Si-Si network strain but it reveals a derelaxation of the bond angle strain. Since the state of relaxation after annealing is very similar for pure and hydrogenated materials, our results give strong experimental support to the predicted configurational gap between a-Si and crystalline silicon

A formalization for multi-agent decision support in cooperative environments. A framework for situated agents

La tesis propone un marco de trabajo para el soporte de la toma de decisiones adecuado para soportar la ejecución distribuida de acciones cooperativas en entornos multi-agente dinámicos y complejos. Soporte para la toma de decisiones es un proceso que intenta mejorar la ejecución de la toma de decisiones en escenarios cooperativos. Este proceso ocurre continuamente en la vida diaria. Los humanos, por ejemplo, deben tomar decisiones acerca de que ropa usar, que comida comer, etc. En este sentido, un agente es definido como cualquier cosa que está situada en un entorno y que actúa, basado en su observación, su interpretación y su conocimiento acerca de su situación en tal entorno para lograr una acción en particular.Por lo tanto, para tomar decisiones, los agentes deben considerar el conocimiento que les permita ser consientes en que acciones pueden o no ejecutar. Aquí, tal proceso toma en cuenta tres parámetros de información con la intención de personificar a un agente en un entorno típicamente físico. Así, el mencionado conjunto de información es conocido como ejes de decisión, los cuales deben ser tomados por los agentes para decidir si pueden ejecutar correctamente una tarea propuesta por otro agente o humano. Los agentes, por lo tanto, pueden hacer mejores decisiones considerando y representando apropiadamente tal información. Los ejes de decisión, principalmente basados en: las condiciones ambientales, el conocimiento físico y el valor de confianza del agente, provee a los sistemas multi-agente un confiable razonamiento para alcanzar un factible y exitoso rendimiento cooperativo.Actualmente, muchos investigadores tienden a generar nuevos avances en la tecnología agente para incrementar la inteligencia, autonomía, comunicación y auto-adaptación en escenarios agentes típicamente abierto y distribuidos. En este sentido, esta investigación intenta contribuir en el desarrollo de un nuevo método que impacte tanto en las decisiones individuales como colectivas de los sistemas multi-agente. Por lo tanto, el marco de trabajo propuesto ha sido utilizado para implementar las acciones concretas involucradas en el campo de pruebas del fútbol robótico. Este campo emula los juegos de fútbol real, donde los agentes deben coordinarse, interactuar y cooperar entre ellos para solucionar tareas complejas dentro de un escenario dinámicamente cambiante y competitivo, tanto para manejar el diseño de los requerimientos involucrados en las tareas como para demostrar su efectividad en trabajos colectivos. Es así que los resultados obtenidos tanto en el simulador como en el campo real de experimentación, muestran que el marco de trabajo para el soporte de decisiones propuesto para agentes situados es capaz de mejorar la interacción y la comunicación, reflejando en un adecuad y confiable trabajo en equipo dentro de entornos impredecibles, dinámicos y competitivos. Además, los experimentos y resultados también muestran que la información seleccionada para generar los ejes de decisión para situar a los agentes, es útil cuando tales agentes deben ejecutar una acción o hacer un compromiso en cada momento con la intención de cumplir exitosamente un objetivo colectivo. Finalmente, algunas conclusiones enfatizando las ventajas y utilidades del trabajo propuesto en la mejora del rendimiento colectivo de los sistemas multi-agente en situaciones tales como tareas coordinadas y asignación de tareas son presentadas.