Experience generalization for multi-agent reinforcement learning

On-line learning methods have been applied successfully in multi-agent systems to achieve coordination among agents. Learning in multi-agent systems implies in a non-stationary scenario perceived by the agents, since the behavior of other agents may change as they simultaneously learn how to improve their actions. Non-stationary scenarios can be modeled as Markov Games, which can be solved using the Minimax-Q algorithm a combination of Q-learning (a Reinforcement Learning (RL) algorithm which directly learns an optimal control policy) and the Minimax algorithm. However, finding optimal control policies using any RL algorithm (Q-learning and Minimax-Q included) can be very time consuming. Trying to improve the learning time of Q-learning, we considered the QS-algorithm. in which a single experience can update more than a single action value by using a spreading function. In this paper, we contribute a Minimax-QS algorithm which combines the Minimax-Q algorithm and the QS-algorithm. We conduct a series of empirical evaluation of the algorithm in a simplified simulator of the soccer domain. We show that even using a very simple domain-dependent spreading function, the performance of the learning algorithm can be improved.

New Insights on Nontechnical Losses Characterization Through Evolutionary-Based Feature Selection

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Reactive power dispatch and planning using a non-linear branch-and-bound algorithm

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

O problema do caixeiro viajante alugador : um estudo algorítmico

The Car Rental Salesman Problem (CaRS) is a variant of the classical Traveling Salesman Problem which was not described in the literature where a tour of visits can be decomposed into contiguous paths that may be performed in different rental cars. The aim is to determine the Hamiltonian cycle that results in a final minimum cost, considering the cost of the route added to the cost of an expected penalty paid for each exchange of vehicles on the route. This penalty is due to the return of the car dropped to the base. This paper introduces the general problem and illustrates some examples, also featuring some of its associated variants. An overview of the complexity of this combinatorial problem is also outlined, to justify their classification in the NPhard class. A database of instances for the problem is presented, describing the methodology of its constitution. The presented problem is also the subject of a study based on experimental algorithmic implementation of six metaheuristic solutions, representing adaptations of the best of state-of-the-art heuristic programming. New neighborhoods, construction procedures, search operators, evolutionary agents, cooperation by multi-pheromone are created for this problem. Furtermore, computational experiments and comparative performance tests are conducted on a sample of 60 instances of the created database, aiming to offer a algorithm with an efficient solution for this problem. These results will illustrate the best performance reached by the transgenetic algorithm in all instances of the dataset

O problema do caixeiro alugador com coleta de bonus: um estudo algoritmico

This paper introduces a new variant of the Traveling Car Renter Problem, named Prizecollecting Traveling Car Renter Problem. In this problem, a set of vertices, each associated with a bonus, and a set of vehicles are given. The objective is to determine a cycle that visits some vertices collecting, at least, a pre-defined bonus, and minimizing the cost of the tour that can be traveled with different vehicles. A mathematical formulation is presented and implemented in a solver to produce results for sixty-two instances. The proposed problem is also subject of an experimental study based on the algorithmic application of four metaheuristics representing the best adaptations of the state of the art of the heuristic programming.We also provide new local search operators which exploit the neighborhoods of the problem, construction procedures and adjustments, created specifically for the addressed problem. Comparative computational experiments and performance tests are performed on a sample of 80 instances, aiming to offer a competitive algorithm to the problem. We conclude that memetic algorithms, computational transgenetic and a hybrid evolutive algorithm are competitive in tests performed

Evolutionary and biogeographic history of the subfamily Neoplecostominae (Siluriformes: Loricariidae)

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Efficient robust optimization based on polynomial chaos and tabu search algorithm

The study of robust design methodologies and techniques has become a new topical area in design optimizations in nearly all engineering and applied science disciplines in the last 10 years due to inevitable and unavoidable imprecision or uncertainty which is existed in real word design problems. To develop a fast optimizer for robust designs, a methodology based on polynomial chaos and tabu search algorithm is proposed. In the methodology, the polynomial chaos is employed as a stochastic response surface model of the objective function to efficiently evaluate the robust performance parameter while a mechanism to assign expected fitness only to promising solutions is introduced in tabu search algorithm to minimize the requirement for determining robust metrics of intermediate solutions. The proposed methodology is applied to the robust design of a practical inverse problem with satisfactory results.

Effects of cement thickness and bonding on the failure loads of CAD/CAM ceramic crowns: Multi-physics FEA modeling and monotonic testing

Objective. To determine the influence of cement thickness and ceramic/cement bonding on stresses and failure of CAD/CAM crowns, using both multi-physics finite element analysis and monotonic testing.Methods. Axially symmetric FEA models were created for stress analysis of a stylized monolithic crown having resin cement thicknesses from 50 to 500 mu m under occlusal loading. Ceramic-cement interface was modeled as bonded or not-bonded (cement-dentin as bonded). Cement polymerization shrinkage was simulated as a thermal contraction. Loads necessary to reach stresses for radial cracking from the intaglio surface were calculated by FEA. Experimentally, feldspathic CAD/CAM crowns based on the FEA model were machined having different occlusal cementation spaces, etched and cemented to dentin analogs. Non-bonding of etched ceramic was achieved using a thin layer of poly(dimethylsiloxane). Crowns were loaded to failure at 5 N/s, with radial cracks detected acoustically.Results. Failure loads depended on the bonding condition and the cement thickness for both FEA and physical testing. Average fracture loads for bonded crowns were: 673.5 N at 50 mu m cement and 300.6 N at 500 mu m. FEA stresses due to polymerization shrinkage increased with the cement thickness overwhelming the protective effect of bonding, as was also seen experimentally. At 50 mu m cement thickness, bonded crowns withstood at least twice the load before failure than non-bonded crowns.Significance. Occlusal "fit" can have structural implications for CAD/CAM crowns; pre-cementation spaces around 50-100 mu m being recommended from this study. Bonding benefits were lost at thickness approaching 450-500 mu m due to polymerization shrinkage stresses. (C) 2012 Academy of Dental Materials. Published by Elsevier Ltd. All rights reserved.

Spatial electric load forecasting using an evolutionary heuristic

A method for spatial electric load forecasting using elements from evolutionary algorithms is presented. The method uses concepts from knowledge extraction algorithms and linguistic rules' representation to characterize the preferences for land use into a spatial database. The future land use preferences in undeveloped zones in the electrical utility service area are determined using an evolutionary heuristic, which considers a stochastic behavior by crossing over similar rules. The method considers development of new zones and also redevelopment of existing ones. The results are presented in future preference maps. The tests in a real system from a midsized city show a high rate of success when results are compared with information gathered from the utility planning department. The most important features of this method are the need for few data and the simplicity of the algorithm, allowing for future scalability.

An improved tabu-based vector optimal algorithm for design optimizations of electromagnetic devices

This paper introduces an improved tabu-based vector optimal algorithm for multiobjective optimal designs of electromagnetic devices. The improvements include a division of the entire search process, a new method for fitness assignment, a novel scheme for the generation and selection of neighborhood solutions, and so forth. Numerical results on a mathematical function and an engineering multiobjective design problem demonstrate that the proposed method can produce virtually the exact Pareto front, in both parameter and objective spaces, even though the iteration number used by it is only about 70% of that required by its ancestor.

Paracoccidioidomycosis: A comparative study of the evolutionary serologic, clinical and radiologic results for patients treated with ketoconazole or amphotericin B plus sulfonamides

A comparative study of two groups of patients with paracoccidioidomycosis was carried out with the objective of comparing the evolutionary serologic, clinical and radiologic results after 6, 12, 15 and 18 months of treatment with ketoconazole (22 patients) or amphotericin B plus sulfonamides (32 patients). The serologic data analyzed as a whole showed a tendency to sharper drops in antibody titers in the patients treated with ketoconazole. Clinically patients treated with ketoconazole fared better but the differences were not statistically significant. No statistical difference was detected between groups in terms of the results of radiologic evolution. © 1985 Martinus Nijhoff/Dr W. Junk Publishers.

Uma metodologia para o controle de seguranca dinamica de sistemas de energia eletrica

This work presents an algorithm for the security control of electric power systems using control actions like generation reallocation, determined by sensitivity analysis (linearized model) and optimization by neural networks. The model is developed taking into account the dynamic network aspects. The preventive control methodology is developed by means of sensitivity analysis of the security margin related with the mechanical power of the system synchronous machines. The reallocation power in each machine is determined using neural networks. The neural network used in this work is of Hopfield type. These networks are dedicated electric circuits which simulate the constraint set and the objective function of an optimization problem. The advantage of using these networks is the higher speed in getting the solutions when compared to conventional optimization algorithms due to the great convergence rate of the process and the facility of the method parallelization. Then, the objectives are: formulate and investigate these networks implementations in determining. The generation reallocation in digital computers. Aiming to illustrate the proposed methodology an application considering a multi-machine system is presented.

Evolutionary heuristic to determine future land use

In the spatial electric load forecasting, the future land use determination is one of the most important tasks, and one of the most difficult, because of the stochastic nature of the city growth. This paper proposes a fast and efficient algorithm to find out the future land use for the vacant land in the utility service area, using ideas from knowledge extraction and evolutionary algorithms. The methodology was implemented into a full simulation software for spatial electric load forecasting, showing a high rate of success when the results are compared to information gathered from specialists. The importance of this methodology lies in the reduced set of data needed to perform the task and the simplicity for implementation, which is a great plus for most of the electric utilities without specialized tools for this planning activity. © 2008 IEEE.

Multi-areas optimal reactive power flow

This paper describes a method for the decentralized solution of the optimal reactive power flow (ORPF) problem in interconnected power systems. The ORPF model is solved in a decentralized framework, consisting of regions, where the transmission system operator in each area operates its system independently of the other areas, obtaining an optimal coordinated but decentralized solution. The proposed scheme is based on an augmented Lagrangian approach using the auxiliary problem principle (APP). An implementation of an interior point method is described to solve the decoupled problem in each area. The described method is successfully implemented and tested using the IEEE two area RTS 96 test system. Numerical results comparing the solutions obtained by the traditional and the proposed decentralized methods are presented for validation. ©2008 IEEE.

An efficient codification to solve distribution network reconfiguration for loss reduction problem

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)