Intelligent decision making in electricity markets: simulated annealing Q-Learning

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is a multi-agent electricity market simulator to model market players and simulate their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. MASCEM is integrated with ALBidS, a system that provides several dynamic strategies for agents’ behavior. This paper presents a method that aims at enhancing ALBidS competence in endowing market players with adequate strategic bidding capabilities, allowing them to obtain the higher possible gains out of the market. This method uses a reinforcement learning algorithm to learn from experience how to choose the best from a set of possible actions. These actions are defined accordingly to the most probable points of bidding success. With the purpose of accelerating the convergence process, a simulated annealing based algorithm is included.

Intelligent electric vehicle heuristic for energy resource management using simulated annealing

The smart grid concept appears as a suitable solution to guarantee the power system operation in the new electricity paradigm with electricity markets and integration of large amounts of Distributed Energy Resources (DERs). Virtual Power Player (VPP) will have a significant importance in the management of a smart grid. In the context of this new paradigm, Electric Vehicles (EVs) rise as a good available resource to be used as a DER by a VPP. This paper presents the application of the Simulated Annealing (SA) technique to solve the Energy Resource Management (ERM) of a VPP. It is also presented a new heuristic approach to intelligently handle the charge and discharge of the EVs. This heuristic process is incorporated in the SA technique, in order to improve the results of the ERM. The case study shows the results of the ERM for a 33-bus distribution network with three different EVs penetration levels, i. e., with 1000, 2000 and 3000 EVs. The results of the proposed adaptation of the SA technique are compared with a previous SA version and a deterministic technique.

Intelligent energy resource management considering vehicle-to-grid: A simulated annealing approach

This paper proposes a simulated annealing (SA) approach to address energy resources management from the point of view of a virtual power player (VPP) operating in a smart grid. Distributed generation, demand response, and gridable vehicles are intelligently managed on a multiperiod basis according to V2G user´s profiles and requirements. Apart from using the aggregated resources, the VPP can also purchase additional energy from a set of external suppliers. The paper includes a case study for a 33 bus distribution network with 66 generators, 32 loads, and 1000 gridable vehicles. The results of the SA approach are compared with a methodology based on mixed-integer nonlinear programming. A variation of this method, using ac load flow, is also used and the results are compared with the SA solution using network simulation. The proposed SA approach proved to be able to obtain good solutions in low execution times, providing VPPs with suitable decision support for the management of a large number of distributed resources.

Simulated annealing metaheuristic to solve the optimal power flow

The optimal power flow problem has been widely studied in order to improve power systems operation and planning. For real power systems, the problem is formulated as a non-linear and as a large combinatorial problem. The first approaches used to solve this problem were based on mathematical methods which required huge computational efforts. Lately, artificial intelligence techniques, such as metaheuristics based on biological processes, were adopted. Metaheuristics require lower computational resources, which is a clear advantage for addressing the problem in large power systems. This paper proposes a methodology to solve optimal power flow on economic dispatch context using a Simulated Annealing algorithm inspired on the cooling temperature process seen in metallurgy. The main contribution of the proposed method is the specific neighborhood generation according to the optimal power flow problem characteristics. The proposed methodology has been tested with IEEE 6 bus and 30 bus networks. The obtained results are compared with other wellknown methodologies presented in the literature, showing the effectiveness of the proposed method.

Seller, buyer and VPP agents behavior on a simulated electricity market

Competitive electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is an electricity market simulator able to model market players and simulate their operation in the market. As market players are complex entities, having their characteristics and objectives, making their decisions and interacting with other players, a multi-agent architecture is used and proved to be adequate. MASCEM players have learning capabilities and different risk preferences. They are able to refine their strategies according to their past experience (both real and simulated) and considering other agents’ behavior. Agents’ behavior is also subject to its risk preferences.

Planning, Cooperation and Competition: A case study in industry and construction in a Portuguese Region

In the present paper we analyzed the behavior of firms in the construction and manufacturing sectors, located in the region of Vale do Sousa, in the north of Portugal. From the literature, even existing some disagreements, it is possible to conclude that planning is crucial for firms survival and growth. Cooperation is another aspect that the literature presents as an important factor for firms sustainability. It also plays a major role in competition, since firms are adopting coopetition strategies. By studying a sample of 251 firms, it was possible to realize, that the majority started their business without a formal planning, and they keep going without using it. In cooperation aspects, there is a lack of cooperation. It was possible to verify, that existing cooperation has some evidence but at a vertical level. These vertical relations were also identified in stakeholder’s involvement.

Economic, environmental and mixed objective functions in non-linear process optimization using simulated annealing and tabu search

Screening of topologies developed by hierarchical heuristic procedures can be carried out by comparing their optimal performance. In this work we will be exploiting mono-objective process optimization using two algorithms, simulated annealing and tabu search, and four different objective functions: two of the net present value type, one of them including environmental costs and two of the global potential impact type. The hydrodealkylation of toluene to produce benzene was used as case study, considering five topologies with different complexities mainly obtained by including or not liquid recycling and heat integration. The performance of the algorithms together with the objective functions was observed, analyzed and discussed from various perspectives: average deviation of results for each algorithm, capacity for producing high purity product, screening of topologies, objective functions robustness in screening of topologies, trade-offs between economic and environmental type objective functions and variability of optimum solutions.

Evaluation of distinct input methods of an intelligent wheelchair in simulated and real environments: a performance and usability study

This paper focuses on evaluating the usability of an Intelligent Wheelchair (IW) in both real and simulated environments. The wheelchair is controlled at a high-level by a flexible multimodal interface, using voice commands, facial expressions, head movements and joystick as its main inputs. A Quasi-experimental design was applied including a deterministic sample with a questionnaire that enabled to apply the System Usability Scale. The subjects were divided in two independent samples: 46 individuals performing the experiment with an Intelligent Wheelchair in a simulated environment (28 using different commands in a sequential way and 18 with the liberty to choose the command); 12 individuals performing the experiment with a real IW. The main conclusion achieved by this study is that the usability of the Intelligent Wheelchair in a real environment is higher than in the simulated environment. However there were not statistical evidences to affirm that there are differences between the real and simulated wheelchairs in terms of safety and control. Also, most of users considered the multimodal way of driving the wheelchair very practical and satisfactory. Thus, it may be concluded that the multimodal interfaces enables very easy and safe control of the IW both in simulated and real environments.

Quantity-setting competition with differentiated goods under uncertain demand

We consider a quantity-setting duopoly model, and we study the decision to move first or second, by assuming that the firms produce differentiated goods and that there is some demand uncertainty. The competitive phase consists of two periods, and in either period, the firms can make a production decision that is irreversible. As far as the firms are allowed to choose (non-cooperatively) the period they make the decision, we study the circumstances that favour sequential rather than simultaneous decisions.

Cournot competition between a non-profit firm and a for-profit firm with uncertainty

In this paper, we consider a Cournot competition between a nonprofit firm and a for-profit firm in a homogeneous goods market, with uncertain demand. Given an asymmetric tax schedule, we compute explicitly the Bayesian-Nash equilibrium. Furthermore, we analyze the effects of the tax rate and the degree of altruistic preference on market equilibrium outcomes.

Simultaneous decisions or leadership in an international competition

In this paper, we study an international market with demand uncertainty. The model has two stages. In the first stage, the home government chooses an import tariff to maximize the revenue. Then, the firms engage in a Cournot or in a Stackelberg competition. The uncertainty is resolved between the decisions made by the home government and by the firms. We compare the results obtained in the three different ways of moving on the decision make of the firms.

Maximum-revenue tariff with different roles in a price-setting competition

Published also at Lecture Notes in Engineering and Computer Science

A Hybrid Simulated Annealing approach to handle Energy Resource Management considering an intensive use of Electric Vehicles

The massification of electric vehicles (EVs) can have a significant impact on the power system, requiring a new approach for the energy resource management. The energy resource management has the objective to obtain the optimal scheduling of the available resources considering distributed generators, storage units, demand response and EVs. The large number of resources causes more complexity in the energy resource management, taking several hours to reach the optimal solution which requires a quick solution for the next day. Therefore, it is necessary to use adequate optimization techniques to determine the best solution in a reasonable amount of time. This paper presents a hybrid artificial intelligence technique to solve a complex energy resource management problem with a large number of resources, including EVs, connected to the electric network. The hybrid approach combines simulated annealing (SA) and ant colony optimization (ACO) techniques. The case study concerns different EVs penetration levels. Comparisons with a previous SA approach and a deterministic technique are also presented. For 2000 EVs scenario, the proposed hybrid approach found a solution better than the previous SA version, resulting in a cost reduction of 1.94%. For this scenario, the proposed approach is approximately 94 times faster than the deterministic approach.

Simulated Annealing Approach Applied to the Energy Resource Management Considering Demand Response for Electric Vehicle

The aggregation and management of Distributed Energy Resources (DERs) by an Virtual Power Players (VPP) is an important task in a smart grid context. The Energy Resource Management (ERM) of theses DERs can become a hard and complex optimization problem. The large integration of several DERs, including Electric Vehicles (EVs), may lead to a scenario in which the VPP needs several hours to have a solution for the ERM problem. This is the reason why it is necessary to use metaheuristic methodologies to come up with a good solution with a reasonable amount of time. The presented paper proposes a Simulated Annealing (SA) approach to determine the ERM considering an intensive use of DERs, mainly EVs. In this paper, the possibility to apply Demand Response (DR) programs to the EVs is considered. Moreover, a trip reduce DR program is implemented. The SA methodology is tested on a 32-bus distribution network with 2000 EVs, and the SA results are compared with a deterministic technique and particle swarm optimization results.

Multi-Agent Based Smart Grid Management and Simulation: Situation Awareness and Learning in a Test Bed with Simulated and Real Installations and Players

The rising usage of distributed energy resources has been creating several problems in power systems operation. Virtual Power Players arise as a solution for the management of such resources. Additionally, approaching the main network as a series of subsystems gives birth to the concepts of smart grid and micro grid. Simulation, particularly based on multi-agent technology is suitable to model all these new and evolving concepts. MASGriP (Multi-Agent Smart Grid simulation Platform) is a system that was developed to allow deep studies of the mentioned concepts. This paper focuses on a laboratorial test bed which represents a house managed by a MASGriP player. This player is able to control a real installation, responding to requests sent by the system operators and reacting to observed events depending on the context.