Intelligent decision making in electricity markets: simulated annealing Q-Learning

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is a multi-agent electricity market simulator to model market players and simulate their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. MASCEM is integrated with ALBidS, a system that provides several dynamic strategies for agents’ behavior. This paper presents a method that aims at enhancing ALBidS competence in endowing market players with adequate strategic bidding capabilities, allowing them to obtain the higher possible gains out of the market. This method uses a reinforcement learning algorithm to learn from experience how to choose the best from a set of possible actions. These actions are defined accordingly to the most probable points of bidding success. With the purpose of accelerating the convergence process, a simulated annealing based algorithm is included.

Intelligent electric vehicle heuristic for energy resource management using simulated annealing

The smart grid concept appears as a suitable solution to guarantee the power system operation in the new electricity paradigm with electricity markets and integration of large amounts of Distributed Energy Resources (DERs). Virtual Power Player (VPP) will have a significant importance in the management of a smart grid. In the context of this new paradigm, Electric Vehicles (EVs) rise as a good available resource to be used as a DER by a VPP. This paper presents the application of the Simulated Annealing (SA) technique to solve the Energy Resource Management (ERM) of a VPP. It is also presented a new heuristic approach to intelligently handle the charge and discharge of the EVs. This heuristic process is incorporated in the SA technique, in order to improve the results of the ERM. The case study shows the results of the ERM for a 33-bus distribution network with three different EVs penetration levels, i. e., with 1000, 2000 and 3000 EVs. The results of the proposed adaptation of the SA technique are compared with a previous SA version and a deterministic technique.

Intelligent energy resource management considering vehicle-to-grid: A simulated annealing approach

This paper proposes a simulated annealing (SA) approach to address energy resources management from the point of view of a virtual power player (VPP) operating in a smart grid. Distributed generation, demand response, and gridable vehicles are intelligently managed on a multiperiod basis according to V2G user´s profiles and requirements. Apart from using the aggregated resources, the VPP can also purchase additional energy from a set of external suppliers. The paper includes a case study for a 33 bus distribution network with 66 generators, 32 loads, and 1000 gridable vehicles. The results of the SA approach are compared with a methodology based on mixed-integer nonlinear programming. A variation of this method, using ac load flow, is also used and the results are compared with the SA solution using network simulation. The proposed SA approach proved to be able to obtain good solutions in low execution times, providing VPPs with suitable decision support for the management of a large number of distributed resources.

Simulated annealing metaheuristic to solve the optimal power flow

The optimal power flow problem has been widely studied in order to improve power systems operation and planning. For real power systems, the problem is formulated as a non-linear and as a large combinatorial problem. The first approaches used to solve this problem were based on mathematical methods which required huge computational efforts. Lately, artificial intelligence techniques, such as metaheuristics based on biological processes, were adopted. Metaheuristics require lower computational resources, which is a clear advantage for addressing the problem in large power systems. This paper proposes a methodology to solve optimal power flow on economic dispatch context using a Simulated Annealing algorithm inspired on the cooling temperature process seen in metallurgy. The main contribution of the proposed method is the specific neighborhood generation according to the optimal power flow problem characteristics. The proposed methodology has been tested with IEEE 6 bus and 30 bus networks. The obtained results are compared with other wellknown methodologies presented in the literature, showing the effectiveness of the proposed method.

Economic, environmental and mixed objective functions in non-linear process optimization using simulated annealing and tabu search

Screening of topologies developed by hierarchical heuristic procedures can be carried out by comparing their optimal performance. In this work we will be exploiting mono-objective process optimization using two algorithms, simulated annealing and tabu search, and four different objective functions: two of the net present value type, one of them including environmental costs and two of the global potential impact type. The hydrodealkylation of toluene to produce benzene was used as case study, considering five topologies with different complexities mainly obtained by including or not liquid recycling and heat integration. The performance of the algorithms together with the objective functions was observed, analyzed and discussed from various perspectives: average deviation of results for each algorithm, capacity for producing high purity product, screening of topologies, objective functions robustness in screening of topologies, trade-offs between economic and environmental type objective functions and variability of optimum solutions.

Self-mated tribological systems based on multilayer micro/nanocrystalline CVD diamond coatings

The tribological response of multilayer micro/nanocrystalline diamond coatings grown by the hot filament CVD technique is investigated. These multigrade systems were tailored to comprise a starting microcrystalline diamond (MCD) layer with high adhesion to a silicon nitride (Si3N4) ceramic substrate, and a top nanocrystalline diamond (NCD) layer with reduced surface roughness. Tribological tests were carried out with a reciprocating sliding configuration without lubrication. Such composite coatings exhibit a superior critical load before delamination (130–200 N), when compared to the mono- (60–100 N) and bilayer coatings (110 N), considering ∼10 µm thick films. Regarding the friction behaviour, a short-lived initial high friction coefficient was followed by low friction regimes (friction coefficients between 0.02 and 0.09) as a result of the polished surfaces tailored by the tribological solicitation. Very mild to mild wear regimes (wear coefficient values between 4.1×10−8 and 7.7×10−7 mm3 N−1 m−1) governed the wear performance of the self-mated multilayer coatings when subjected to high-load short-term tests (60–200 N; 2 h; 86 m) and medium-load endurance tests (60 N; 16 h; 691 m).

A Hybrid Simulated Annealing approach to handle Energy Resource Management considering an intensive use of Electric Vehicles

The massification of electric vehicles (EVs) can have a significant impact on the power system, requiring a new approach for the energy resource management. The energy resource management has the objective to obtain the optimal scheduling of the available resources considering distributed generators, storage units, demand response and EVs. The large number of resources causes more complexity in the energy resource management, taking several hours to reach the optimal solution which requires a quick solution for the next day. Therefore, it is necessary to use adequate optimization techniques to determine the best solution in a reasonable amount of time. This paper presents a hybrid artificial intelligence technique to solve a complex energy resource management problem with a large number of resources, including EVs, connected to the electric network. The hybrid approach combines simulated annealing (SA) and ant colony optimization (ACO) techniques. The case study concerns different EVs penetration levels. Comparisons with a previous SA approach and a deterministic technique are also presented. For 2000 EVs scenario, the proposed hybrid approach found a solution better than the previous SA version, resulting in a cost reduction of 1.94%. For this scenario, the proposed approach is approximately 94 times faster than the deterministic approach.

Simulated Annealing Approach Applied to the Energy Resource Management Considering Demand Response for Electric Vehicle

The aggregation and management of Distributed Energy Resources (DERs) by an Virtual Power Players (VPP) is an important task in a smart grid context. The Energy Resource Management (ERM) of theses DERs can become a hard and complex optimization problem. The large integration of several DERs, including Electric Vehicles (EVs), may lead to a scenario in which the VPP needs several hours to have a solution for the ERM problem. This is the reason why it is necessary to use metaheuristic methodologies to come up with a good solution with a reasonable amount of time. The presented paper proposes a Simulated Annealing (SA) approach to determine the ERM considering an intensive use of DERs, mainly EVs. In this paper, the possibility to apply Demand Response (DR) programs to the EVs is considered. Moreover, a trip reduce DR program is implemented. The SA methodology is tested on a 32-bus distribution network with 2000 EVs, and the SA results are compared with a deterministic technique and particle swarm optimization results.

A New Heuristic Providing an Effective Initial Solution for a Simulated Annealing approach to Energy Resource Scheduling in Smart Grids

An intensive use of dispersed energy resources is expected for future power systems, including distributed generation, especially based on renewable sources, and electric vehicles. The system operation methods and tool must be adapted to the increased complexity, especially the optimal resource scheduling problem. Therefore, the use of metaheuristics is required to obtain good solutions in a reasonable amount of time. This paper proposes two new heuristics, called naive electric vehicles charge and discharge allocation and generation tournament based on cost, developed to obtain an initial solution to be used in the energy resource scheduling methodology based on simulated annealing previously developed by the authors. The case study considers two scenarios with 1000 and 2000 electric vehicles connected in a distribution network. The proposed heuristics are compared with a deterministic approach and presenting a very small error concerning the objective function with a low execution time for the scenario with 2000 vehicles.

Cost dependent strategy for electricity markets bidding based on adaptive reinforcement learning

Electricity markets are complex environments, involving a large number of different entities, playing in a dynamic scene to obtain the best advantages and profits. MASCEM is a multi-agent electricity market simulator to model market players and simulate their operation in the market. Market players are entities with specific characteristics and objectives, making their decisions and interacting with other players. MASCEM provides several dynamic strategies for agents’ behavior. This paper presents a method that aims to provide market players with strategic bidding capabilities, allowing them to obtain the higher possible gains out of the market. This method uses a reinforcement learning algorithm to learn from experience how to choose the best from a set of possible bids. These bids are defined accordingly to the cost function that each producer presents.

Optimization techniques for power distribution planning with uncertainties: a comparative study

Important research effort has been devoted to the topic of optimal planning of distribution systems. The non linear nature of the system, the need to consider a large number of scenarios and the increasing necessity to deal with uncertainties make optimal planning in distribution systems a difficult task. Heuristic techniques approaches have been proposed to deal with these issues, overcoming some of the inherent difficulties of classic methodologies. This paper considers several methodologies used to address planning problems of electrical power distribution networks, namely mixedinteger linear programming (MILP), ant colony algorithms (AC), genetic algorithms (GA), tabu search (TS), branch exchange (BE), simulated annealing (SA) and the Bender´s decomposition deterministic non-linear optimization technique (BD). Adequacy of theses techniques to deal with uncertainties is discussed. The behaviour of each optimization technique is compared from the point of view of the obtained solution and of the methodology performance. The paper presents results of the application of these optimization techniques to a real case of a 10-kV electrical distribution system with 201 nodes that feeds an urban area.

Prova de esforço e factores de risco cardiovascular: um estudo piloto

As doenças cardiovasculares (DCV’s) são a maior causa de mortalidade e morbilidade em Portugal. O seu elevado impacto passa pelo desconhecimento, sub-diagnóstico, elevada prevalência e descontrolo dos seus principais factores de risco (clássicos e novos marcadores bioquímicos). Para o diagnóstico de uma das vertentes da doença cardiovascular, a doença cardíaca isquémica, a prova de esforço (PE) é o exame não invasivo, de baixo custo, com reduzida taxa de complicações e de fácil execução, mais usado na clínica. O objectivo deste estudo é averiguar se existe relação entre a prova de esforço, os factores de risco cardiovascular (FR’s) e alguns dos seus marcadores bioquímicos. Com vista a alcançar objectivo realizou-se um estudo prospectivo, longitudinal e descritivo, na Esferasaúde (Maia), entre Janeiro e Maio de 2011. Foram recolhidos dados, por inquérito, referentes a: biografia, antropometria, FR’s, medicação, PE e análises clínicas. Tendo sido incluídos todos os indivíduos (idade ≥ 18 anos) que tenham realizado prova de esforço e análises na unidade citada e com diferença temporal máxima de 2 meses, pelo método de amostragem dirigida e intencional. A dimensão amostral situou-se nos 30 elementos, sendo que 19 eram do género masculino. A média de idade foi 49,43±15,39 anos. Estimou-se a prevalência de FR’s e de indivíduos com valores dos marcadores bioquímicos anormais. Dois dos indivíduos apresentavam história de DCV’s e três deles PE positiva. Foram efectuadas diversas tentativas de associação entre as variáveis integradas no estudo - DCV e FR’s; PE e FR’s; PE e marcadores bioquímicos; capacidade de esforço e FR’s, género e resultado PE. Nenhuma relação se revelou significativa, com excepção para dois casos: relação entre as DCV’s e o aparecimento de alterações na PE (p = 0,002) e associação entre PE e colesterol HDL (p=0,040). Para α de 5%. Conclui-se que não existe relação aparente entre a prova de esforço, a existência de doença cardiovascular, os seus factores de risco e marcadores bioquímicos.

Electroanalysis of urinary L-dopa using tyrosinase immobilized on gold nanoelectrode ensembles

The performance of an amperometric biosensor constructed by associating tyrosinase (Tyr) enzyme with the advantages of a 3D gold nanoelectrode ensemble (GNEE) is evaluated in a flow-injection analysis (FIA) system for the analysis of l-dopa. GNEEs were fabricated by electroless deposition of the metal within the pores of polycarbonate track-etched membranes. A simple solvent etching procedure based on the solubility of polycarbonate membranes is adopted for the fabrication of the 3D GNEE. Afterward, enzyme was immobilized onto preformed self-assembled monolayers of cysteamine on the 3D GNEEs (GNEE-Tyr) via cross-linking with glutaraldehyde. The experimental conditions of the FIA system, such as the detection potential (−0.200 V vs. Ag/AgCl) and flow rates (1.0 mL min−1) were optimized. Analytical responses for l-dopa were obtained in a wide concentration range between 1 × 10−8 mol L−1 and 1 × 10−2 mol L−1. The limit of quantification was found to be 1 × 10−8 mol L−1 with a resultant % RSD of 7.23% (n = 5). The limit of detection was found to be 1 × 10−9 mol L−1 (S/N = 3). The common interfering compounds, namely glucose (10 mmol L−1), ascorbic acid (10 mmol L−1), and urea (10 mmol L−1), were studied. The recovery of l-dopa (1 × 10−7 mol L−1) from spiked urine samples was found to be 96%. Therefore, the developed method is adequate to be applied in the clinical analysis.

Energy resource management in smart grids considering an Intensive use of electric vehicles

The introduction of electricity markets and integration of Distributed Generation (DG) have been influencing the power system’s structure change. Recently, the smart grid concept has been introduced, to guarantee a more efficient operation of the power system using the advantages of this new paradigm. Basically, a smart grid is a structure that integrates different players, considering constant communication between them to improve power system operation and management. One of the players revealing a big importance in this context is the Virtual Power Player (VPP). In the transportation sector the Electric Vehicle (EV) is arising as an alternative to conventional vehicles propel by fossil fuels. The power system can benefit from this massive introduction of EVs, taking advantage on EVs’ ability to connect to the electric network to charge, and on the future expectation of EVs ability to discharge to the network using the Vehicle-to-Grid (V2G) capacity. This thesis proposes alternative strategies to control these two EV modes with the objective of enhancing the management of the power system. Moreover, power system must ensure the trips of EVs that will be connected to the electric network. The EV user specifies a certain amount of energy that will be necessary to charge, in order to ensure the distance to travel. The introduction of EVs in the power system turns the Energy Resource Management (ERM) under a smart grid environment, into a complex problem that can take several minutes or hours to reach the optimal solution. Adequate optimization techniques are required to accommodate this kind of complexity while solving the ERM problem in a reasonable execution time. This thesis presents a tool that solves the ERM considering the intensive use of EVs in the smart grid context. The objective is to obtain the minimum cost of ERM considering: the operation cost of DG, the cost of the energy acquired to external suppliers, the EV users payments and remuneration and penalty costs. This tool is directed to VPPs that manage specific network areas, where a high penetration level of EVs is expected to be connected in these areas. The ERM is solved using two methodologies: the adaptation of a deterministic technique proposed in a previous work, and the adaptation of the Simulated Annealing (SA) technique. With the purpose of improving the SA performance for this case, three heuristics are additionally proposed, taking advantage on the particularities and specificities of an ERM with these characteristics. A set of case studies are presented in this thesis, considering a 32 bus distribution network and up to 3000 EVs. The first case study solves the scheduling without considering EVs, to be used as a reference case for comparisons with the proposed approaches. The second case study evaluates the complexity of the ERM with the integration of EVs. The third case study evaluates the performance of scheduling with different control modes for EVs. These control modes, combined with the proposed SA approach and with the developed heuristics, aim at improving the quality of the ERM, while reducing drastically its execution time. The proposed control modes are: uncoordinated charging, smart charging and V2G capability. The fourth and final case study presents the ERM approach applied to consecutive days.

Oxidative Leaching of metals from electronic waste with solutions based on quaternary ammonium salts

The treatment of electric and electronic waste (WEEE) is a problem which receives ever more attention. An inadequate treatment results in harmful products ending up in the environment. This project intends to investigate the possibilities of an alternative route for recycling of metals from printed circuit boards (PCBs) obtained from rejected computers. The process is based on aqueous solutions composed of an etchant, either 0.2 M CuCl2.2H2O or 0.2 M FeCl3.6H2O, and a quaternary ammonium salt (quat) such as choline chloride or chlormequat. These solutions are reminiscent of deep eutectic solvents (DES) based on quats. DES are quite similar to ionic liquids (ILs) and are used as well as alternative solvents with a great diversity of physical properties, making them attractive for replacement of hazardous, volatile solvents (e.g. VOCs). A remarkable difference between genuine DES and ILs with the solutions used in this project is the addition of rather large quantities of water. It is shown the presence of water has a lot of advantages on the leaching of metals, while the properties typical for DES still remain. The oxidizing capacities of Cu(II) stem from the existence of a stable Cu(I) component in quat based DES and thus the leaching stems from the activity of the Cu(II)/Cu(I) redox couple. The advantage of Fe(III) in combination with DES is the fact that the Fe(III)/Fe(II) redox couple becomes reversible, which is not true in pure water. This opens perspectives for regeneration of the etching solution. In this project the leaching of copper was studied as a function of gradual increasing water content from 0 - 100w% with the same concentration of copper chloride or iron(III) chloride at room temperature and 80ºC. The solutions were also tested on real PCBs. At room temperature a maximum leaching effect for copper was obtained with 30w% choline chloride with 0.2 M CuCl2.2H2O. The leaching effect is still stronger at 80°C, b ut of course these solutions are more energy consuming. For aluminium, tin, zinc and lead, the leaching was faster at 80ºC. Iron and nickel dissolved easily at room temperature. The solutions were not able to dissolve gold, silver, rhodium and platinum.