Reinforcement Learning Based on the Bayesian Theorem for Electricity Markets Decision Support

This paper presents the applicability of a reinforcement learning algorithm based on the application of the Bayesian theorem of probability. The proposed reinforcement learning algorithm is an advantageous and indispensable tool for ALBidS (Adaptive Learning strategic Bidding System), a multi-agent system that has the purpose of providing decision support to electricity market negotiating players. ALBidS uses a set of different strategies for providing decision support to market players. These strategies are used accordingly to their probability of success for each different context. The approach proposed in this paper uses a Bayesian network for deciding the most probably successful action at each time, depending on past events. The performance of the proposed methodology is tested using electricity market simulations in MASCEM (Multi-Agent Simulator of Competitive Electricity Markets). MASCEM provides the means for simulating a real electricity market environment, based on real data from real electricity market operators.

Sorption of phenanthrene on agricultural soils

Polyaromatic hydrocarbon (PAH) sorption to soil is a key process deciding the transport and fate of PAH, and potential toxic impacts in the soil and groundwater ecosystems, for example in connection with atmospheric PAH deposition on soils. There are numerous studies on PAH sorption in relatively low organic porous media such as urban soils and groundwater sediments, but less attention has been given to cultivated soils. In this study, the phenanthrene partition coefficient, KD (liter per kilogram), was measured on 143 cultivated Danish soils (115 topsoils, 0–0.25-m soil depth and 28 subsoils, 0.25–1-m depth) by the single-point adsorption method. The organic carbon partition coefficient, KOC (liter per kilogram) for topsoils was found generally to fall between the KOC values estimated by the two most frequently used models for PAH partitioning, the Abdul et al. (Hazardous Waste & Hazardous Materials 4(3):211– 222, 1987) model and Karickhoff et al. (Water Research 13:241–248, 1979) model. A less-recognized model by Karickhoff (Chemosphere 10:833–846, 1981), yielding a KOC of 14,918 Lkg−1, closely corresponded to the average measured KOC value for the topsoils, and this model is therefore recommended for prediction of phenanthrene mobility in cultivated topsoils. For lower subsoils (0.25–1-m depth), the KOC values were closer to and mostly below the estimate by the Abdul et al. (Hazardous Waste & Hazardous Materials 4(3):211–222, 1987) model. This implies a different organic matter composition and higher PAH sorption strength in cultivated topsoils, likely due to management effects including more rapid carbon turnover. Finally, we applied the recent Dexter et al. (Geoderma 144:620–627, 2008) theorem, and calculated the complexed organic carbon and non-complexed organic carbon fractions (COC and NCOC, grams per gram). Multiple regression analyses showed that the NCOC-based phenanthrene partition coefficient (KNCOC) could be markedly higher than the COCbased partition coefficient (KCOC) for soils with a clay/OC ratio <10. This possibly higher PAH sorption affinity to the NCOC fraction needs further investigations to develop more realistic and accurate models for PAH mobility and effects in the environment, also with regard to colloid-facilitated PAH transport.

Observability of nonlinear fractional dynamical systems

We study the observability of linear and nonlinear fractional differential systems of order 0 < α < 1 by using the Mittag-Leffler matrix function and the application of Banach’s contraction mapping theorem. Several examples illustrate the concepts.

Controllability results for impulsive mixed type functional integro-differential evolution equations with nonlocal conditions

In this paper, we establish the controllability for a class of abstract impulsive mixed-type functional integro-differential equations with finite delay in a Banach space. Some sufficient conditions for controllability are obtained by using the Mönch fixed point theorem via measures of noncompactness and semigroup theory. Particularly, we do not assume the compactness of the evolution system. An example is given to illustrate the effectiveness of our results.

Classification of some penalty methods

Optimization problems arise in science, engineering, economy, etc. and we need to find the best solutions for each reality. The methods used to solve these problems depend on several factors, including the amount and type of accessible information, the available algorithms for solving them, and, obviously, the intrinsic characteristics of the problem. There are many kinds of optimization problems and, consequently, many kinds of methods to solve them. When the involved functions are nonlinear and their derivatives are not known or are very difficult to calculate, these methods are more rare. These kinds of functions are frequently called black box functions. To solve such problems without constraints (unconstrained optimization), we can use direct search methods. These methods do not require any derivatives or approximations of them. But when the problem has constraints (nonlinear programming problems) and, additionally, the constraint functions are black box functions, it is much more difficult to find the most appropriate method. Penalty methods can then be used. They transform the original problem into a sequence of other problems, derived from the initial, all without constraints. Then this sequence of problems (without constraints) can be solved using the methods available for unconstrained optimization. In this chapter, we present a classification of some of the existing penalty methods and describe some of their assumptions and limitations. These methods allow the solving of optimization problems with continuous, discrete, and mixing constraints, without requiring continuity, differentiability, or convexity. Thus, penalty methods can be used as the first step in the resolution of constrained problems, by means of methods that typically are used by unconstrained problems. We also discuss a new class of penalty methods for nonlinear optimization, which adjust the penalty parameter dynamically.

Métodos de penalidade exacta para resolução de problemas de optimização não linear

In this work we present a classification of some of the existing Penalty Methods (denominated the Exact Penalty Methods) and describe some of its limitations and estimated. With these methods we can solve problems of optimization with continuous, discrete and mixing constrains, without requiring continuity, differentiability or convexity. The boarding consists of transforming the original problem, in a sequence of problems without constrains, derivate of the initial, making possible its resolution for the methods known for this type of problems. Thus, the Penalty Methods can be used as the first step for the resolution of constrained problems for methods typically used in by unconstrained problems. The work finishes discussing a new class of Penalty Methods, for nonlinear optimization, that adjust the penalty parameter dynamically.