Reinforcement Learning Based on the Bayesian Theorem for Electricity Markets Decision Support

This paper presents the applicability of a reinforcement learning algorithm based on the application of the Bayesian theorem of probability. The proposed reinforcement learning algorithm is an advantageous and indispensable tool for ALBidS (Adaptive Learning strategic Bidding System), a multi-agent system that has the purpose of providing decision support to electricity market negotiating players. ALBidS uses a set of different strategies for providing decision support to market players. These strategies are used accordingly to their probability of success for each different context. The approach proposed in this paper uses a Bayesian network for deciding the most probably successful action at each time, depending on past events. The performance of the proposed methodology is tested using electricity market simulations in MASCEM (Multi-Agent Simulator of Competitive Electricity Markets). MASCEM provides the means for simulating a real electricity market environment, based on real data from real electricity market operators.

Experimental Reinforcement of Wood Beams

Rehabilitation is becoming more and more usual in the construction sector in Portugal. The introduction of newer construction materials and technical know-how of integrating different materials for achieving desired engineering goals is an important step to the development of the sector. Wood industry is also getting more and more adapted to composite technologies with the introduction of the so called “highly engineered wood products” and with the use of modification treatments. This work is an attempt to explain the viability of using stainless steel and glass fibre reinforced polymer (GFRP) as reinforcements in wood beams. This thesis specifically focuses on the flexural behaviour of Portuguese Pine unmodified and modified wood beams. Two types of modification were used: 1,3-dimethylol-4,5- dihydroxyethyleneurea (DMDHEU) resin and amid wax. The behaviour of the material was analysed with a nonlinear model. The latter model simulates the behaviour of the reinforced wood beams under flexural loading. Small-scale beams (1:15) were experimented in flexural bending and the experimental results obtained were compared with the analytical model results. The experiments confirm the viability of the reinforcing schemes and the working procedures. Experimental results showed fair agreement with the nonlinear model. A strength increase between 15% and 80% was achieved. Stiffness increased by 40% to 50% in beams reinforced with steel but no significant increase was achieved with the glass fibre reinforcement.

Hierarchical Reinforcement Learning in Behavior and the Brain

Dissertation presented to obtain the Ph.D degree in Biology, Neuroscience

Design formula for the flexural strengthening of RC beams using prestressed CFRP reinforcement

Applying a certain prestress level to the carbon fiber reinforced polymer (CFRP) reinforcement according to either externally bonded reinforcing (EBR) or near surface mounted (NSM) techniques can mobilize the strengthening potentialities of this high tensile strength composite material. For the prediction of the flexural behavior of reinforced concrete (RC) structures strengthened with prestressed EBR or NSM CFRPs, however, simplified analytical and design formulations still need to be developed as a guidance for engineers to design this type of strengthened structures by hand calculation without any programming help. Hence, the current work aims to briefly explain a developed simplified analytical approach, with a design framework, to predict the flexural behavior of RC beams flexurally strengthened with either prestressed EBR or NSM CFRP reinforcements. Moreover, an upper limit for the prestress level is proposed in order to optimize the ductility performance of the NSM prestressing technique. The good predictive performance of the analytical approaches was appraised by simulating the results of experimental programs composed of RC beams strengthened with prestressed NSM CFRP reinforcements.

High performance fiber reinforced concrete for the shear reinforcement: experimental and numerical research

High performance fiber reinforced concrete (HPFRC) is developing rapidly to a modern structural material with unique rheological and mechanical characteristics. Despite applying several methodologies to achieve self15 compacting requirements, some doubts still remain regarding the most convenient strategy for developing a HPFRC. In the present study, an innovative mix design method is proposed for the development of high17 performance concrete reinforced with a relatively high dosage of steel fibers. The material properties of the developed concrete are assessed, and the concrete structural behavior is characterized under compressive, flexural and shear loading. This study better clarifies the significant contribution of fibers for shear resistance of concrete elements. This paper further discusses a FEM-based simulation, aiming to address the possibility of calibrating the constitutive model parameters related to fracture modes I and II.

Assessment of the effectiveness of steel fibre reinforcement for the punching resistance of flat slabs by experimental research and design approach

The present paper deals with the experimental assessment of the effectiveness of steel fibre reinforcement in terms of punching resistance of centrically loaded flat slabs, and to the development of an analytical model capable of predicting the punching behaviour of this type of structures. For this purpose, eight slabs of 2550 x 2550 x 150 mm3 dimensions were tested up to failure, by investigating the influence of the content of steel fibres (0, 60, 75 and 90 kg/m3) and concrete strength class (50 and 70 MPa). Two reference slabs without fibre reinforcement, one for each concrete strength class, and one slab for each fibre content and each strength class compose the experimental program. All slabs were flexurally reinforced with a grid of ribbed steel bars in a percentage to assure punching failure mode for the reference slabs. Hooked ends steel fibres provided the unique shear reinforcement. The results have revealed that steel fibres are very effective in converting brittle punching failure into ductile flexural failure, by increasing both the ultimate load and deflection, as long as adequate fibre reinforcement is assured. An analytical model was developed based on the most recent concepts proposed by the fib Mode Code 2010 for predicting the punching resistance of flat slabs and for the characterization of the behaviour of fibre reinforced concrete. The most refined version of this model was capable of predicting the punching resistance of the tested slabs with excellent accuracy and coefficient of variation of about 5%.

Transfer zone of prestressed CFRP reinforcement applied according to NSM technique for strengthening of RC structures

This study presents an experimental program to assess the tensile strain distribution along prestressed carbon fiber reinforced polymer (CFRP) reinforcement flexurally applied on the tensile surface of RC beams according to near surface mounted (NSM) technique. Moreover, the current study aims to propose an analytical formulation, with a design framework, for the prediction of distribution of CFRP tensile strain and bond shear stress and, additionally, the prestress transfer length. After demonstration the good predictive performance of the proposed analytical approach, parametric studies were carried out to analytically evaluate the influence of the main material properties, and CFRP and groove cross section on the distribution of the CFRP tensile strain and bond shear stress, and on the prestress transfer length. The proposed analytical approach can also predict the evolution of the prestress transfer length during the curing time of the adhesive by considering the variation of its elasticity modulus during this period.

Natural fibers for soil reinforcement

The reinforcement of soil is defined as an effective and reliable technique to improve strength and stability. For this propose, the use of natural fibers has been commonly. Over the past years, a series of studies have been performed in order to investigate the influence of randomly oriented fibers, especially for compressible clayey soils. However, less attention has been given to the reinforcing of sandy materials, as well as the use of oriented fibers meshes in order to improve mechanical behaviour. The main aim of this study is to identify the influence that different percentages of fibers, as well as the use of meshes of oriented fibers, has on soil mechanical behaviour. For this purpose, unconfined compression tests with local strain measurements were performed on a silty sand reinforced with Sisal fibers and a comparative study between randomly oriented and 0° and 90° fibers is presented.

On the role of temporal context in human reinforcement learning

Magdeburg, Univ., Fak. für Naturwiss., Diss., 2011

Hybrid coordination of reinforcement learning-based behaviors for AUV control

This paper proposes a hybrid coordination method for behavior-based control architectures. The hybrid method takes advantages of the robustness and modularity in competitive approaches as well as optimized trajectories in cooperative ones. This paper shows the feasibility of applying this hybrid method with a 3D-navigation to an autonomous underwater vehicle (AUV). The behaviors are learnt online by means of reinforcement learning. A continuous Q-learning implemented with a feed-forward neural network is employed. Realistic simulations were carried out. The results obtained show the good performance of the hybrid method on behavior coordination as well as the convergence of the behaviors

A behavior-based scheme using reinforcement learning for autonomous underwater vehicles

This paper presents a hybrid behavior-based scheme using reinforcement learning for high-level control of autonomous underwater vehicles (AUVs). Two main features of the presented approach are hybrid behavior coordination and semi on-line neural-Q_learning (SONQL). Hybrid behavior coordination takes advantages of robustness and modularity in the competitive approach as well as efficient trajectories in the cooperative approach. SONQL, a new continuous approach of the Q_learning algorithm with a multilayer neural network is used to learn behavior state/action mapping online. Experimental results show the feasibility of the presented approach for AUVs

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Management of corn stalk waste as reinforcement for polypropylene injection moulded composites

The main objective of this study was the management of corn stalk waste as reinforcement for polypropylene (PP) injection moulded composites as an alternative to wood flour and fibers. In the first step, corn stalk waste was subjected to various treatments, and four different corn stalk derivatives (flour and fibers) able to be used as reinforcement of composite materials were prepared and characterized. These derivatives are corn stalk flour, thermo-mechanical, semi-chemical, and chemical fibers. They were characterized in terms of their yield, lignin content, Kappa number, fiber length/diameter ratio, fines, coarseness, viscosity, and the length at the break of a standard sheet of paper. Results showed that the corn stalk derivatives have different physico-chemical properties. In the second step, the prepared flour and fibers were explored as a reinforcing element for PP composites. Coupled and non-coupled PP composites were prepared and tested for tensile properties. For overall trend, with the addition of a coupling agent, tensile properties of composites significantly improved, as compared with non-coupled samples. In addition, a morphological study revealed the positive effect of the coupling agent on the interfacial bonding. The composites prepared with semichemical fiber gave better results in comparison with the rest of the corn stalk derivatives due to its chemical characteristics