Impact of Bottom Flange Confinement Reinforcement on Performance of Prestressed Concrete Bridge Girders

For many years AASHTO provided no recommendation to state DOT’s on bottom flange confinement reinforcement for their bridge superstructures. The 1996 edition of AASHTO Standard Specification for Highway Bridges stated that nominal reinforcement be placed to enclose the prestressing steel from the end of the girder for at least a distance equal to the girder’s height. A few years later the 2004 AASHTO LRFD Bridge Design Specification changed the distance over which the confinement was to be distributed from 1.0h to 1.5h, and gave minimum requirements for the amount of steel to be used, No.3 bars, and their maximum spacing, not to exceed 6”. Research was undertaken to study what impact, if any, confinement reinforcement has on the performance of prestressed concrete bridge girders. Of particular interest was the effect confinement had on the transfer length, development length, and vertical shear capacity of the fore mentioned members. First, an analytical investigation was performed on the subject, and then an experimental investigation followed which consisted of designing, fabricating, and testing eight tee-girders and three NU1100 girders with particular attention paid to the amount and distribution of confinement reinforcement placed at the end of each girder. The results of the study show: 1) neither the amount or distribution of confinement reinforcement had a significant effect on the initial or final transfer length of the prestress strands; 2) at the AASHTO calculated development length, no significant impact from confinement was found on either the nominal flexural capacity of bridge girders or bond capacity of the prestressing steel; 3) the effects from varied confinement reinforcement on the shear resistance of girders tested was negligible, however, distribution of confinement did show to have an impact on the prestressed strands’ bond capacity; 4) confinement distribution across the entire girder did increase ductility and reduced cracking under extreme loading conditions.

Evaluation of Vickers hardness of different types of acrylic denture base resins with and without glass fibre reinforcement

Objective: To evaluate the Vickers hardness of different acrylic resins for denture bases with and without the addition of glass fibres. Background: It has been suggested that different polymerisation methods, as well as the addition of glass fibre (FV) might improve the hardness of acrylic. Materials and methods: Five types of acrylic resin were tested: Vipi Wave (VW), microwave polymerisation; Vipi Flash (VF), auto-polymerisation; Lucitone (LT), QC20 (QC) and Vipi Cril (VC), conventional heat-polymerisation, all with or without glass fibre reinforcement (GFR) and distributed into 10 groups (n = 12). Specimens were then submitted to Vickers hardness testing with a 25-g load for 30 s. All data were submitted to ANOVA and Tukey's HSD test. Results: A significant statistical difference was observed with regard to the polymerisation method and the GFR (p < 0.05). Without the GFR, the acrylic resin VC presented the highest hardness values, and VF and LT presented the lowest. In the presence of GFR, VC resin still presented the highest Vickers hardness values, and VF and QC presented the lowest. Conclusions: The acrylic resin VC and VW presented higher hardness values than VF and QC resins. Moreover, GFR increased the Vickers hardness of resins VW, VC and LT.

Modelling shared attention through relational reinforcement learning

Shared attention is a type of communication very important among human beings. It is sometimes reserved for the more complex form of communication being constituted by a sequence of four steps: mutual gaze, gaze following, imperative pointing and declarative pointing. Some approaches have been proposed in Human-Robot Interaction area to solve part of shared attention process, that is, the most of works proposed try to solve the first two steps. Models based on temporal difference, neural networks, probabilistic and reinforcement learning are methods used in several works. In this article, we are presenting a robotic architecture that provides a robot or agent, the capacity of learning mutual gaze, gaze following and declarative pointing using a robotic head interacting with a caregiver. Three learning methods have been incorporated to this architecture and a comparison of their performance has been done to find the most adequate to be used in real experiment. The learning capabilities of this architecture have been analyzed by observing the robot interacting with the human in a controlled environment. The experimental results show that the robotic head is able to produce appropriate behavior and to learn from sociable interaction.

Non-Dominated Sorting Genetic Algorithm Based on Reinforcement Learning to Optimization of Broad-Band Reflector Antennas Satellite

This paper aims to provide an improved NSGA-II (Non-Dominated Sorting Genetic Algorithm-version II) which incorporates a parameter-free self-tuning approach by reinforcement learning technique, called Non-Dominated Sorting Genetic Algorithm Based on Reinforcement Learning (NSGA-RL). The proposed method is particularly compared with the classical NSGA-II when applied to a satellite coverage problem. Furthermore, not only the optimization results are compared with results obtained by other multiobjective optimization methods, but also guarantee the advantage of no time-spending and complex parameter tuning.

Involvement of the basolateral complex and central nucleus of amygdala in the omission effects of different magnitudes of reinforcement

Evidence from appetitive Pavlovian and instrumental conditioning studies suggest that the amygdala is involved in modulation of responses correlated with motivational states, and therefore, to the modulation of processes probably underlying reinforcement omission effects. The present study aimed to clarify whether or not the mechanisms related to reinforcement omission effects of different magnitudes depend on basolateral complex and central nucleus of amygdala. Rats were trained on a fixed-interval 12 s with limited hold 6 s signaled schedule in which correct responses were always followed by one of two reinforcement magnitudes. Bilateral lesions of the basolateral complex and central nucleus were made after acquisition of stable performance. After postoperative recovery, the training was changed from 100% to 50% reinforcement schedules. The results showed that lesions of the basolateral complex and central nucleus did not eliminate or reduce, but interfere with reinforcement omission effects. The response from rats of both the basolateral complex and central nucleus lesioned group was higher relative to that of the rats of their respective sham-lesioned groups after reinforcement omission. Thus, the lesioned rats were more sensitive to the omission effect. Moreover, the basolateral complex lesions prevented the magnitude effect on reinforcement omission effects. Basolateral complex lesioned rats showed no differential performance following omission of larger and smaller reinforcement magnitude. Thus, the basolateral complex is involved in incentive processes relative to omission of different reinforcement magnitudes. Therefore, it is possible that reinforcement omission effects are modulated by brain circuitry which involves amygdala. (C) 2012 Elsevier B.V. All rights reserved.

Role of the amygdala in the reinforcement omission effect

The reinforcement omission effect (ROE) has been attributed to both motivational and attentional consequences of surprising reinforcement omission. Recent evidence suggests that the basolateral complex of the amygdala is involved in motivational components related to reinforcement value, whereas the central nucleus of the amygdala is involved in the processing of the attentional consequences of surprise. This study was designed to verify whether the mechanisms involved in the ROE depend on the integrity of either the basolateral amygdala complex or central nucleus of the amygdala. The ROE was evaluated in rats with lesions of either the central nucleus or basolateral complex of the amygdala and trained on a fixed-interval schedule procedure (Experiment 1) and fixed-interval with limited hold signaled schedule procedure (Experiment 2). The results of Experiment 1 showed that sham-operated rats and rats with lesions of either the central nucleus or basolateral area displayed the ROE. In contrast, in Experiment 2, subjects with lesions of the central nucleus or basolateral complex of the amygdala exhibited a smaller ROE compared with sham-operated subjects. Thus, the effects of selective lesions of amygdala subregions on the ROE in rats depended on the training procedure. Furthermore, the absence of differences between the lesioned groups in either experiment did not allow the dissociation of attentional or motivational components of the ROE with functions of specific areas of the amygdala. Thus, results did not show a functional double-dissociation between the central nucleus and basolateral area in the ROE.

Design of compression reinforcement in reinforced concrete membrane

This paper presents a method to design membrane elements of concrete with orthogonal mesh of reinforcement which are subject to compressive stress. Design methods, in general, define how to quantify the reinforcement necessary to support the tension stress and verify if the compression in concrete is within the strength limit. In case the compression in membrane is excessive, it is possible to use reinforcements subject to compression. However, there is not much information in the literature about how to design reinforcement for these cases. For that, this paper presents a procedure which uses the model based on Baumann's [1] criteria. The strength limits used herein are those recommended by CEB [3], however, a model is proposed in which this limit varies according to the tensile strain which occur perpendicular to compression. This resistance model is based on concepts proposed by Vecchio e Collins [2].

Aktionenlernen mit Selbstorganisierenden Karten und Reinforcement Learning

Die vorliegende Arbeit beschäftigt sich mit der Entwicklung eines Funktionsapproximators und dessen Verwendung in Verfahren zum Lernen von diskreten und kontinuierlichen Aktionen: 1. Ein allgemeiner Funktionsapproximator – Locally Weighted Interpolating Growing Neural Gas (LWIGNG) – wird auf Basis eines Wachsenden Neuralen Gases (GNG) entwickelt. Die topologische Nachbarschaft in der Neuronenstruktur wird verwendet, um zwischen benachbarten Neuronen zu interpolieren und durch lokale Gewichtung die Approximation zu berechnen. Die Leistungsfähigkeit des Ansatzes, insbesondere in Hinsicht auf sich verändernde Zielfunktionen und sich verändernde Eingabeverteilungen, wird in verschiedenen Experimenten unter Beweis gestellt. 2. Zum Lernen diskreter Aktionen wird das LWIGNG-Verfahren mit Q-Learning zur Q-LWIGNG-Methode verbunden. Dafür muss der zugrunde liegende GNG-Algorithmus abgeändert werden, da die Eingabedaten beim Aktionenlernen eine bestimmte Reihenfolge haben. Q-LWIGNG erzielt sehr gute Ergebnisse beim Stabbalance- und beim Mountain-Car-Problem und gute Ergebnisse beim Acrobot-Problem. 3. Zum Lernen kontinuierlicher Aktionen wird ein REINFORCE-Algorithmus mit LWIGNG zur ReinforceGNG-Methode verbunden. Dabei wird eine Actor-Critic-Architektur eingesetzt, um aus zeitverzögerten Belohnungen zu lernen. LWIGNG approximiert sowohl die Zustands-Wertefunktion als auch die Politik, die in Form von situationsabhängigen Parametern einer Normalverteilung repräsentiert wird. ReinforceGNG wird erfolgreich zum Lernen von Bewegungen für einen simulierten 2-rädrigen Roboter eingesetzt, der einen rollenden Ball unter bestimmten Bedingungen abfangen soll.

Reinforcement Lernen mit Regularisierungsnetzwerken

Die Arbeit behandelt das Problem der Skalierbarkeit von Reinforcement Lernen auf hochdimensionale und komplexe Aufgabenstellungen. Unter Reinforcement Lernen versteht man dabei eine auf approximativem Dynamischen Programmieren basierende Klasse von Lernverfahren, die speziell Anwendung in der Künstlichen Intelligenz findet und zur autonomen Steuerung simulierter Agenten oder realer Hardwareroboter in dynamischen und unwägbaren Umwelten genutzt werden kann. Dazu wird mittels Regression aus Stichproben eine Funktion bestimmt, die die Lösung einer "Optimalitätsgleichung" (Bellman) ist und aus der sich näherungsweise optimale Entscheidungen ableiten lassen. Eine große Hürde stellt dabei die Dimensionalität des Zustandsraums dar, die häufig hoch und daher traditionellen gitterbasierten Approximationsverfahren wenig zugänglich ist. Das Ziel dieser Arbeit ist es, Reinforcement Lernen durch nichtparametrisierte Funktionsapproximation (genauer, Regularisierungsnetze) auf -- im Prinzip beliebig -- hochdimensionale Probleme anwendbar zu machen. Regularisierungsnetze sind eine Verallgemeinerung von gewöhnlichen Basisfunktionsnetzen, die die gesuchte Lösung durch die Daten parametrisieren, wodurch die explizite Wahl von Knoten/Basisfunktionen entfällt und so bei hochdimensionalen Eingaben der "Fluch der Dimension" umgangen werden kann. Gleichzeitig sind Regularisierungsnetze aber auch lineare Approximatoren, die technisch einfach handhabbar sind und für die die bestehenden Konvergenzaussagen von Reinforcement Lernen Gültigkeit behalten (anders als etwa bei Feed-Forward Neuronalen Netzen). Allen diesen theoretischen Vorteilen gegenüber steht allerdings ein sehr praktisches Problem: der Rechenaufwand bei der Verwendung von Regularisierungsnetzen skaliert von Natur aus wie O(n**3), wobei n die Anzahl der Daten ist. Das ist besonders deswegen problematisch, weil bei Reinforcement Lernen der Lernprozeß online erfolgt -- die Stichproben werden von einem Agenten/Roboter erzeugt, während er mit der Umwelt interagiert. Anpassungen an der Lösung müssen daher sofort und mit wenig Rechenaufwand vorgenommen werden. Der Beitrag dieser Arbeit gliedert sich daher in zwei Teile: Im ersten Teil der Arbeit formulieren wir für Regularisierungsnetze einen effizienten Lernalgorithmus zum Lösen allgemeiner Regressionsaufgaben, der speziell auf die Anforderungen von Online-Lernen zugeschnitten ist. Unser Ansatz basiert auf der Vorgehensweise von Recursive Least-Squares, kann aber mit konstantem Zeitaufwand nicht nur neue Daten sondern auch neue Basisfunktionen in das bestehende Modell einfügen. Ermöglicht wird das durch die "Subset of Regressors" Approximation, wodurch der Kern durch eine stark reduzierte Auswahl von Trainingsdaten approximiert wird, und einer gierigen Auswahlwahlprozedur, die diese Basiselemente direkt aus dem Datenstrom zur Laufzeit selektiert. Im zweiten Teil übertragen wir diesen Algorithmus auf approximative Politik-Evaluation mittels Least-Squares basiertem Temporal-Difference Lernen, und integrieren diesen Baustein in ein Gesamtsystem zum autonomen Lernen von optimalem Verhalten. Insgesamt entwickeln wir ein in hohem Maße dateneffizientes Verfahren, das insbesondere für Lernprobleme aus der Robotik mit kontinuierlichen und hochdimensionalen Zustandsräumen sowie stochastischen Zustandsübergängen geeignet ist. Dabei sind wir nicht auf ein Modell der Umwelt angewiesen, arbeiten weitestgehend unabhängig von der Dimension des Zustandsraums, erzielen Konvergenz bereits mit relativ wenigen Agent-Umwelt Interaktionen, und können dank des effizienten Online-Algorithmus auch im Kontext zeitkritischer Echtzeitanwendungen operieren. Wir demonstrieren die Leistungsfähigkeit unseres Ansatzes anhand von zwei realistischen und komplexen Anwendungsbeispielen: dem Problem RoboCup-Keepaway, sowie der Steuerung eines (simulierten) Oktopus-Tentakels.

Life cycle assessment in construction field: A life cycle cost analysis of reinforcement concrete bridge

The present work is included in the context of the assessment of sustainability in the construction field and is aimed at estimating and analyzing life cycle cost of the existing reinforced concrete bridge “Viadotto delle Capre” during its entire life. This was accomplished by a comprehensive data collection and results evaluation. In detail, the economic analysis of the project is performed. The work has investigated possible design alternatives for maintenance/rehabilitation and end-of-life operations, when structural, functional, economic and also environmental requirements have to be fulfilled. In detail, the economic impact of different design options for the given reinforced concrete bridge have been assessed, whereupon the most economically, structurally and environmentally efficient scenario was chosen. The Integrated Life-Cycle Analysis procedure and Environmental Impact Assessment were also discussed in this work. The scope of this thesis is to illustrate that Life Cycle Cost analysis as part of Life Cycle Assessment approach could be effectively used to drive the design and management strategy of new and existing structures. The final objective of this contribution is to show how an economic analysis can influence decision-making in the definition of the most sustainable design alternatives. The designers can monitor the economic impact of different design strategies in order to identify the most appropriate option.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Gradient estimation in dendritic reinforcement learning

We study synaptic plasticity in a complex neuronal cell model where NMDA-spikes can arise in certain dendritic zones. In the context of reinforcement learning, two kinds of plasticity rules are derived, zone reinforcement (ZR) and cell reinforcement (CR), which both optimize the expected reward by stochastic gradient ascent. For ZR, the synaptic plasticity response to the external reward signal is modulated exclusively by quantities which are local to the NMDA-spike initiation zone in which the synapse is situated. CR, in addition, uses nonlocal feedback from the soma of the cell, provided by mechanisms such as the backpropagating action potential. Simulation results show that, compared to ZR, the use of nonlocal feedback in CR can drastically enhance learning performance. We suggest that the availability of nonlocal feedback for learning is a key advantage of complex neurons over networks of simple point neurons, which have previously been found to be largely equivalent with regard to computational capability.