Reinforcement learning with reference tracking control in continuous state spaces

The contribution described in this paper is an algorithm for learning nonlinear, reference tracking, control policies given no prior knowledge of the dynamical system and limited interaction with the system through the learning process. Concepts from the field of reinforcement learning, Bayesian statistics and classical control have been brought together in the formulation of this algorithm which can be viewed as a form of indirect self tuning regulator. On the task of reference tracking using a simulated inverted pendulum it was shown to yield generally improved performance on the best controller derived from the standard linear quadratic method using only 30 s of total interaction with the system. Finally, the algorithm was shown to work on the simulated double pendulum proving its ability to solve nontrivial control tasks. © 2011 IEEE.

Optimization of thermal material in a flux pump system with high temperature superconductor

Superconductors are known for the ability to trap magnetic field. A thermally actuated magnetization (TAM) flux pump is a system that utilizes the thermal material to generate multiple small magnetic pulses resulting in a high magnetization accumulated in the superconductor. Ferrites are a good thermal material candidate for the future TAM flux pumps because the relative permeability of ferrite changes significantly with temperature, particularly around the Curie temperature. Several soft ferrites have been specially synthesized to reduce the cost and improve the efficiency of the TAM flux pump. Various ferrite compositions have been tested under a temperature variation ranging from 77K to 300K. The experimental results of the synthesized soft ferrites-Cu 0.3 Zn 0.7Ti 0.04Fe 1.96O 4, including the Curie temperature, magnetic relative permeability and the volume magnetization (emu/cm3), are presented in this paper. The results are compared with original thermal material, gadolinium, used in the TAM flux pump system.-Cu 0.3 Zn 0.7Ti 0.04 Fe 1.96O 4 holds superior characteristics and is believed to be a suitable material for next generation TAM flux pump. © 2011 IEEE.

Construction site image retrieval based on material cluster recognition

The capability to automatically identify shapes, objects and materials from the image content through direct and indirect methodologies has enabled the development of several civil engineering related applications that assist in the design, construction and maintenance of construction projects. This capability is a product of the technological breakthroughs in the area of image processing that has allowed for the development of a large number of digital imaging applications in all industries. In this paper, an automated and content based construction site image retrieval method is presented. This method is based on image retrieval techniques, and specifically those related with material and object identification and matches known material samples with material clusters within the image content. The results demonstrate the suitability of this method for construction site image retrieval purposes and reveal the capability of existing image processing technologies to accurately identify a wealth of materials from construction site images.

Automated Detection of Exposed Reinforcement in Post-Earthquake Safety and Structural Evaluations

Post-earthquake structural safety evaluations are currently performed manually by a team of certified inspectors and/or structural engineers. This process is time-consuming and costly, keeping owners and occupants from returning to their businesses and homes. Automating these evaluations would enable faster, and potentially more consistent, relief and response processes. In order to do this, the detection of exposed reinforcing steel is of utmost significance. This paper presents a novel method of detecting exposed reinforcement in concrete columns for the purpose of advancing practices of structural and safety evaluation of buildings after earthquakes. Under this method, the binary image of the reinforcing area is first isolated using a state-of-the-art adaptive thresholding technique. Next, the ribbed regions of the reinforcement are detected by way of binary template matching. Finally, vertical and horizontal profiling are applied to the processed image in order to filter out any superfluous pixels and take into consideration the size of reinforcement bars in relation to that of the structural element within which they reside. The final result is the combined binary image disclosing only the regions containing rebar overlaid on top of the original image. The method is tested on a set of images from the January 2010 earthquake in Haiti. Preliminary test results convey that most exposed reinforcement could be properly detected in images of moderately-to-severely damaged concrete columns.

Automated Detection of Exposed Reinforcement in Post-Earthquake Safety and Structural Evaluations

The world is at the threshold of emerging technologies, where new systems in construction, materials, and civil and architectural design are poised to make the world better from a structural and construction perspective. Exciting developments, that are too many to name individually, take place yearly, affecting design considerations and construction practices. This edited book brings together modern methods and advances in structural engineering and construction, fulfilling the mission of ISEC Conferences, which is to enhance communication and understanding between structural and construction engineers for successful design and construction of engineering projects. The articles in this book are those accepted for publication and presentation at the 6th International Structural Engineering and Construction Conference in Zurich. The 6th ISEC Conference in Zurich, Switzerland, follows the overwhelming reception and success of previous ISEC conference in Las Vegas, USA in 2009; Melbourne, Australia in 2007; Shunan, Japan in 2005; Rome, Italy in 2003; and Honolulu, USA in 2001. Many topics are covered in this book, ranging from legal affairs and contracting, to innovations and risk analysis in infrastructure projects, analysis and design of structural systems, materials, architecture, and construction. The articles here are a lasting testimony to the excellent research being undertaken around the world. These articles provide a platform for the exchange of ideas, research efforts and networking in the structural engineering and construction communities. We congratulate and thank the authors for these articles that were selected after intensive peer-review, and our gratitude extends to all reviewers and members of the International Technical Committee. It is their combined contributions that have made this book a reality.

Material-Based Construction Site Image Retrieval

The technological advancements in digital imaging, the widespread popularity of digital cameras, and the increasing demand by owners and contractors for detailed and complete site photograph logs have triggered an ever-increasing growth in the rate of construction image data collection, with thousands of images being stored for each project. However, the sheer volume of images and the difficulties in accurately and manually indexing them have generated a pressing need for methods that can index and retrieve images with minimal or no user intervention. This paper reports recent developments from research efforts in the indexing and retrieval of construction site images in architecture, engineering, construction, and facilities management image database systems. The limitations and benefits of the existing methodologies will be presented, as well as an explanation of the reasons for the development of a novel image retrieval approach that not only can recognize construction materials within the image content in order to index images, but also can be compatible with existing retrieval methods, enabling enhanced results.

A high performance oxygen storage material for chemical looping processes with CO2 capture

Chemical looping combustion (CLC) is a novel combustion technology that involves cyclic reduction and oxidation of oxygen storage materials to provide oxygen for the combustion of fuels to CO2 and H2O, whilst giving a pure stream of CO2 suitable for sequestration or utilisation. Here, we report a method for preparing of oxygen storage materials from layered double hydroxides (LDHs) precursors and demonstrate their applications in the CLC process. The LDHs precursor enables homogeneous mixing of elements at the molecular level, giving a high degree of dispersion and high-loading of active metal oxide in the support after calcination. Using a Cu-Al LDH precursor as a prototype, we demonstrate that rational design of oxygen storage materials by material chemistry significantly improved the reactivity and stability in the high temperature redox cycles. We discovered that the presence of sodium-containing species were effective in inhibiting the formation of copper aluminates (CuAl2O4 or CuAlO 2) and stabilising the copper phase in an amorphous support over multiple redox cycles. A representative nanostructured Cu-based oxygen storage material derived from the LDH precursor showed stable gaseous O2 release capacity (∼5 wt%), stable oxygen storage capacity (∼12 wt%), and stable reaction rates during reversible phase changes between CuO-Cu 2O-Cu at high temperatures (800-1000 °C). We anticipate that the strategy can be extended to manufacture a variety of metal oxide composites for applications in novel high temperature looping cycles for clean energy production and CO2 capture. © The Royal Society of Chemistry 2013.

Material efficiency: providing material services with less material production.

Material efficiency, as discussed in this Meeting Issue, entails the pursuit of the technical strategies, business models, consumer preferences and policy instruments that would lead to a substantial reduction in the production of high-volume energy-intensive materials required to deliver human well-being. This paper, which introduces a Discussion Meeting Issue on the topic of material efficiency, aims to give an overview of current thinking on the topic, spanning environmental, engineering, economics, sociology and policy issues. The motivations for material efficiency include reducing energy demand, reducing the emissions and other environmental impacts of industry, and increasing national resource security. There are many technical strategies that might bring it about, and these could mainly be implemented today if preferred by customers or producers. However, current economic structures favour the substitution of material for labour, and consumer preferences for material consumption appear to continue even beyond the point at which increased consumption provides any increase in well-being. Therefore, policy will be required to stimulate material efficiency. A theoretically ideal policy measure, such as a carbon price, would internalize the externality of emissions associated with material production, and thus motivate change directly. However, implementation of such a measure has proved elusive, and instead the adjustment of existing government purchasing policies or existing regulations-- for instance to do with building design, planning or vehicle standards--is likely to have a more immediate effect.

Transitions to material efficiency in the UK steel economy.

Steel production is energy intensive so already has achieved impressive levels of energy efficiency. If the emissions associated with steel must be reduced in line with the requirements of the UK Climate Change Act, demand for new steel must be reduced. The strategies of 'material efficiency' aim to achieve such a reduction, while delivering the same final services. To meet the emissions targets set into UK law, UK consumption of steel must be reduced to 30 per cent of present levels by 2050. Previous work has revealed six strategies that could contribute to this target, and this paper presents an approximate analysis of the required transition. A macro-economic analysis of steel in the UK shows that while the steel industry is relatively small, the construction and manufacturing sectors are large, and it would be politically unacceptable to pursue options that lead to a major contraction in other sectors. Alternative business models are therefore required, and these are explored through four representative products--one for each final sector with particular emphasis given to options for reducing product weight, and extending product life. Preliminary evidence on the triggers that would lead to customers preferring these options is presented and organized in order to predict required policy measures. The estimated analysis of transitions explored in this paper is used to define target questions for future research in the area.

The roles of energy and material efficiency in meeting steel industry CO2 targets.

Identifying strategies for reducing greenhouse gas emissions from steel production requires a comprehensive model of the sector but previous work has either failed to consider the whole supply chain or considered only a subset of possible abatement options. In this work, a global mass flow analysis is combined with process emissions intensities to allow forecasts of future steel sector emissions under all abatement options. Scenario analysis shows that global capacity for primary steel production is already near to a peak and that if sectoral emissions are to be reduced by 50% by 2050, the last required blast furnace will be built by 2020. Emissions reduction targets cannot be met by energy and emissions efficiency alone, but deploying material efficiency provides sufficient extra abatement potential.

The incentives for supply chain collaboration to improve material efficiency in the use of steel: An analysis using input output techniques

In the face of increasing demand and limited emission reduction opportunities, the steel industry will have to look beyond its process emissions to bear its share of emission reduction targets. One option is to improve material efficiency - reducing the amount of metal required to meet services. In this context, the purpose of this paper is to explore why opportunities to improve material efficiency through upstream measures such as yield improvement and lightweighting might remain underexploited by industry. Established input-output techniques are applied to the GTAP 7 multi-regional input-output model to quantify the incentives for companies in key steel-using sectors (such as property developers and automotive companies) to seek opportunities to improve material efficiency in their upstream supply chains under different short-run carbon price scenarios. Because of the underlying assumptions, the incentives are interpreted as overestimates. The principal result of the paper is that these generous estimates of the incentives for material efficiency caused by a carbon price are offset by the disincentives to material efficiency caused by labour taxes. Reliance on a carbon price alone to deliver material efficiency would therefore be misguided and additional policy interventions to support material efficiency should be considered. © 2013 Elsevier B.V.

Thermal material with low curie temperature in a thermally actuated superconducting flux pump system

A thermally actuated flux pump is an efficient method to magnetize the high-temperature superconductor (HTS) bulk without applying a strong magnetic field. A thermal material is employed as a magnetic switch, which decides the efficiency of the system. To measure the Curie temperatures of those samples without destroying them, the nondestructive Curie temperature (NDT) measurement was developed. The Curie temperature of gadolinium (Gd) was measured by the NDT method and compared to the results from superconducting quantum interference device (SQUID). Because the SQUID tests require the sample to be cut into small piece, a constant shape of the testing sample could not be guaranteed. The demagnetizing effect was considered to remove the shape effect. The intrinsic permeability was modified from the apparent susceptibility by considering demagnetization. A thermal material with low Curie temperature, Mg 0.15Cu0.15Zn0.7Ti0.04Fe 1.96O4, was synthesized and its performance was tested and compared with previous thermal materials. Comparisons of three thermal materials, including the Curie temperature and the permeability, will be detailed in the paper. © 2002-2011 IEEE.

Dopamine and performance in a reinforcement learning task: evidence from Parkinson's disease.

The role dopamine plays in decision-making has important theoretical, empirical and clinical implications. Here, we examined its precise contribution by exploiting the lesion deficit model afforded by Parkinson's disease. We studied patients in a two-stage reinforcement learning task, while they were ON and OFF dopamine replacement medication. Contrary to expectation, we found that dopaminergic drug state (ON or OFF) did not impact learning. Instead, the critical factor was drug state during the performance phase, with patients ON medication choosing correctly significantly more frequently than those OFF medication. This effect was independent of drug state during initial learning and appears to reflect a facilitation of generalization for learnt information. This inference is bolstered by our observation that neural activity in nucleus accumbens and ventromedial prefrontal cortex, measured during simultaneously acquired functional magnetic resonance imaging, represented learnt stimulus values during performance. This effect was expressed solely during the ON state with activity in these regions correlating with better performance. Our data indicate that dopamine modulation of nucleus accumbens and ventromedial prefrontal cortex exerts a specific effect on choice behaviour distinct from pure learning. The findings are in keeping with the substantial other evidence that certain aspects of learning are unaffected by dopamine lesions or depletion, and that dopamine plays a key role in performance that may be distinct from its role in learning.