Feature Search in the Grassmanian in Online Reinforcement Learning

We consider the problem of finding the best features for value function approximation in reinforcement learning and develop an online algorithm to optimize the mean square Bellman error objective. For any given feature value, our algorithm performs gradient search in the parameter space via a residual gradient scheme and, on a slower timescale, also performs gradient search in the Grassman manifold of features. We present a proof of convergence of our algorithm. We show empirical results using our algorithm as well as a similar algorithm that uses temporal difference learning in place of the residual gradient scheme for the faster timescale updates.

An 8-to-1 bit 1-MS/s SAR ADC With VGA and Integrated Data Compression for Neural Recording

Low power consumption per channel and data rate minimization are two key challenges which need to be addressed in future generations of neural recording systems (NRS). Power consumption can be reduced by avoiding unnecessary processing whereas data rate is greatly decreased by sending spike time-stamps along with spike features as opposed to raw digitized data. Dynamic range in NRS can vary with time due to change in electrode-neuron distance or background noise, which demands adaptability. An analog-to-digital converter (ADC) is one of the most important blocks in a NRS. This paper presents an 8-bit SAR ADC in 0.13-mu m CMOS technology along with input and reference buffer. A novel energy efficient digital-to-analog converter switching scheme is proposed, which consumes 37% less energy than the present state-of-the-art. The use of a ping-pong input sampling scheme is emphasized for multichannel input to alleviate the bandwidth requirement of the input buffer. To reduce the data rate, the A/D process is only enabled through the in-built background noise rejection logic to ensure that the noise is not processed. The ADC resolution can be adjusted from 8 to 1 bit in 1-bit step based on the input dynamic range. The ADC consumes 8.8 mu W from 1 V supply at 1 MS/s speed. It achieves effective number of bits of 7.7 bits and FoM of 42.3 fJ/conversion-step.

Effect of TiN particulate reinforcement on corrosive behaviour of aluminium 6061 composites in chloride medium

In the present investigation, the corrosive behaviour of Al 6061-TiN particulate composites prepared by liquid metallurgy has been studied in chloride medium using electroanalytical techniques such as Tafel, cyclic polarization and electrochemical impedance spectroscopy (EIS). Surface morphology of the sample electrodes was examined using scanning electron micrography and energy dispersive X-ray methods. X-ray diffraction technique was used to confirm inclusion of TiN particulates in the matrix alloy and identify the alloying elements and intermetallic compounds in the Al 6061 composites. Polarization studies indicate an increase in the corrosion resistance in composites compared to the matrix alloy. EIS study reveals that the polarization resistance (R (p)) increases with increase in TiN content in composites, thus confirming improved corrosion resistance in composites. The observed decrease in corrosion rate in the case of composites is due to decoupling between TiN particles and Al 6061 alloy. It is understood that after the initiation of corrosion, interfacial corrosion products may have decoupled the conducting ceramic TiN from Al 6061 matrix alloy thus eliminating the galvanic effect between them.

On the absence of shear cracking and grain boundary cavitation in secondary tensile regions of Ti-6Al-4V-0.1B alloy during hot (alpha plus beta)-compression

Deformation instabilities, such as shear cracking and grain boundary cavitation, which are observed in the secondary tensile region of Ti-6Al-4V alloy during compressive deformation in the (+)-phase field, do not form in Ti-6Al-4V-0.1B alloy when processed under the same conditions. This has been attributed to the microstructural modifications, e.g. the absence of grain boundary and adjacent grain boundary retained layers and a lower proportion of 90(o)-misoriented -colonies that occur with boron addition.

Crystallographic texture and microstructure evolution during hot compression of Ti-6Al-4V-0.1B alloy in the (alpha plus beta)-regime

Microstructure and texture are known to undergo drastic modifications due to trace hypoeutectic boron addition (similar to 0.1wt.%) for various titanium alloys e.g. Ti-6Al-4V. The deformation behaviour of such an alloy Ti-6Al-4V-0.1B is investigated in the (+) phase field and compared against that of the base alloy Ti-6Al-4V studied under selfsame conditions. The deformation microstructures for the two alloys display bending and kinking of lamellae in near and softening via globularization of lamella in near phase regimes, respectively. The transition temperature at which pure slip based deformation changes to softening is lower for the boron added alloy. The presence of TiB particles is largely held attributable for the early softening of Ti-6Al-4V-0.1B alloy. The compression texture of both the alloys carry signature of pure phase defamation at lower temperature and phase transformation near the transus temperature. Texture is influenced by a complex interplay of the deformation and transformation processes in the intermediate temperature range. The contribution from phase transformation is prominent for Ti-6Al-4V-0.1B alloy at comparatively lower temperature.

Micropillar and macropillar compression responses of magnesium single crystals oriented for single slip or extension twinning

Uniaxial compression experiments were conducted on two magnesium (Mg) single crystals whose crystallographic orientations facilitate the deformation either by basal slip or by extension twinning. Specimen size effects were examined by conducting experiments on mu m- and mm-sized samples. A marked specimen size effect was noticed, with micropillars exhibiting significantly higher flow stress than bulk samples. Further, it is observed that the twin nucleation stress exerts strong size dependence, with micropillars requiring substantially higher stress than the bulk samples. The flow curves obtained on the bulk samples are smooth whereas those obtained from micropillars exhibit intermittent and precipitous stress drops. Electron backscattered diffraction and microstructural analyses of the deformed samples reveal that the plastic deformation in basal slip oriented crystals occurs only by slip while twin oriented crystals deform by both slip and twinning modes. The twin oriented crystals exhibit a higher strain hardening during plastic deformation when compared to the single slip oriented crystals. The strain hardening rate, theta, of twin oriented crystals is considerably greater in micropillars compared to the bulk single crystals, suggesting the prevalence of different work hardening mechanisms at these different sample sizes. (C) 2013 Acta Materialia Inc. Published by Elsevier Ltd. All rights reserved.

Analytical model for stress-strain response of plastic waste mixed soil

Recycling plastic water bottles has become one of the major challenges world wide. The present study provides an approach for the use of plastic waste as reinforcement material in soil, which can be used for ground improvement, subbases, and subgrade preparation in road construction. The experimental results are presented in the form of stress-strain-pore water pressure response and compression paths. On the basis of experimental test results, it is observed that the strength of soil is improved and compressibility reduced significantly with the addition of a small percentage of plastic waste to the soil. In this paper, an analytical model is proposed to evaluate the response of plastic waste mixed soil. It is noted that the model captures the stress-strain and pore water pressure response of all percentages of plastic waste adequately. The paper also provides a comparative study of failure stress obtained from different published models and the proposed model, which are compared with experimental results. The improvement in strength attributable to the inclusion of plastic waste can be advantageously used in ground improvement projects.

Microstructure Development, Nanomechanical, and Dynamic Compression Properties of Spark Plasma Sintered TiB2-Ti-Based Homogeneous and Bi-layered Composites

The growing threats due to increased use of small-caliber armor piercing projectiles demand the development of new light-weight body armor materials. In this context, TiB2 appears to be a promising ceramic material. However, poor sinterability and low fracture toughness remain two major issues for TiB2. In order to address these issues together, Ti as a sinter-aid is used to develop TiB2-(x wt pct Ti), (x = 10, 20) homogeneous composites and a bi-layered composite (BLC) with each layer having Ti content of 10 and 20 wt pct. The present study uniquely demonstrates the efficacy of two-stage spark plasma sintering route to develop dense TiB2-Ti composites with an excellent combination of nanoscale hardness (similar to 36 GPa) and indentation fracture toughness (similar to 12 MPa m(1/2)). In case of BLC, these properties are not compromised w.r.t. homogeneous composites, suggesting the retention of baseline material properties even in the bi-layer design due to optimal relief of residual stresses. The better indentation toughness of TiB2-(10 wt pct Ti) and TiB2-(20 wt pct Ti) composites can be attributed to the observed crack deflection/arrest, indicating better damage tolerance. Transmission electron microscope investigation reveals the presence of dense dislocation networks and deformation twins in alpha-Ti at the grain boundaries and triple pockets, surrounded by TiB2 grains. The dynamic strength of around 4 GPa has been measured using Split Hopkinson Pressure Bar tests in a reproducible manner at strain rates of the order of 600 s(-1). The damage progression under high strain rate has been investigated by acquiring real time images for the entire test duration using ultra-high speed imaging. An attempt has been made to establish microstructure-property correlation and a simple analysis based on Mohr-Coulomb theory is used to rationalize the measured strength properties.

Nano-ZnO particle addition to monolithic magnesium for enhanced tensile and compressive response

In this study, the effects of nanoscale ZnO reinforcement on the room temperature tensile and compressive response of monolithic Mg were studied. Experimental observations indicated strength properties improvement due to nanoscale ZnO addition. A maximum increment in tensile yield strength by similar to 55% and compressive yield strength by 90% (with reduced tension-compression asymmetry) was achieved when 0.8 vol.% ZnO nanoparticles were added to Mg. While the fracture strain values under tensile loads were found to increase significantly (by similar to 95%, in case of Mg-0.48ZnO), it remained largely unaffected under compressive loads. The microstructural characteristics studied in order to comprehend the mechanical response showed significant grain refinement due to grain boundary pinning effect of nano-ZnO particles which resulted in strengthening of Mg. Texture analysis using X-ray and EBSD methods indicated weakening of basal fibre texture in Mg/ZnO nanocomposites which contributed towards the reduction in tension-compression yield asymmetry and enhancement in tensile ductility when compared to pure Mg. (C) 2014 Elsevier B.V. All rights reserved.

MULTI-PITCH TRACKING USING GAUSSIAN MIXTURE MODEL WITH TIME VARYING PARAMETERS AND GRATING COMPRESSION TRANSFORM

Grating Compression Transform (GCT) is a two-dimensional analysis of speech signal which has been shown to be effective in multi-pitch tracking in speech mixtures. Multi-pitch tracking methods using GCT apply Kalman filter framework to obtain pitch tracks which requires training of the filter parameters using true pitch tracks. We propose an unsupervised method for obtaining multiple pitch tracks. In the proposed method, multiple pitch tracks are modeled using time-varying means of a Gaussian mixture model (GMM), referred to as TVGMM. The TVGMM parameters are estimated using multiple pitch values at each frame in a given utterance obtained from different patches of the spectrogram using GCT. We evaluate the performance of the proposed method on all voiced speech mixtures as well as random speech mixtures having well separated and close pitch tracks. TVGMM achieves multi-pitch tracking with 51% and 53% multi-pitch estimates having error <= 20% for random mixtures and all-voiced mixtures respectively. TVGMM also results in lower root mean squared error in pitch track estimation compared to that by Kalman filtering.

Adaptive Sleep-Wake Control using Reinforcement Learning in Sensor Networks

The aim in this paper is to allocate the `sleep time' of the individual sensors in an intrusion detection application so that the energy consumption from the sensors is reduced, while keeping the tracking error to a minimum. We propose two novel reinforcement learning (RL) based algorithms that attempt to minimize a certain long-run average cost objective. Both our algorithms incorporate feature-based representations to handle the curse of dimensionality associated with the underlying partially-observable Markov decision process (POMDP). Further, the feature selection scheme used in our algorithms intelligently manages the energy cost and tracking cost factors, which in turn assists the search for the optimal sleeping policy. We also extend these algorithms to a setting where the intruder's mobility model is not known by incorporating a stochastic iterative scheme for estimating the mobility model. The simulation results on a synthetic 2-d network setting are encouraging.

Mixed Mode Analysis of Obliquely Oriented Edge Cracked Semicircular Disk Under Compression

A new mixed-mode compression fracture specimen, obliquely oriented edge cracked semicircular disk (OECSD) is analyzed by extending pure opening mode configuration of edge cracked semicircular disk (ECSD) under Hertzian compression. Photoelastic experiments are conducted on two different specimens of OECSD of same size and different crack lengths and inclinations. Finite element method (FEM) is used to solve a number of cases of the problem varying crack length and crack inclination. FE results show a good match with experiments. Inclination of edge crack in OECSD can be so made as to obtain any mode-mixity ratio between zero and one and beyond for any crack length. The new specimen can be used for fracture testing under compression more conveniently than the existing ones in several ways.

Dynamic compression behavior of reactive spark plasma sintered ultrafine grained (Hf, Zr)B-2-SiC composites

This paper reports the dynamic compression behavior of ultrafine grained (Hf, Zr)B-2-SiC composites, sintered using reactive spark plasma sintering at 1600 degrees C for 10 min. Dynamic strength of similar to 2.3 GPa has been measured using Split Hopkinson Pressure Bar (SHPB) tests in a reproducible manner at strain rates of 800-1300 s(-1). A comparison with competing boride based armor ceramics, in reference to the spectrum of properties evaluated, establishes the potential of (Hf, Zr)B-2-SiC composites for armor applications. (C) 2015 Elsevier Ltd and Techna Group S.r.l. All rights reserved.

FRP-Confined Clay Brick Masonry Assemblages under Axial Compression: Experimental and Analytical Investigations

Unreinforced masonry (URM) structures that are in need of repair and rehabilitation constitute a significant portion of building stock worldwide. The successful application of fiber-reinforced polymers (FRP) for repair and retrofitting of reinforced-concrete (RC) structures has opened new avenues for strengthening URM structures with FRP materials. The present study analyzes the behavior of FRP-confined masonry prisms under monotonic axial compression. Masonry comprising of burnt clay bricks and cement-sand mortar (generally adopted in the Indian subcontinent) having E-b/E-m ratio less than one is employed in the study. The parameters considered in the study are, (1) masonry bonding pattern, (2) inclination of loading axis to the bed joint, (3) type of FRP (carbon FRP or glass FRP), and (4) grade of FRP fabric. The performance of FRP-confined masonry prisms is compared with unconfined masonry prisms in terms of compressive strength, modulus of elasticity and stress-strain response. The results showed an enhancement in compressive strength, modulus of elasticity, strain at peak stress, and ultimate strain for FRP-confined masonry prisms. The FRP confinement of masonry resulted in reducing the influence of the inclination of the loading axis to the bed joint on the compressive strength and failure pattern. Various analytical models available in the literature for the prediction of compressive strength of FRP-confined masonry are assessed. New coefficients are generated for the analytical model by appending experimental results of the current study with data available in the literature. (C) 2014 American Society of Civil Engineers.

Multi-agent Reinforcement Learning for Traffic Signal Control

Optimal control of traffic lights at junctions or traffic signal control (TSC) is essential for reducing the average delay experienced by the road users amidst the rapid increase in the usage of vehicles. In this paper, we formulate the TSC problem as a discounted cost Markov decision process (MDP) and apply multi-agent reinforcement learning (MARL) algorithms to obtain dynamic TSC policies. We model each traffic signal junction as an independent agent. An agent decides the signal duration of its phases in a round-robin (RR) manner using multi-agent Q-learning with either is an element of-greedy or UCB 3] based exploration strategies. It updates its Q-factors based on the cost feedback signal received from its neighbouring agents. This feedback signal can be easily constructed and is shown to be effective in minimizing the average delay of the vehicles in the network. We show through simulations over VISSIM that our algorithms perform significantly better than both the standard fixed signal timing (FST) algorithm and the saturation balancing (SAT) algorithm 15] over two real road networks.