An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.

Exploiting coherence for the simultaneous discovery of latent facets and associated sentiments

Facet-based sentiment analysis involves discovering the latent facets, sentiments and their associations. Traditional facet-based sentiment analysis algorithms typically perform the various tasks in sequence, and fail to take advantage of the mutual reinforcement of the tasks. Additionally,inferring sentiment levels typically requires domain knowledge or human intervention. In this paper, we propose aseries of probabilistic models that jointly discover latent facets and sentiment topics, and also order the sentiment topics with respect to a multi-point scale, in a language and domain independent manner. This is achieved by simultaneously capturing both short-range syntactic structure and long range semantic dependencies between the sentiment and facet words. The models further incorporate coherence in reviews, where reviewers dwell on one facet or sentiment level before moving on, for more accurate facet and sentiment discovery. For reviews which are supplemented with ratings, our models automatically order the latent sentiment topics, without requiring seed-words or domain-knowledge. To the best of our knowledge, our work is the first attempt to combine the notions of syntactic and semantic dependencies in the domain of review mining. Further, the concept of facet and sentiment coherence has not been explored earlier either. Extensive experimental results on real world review data show that the proposed models outperform various state of the art baselines for facet-based sentiment analysis.

Effect of frequency on seismic response of reinforced soil slopes in shaking table tests

This paper studies the effect of frequency of base shaking on the dynamic response of unreinforced and reinforced soil slopes through a series of shaking table tests. Slopes were constructed using clayey sand and geogrids were used for reinforcing the slopes. Two different slope angles 45 degrees and 60 degrees were used in tests and the quantity and location of reinforcement is varied in different tests. Acceleration of shaking is kept constant as 0.3 g in all the tests to maximize the response and the frequency of shaking was 2 Hz, 5 Hz and 7 Hz in different tests. The slope is instrumented with ultrasonic displacement sensors and accelerometers at different elevations. The response of different slopes is compared in terms of the deformation of the slope and acceleration amplifications measured at different elevations. It is observed that the displacements at all elevations increased with increase in frequency for all slopes, whereas the effect of frequency on acceleration amplifications is not significant for reinforced slopes. Results showed that the acceleration and displacement response is not increasing proportionately with the increase in the frequency, suggesting that the role of frequency in the seismic response is very important. Reinforced slopes showed lesser displacements compared to unreinforced slopes at all frequency levels. (C) 2012 Elsevier Ltd. All rights reserved.

Analysis of fatigue crack growth in RC beams

In this work, a fatigue crack propagation model developed using dimensional analysis for plain concrete is used in conjunction with the steel closing force to predict the crack growth behavior of reinforced concrete beams. A numerical procedure is followed using the proposed model to compute the fatigue life of RC beams and the dissipated energy in the steel reinforcement due to shake down behavior. Through a sensitivity study, it is found that the structural size is the most sensitive parameter on which the crack growth rate is dependent. Furthermore, the moment carrying capacity of an RC beam is computed as function of crack size by considering the effect of bond slip.

Modeling the dynamic response of wrap-faced reinforced soil retaining walls

This paper describes the development of a numerical model for simulating the shaking table tests on wrap-faced reinforced soil retaining walls. Some of the physical model tests carried out on reinforced soil retaining walls subjected to dynamic excitation through uniaxial shaking tests are briefly discussed. Models of retaining walls are constructed in a perspex box with geotextile reinforcement using the wraparound technique with dry sand backfill and instrumented with displacement sensors, accelerometers, and soil pressure sensors. Results showed that the displacements decrease with the increase in number of reinforcement layers, whereas acceleration amplifications were not affected significantly. Numerical modeling of these shaking table tests is carried out using the Fast Lagrangian Analysis of Continua program. The numerical model is validated by comparing the results with experiments on physical models. Responses of wrap-faced walls with varying numbers of reinforcement layers are compared. Sensitivity analysis performed on the numerical models showed that the friction and dilation angle of backfill material and stiffness properties of the geotextile-soil interface are the most affecting parameters for the model response.

GYROSONICS: SIGNATURE ANALYSIS AND REDUCED-ORDER MODELS

In this paper, the authors study the structure of a novel binaural sound with a certain phase and amplitude modulation and the response to this excitation when it is applied to natural rewarding circuit of human brain through auditory neural pathways. This novel excitation, also referred to as gyrosonic excitation in this work, has been found to have interesting effects such as stabilization effects on the left and right hemispheric brain signaling as captured by Galvanic Skin Resistance (GSR) measurements, control of cardiac rhythms (observed from ECG signals), mitigation of psychosomatic syndrome, and mitigation of migraine pain. Experimental data collected from human subjects are presented, and these data are examined to categorize the extent of systems disorder and reinforcement reward due to the gyrosonic stimulus. A multi-path reduced-order model has been developed to analyze the GSR signals. The filtered results are indicative of complicated reinforcing reward patterns due to the gyrosonic stimulation when it is used as a control input for patients with psychosomatic and cardiac disorders.

Effect of hybridizing micron-sized Ti with nano-sized SiC on the microstructural evolution and mechanical response of Mg-5.6Ti composite

In this work, the effect of hybridizing micro-Ti with nano-SiC particulates on the microstructural and the mechanical behaviour of Mg-5.6Ti composite were investigated. Mg materials containing micron-sized Ti particulates hybridized with different amounts of nano-size SiC particulates were synthesized using the disintegrated melt deposition method followed by hot extrusion. The microstructural and mechanical behaviour of the developed Mg hybrid composites were studied in comparison with Mg-5.6Ti. Microstructural characterization revealed grain refinement attributed to the presence of uniformly distributed micro-Ti particles embedded with nano-SiC particulates. Electron back scattered diffraction (EBSD) analyses of Mg-(5.6Ti + 1.0SiC)(BM) hybrid composite showed relatively more localized recrystallized grains and lesser tensile twin fraction, when compared to Mg-5.6Ti. The evaluation of mechanical properties indicated that the best combination of strength and ductility was observed in the Mg-(5.6Ti + 1.0SiC)(BM) hybrid composites. The superior strength properties of the Mg-(5.6Ti + x-SiC)(BM) hybrid composites when compared to Mg-5.6Ti is attributed to the presence of nano-reinforcements, the uniform distribution of the hybridized particles and the better interfacial bonding between the matrix and the reinforcement particles, achieved by nano-SiC addition.

Numerical analysis of geocell reinforced ballast overlying soft clay subgrade

Geotextiles and geogrids have been in use for several decades in variety of geo-structure applications including foundation of embankments, retaining walls, pavements. Geocells is one such variant in geosynthetic reinforcement of recent years, which provides a three dimensional confinement to the infill material. Although extensive research has been carried on geocell reinforced sand, clay and layered soil subgrades, limited research has been reported on the aggregates/ballast reinforced with geocells. This paper presents the behavior of a railway sleeper subjected to monotonic loading on geocell reinforced aggregates, of size ranging from 20 to 75 mm, overlying soft clay subgrades. Series of tests were conducted in a steel test tank of dimensions 700 mm x 300 mm x 700 mm. In addition to the laboratory model tests, numerical simulations were performed using a finite difference code to predict the behavior of geocell reinforced ballast. The results from numerical simulations were compared with the experimental data. The numerical and experimental results manifested the importance that the geocell reinforcement has a significant effect on the ballast behaviour. The results depicted that the stiffness of underlying soft clay subgrade has a significant influence on the behavior of the geocell-aggregate composite material in redistributing the loading system.

Bactericidal effect of silver-reinforced carbon nanotube and hydroxyapatite composites

Bacterial infection remains an important risk factor after orthopedic surgery. The present paper reports the synthesis of hydroxyapatite-silver (HA-Ag) and carbon nanotube-silver (CNT-Ag) composites via spark plasma sintering (SPS) route. The retention of the initial phases after SPS was confirmed by phase analysis using X-ray diffraction and Raman spectroscopy. Energy dispersive spectrum analysis showed that Ag was distributed uniformly in the CNT/HA matrix. The breakage of CNTs into spheroid particles at higher temperatures (1700 degrees C) is attributed to the Rayleigh instability criterion. Mechanical properties (hardness and elastic modulus) of the samples were evaluated using nanoindentation testing. Ag reinforcement resulted in the enhancement of hardness (by similar to 15%) and elastic modulus (similar to 5%) of HA samples, whereas Ag reinforcement in CNT, Ag addition does not have much effect on hardness (0.3 GPa) and elastic modulus (5 GPa). The antibacterial tests performed using Escherichia coli and Staphylococcus epidermidis showed significant decrease (by similar to 65-86%) in the number of adhered bacteria in HA/CNT composites reinforced with 5% Ag nanoparticles. Thus, Ag-reinforced HA/CNT can serve as potential antibacterial biocomposites.

Analysis of fatigue crack growth in reinforced concrete beams

Mechanical behavior of reinforced concrete members is influenced by the action of unknown crack bridging reactions of rebars. Under cyclic loading, due to progressive growth of cracks, this bridging action contributes to the overall strength, stiffness and hysteretic behavior of the member. In this work, fatigue behavior of reinforced concrete beams are studied using a crack propagation law, developed using dimensional analysis for plain concrete with the effect of reinforcement being simulated through constraint exerted on the crack opening. The parameters considered in the model are fracture toughness, crack length, loading ratio and structural size. A numerical procedure is followed to compute fatigue life of RC beams and the dissipated energy in the steel reinforcement due to the shake down phenomenon under cyclic loading. Through a sensitivity study, it is concluded that the structural size is the most sensitive parameter in the fatigue crack propagation phenomenon. Furthermore, the residual moment carrying capacity of an RC member is determined as a function of crack extension by including the bond-slip mechanism.

Microstructural evolution and mechanical properties of Mg composites containing nano-B4C hybridized micro-Ti particulates

In this work, the microstructural evolution and mechanical properties of extruded Mg composites containing micro-Ti particulates hybridized with varying contents of nano-B4C are investigated, and compared with Mg-5.6Ti. Microstructural characterization showed the presence of uniformly distributed micro-Ti particles embedded with nano-B4C particulates that resulted in significant grain refinement. Electron back scattered diffraction (EBSD) analyses of Mg-(5.6Ti + x-B4C)(BM) hybrid composites showed that the addition of hybridized particle resulted in relatively more recrystallized grains, realignment of basal planes and extension of weak basal fibre texture when compared to Mg-5.6Ti. The evaluation of mechanical properties indicated improved strength with ductility retention in Mg-(5.6Ti + x-B4C)(BM) hybrid composites. When compared to Mg-5.6Ti, the superior strength properties of the Mg-(5.6Ti + xB(4)C)(BM) hybrid composites are attributed to the presence of nano-reinforcements, the uniform distribution of the hybridized particles, better interfacial bonding between the matrix and the reinforcement particles and the matrix grain refinement achieved by nano-B4C addition. The ductility enhancement obtained in hybrid composites can be attributed to the fibre texture spread and favourable basal plane orientation achieved due to nano B4C addition. (C) 2013 Elsevier B.V. All rights reserved.

Target reliability-based optimisation for internal seismic stability of reinforced soil structures

In this paper, an approach for target component and system reliability-based design optimisation (RBDO) to evaluate safety for the internal seismic stability of geosynthetic-reinforced soil (GRS) structures is presented. Three modes of failure are considered: tension failure of the bottom-most layer of reinforcement, pullout failure of the topmost layer of reinforcement, and total pullout failure of all reinforcement layers. The analysis is performed by treating backfill properties, geometric and strength properties of reinforcement as random variables. The optimum number of reinforcement layers and optimum pullout length needed to maintain stability against tension failure, pullout failure and total pullout failure for different coefficients of variation of friction angle of the backfill, design strength of the reinforcement and horizontal seismic acceleration coefficients by targeting various system reliability indices are proposed. The results provide guidelines for the total length of reinforcement required, considering the variability of backfill as well as seismic coefficients. One illustrative example is presented to explain the evaluation of reliability for internal stability of reinforced soil structures using the proposed approach. In the second illustration (the stability of five walls), the Kushiro wall subjected to the Kushiro-Oki earthquake, the Seiken wall subjected to the Chiba-ken Toho-Oki earthquake, the Ta Kung wall subjected to the Ji-Ji earthquake, and the Gould and Valencia walls subjected to Northridge earthquake are re-examined.

Analytical model for stress-strain response of plastic waste mixed soil

Recycling plastic water bottles has become one of the major challenges world wide. The present study provides an approach for the use of plastic waste as reinforcement material in soil, which can be used for ground improvement, subbases, and subgrade preparation in road construction. The experimental results are presented in the form of stress-strain-pore water pressure response and compression paths. On the basis of experimental test results, it is observed that the strength of soil is improved and compressibility reduced significantly with the addition of a small percentage of plastic waste to the soil. In this paper, an analytical model is proposed to evaluate the response of plastic waste mixed soil. It is noted that the model captures the stress-strain and pore water pressure response of all percentages of plastic waste adequately. The paper also provides a comparative study of failure stress obtained from different published models and the proposed model, which are compared with experimental results. The improvement in strength attributable to the inclusion of plastic waste can be advantageously used in ground improvement projects.

The influence of mesoporous silica in low T-g cyclic olefin copolymer nanocomposite films: Mechanical and moisture barrier studies

In this study, mesoporous silica-cyclic olefin copolymer nanocomposite films were fabricated by solution casting. With an increase in silica loading, the stiffness of the matrix increased. The nanocomposite film shows increased strain to failure with moisture after aging by matrix plasticization. The storage modulus and loss factor for samples with silica content show better results compared with pristine polymer, as indicated by dynamic mechanical analysis. The interaction between filler-polymer chain exhibit hydrophobicity compared to the neat polymer. Water absorption studies at room temperature and near the T-g of the polymer (similar to 64 degrees C) were carried out. The nanocomposites up to 4 wt% filler reduces the water diffusion by forming hydrogen and chemical bonding. The result by calcium degradation test method for moisture permeability and Schottky structured organic device encapsulation under weathering condition confirms the effective reinforcement effect of silica particles in the matrix. (C) 2014 Elsevier Ltd. All rights reserved.