Multi-agent Reinforcement Learning for Traffic Signal Control

Optimal control of traffic lights at junctions or traffic signal control (TSC) is essential for reducing the average delay experienced by the road users amidst the rapid increase in the usage of vehicles. In this paper, we formulate the TSC problem as a discounted cost Markov decision process (MDP) and apply multi-agent reinforcement learning (MARL) algorithms to obtain dynamic TSC policies. We model each traffic signal junction as an independent agent. An agent decides the signal duration of its phases in a round-robin (RR) manner using multi-agent Q-learning with either is an element of-greedy or UCB 3] based exploration strategies. It updates its Q-factors based on the cost feedback signal received from its neighbouring agents. This feedback signal can be easily constructed and is shown to be effective in minimizing the average delay of the vehicles in the network. We show through simulations over VISSIM that our algorithms perform significantly better than both the standard fixed signal timing (FST) algorithm and the saturation balancing (SAT) algorithm 15] over two real road networks.

A hybrid method for stochastic response analysis of a vibrating structure

Response analysis of a linear structure with uncertainties in both structural parameters and external excitation is considered here. When such an analysis is carried out using the spectral stochastic finite element method (SSFEM), often the computational cost tends to be prohibitive due to the rapid growth of the number of spectral bases with the number of random variables and the order of expansion. For instance, if the excitation contains a random frequency, or if it is a general random process, then a good approximation of these excitations using polynomial chaos expansion (PCE) involves a large number of terms, which leads to very high cost. To address this issue of high computational cost, a hybrid method is proposed in this work. In this method, first the random eigenvalue problem is solved using the weak formulation of SSFEM, which involves solving a system of deterministic nonlinear algebraic equations to estimate the PCE coefficients of the random eigenvalues and eigenvectors. Then the response is estimated using a Monte Carlo (MC) simulation, where the modal bases are sampled from the PCE of the random eigenvectors estimated in the previous step, followed by a numerical time integration. It is observed through numerical studies that this proposed method successfully reduces the computational burden compared with either a pure SSFEM of a pure MC simulation and more accurate than a perturbation method. The computational gain improves as the problem size in terms of degrees of freedom grows. It also improves as the timespan of interest reduces.

Influence of soil pedological properties on termite mound stability

This study investigated the influence of soil properties on the density and shape of epigeous fungus-growing termite nests in a dry deciduous forest in Karnataka, India. In this environment, Odontotermes obesus produces cathedral shaped mounds. Their density, shape (height and volume) and soil physicochemical properties were analyzed in ferralsol and vertisol environments. No significant difference was observed in O. obesus mound density (n = 2.7 mound ha(-1) on average in the vertisol and ferralsol areas). This study also showed that O. obesus has a limited effect on soil physical properties. No differences in soil particle size, pH, or the C:N ratio and base saturation were measured whereas the C and N contents were reduced and CEC was higher in termite nest soils in both environments. Clay mineralogical composition was also measured, and showed the presence of higher amounts of smectite clays in termite nest soils, which thus explained the increasing CEC despite the reduced C and N content. However, the main difference was the shape of the termite mounds. The degradation of the nests created a hillock of eroded soil at the base of termite mounds in the vertisol while only a thin layer of eroded soil was observed in the ferralsol. The increased degradation of termite mounds in the vertisol is explained by the presence of smectites (2:1 swelling clays), which confer macroscopic swelling and shrinking characteristics to the soil. Soil shrinkage during the dry season leads to the formation of deep cracks in the termite mounds that allow rain to rapidly penetrate inside the mound wall and then breakdown unstable aggregates. In conclusion, it appears that despite a similar abundance, termite mound properties depend to a large extent on the soil properties of their environments. (C) 2015 Elsevier B.V. All rights reserved.

Ray tracing based path-length calculations for polarized light tomographic imaging

A ray tracing based path length calculation is investigated for polarized light transport in a pixel space. Tomographic imaging using polarized light transport is promising for applications in optical projection tomography of small animal imaging and turbid media with low scattering. Polarized light transport through a medium can have complex effects due to interactions such as optical rotation of linearly polarized light, birefringence, diattenuation and interior refraction. Here we investigate the effects of refraction of polarized light in a non-scattering medium. This step is used to obtain the initial absorption estimate. This estimate can be used as prior in Monte Carlo (MC) program that simulates the transport of polarized light through a scattering medium to assist in faster convergence of the final estimate. The reflectance for p-polarized (parallel) and s-polarized (perpendicular) are different and hence there is a difference in the intensities that reach the detector end. The algorithm computes the length of the ray in each pixel along the refracted path and this is used to build the weight matrix. This weight matrix with corrected ray path length and the resultant intensity reaching the detector for each ray is used in the algebraic reconstruction (ART) method. The proposed method is tested with numerical phantoms for various noise levels. The refraction errors due to regions of different refractive index are discussed, the difference in intensities with polarization is considered. The improvements in reconstruction using the correction so applied is presented. This is achieved by tracking the path of the ray as well as the intensity of the ray as it traverses through the medium.

A Computational Study of the Factors Influencing the PVC-Triggering Ability of a Cluster of Early Afterdepolarization-Capable Myocytes

Premature ventricular complexes (PVCs), which are abnormal impulse propagations in cardiac tissue, can develop because of various reasons including early afterdepolarizations (EADs). We show how a cluster of EAD-generating cells (EAD clump) can lead to PVCs in a model of cardiac tissue, and also investigate the factors that assist such clumps in triggering PVCs. In particular, we study, through computer simulations, the effects of the following factors on the PVC-triggering ability of an EAD clump: (1) the repolarization reserve (RR) of the EAD cells; (2) the size of the EAD clump; (3) the coupling strength between the EAD cells in the clump; and (4) the presence of fibroblasts in the EAD clump. We find that, although a low value of RR is necessary to generate EADs and hence PVCs, a very low value of RR leads to low-amplitude EAD oscillations that decay with time and do not lead to PVCs. We demonstrate that a certain threshold size of the EAD clump, or a reduction in the coupling strength between the EAD cells, in the clump, is required to trigger PVCs. We illustrate how randomly distributed inexcitable obstacles, which we use to model collagen deposits, affect PVC-triggering by an EAD clump. We show that the gap-junctional coupling of fibroblasts with myocytes can either assist or impede the PVC-triggering ability of an EAD clump, depending on the resting membrane potential of the fibroblasts and the coupling strength between the myocyte and fibroblasts. We also find that the triggering of PVCs by an EAD clump depends sensitively on factors like the pacing cycle length and the distribution pattern of the fibroblasts.

Weakly corrected numerical solutions to stochastically driven nonlinear dynamical systems

A method to weakly correct the solutions of stochastically driven nonlinear dynamical systems, herein numerically approximated through the Eule-Maruyama (EM) time-marching map, is proposed. An essential feature of the method is a change of measures that aims at rendering the EM-approximated solution measurable with respect to the filtration generated by an appropriately defined error process. Using Ito's formula and adopting a Monte Carlo (MC) setup, it is shown that the correction term may be additively applied to the realizations of the numerically integrated trajectories. Numerical evidence, presently gathered via applications of the proposed method to a few nonlinear mechanical oscillators and a semi-discrete form of a 1-D Burger's equation, lends credence to the remarkably improved numerical accuracy of the corrected solutions even with relatively large time step sizes. (C) 2015 Elsevier Inc. All rights reserved.

Probing Photoexcited Carriers in a Few-Layer MoS2 Laminate by Time-Resolved Optical Pump-Terahertz Probe Spectroscopy

We report the dynamics of photoinduced carriers in a free-standing MoS2 laminate consisting of a few layers (1-6 layers) using time-resolved optical pump-terahertz probe spectroscopy. Upon photoexcitation with the 800 nm pump pulse, the terahertz conductivity increases due to absorption by the photoinduced charge carriers. The relaxation of the non-equilibrium carriers shows fast as well as slow decay channels, analyzed using a rate equation model incorporating defect-assisted Auger scattering of photoexcited electrons, holes, and excitons. The fast relaxation time occurs due to the capture of electrons and holes by defects via Auger processes, resulting in nonradiative recombination. The slower relaxation arises since the excitons are bound to the defects, preventing the defect-assisted Auger recombination of the electrons and the holes. Our results provide a comprehensive understanding of the non-equilibrium carrier kinetics in a system of unscreened Coulomb interactions, where defect-assisted Auger processes dominate and should be applicable to other 2D systems.

Investigations on structural, magnetic and electronic structure of Gd-doped ZnO nanostructures synthesized using sol-gel technique

GdxZn1-xO (x = 0, 0.02, 0.04 and 0.06) nanostructures have been synthesized using sol-gel technique and characterized to understand their structural and magnetic properties. X-ray diffraction (XRD) results show that Gd (0, 2, 4 and 6 %)-doped ZnO nanostructures crystallized in the wurtzite structure having space group C3(v) (P6(3)mc). Photoluminescence and Raman studies of Gd-doped ZnO powder show the formation of singly ionized oxygen vacancies. X-ray absorption spectroscopy reveals that Gd replaces the Zn atoms in the host lattice and maintains the crystal symmetry with slight lattice distortion. Gd L-3-edge spectra reveal charge transfer between Zn and Gd dopant ions. O K-edge spectra also depict the charge transfer through the oxygen bridge (Gd-O-Zn). Weak magnetic ordering is observed in all Gd-doped ZnO samples.

Isotopes in cosmochemistry: recipe for a Solar System

Extreme isotopic variations among extraterrestrial materials provide great insights into the origin and evolution of the Solar System. In this tutorial review, we summarize how the measurement of isotope ratios can expand our knowledge of the processes that took place before and during the formation of our Solar System and its subsequent early evolution. The continuous improvement of mass spectrometers with high precision and increased spatial resolution, including secondary ion mass spectrometry (SIMS), thermal ionization mass spectrometry (TIMS) and multi collector-inductively coupled plasma-mass spectrometry (MC-ICP-MS), along with the ever growing amounts of available extraterrestrial samples have significantly increased the temporal and spatial constraints on the sequence of events that took place since and before the formation of the first Solar System condensates (i.e., Ca-Al-rich inclusions). Grains sampling distinct stellar environments with a wide range of isotopic compositions were admixed to, but possibly not fully homogenized in, the Sun's parent molecular cloud or the nascent Solar System. Before, during and after accretion of the nebula, as well as the formation and subsequent evolution of planetesimals and planets, chemical and physical fractionation processes irrevocably changed the chemical and isotopic compositions of all Solar System bodies. Since the formation of the first Solar System minerals and rocks 4.568 Gyr ago, short-and long-lived radioactive decay and cosmic ray interaction also contributed to the modification of the isotopic framework of the Solar System, and permit to trace the formation and evolution of directly accessible and inferred planetary and stellar isotopic reservoirs.