Multi-agent Reinforcement Learning for Traffic Signal Control

Optimal control of traffic lights at junctions or traffic signal control (TSC) is essential for reducing the average delay experienced by the road users amidst the rapid increase in the usage of vehicles. In this paper, we formulate the TSC problem as a discounted cost Markov decision process (MDP) and apply multi-agent reinforcement learning (MARL) algorithms to obtain dynamic TSC policies. We model each traffic signal junction as an independent agent. An agent decides the signal duration of its phases in a round-robin (RR) manner using multi-agent Q-learning with either is an element of-greedy or UCB 3] based exploration strategies. It updates its Q-factors based on the cost feedback signal received from its neighbouring agents. This feedback signal can be easily constructed and is shown to be effective in minimizing the average delay of the vehicles in the network. We show through simulations over VISSIM that our algorithms perform significantly better than both the standard fixed signal timing (FST) algorithm and the saturation balancing (SAT) algorithm 15] over two real road networks.

Selective localisation of multi walled carbon nanotubes in polypropylene/natural rubber blends to reduce the percolation threshold

Polypropylene and natural rubber blends with multiwalled carbon nanotube (PP/NR + MWCNT nanocomposites) were prepared by melt mixing. The melt rheological behaviour of neat PP and PP/NR blends filled with different loadings (1, 3, 5, 7 wt%) of MWCNT was studied. The effect of PP/NR blends (with compositions, 80/20,50/50, 20/80 by wt) on the rheological percolation threshold was investigated. It was found that blending PP with NR (80/20 and 50/50 composition) reduced the rheological percolation threshold from 5 wt% to 3 wt% MWCNT. The melt rheological behaviour of the MWCNT filled PP/NR blends was correlated with the morphology observations from high resolution transmission electron microscopic (HRTEM) images. In predicting the thermodynamically favoured location of MWCNT in PP/NR blend, the specific interaction of phospholipids in NR phase with MWCNTs was considered quantitatively. The MWCNTs were selectively localised in the NR phase. The percolation mechanism in MWCNT filled PP/NR blends was discussed and for each blend composition, the percolation mechanism was found to be different. (C) 2015 Elsevier Ltd. All rights reserved.

Vacancy mediated clipping of multi-layered graphene: A precursor for 1, 2 and 3D carbon structures

Using first principles calculations, we show that the overlapping defects in bi-layer graphene (both AA-and AB-stacked) interact forming inter-layer covalent bonds, giving rise to two-dimensional (2D) clipped structures, without explicit use of functional groups. These clipped structures can be transformed into one-dimensional (1D) double wall nanotubes (DWCNT) or multi-layered three dimensional (3D) bulk structures. These clipped structures show good mechanical strength due to covalent bonding between multi-layers. Clipping also provides a unique way to simultaneously harness the conductivity of both walls of a double wall nanotube through covalently bonded scattering junctions. With additional conducting channels and improved mechanical stability, these clipped structures can lead to a myriad of applications in novel devices. (C) 2015 Elsevier Ltd. All rights reserved.

Spectral Clustering with Jensen-type kernels and their multi-point extensions

Motivated by multi-distribution divergences, which originate in information theory, we propose a notion of `multipoint' kernels, and study their applications. We study a class of kernels based on Jensen type divergences and show that these can be extended to measure similarity among multiple points. We study tensor flattening methods and develop a multi-point (kernel) spectral clustering (MSC) method. We further emphasize on a special case of the proposed kernels, which is a multi-point extension of the linear (dot-product) kernel and show the existence of cubic time tensor flattening algorithm in this case. Finally, we illustrate the usefulness of our contributions using standard data sets and image segmentation tasks.

COMPOSITE AND MULTI-SCALE COMPLIANT MECHANISMS FOR MANIPULATION AND MECHANICAL CHARACTERIZATION

In this paper, we integrate two or more compliant mechanisms to get enhanced functionality for manipulating and mechanically characterizing the grasped objects of varied size (cm to sub-mm), stiffness (1e5 to 10 N/m), and materials (cement to biological cells). The concepts of spring-lever (SL) model, stiffness maps, and non-dimensional kinetoelastostatic maps are used to design composite and multi-scale compliant mechanisms. Composite compliant mechanisms comprise two or more different mechanisms within a single elastic continuum while multi-scale ones possess the additional feature of substantial difference in the sizes of the mechanisms that are combined into one. We present three applications: (i) a composite compliant device to measure the failure load of the cement samples; (ii) a composite multi-scale compliant gripper to measure the bulk stiffness of zebrafish embryos; and (iii) a compliant gripper combined with a negative-stiffness element to reduce the overall stiffness. The prototypes of all three devices are made and tested. The cement sample needed a breaking force of 22.5 N; the zebrafish embryo is found to have bulk stiffness of about 10 N/m; and the stiffness of a compliant gripper was reduced by 99.8 % to 0.2 N/m.

Stimuli-responsive weak polyelectrolyte multilayer films: A thin film platform for self triggered multi-drug delivery

Polyelectrolyte multilayer (PEM) thin film composed of weak polyelectrolytes was designed by layer-by-layer (LbL) assembly of poly(allylamine hydrochloride) (PAH) and poly(methacrylic acid) (PMA) for multi-drug delivery applications. Environmental stimuli such as pH and ionic strength showed significant influence in changing the film morphology from pore-free smooth structure to porous structure and favored triggered release of loaded molecules. The film was successfully loaded with bovine serum albumin (BSA) and ciprofloxacin hydrochloride (CH) by modulating the porous polymeric network of the film. Release studies showed that the amount of release could be easily controlled by changing the environmental conditions such as pH and ionic strength. Sustained release of loaded molecules was observed up to 8 h. The fabricated films were found to be biocompatible with epithelial cells during in-vitro cell culture studies. PEM film reported here not only has the potential to be used as self-responding thin film platform for transdermal drug delivery, but also has the potential for further development in antimicrobial or anti-inflammatory coatings on implants and drug-releasing coatings for stents. (C) 2015 Elsevier B.V. All rights reserved.

A recursive multi-scaling approach to regional flood frequency analysis

Scaling approaches are widely used by hydrologists for Regional Frequency Analysis (RFA) of floods at ungauged/sparsely gauged site(s) in river basins. This paper proposes a Recursive Multi-scaling (RMS) approach to RFA that overcomes limitations of conventional simple- and multi-scaling approaches. The approach involves identification of a separate set of attributes corresponding to each of the sites (being considered in the study area/region) in a recursive manner according to their importance, and utilizing those attributes to construct effective regional regression relationships to estimate statistical raw moments (SMs) of peak flows. The SMs are then utilized to arrive at parameters of flood frequency distribution and quantile estimate(s) corresponding to target return period(s). Effectiveness of the RMS approach in arriving at flood quantile estimates for ungauged sites is demonstrated through leave-one-out cross-validation experiment on watersheds in Indiana State, USA. Results indicate that the approach outperforms index-flood based Region-of-Influence approach, simple- and multi-scaling approaches and a multiple linear regression method. (C) 2015 Elsevier B.V. All rights reserved.

Multi-dimensional Goodness-of-Fit Tests for Spectrum Sensing Based on Stochastic Distances

In this paper, we study two multi-dimensional Goodness-of-Fit tests for spectrum sensing in cognitive radios. The multi-dimensional scenario refers to multiple CR nodes, each with multiple antennas, that record multiple observations from multiple primary users for spectrum sensing. These tests, viz., the Interpoint Distance (ID) based test and the h, f distance based tests are constructed based on the properties of stochastic distances. The ID test is studied in detail for a single CR node case, and a possible extension to handle multiple nodes is discussed. On the other hand, the h, f test is applicable in a multi-node setup. A robustness feature of the KL distance based test is discussed, which has connections with Middleton's class A model. Through Monte-Carlo simulations, the proposed tests are shown to outperform the existing techniques such as the eigenvalue ratio based test, John's test, and the sphericity test, in several scenarios.

A fast and accurate performance analysis of beaconless IEEE 802.15.4 multi-hop networks

We develop an approximate analytical technique for evaluating the performance of multi-hop networks based on beaconless IEEE 802.15.4 ( the ``ZigBee'' PHY and MAC), a popular standard for wireless sensor networks. The network comprises sensor nodes, which generate measurement packets, relay nodes which only forward packets, and a data sink (base station). We consider a detailed stochastic process at each node, and analyse this process taking into account the interaction with neighbouring nodes via certain time averaged unknown variables (e.g., channel sensing rates, collision probabilities, etc.). By coupling the analyses at various nodes, we obtain fixed point equations that can be solved numerically to obtain the unknown variables, thereby yielding approximations of time average performance measures, such as packet discard probabilities and average queueing delays. The model incorporates packet generation at the sensor nodes and queues at the sensor nodes and relay nodes. We demonstrate the accuracy of our model by an extensive comparison with simulations. As an additional assessment of the accuracy of the model, we utilize it in an algorithm for sensor network design with quality-of-service (QoS) objectives, and show that designs obtained using our model actually satisfy the QoS constraints (as validated by simulating the networks), and the predictions are accurate to well within 10% as compared to the simulation results in a regime where the packet discard probability is low. (C) 2015 Elsevier B.V. All rights reserved.

Amine Functionalized polyaniline grafted to exfoliated graphite oxide: Synthesis, characterization and multi-element sensor studies

An amine functionalized polyaniline (AMPANI) derivative has been grafted onto exfoliated graphite oxide (EGO). The synthesis involved the in-situ chemical oxidative polymerization of functionalized aniline monomer in the presence of EGO with diaminobenzene acting as a bridging ligand to yield EGAMPANI. The synthesized compound was characterized by FT-IR and FT-Raman spectroscopy as well as thermogravimetric and X-ray diffraction analysis. The EGAMPANI was then used to modify a carbon paste electrode (CPE), which was applied for multi-elemental sensing of Pb2+, Cd2+ and Hg2+ ions using differential pulse anodic stripping voltammetty. The limits of detection achieved using the EGAMPANI modified CPE were 22 x 10(-6) M for Hg2+ ion, 1.2 x 10(-6) M for Cd2+ ion and 9.8 x 10(-7) M for Pb2+ ion. (C) 2015 Elsevier B.V. All rights reserved.

MULTI-INSTRUMENT DETECTION IN POLYPHONIC MUSIC USING GAUSSIAN MIXTURE BASED FACTORIAL HMM

We formulate the problem of detecting the constituent instruments in a polyphonic music piece as a joint decoding problem. From monophonic data, parametric Gaussian Mixture Hidden Markov Models (GM-HMM) are obtained for each instrument. We propose a method to use the above models in a factorial framework, termed as Factorial GM-HMM (F-GM-HMM). The states are jointly inferred to explain the evolution of each instrument in the mixture observation sequence. The dependencies are decoupled using variational inference technique. We show that the joint time evolution of all instruments' states can be captured using F-GM-HMM. We compare performance of proposed method with that of Student's-t mixture model (tMM) and GM-HMM in an existing latent variable framework. Experiments on two to five polyphony with 8 instrument models trained on the RWC dataset, tested on RWC and TRIOS datasets show that F-GM-HMM gives an advantage over the other considered models in segments containing co-occurring instruments.

Anthropogenic aerosol fraction over the Indian region: model simulations versus multi-satellite data analysis

Anthropogenic aerosols play a crucial role in our environment, climate, and health. Assessment of spatial and temporal variation in anthropogenic aerosols is essential to determine their impact. Aerosols are of natural and anthropogenic origin and together constitute a composite aerosol system. Information about either component needs elimination of the other from the composite aerosol system. In the present work we estimated the anthropogenic aerosol fraction (AF) over the Indian region following two different approaches and inter-compared the estimates. We espouse multi-satellite data analysis and model simulations (using the CHIMERE Chemical transport model) to derive natural aerosol distribution, which was subsequently used to estimate AF over the Indian subcontinent. These two approaches are significantly different from each other. Natural aerosol satellite-derived information was extracted in terms of optical depth while model simulations yielded mass concentration. Anthropogenic aerosol fraction distribution was studied over two periods in 2008: premonsoon (March-May) and winter (November-February) in regard to the known distinct seasonality in aerosol loading and type over the Indian region. Although both techniques have derived the same property, considerable differences were noted in temporal and spatial distribution. Satellite retrieval of AF showed maximum values during the pre-monsoon and summer months while lowest values were observed in winter. On the other hand, model simulations showed the highest concentration of AF in winter and the lowest during pre-monsoon and summer months. Both techniques provided an annual average AF of comparable magnitude (similar to 0.43 +/- 0.06 from the satellite and similar to 0.48 +/- 0.19 from the model). For winter months the model-estimated AF was similar to 0.62 +/- 0.09, significantly higher than that (0.39 +/- 0.05) estimated from the satellite, while during pre-monsoon months satellite-estimated AF was similar to 0.46 +/- 0.06 and the model simulation estimation similar to 0.53 +/- 0.14. Preliminary results from this work indicate that model-simulated results are nearer to the actual variation as compared to satellite estimation in view of general seasonal variation in aerosol concentrations.

Toward a High-Resolution Monitoring of Continental Surface Water Extent and Dynamics, at Global Scale: from GIEMS (Global Inundation Extent from Multi-Satellites) to SWOT (Surface Water Ocean Topography)

Up to now, high-resolution mapping of surface water extent from satellites has only been available for a few regions, over limited time periods. The extension of the temporal and spatial coverage was difficult, due to the limitation of the remote sensing technique e.g., the interaction of the radiation with vegetation or cloud for visible observations or the temporal sampling with the synthetic aperture radar (SAR)]. The advantages and the limitations of the various satellite techniques are reviewed. The need to have a global and consistent estimate of the water surfaces over long time periods triggered the development of a multi-satellite methodology to obtain consistent surface water all over the globe, regardless of the environments. The Global Inundation Extent from Multi-satellites (GIEMS) combines the complementary strengths of satellite observations from the visible to the microwave, to produce a low-resolution monthly dataset () of surface water extent and dynamics. Downscaling algorithms are now developed and applied to GIEMS, using high-spatial-resolution information from visible, near-infrared, and synthetic aperture radar (SAR) satellite images, or from digital elevation models. Preliminary products are available down to 500-m spatial resolution. This work bridges the gaps and prepares for the future NASA/CNES Surface Water Ocean Topography (SWOT) mission to be launched in 2020. SWOT will delineate surface water extent estimates and their water storage with an unprecedented spatial resolution and accuracy, thanks to a SAR in an interferometry mode. When available, the SWOT data will be adopted to downscale GIEMS, to produce a long time series of water surfaces at global scale, consistent with the SWOT observations.

Engineering a multi-biofunctional composite using poly(ethylenimine) decorated graphene oxide for bone tissue regeneration

Toward preparing strong multi-biofunctional materials, poly(ethylenimine) (PEI) conjugated graphene oxide (GO_PEI) was synthesized using poly(acrylic acid) (PAA) as a spacer and incorporated in poly( e-caprolactone) (PCL) at different fractions. GO_PEI significantly promoted the proliferation and formation of focal adhesions in human mesenchymal stem cells (hMSCs) on PCL. GO_PEI was highly potent in inducing stem cell osteogenesis leading to near doubling of alkaline phosphatase expression and mineralization over neat PCL with 5% filler content and was approximate to 50% better than GO. Remarkably, 5% GO_ PEI was as potent as soluble osteoinductive factors. Increased adsorption of osteogenic factors due to the amine and oxygen containing functional groups on GO_ PEI augment stem cell differentiation. GO_ PEI was also highly efficient in imparting bactericidal activity with 85% reduction in counts of E. coli colonies compared to neat PCL at 5% filler content and was more than twice as efficient as GO. This may be attributed to the synergistic effect of the sharp edges of the particles along with the presence of the different chemical moieties. Thus, GO_ PEI based polymer composites can be utilized to prepare bioactive resorbable biomaterials as an alternative to using labile biomolecules for fabricating orthopedic devices for fracture fixation and tissue engineering.

Multi-scale vulnerability assessment for adaptation planning

Vulnerability of communities and natural ecosystems, to potential impacts of climate change in developing countries like India, and the need for adaptation are rapidly emerging as central issues in the debate around policy responses to climate change. The present study presents an approach to identify and prioritize the most vulnerable districts, villages and households in Karnataka State, through a multi-scale assessment of inherent vulnerability to current climate variability. It also identifies the drivers of inherent vulnerability, thereby providing a tool for developing and mainstreaming adaptation strategies, in ongoing developmental or dedicated adaptation programmes. The multi-scale assessment was made for all 30 districts at the state level in Karnataka, about 1220 villages in Chikballapur district, and at the household level for two villages - Gundlapalli and Saddapalli - in Bagepalli taluk of Chikballapur district. At the district, village and household levels, low levels of education and skills are the dominant factors contributing to vulnerability. At the village and household level, the lack of income diversification and livelihood support institutions are key drivers of vulnerability. The approach of multi-scale vulnerability assessment facilitates identification and prioritization of the drivers of vulnerability at different scales, to focus adaptation interventions to address these drivers.