Adaptive Sleep-Wake Control using Reinforcement Learning in Sensor Networks

The aim in this paper is to allocate the `sleep time' of the individual sensors in an intrusion detection application so that the energy consumption from the sensors is reduced, while keeping the tracking error to a minimum. We propose two novel reinforcement learning (RL) based algorithms that attempt to minimize a certain long-run average cost objective. Both our algorithms incorporate feature-based representations to handle the curse of dimensionality associated with the underlying partially-observable Markov decision process (POMDP). Further, the feature selection scheme used in our algorithms intelligently manages the energy cost and tracking cost factors, which in turn assists the search for the optimal sleeping policy. We also extend these algorithms to a setting where the intruder's mobility model is not known by incorporating a stochastic iterative scheme for estimating the mobility model. The simulation results on a synthetic 2-d network setting are encouraging.

Multi-agent Reinforcement Learning for Traffic Signal Control

Optimal control of traffic lights at junctions or traffic signal control (TSC) is essential for reducing the average delay experienced by the road users amidst the rapid increase in the usage of vehicles. In this paper, we formulate the TSC problem as a discounted cost Markov decision process (MDP) and apply multi-agent reinforcement learning (MARL) algorithms to obtain dynamic TSC policies. We model each traffic signal junction as an independent agent. An agent decides the signal duration of its phases in a round-robin (RR) manner using multi-agent Q-learning with either is an element of-greedy or UCB 3] based exploration strategies. It updates its Q-factors based on the cost feedback signal received from its neighbouring agents. This feedback signal can be easily constructed and is shown to be effective in minimizing the average delay of the vehicles in the network. We show through simulations over VISSIM that our algorithms perform significantly better than both the standard fixed signal timing (FST) algorithm and the saturation balancing (SAT) algorithm 15] over two real road networks.

Biocompatibility property of 100% strontium-substituted SiO2-Al2O3-P2O5-CaO-CaF2 glass ceramics over 26 weeks implantation in rabbit model: Histology and micro-Computed Tomography analysis

One of the desired properties for any new biomaterial composition is its long-term stability in a suitable animal model and such property cannot be appropriately assessed by performing short-term implantation studies. While hydroxyapatite (HA) or bioglass coated metallic biomaterials are being investigated for in vivo biocompatibility properties, such study is not extensively being pursued for bulk glass ceramics. In view of their inherent brittle nature, the implant stability as well as impact of long-term release of metallic ions on bone regeneration have been a major concern. In this perspective, the present article reports the results of the in vivo implantation experiments carried out using 100% strontium (Sr)-substituted glass ceramics with the nominal composition of 4.5 SiO2-3Al(2)O(3)-1.5P(2)O(5)-3SrO-2SrF(2) for 26 weeks in cylindrical bone defects in rabbit model. The combination of histological and micro-computed tomography analysis provided a qualitative and quantitative understanding of the bone regeneration around the glass ceramic implants in comparison to the highly bioactive HA bioglass implants (control). The sequential polychrome labeling of bone during in vivo osseointegration using three fluorochromes followed by fluorescence microscopy observation confirmed homogeneous bone formation around the test implants. The results of the present study unequivocally confirm the long-term implant stability as well as osteoconductive property of 100% Sr-substituted glass ceramics, which is comparable to that of a known bioactive implant, that is, HA-based bioglass. (c) 2014 Wiley Periodicals, Inc. J Biomed Mater Res Part B: Appl Biomater, 103B: 1168-1179, 2015.

Accelerating Information Diffusion in Social Networks Under the Susceptible-Infected-Susceptible Epidemic Model

Standard Susceptible-Infected-Susceptible (SIS) epidemic models assume that a message spreads from the infected to the susceptible nodes due to only susceptible-infected epidemic contact. We modify the standard SIS epidemic model to include direct recruitment of susceptible individuals to the infected class at a constant rate (independent of epidemic contacts), to accelerate information spreading in a social network. Such recruitment can be carried out by placing advertisements in the media. We provide a closed form analytical solution for system evolution in the proposed model and use it to study campaigning in two different scenarios. In the first, the net cost function is a linear combination of the reward due to extent of information diffusion and the cost due to application of control. In the second, the campaign budget is fixed. Results reveal the effectiveness of the proposed system in accelerating and improving the extent of information diffusion. Our work is useful for devising effective strategies for product marketing and political/social-awareness/crowd-funding campaigns that target individuals in a social network.

Impact of lifetime control on the threshold of quantum dot lasers

Despite significant improvements in their properties as emitters, colloidal quantum dots have not had much success in emerging as suitable materials for laser applications. Gain in most colloidal systems is short lived, and needs to compete with biexcitonic decay. This has necessitated the use of short pulsed lasers to pump quantum dots to thresholds needed for amplified spontaneous emission or lasing. Continuous wave pumping of gain that is possible in some inorganic phosphors has therefore remained a very distant possibility for quantum dots. Here, we demonstrate that trilayer heterostructures could provide optimal conditions for demonstration of continuous wave lasing in colloidal materials. The design considerations for these materials are discussed in terms of a kinetic model. The electronic structure of the proposed dot architectures is modeled within effective mass theory.

Effect of combined treatment with zoledronic acid and propranolol on mechanical strength in an rat model of disuse osteoporosis

Objectives: A model that uses right hind-limb unloading of rats is used to study the consequences of skeletal unloading during various conditions like space flights and prolonged bed rest in elderly. This study was aimed to investigate the additive effects of antiresorptive agent zoledronic acid (ZOL), alone and in combination with propranolol (PRO) in a rat model of disuse osteoporosis. Methods: In the present study, 3-month-old male Wistar rats had their right hind-limb immobilized (RHLI) for 10 weeks to induce osteopenia, then were randomized into four groups: 1-RHLI positive control, 2-RHLI plus ZOL (50 mu g/kg, i.v. single dose), 3-RHLI plus PRO (0.1 mg/kg, s.c. 5 days per week), 4-RHLI plus PRO (0.1 mg/kg, s.c. 5 days per week) plus ZOL (50 mu g/kg, i.v. single dose) for another 10 weeks. One group of non-immobilized rats was used as negative control. At the end of treatment, the femurs were removed and tested for bone porosity, bone mechanical properties, and bone dry and ash weight. Results: With respect to improvement in the mechanical strength of the femoral mid-shaft, the combination treatment with ZOL plus PRO was more effective than ZOL or PRO monotherapy. Moreover, combination therapy using ZOL plus PRO was more effective in improving dry bone weight and preserved the cortical bone porosity better than monotherapy using ZOL or PRO in right hind-limb immobilized rats. Conclusions: These data suggest that this combined treatment with ZOL plus PRO should be recommended for the treatment of disuse osteoporosis. (C) 2014 Elsevier Editora Ltda. All rights reserved.

Use of polydispersity index as control parameter to study melting/freezing of Lennard-Jones system: Comparison among predictions of bifurcation theory with Lindemann criterion, inherent structure analysis and Hansen-Verlet rule

Using polydispersity index as an additional order parameter we investigate freezing/melting transition of Lennard-Jones polydisperse systems (with Gaussian polydispersity in size), especially to gain insight into the origin of the terminal polydispersity. The average inherent structure (IS) energy and root mean square displacement (RMSD) of the solid before melting both exhibit quite similar polydispersity dependence including a discontinuity at solid-liquid transition point. Lindemann ratio, obtained from RMSD, is found to be dependent on temperature. At a given number density, there exists a value of polydispersity index (delta (P)) above which no crystalline solid is stable. This transition value of polydispersity(termed as transition polydispersity, delta (P) ) is found to depend strongly on temperature, a feature missed in hard sphere model systems. Additionally, for a particular temperature when number density is increased, delta (P) shifts to higher values. This temperature and number density dependent value of delta (P) saturates surprisingly to a value which is found to be nearly the same for all temperatures, known as terminal polydispersity (delta (TP)). This value (delta (TP) similar to 0.11) is in excellent agreement with the experimental value of 0.12, but differs from hard sphere transition where this limiting value is only 0.048. Terminal polydispersity (delta (TP)) thus has a quasiuniversal character. Interestingly, the bifurcation diagram obtained from non-linear integral equation theories of freezing seems to provide an explanation of the existence of unique terminal polydispersity in polydisperse systems. Global bond orientational order parameter is calculated to obtain further insights into mechanism for melting.

A mean-reverting stochastic model for the political business cycle

In this article, we look at the political business cycle problem through the lens of uncertainty. The feedback control used by us is the famous NKPC with stochasticity and wage rigidities. We extend the New Keynesian Phillips Curve model to the continuous time stochastic set up with an Ornstein-Uhlenbeck process. We minimize relevant expected quadratic cost by solving the corresponding Hamilton-Jacobi-Bellman equation. The basic intuition of the classical model is qualitatively carried forward in our set up but uncertainty also plays an important role in determining the optimal trajectory of the voter support function. The internal variability of the system acts as a base shifter for the support function in the risk neutral case. The role of uncertainty is even more prominent in the risk averse case where all the shape parameters are directly dependent on variability. Thus, in this case variability controls both the rates of change as well as the base shift parameters. To gain more insight we have also studied the model when the coefficients are time invariant and studied numerical solutions. The close relationship between the unemployment rate and the support function for the incumbent party is highlighted. The role of uncertainty in creating sampling fluctuation in this set up, possibly towards apparently anomalous results, is also explored.

Campaigning in Heterogeneous Social Networks: Optimal Control of SI Information Epidemics

We study the optimal control problem of maximizing the spread of an information epidemic on a social network. Information propagation is modeled as a susceptible-infected (SI) process, and the campaign budget is fixed. Direct recruitment and word-of-mouth incentives are the two strategies to accelerate information spreading (controls). We allow for multiple controls depending on the degree of the nodes/individuals. The solution optimally allocates the scarce resource over the campaign duration and the degree class groups. We study the impact of the degree distribution of the network on the controls and present results for Erdos-Renyi and scale-free networks. Results show that more resource is allocated to high-degree nodes in the case of scale-free networks, but medium-degree nodes in the case of Erdos-Renyi networks. We study the effects of various model parameters on the optimal strategy and quantify the improvement offered by the optimal strategy over the static and bang-bang control strategies. The effect of the time-varying spreading rate on the controls is explored as the interest level of the population in the subject of the campaign may change over time. We show the existence of a solution to the formulated optimal control problem, which has nonlinear isoperimetric constraints, using novel techniques that is general and can be used in other similar optimal control problems. This work may be of interest to political, social awareness, or crowdfunding campaigners and product marketing managers, and with some modifications may be used for mitigating biological epidemics.

Optimal Guidance for Accurate Lunar Soft Landing with Minimum Fuel Consumption using Model Predictive Static Programming

In this paper the soft lunar landing with minimum fuel expenditure is formulated as a nonlinear optimal guidance problem. The realization of pinpoint soft landing with terminal velocity and position constraints is achieved using Model Predictive Static Programming (MPSP). The high accuracy of the terminal conditions is ensured as the formulation of the MPSP inherently poses final conditions as a set of hard constraints. The computational efficiency and fast convergence make the MPSP preferable for fixed final time onboard optimal guidance algorithm. It has also been observed that the minimum fuel requirement strongly depends on the choice of the final time (a critical point that is not given due importance in many literature). Hence, to optimally select the final time, a neural network is used to learn the mapping between various initial conditions in the domain of interest and the corresponding optimal flight time. To generate the training data set, the optimal final time is computed offline using a gradient based optimization technique. The effectiveness of the proposed method is demonstrated with rigorous simulation results.

Control of vortex shedding at moderate Reynolds numbers

The suppression method of vortex shedding from a circular cylinder has been studied experimentally in the Reynolds number range from 300 to 1600. The test is performed in a water channel. The model cylinder is 1 cm in diameter and 38 cm in length. A row of small rods of 0.18 cm in diameter and 1.5 cm in length are perpendicularly connected to the surface of the model cylinder and distributed along the meridian, The distance between the neighboring rods and the angle of attack of the rods can be changed so that the suppression effect on vortex shedding can be adjusted. The results show that vortex shedding can be suppressed effectively if the distance between the neighboring rods is smaller than 3 times and the cylinder diameter and the angle of attack is in the range of 30degreesless than or equal tobeta<90&DEG;.

Simulation of the Load-Unload Response Ratio and Critical Sensitivity in the Lattice Solid Model

The Load-Unload Response Ratio (LURR) method is an intermediate-term earthquake prediction approach that has shown considerable promise. It involves calculating the ratio of a specified energy release measure during loading and unloading where loading and unloading periods are determined from the earth tide induced perturbations in the Coulomb Failure Stress on optimally oriented faults. In the lead-up to large earthquakes, high LURR values are frequently observed a few months or years prior to the event. These signals may have a similar origin to the observed accelerating seismic moment release (AMR) prior to many large earthquakes or may be due to critical sensitivity of the crust when a large earthquake is imminent. As a first step towards studying the underlying physical mechanism for the LURR observations, numerical studies are conducted using the particle based lattice solid model (LSM) to determine whether LURR observations can be reproduced. The model is initialized as a heterogeneous 2-D block made up of random-sized particles bonded by elastic-brittle links. The system is subjected to uniaxial compression from rigid driving plates on the upper and lower edges of the model. Experiments are conducted using both strain and stress control to load the plates. A sinusoidal stress perturbation is added to the gradual compressional loading to simulate loading and unloading cycles and LURR is calculated. The results reproduce signals similar to those observed in earthquake prediction practice with a high LURR value followed by a sudden drop prior to macroscopic failure of the sample. The results suggest that LURR provides a good predictor for catastrophic failure in elastic-brittle systems and motivate further research to study the underlying physical mechanisms and statistical properties of high LURR values. The results provide encouragement for earthquake prediction research and the use of advanced simulation models to probe the physics of earthquakes.

Investigation of Vaporized Kerosene Injection and Combustion in a Supersonic Model Combustor

Injection and combustion of vaporized kerosene was experimentally investigated in a Mach 2.5 model combustor at various fuel temperatures and injection pressures. A unique kerosene heating and delivery system, which can prepare heated kerosene up to 820 K at a pressure of 5.5 MPa with negligible fuel coking, was developed. A three-species surrogate was employed to simulate the thermophysical properties of kerosene. The calculated thermophysical properties of surrogate provided insight into the fuel flow control in experiments. Kerosene jet structures at various preheat temperatures injecting into both quiescent environment and a Mach 2.5 crossflow were characterized. It was shown that the use ofvaporized kerosene injection holds the potential of enhancing fuel-air mixing and promoting overall burning. Supersonic combustion tests further confirmed the preceding conjecture by comparing the combustor performances of supercritical kerosene with those of liquid kerosene and effervescent atomization with hydrogen barbotage. Under the similar flow conditions and overall kerosene equivalence ratios, experimental results illustrated that the combustion efficiency of supercritical kerosene increased approximately 10-15% over that of liquid kerosene, which was comparable to that of effervescent atomization.

An Electrowetting Model For Rough Surfaces Under Low Voltage

Electrowetting is one of the most effective methods to enhance wettability. A significant change of contact angle for the liquid droplet can result from the surface microstructures and the external electric field, without altering the chemical composition of the system. During the electrowetting process on a rough surface, the droplet exhibits a sharp transition from the Cassie-Baxter to the Wenzel regime at a low critical voltage. In this paper, a theoretical model for electrowetting is put forth to describe the dynamic electrical control of the wetting behavior at the low voltage, considering the surface topography. The theoretical results are found to be in good agreement with the existing experimental results. (c) Koninklijke Brill NV, Leiden, 2008.