Optimal Control of Markov Processes with Age-Dependent Transition Rates

We study optimal control of Markov processes with age-dependent transition rates. The control policy is chosen continuously over time based on the state of the process and its age. We study infinite horizon discounted cost and infinite horizon average cost problems. Our approach is via the construction of an equivalent semi-Markov decision process. We characterise the value function and optimal controls for both discounted and average cost cases.

Metal and Alloy Nanoparticles by Amine-Borane Reduction of Metal Salts by Solid-Phase Synthesis: Atom Economy and Green Process

A new solid state synthetic route has been developed toward metal and bimetallic alloy nanoparticles from metal salts employing amine-boranes, as the reducing agent. During the reduction, amine-borane plays a dual role: acts as a reducing agent and reduces the metal salts to their elemental form and simultaneously generates a stabilizing agent in situ which controls the growth of the particles and stabilizes them in the nanosize regime. Employing different amine-boranes with differing reducing ability (ammonia borane (AB), dimethylamine borane (DMAB), and triethylamine borane (TMAB)) was found to have a profound effect on the particle size and the size distribution. Usage of AB as the reducing agent provided the smallest possible size with best size distribution. Employment of TMAB also afforded similar results; however, when DMAB was used as the reducing agent it resulted in larger sized nanoparticles that are polydisperse too. In the AB mediated reduction, BNHx polymer generated in situ acts as a capping agent whereas, the complexing amine of the other amine-boranes (DMAB and TMAB) play the same role. Employing the solid state route described herein, monometallic Au, Ag, Cu, Pd, and Ir and bimetallic CuAg and CuAu alloy nanoparticles of <10 nm were successfully prepared. Nucleation and growth processes that control the size and the size distribution of the resulting nanoparticles have been elucidated in these systems.

Statistical Methods for Accelerated Life Testing with Particular Reference to Resin Impregnated Paper Bushings

Electrical failure of insulation is known to be an extremal random process wherein nominally identical pro-rated specimens of equipment insulation, at constant stress fail at inordinately different times even under laboratory test conditions. In order to be able to estimate the life of power equipment, it is necessary to run long duration ageing experiments under accelerated stresses, to acquire and analyze insulation specific failure data. In the present work, Resin Impregnated Paper (RIP) a relatively new insulation system of choice used in transformer bushings, is taken as an example. The failure data has been processed using proven statistical methods, both graphical and analytical. The physical model governing insulation failure at constant accelerated stress has been assumed to be based on temperature dependent inverse power law model.

Transmit Power Control Policies for Energy Harvesting Sensors With Retransmissions

This paper addresses the problem of finding outage-optimal power control policies for wireless energy harvesting sensor (EHS) nodes with automatic repeat request (ARQ)-based packet transmissions. The power control policy of the EHS specifies the transmission power for each packet transmission attempt, based on all the information available at the EHS. In particular, the acknowledgement (ACK) or negative acknowledgement (NACK) messages received provide the EHS with partial information about the channel state. We solve the problem of finding an optimal power control policy by casting it as a partially observable Markov decision process (POMDP). We study the structure of the optimal power policy in two ways. First, for the special case of binary power levels at the EHS, we show that the optimal policy for the underlying Markov decision process (MDP) when the channel state is observable is a threshold policy in the battery state. Second, we benchmark the performance of the EHS by rigorously analyzing the outage probability of a general fixed-power transmission scheme, where the EHS uses a predetermined power level at each slot within the frame. Monte Carlo simulation results illustrate the performance of the POMDP approach and verify the accuracy of the analysis. They also show that the POMDP solutions can significantly outperform conventional ad hoc approaches.

Transmit power Control with ARQ in energy harvesting sensors: a decision-theoretic approach

This paper addresses the problem of finding optimal power control policies for wireless energy harvesting sensor (EHS) nodes with automatic repeat request (ARQ)-based packet transmissions. The EHS harvests energy from the environment according to a Bernoulli process; and it is required to operate within the constraint of energy neutrality. The EHS obtains partial channel state information (CSI) at the transmitter through the link-layer ARQ protocol, via the ACK/NACK feedback messages, and uses it to adapt the transmission power for the packet (re)transmission attempts. The underlying wireless fading channel is modeled as a finite state Markov chain with known transition probabilities. Thus, the goal of the power management policy is to determine the best power setting for the current packet transmission attempt, so as to maximize a long-run expected reward such as the expected outage probability. The problem is addressed in a decision-theoretic framework by casting it as a partially observable Markov decision process (POMDP). Due to the large size of the state-space, the exact solution to the POMDP is computationally expensive. Hence, two popular approximate solutions are considered, which yield good power management policies for the transmission attempts. Monte Carlo simulation results illustrate the efficacy of the approach and show that the approximate solutions significantly outperform conventional approaches.

Simultaneous attitude control and trajectory tracking of a micro quadrotor: a SNAC aided nonlinear dynamic inversion approach

This paper presents an advanced single network adaptive critic (SNAC) aided nonlinear dynamic inversion (NDI) approach for simultaneous attitude control and trajectory tracking of a micro-quadrotor. Control of micro-quadrotors is a challenging problem due to its small size, strong coupling in pitch-yaw-roll and aerodynamic effects that often need to be ignored in the control design process to avoid mathematical complexities. In the proposed SNAC aided NDI approach, the gains of the dynamic inversion design are selected in such a way that the resulting controller behaves closely to a pre-synthesized SNAC controller for the output regulation problem. However, since SNAC is based on optimal control theory, it makes the dynamic inversion controller to operate near optimal and enhances its robustness property as well. More important, it retains two major benefits of dynamic inversion: (i) closed form expression of the controller and (ii) easy scalability to command tracking application even without any apriori knowledge of the reference command. Effectiveness of the proposed controller is demonstrated from six degree-of-freedom simulation studies of a micro-quadrotor. It has also been observed that the proposed SNAC aided NDI approach is more robust to modeling inaccuracies, as compared to the NDI controller designed independently from time domain specifications.

Remarkable Regioisomer Control in the Hydrogel Formation from a Two-Component Mixture of Pyridine-End Oligo(p-phenylenevinylene)s and N-Decanoyl-L-alanine

N-Decanoyl-L-alanine (DA) was mixed with either colorless 4,4-bipyridine (BP) or various derivatives such as chromogenic oligo(p-phenylenevinylene) (OPV) functionalized with isomeric pyridine termini in specific molar ratios. This mixtures form salt-type gels in a water/ethanol (2:1, v/v) mixture. The gelation properties of these two-component mixtures could be modulated by variation of the position of the N atom of the end pyridyl groups in OPVs. The presence of acid-base interactions in the self-assembly of these two-component systems leading to gelation was probed in detail by using stoichiometry-dependent UV/Vis and FTIR spectroscopy. Furthermore, temperature-dependent UV/Vis and fluorescence spectroscopy clearly demonstrated a J-type aggregation mode of these gelator molecules during the sol-to-gel transition process. Morphological features and the arrangement of the molecules in the gels were examined by using scanning electron microscopy (SEM), atomic force microscopy (AFM), and X-ray diffraction (XRD) techniques. Calculation of the length of each molecular system by energy minimization in its extended conformation and comparison with the XRD patterns revealed that this class of gelator molecules adopts lamellar organizations. Rheological properties of these two-component systems provided clear evidence that the flow behavior could be modulated by varying the acid/amine ratio. Polarized optical microscopy (POM), differential scanning calorimetry (DSC), and XRD results revealed that the solid-phase behavior of such two-component mixtures (acid/base=2:1) varied significantly upon changing the proton-acceptor part from BP to OPV. Interestingly, the XRD pattern of these acid/base mixtures after annealing at their associated isotropic temperature was significantly different from that of their xerogels.

Size Control and Magnetic Property Trends in Cobalt Ferrite Nanoparticles Synthesized Using an Aqueous Chemical Route

Cobalt ferrite (CoFe2O4) is an engineering material which is used for applications such as magnetic cores, magnetic switches, hyperthermia based tumor treatment, and as contrast agents for magnetic resonance imaging. Utility of ferrites nanoparticles hinges on its size, dispersibility in solutions, and synthetic control over its coercivity. In this work, we establish correlations between room temperature co-precipitation conditions, and these crucial materials parameters. Furthermore, post-synthesis annealing conditions are correlated with morphology, changes in crystal structure and magnetic properties. We disclose the synthesis and process conditions helpful in obtaining easily sinterable CoFe2O4 nanoparticles with coercive magnetic flux density (H-c) in the range 5.5-31.9 kA/m and M-s in the range 47.9-84.9 A.m(2)Kg(-1). At a grain size of similar to 54 +/- 2 nm (corresponding to 1073 K sintering temperature), multi-domain behavior sets in, which is indicated by a decrease in H-c. In addition, we observe an increase in lattice constant with respect to grain size, which is the inverse of what is expected of in ferrites. Our results suggest that oxygen deficiency plays a crucial role in explaining this inverse trend. We expect the method disclosed here to be a viable and scalable alternative to thermal decomposition based CoFe2O4 synthesis. The magnetic trends reported will aid in the optimization of functional CoFe2O4 nanoparticles

Partial Integrated Guidance and Control of Interceptors for High-Speed Ballistic Targets

A new partial integrated guidance and control design approach is proposed in this paper, which combines the benefits of both integrated guidance and control as well as the conventional guidance and control design philosophies. The proposed technique essentially operates in a two-loop structure. In the outer loop, an optimal guidance problem is formulated considering the nonlinear six degrees-of-freedom equation of motion of the interceptor. From this loop, the required pitch and yaw rates are generated by solving a nonlinear suboptimal guidance formulation in a computationally efficient manner while simultaneously assuring roll stabilization. Next, the inner loop tracks these outer loop body rate commands. This manipulation of the six degrees-of-freedom dynamics in both loops preserves the inherent time scale separation property between the translational and rotational dynamics, while retaining the philosophy of integrated guidance and control design as well. Because of this, the tuning process is quite straightforward and nontedious as well. Extensive six degrees-of-freedom simulations studies have been carried out, considering three-dimensional engagement geometry, to demonstrate the effectiveness of the proposed new design approach engaging high-speed ballistic targets. A variety of comparison studies have also been carried out to demonstrate the effectiveness of the proposed approach.

Influence of surface area to volume ratio of fuel particles on gasification process in a fixed bed

The paper addresses the effect of particle size on tar generation in a fixed bed gasification system. Pyrolysis, a diffusion limited process, depends on the heating rate and the surface area of the particle influencing the release of the volatile fraction leaving behind residual char. The flaming time has been estimated for different biomass samples. It is found that the flaming time for wood flakes is almost one fourth than that of coconut shells for same equivalent diameter fuel samples. The particle density of the coconut shell is more than twice that of wood spheres, and almost four times compared with wood flakes; having a significant influence on the flaming time. The ratio of the particle surface area to that of an equivalent diameter is nearly two times higher for flakes compared with wood pieces. Accounting for the density effect, on normalizing with density of the particle, the flaming rate is double in the case of wood flakes or coconut shells compared with the wood sphere for an equivalent diameter. This is due to increased surface area per unit volume of the particle. Experiments are conducted on estimation of tar content in the raw gas for wood flakes and standard wood pieces. It is observed that the tar level in the raw gas is about 80% higher in the case of wood flakes compared with wood pieces. The analysis suggests that the time for pyrolysis is lower with a higher surface area particle and is subjected to fast pyrolysis process resulting in higher tar fraction with low char yield. Increased residence time with staged air flow has a better control on residence time and lower tar in the raw gas. (C) 2014 International Energy Initiative. Published by Elsevier Inc. All rights reserved.

How does spacer length of imidazolium gemini surfactants control the fabrication of 2D-Langmuir films of silver-nanoparticles at the air-water interface?

A series of gemini surfactants based on cationic imidazolium ring as polar headgroup, abbreviated as lm-n-lm], 2Br(-) (n = 2, 5, 6 and 12), was synthesized. Their ability to stabilize silver nanoparticles in aqueous media was investigated. The resulting suspensions were characterized by UV-Vis spectroscopy and transmission electron microscopy (TEM). They exhibit specific morphologies by adopting different supramolecular assemblies in aqueous media depending on the internal packing arrangements and on the number of spacer methylene units -(CH2)(n)-]. Individual colloids were extracted from the aqueous to chloroform layer and spread at the air/water interface to allow the formation of well-defined Langmuir films. By analysis of the surface pressure-area isotherms, the details about the packing behavior and orientation of the imidazolium gemini surfactant capped silver nanoparticles were obtained. Morphological features of the dynamic process of monolayer compression at the air-water interface were elucidated using Brewster angle microscopy (BAM). These monolayers were further transferred on mica sheets by the Langmuir-Blodgett technique at their associated collapse pressure and the morphology of these monolayers was investigated by atomic force microscopy (AFM). The number of spacer methylene units -(CH2)(n)-] of the gemini surfactants exerted critical influence in modulating the characteristics of the resulting Langmuir films. (C) 2014 Elsevier Inc. All rights reserved.

How does spacer length of imidazolium gemini surfactants control the fabrication of 2D-Langmuir films of silver-nanoparticles at the air-water interface?

A series of gemini surfactants based on cationic imidazolium ring as polar headgroup, abbreviated as Im-n-Im], 2Br(-) (n = 2, 5,6 and 12), was synthesized. Their ability to stabilize silver nanoparticles in aqueous media was investigated. The resulting suspensions were characterized by UV-Vis spectroscopy and transmission electron microscopy (TEM). They exhibit specific morphologies by adopting different supramolecular assemblies in aqueous media depending on the internal packing arrangements and on the number of spacer methylene units -(CH2)(n)-]. Individual colloids were extracted from the aqueous to chloroform layer and spread at the air/water interface to allow the formation of well-defined Langmuir films. By analysis of the surface pressure-area isotherms, the details about the packing behavior and orientation of the imidazolium gemini surfactant capped silver nanoparticles were obtained. Morphological features of the dynamic process of monolayer compression at the air-water interface were elucidated using Brewster angle microscopy (BAM). These monolayers were further transferred on mica sheets by the Langmuir-Blodgett technique at their associated collapse pressure and the morphology of these monolayers was investigated by atomic force microscopy (AFM). The number of spacer methylene units (CH2)(n)-] of the gemini surfactants exerted critical influence in modulating the characteristics of the resulting Langmuir films. (C) 2014 Elsevier Inc. All rights reserved.

Adaptive Sleep-Wake Control using Reinforcement Learning in Sensor Networks

The aim in this paper is to allocate the `sleep time' of the individual sensors in an intrusion detection application so that the energy consumption from the sensors is reduced, while keeping the tracking error to a minimum. We propose two novel reinforcement learning (RL) based algorithms that attempt to minimize a certain long-run average cost objective. Both our algorithms incorporate feature-based representations to handle the curse of dimensionality associated with the underlying partially-observable Markov decision process (POMDP). Further, the feature selection scheme used in our algorithms intelligently manages the energy cost and tracking cost factors, which in turn assists the search for the optimal sleeping policy. We also extend these algorithms to a setting where the intruder's mobility model is not known by incorporating a stochastic iterative scheme for estimating the mobility model. The simulation results on a synthetic 2-d network setting are encouraging.

Morphology control synthesis of Au-Cu2S metal-semiconductor hybrid nanostructures by modulating reaction constituents

A facile methodology for synthesizing Au-Cu2S hybrid nanoparticles is presented. Au-Cu2S nanoparticles have application in visible light driven photocatalytic degradation of dyes. Detailed microstructural and compositional characterization illustrated that the hybrid nanoparticles are composed of cube shaped Au-Cu solid solution and hemispherical shaped Cu2S phases. Investigation of nanoparticles extracted at different stages of the synthesis process revealed that the mechanism of formation of hybrid nanoparticles involved initial formation of isolated cube shaped pure Au nanoparticles and Cu-thiolate complex. In the subsequent stages, the Au nanoparticles get adsorbed onto the Cu-thiolate complex which is followed by the decomposition of the Cu-thiolate complex to form Au-Cu2S hybrid nanoparticles. This study also illustrates that an optimum concentration of dodecanethiol is required both for achieving size and morphological uniformity of the participating phases and for their attachment to form a hybrid nanoparticle.

Multi-agent Reinforcement Learning for Traffic Signal Control

Optimal control of traffic lights at junctions or traffic signal control (TSC) is essential for reducing the average delay experienced by the road users amidst the rapid increase in the usage of vehicles. In this paper, we formulate the TSC problem as a discounted cost Markov decision process (MDP) and apply multi-agent reinforcement learning (MARL) algorithms to obtain dynamic TSC policies. We model each traffic signal junction as an independent agent. An agent decides the signal duration of its phases in a round-robin (RR) manner using multi-agent Q-learning with either is an element of-greedy or UCB 3] based exploration strategies. It updates its Q-factors based on the cost feedback signal received from its neighbouring agents. This feedback signal can be easily constructed and is shown to be effective in minimizing the average delay of the vehicles in the network. We show through simulations over VISSIM that our algorithms perform significantly better than both the standard fixed signal timing (FST) algorithm and the saturation balancing (SAT) algorithm 15] over two real road networks.