Joint Power Control, Scheduling and Routing for Multihop Energy Harvesting Sensor Networks

We study wireless multihop energy harvesting sensor networks employed for random field estimation. The sensors sense the random field and generate data that is to be sent to a fusion node for estimation. Each sensor has an energy harvesting source and can operate in two modes: Wake and Sleep. We consider the problem of obtaining jointly optimal power control, routing and scheduling policies that ensure a fair utilization of network resources. This problem has a high computational complexity. Therefore, we develop a computationally efficient suboptimal approach to obtain good solutions to this problem. We study the optimal solution and performance of the suboptimal approach through some numerical examples.

Room-temperature synthesis of gold nanoparticles - Size-control by slow addition

We report a simple and rapid process for the room-temperature synthesis of gold nanoparticles using tannic acid, a green reagent, as both the reducing and stabilising agent. We systematically investigated the effect of pH on the size distribution of nanoparticles synthesized. Based on induction time and zeta- potential measurements, we show that particle size distribution is controlled by a fine balance between the rates of reduction (determined by the initial pH of reactants) and coalescence (determined by the pH of the reaction mixture) in the initial period of growth. This insight led to the optimal batch process for size-controlled synthesis of 2-10 nm gold nanoparticles - slow addition (within 10 minutes) of chloroauric acid into tannic acid.

Optimal explicit guidance of multistage launch vehicle along three-dimensional trajectory

Details of an efficient optimal closed-loop guidance algorithm for a three-dimensional launch are presented with simulation results. Two types of orbital injections, with either true anomaly or argument of perigee being free at injection, are considered. The resulting steering-angle profile under the assumption of uniform gravity lies in a canted plane which transforms a three-dimensional problem into an equivalent two-dimensional one. Effects of thrust are estimated using a series in a recursive way. Encke's method is used to predict the trajectory during powered flight and then to compute the changes due to actual gravity using two gravity-related vectors. Guidance parameters are evaluated using the linear differential correction method. Optimality of the algorithm is tested against a standard ground-based trajectory optimization package. The performance of the algorithm is tested for accuracy, robustness, and efficiency for a sun-synchronous mission involving guidance for a multistage vehicle that requires large pitch and yaw maneuver. To demonstrate applicability of the algorithm to a range of missions, injection into a geostationary transfer orbit is also considered. The performance of the present algorithm is found to be much better than others.

Denumerable state stochastic games with limiting average payoff

We study stochastic games with countable state space, compact action spaces, and limiting average payoff. ForN-person games, the existence of an equilibrium in stationary strategies is established under a certain Liapunov stability condition. For two-person zero-sum games, the existence of a value and optimal strategies for both players are established under the same stability condition.

An actor-critic algorithm with function approximation for discounted cost constrained Markov decision processes

We develop in this article the first actor-critic reinforcement learning algorithm with function approximation for a problem of control under multiple inequality constraints. We consider the infinite horizon discounted cost framework in which both the objective and the constraint functions are suitable expected policy-dependent discounted sums of certain sample path functions. We apply the Lagrange multiplier method to handle the inequality constraints. Our algorithm makes use of multi-timescale stochastic approximation and incorporates a temporal difference (TD) critic and an actor that makes a gradient search in the space of policy parameters using efficient simultaneous perturbation stochastic approximation (SPSA) gradient estimates. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal policy. (C) 2010 Elsevier B.V. All rights reserved.

Optimal pulse spacing for dynamical decoupling in the presence of a purely dephasing spin bath

Maintaining quantum coherence is a crucial requirement for quantum computation; hence protecting quantum systems against their irreversible corruption due to environmental noise is an important open problem. Dynamical decoupling (DD) is an effective method for reducing decoherence with a low control overhead. It also plays an important role in quantum metrology, where, for instance, it is employed in multiparameter estimation. While a sequence of equidistant control pulses the Carr-Purcell-Meiboom-Gill (CPMG) sequence] has been ubiquitously used for decoupling, Uhrig recently proposed that a nonequidistant pulse sequence the Uhrig dynamic decoupling (UDD) sequence] may enhance DD performance, especially for systems where the spectral density of the environment has a sharp frequency cutoff. On the other hand, equidistant sequences outperform UDD for soft cutoffs. The relative advantage provided by UDD for intermediate regimes is not clear. In this paper, we analyze the relative DD performance in this regime experimentally, using solid-state nuclear magnetic resonance. Our system qubits are C-13 nuclear spins and the environment consists of a H-1 nuclear spin bath whose spectral density is close to a normal (Gaussian) distribution. We find that in the presence of such a bath, the CPMG sequence outperforms the UDD sequence. An analogy between dynamical decoupling and interference effects in optics provides an intuitive explanation as to why the CPMG sequence performs better than any nonequidistant DD sequence in the presence of this kind of environmental noise.

An integrated model for optimal reservoir operation for irrigation of multiple crops

An integrated model is developed, based on seasonal inputs of reservoir inflow and rainfall in the irrigated area, to determine the optimal reservoir release policies and irrigation allocations to multiple crops. The model is conceptually made up of two modules, Module 1 is an intraseasonal allocation model to maximize the sum of relative yields of all crops, for a given state of the system, using linear programming (LP). The module takes into account reservoir storage continuity, soil moisture balance, and crop root growth with time. Module 2 is a seasonal allocation model to derive the steady state reservoir operating policy using stochastic dynamic programming (SDP). Reservoir storage, seasonal inflow, and seasonal rainfall are the state variables in the SDP. The objective in SDP is to maximize the expected sum of relative yields of all crops in a year. The results of module 1 and the transition probabilities of seasonal inflow and rainfall form the input for module 2. The use of seasonal inputs coupled with the LP-SDP solution strategy in the present formulation facilitates in relaxing the limitations of an earlier study, while affecting additional improvements. The model is applied to an existing reservoir in Karnataka State, India.

Active vibration suppression of beams with discrete actuators using optimal dynamic inversion

Most of the structural elements like beams, cables etc. are flexible and should be modeled as distributed parameter systems (DPS) to represent the reality better. For large structures, the usual approach of 'modal representation' is not an accurate representation. Moreover, for excessive vibrations (possibly due to strong wind, earthquake etc.), external power source (controller) is needed to suppress it, as the natural damping of these structures is usually small. In this paper, we propose to use a recently developed optimal dynamic inversion technique to design a set of discrete controllers for this purpose. We assume that the control force to the structure is applied through finite number of actuators, which are located at predefined locations in the spatial domain. The method used in this paper determines control forces directly from the partial differential equation (PDE) model of the system. The formulation has better practical significance, both because it leads to a closed form solution of the controller (hence avoids computational issues) as well as because a set of discrete actuators along the spatial domain can be implemented with relative ease (as compared to a continuous actuator)

An optimal dynamic inversion approach for controlling a class of one-dimensional nonlinear distributed parameter systems

Combining the principles of dynamic inversion and optimization theory, a new approach is presented for stable control of a class of one-dimensional nonlinear distributed parameter systems, assuming the availability a continuous actuator in the spatial domain. Unlike the existing approximate-then-design and design-then-approximate techniques, here there is no need of any approximation either of the system dynamics or of the resulting controller. Rather, the control synthesis approach is fairly straight-forward and simple. The controller formulation has more elegance because we can prove the convergence of the controller to its steady state value. To demonstrate the potential of the proposed technique, a real-life temperature control problem for a heat transfer application is solved. It has been demonstrated that a desired temperature profile can be achieved starting from any arbitrary initial temperature profile.

Clan-based evolutionary approach for solving control problems

This paper presents a dan-based evolutionary approach for solving control problems. Three selected control problems, viz. linear-quadratic, harvest, and push-cart problems, are solved using the proposed approach. Results are compared with those of the evolutionary programming (EP) approach. In most of the cases, the proposed approach is successful in obtaining (near) optimal solutions for these selected problems.

Linear dynamics and active control of an elastically supported travelling string

In this paper, the linear dynamics and active control of a string travelling with uniform velocity is presented. Discrete elastic supports are introduced along the length of the string. Finite element formulation is adopted to obtain the governing equations of motion. The velocity of translation introduces gyroscopic terms in the system equations. The effect of translation and the discrete elastic supports on the free vibration solution is studied. The solution is utilized in actively controlling the string vibrations due to an initial disturbance. The control, affected in modal space, is optimal with respect to a quadratic performance index. Numerical results are presented to demonstrate the effectiveness of the control strategy in regulating the travelling string vibrations.

Stochastic differential games: Occupation measure based approach

A new approach based on occupation measures is introduced for studying stochastic differential games. For two-person zero-sum games, the existence of values and optimal strategies for both players is established for various payoff criteria. ForN-person games, the existence of equilibria in Markov strategies is established for various cases.

A reinforcement learning approach to automatic generation control

This paper formulates the automatic generation control (AGC) problem as a stochastic multistage decision problem. A strategy for solving this new AGC problem formulation is presented by using a reinforcement learning (RL) approach This method of obtaining an AGC controller does not depend on any knowledge of the system model and more importantly it admits considerable flexibility in defining the control objective. Two specific RL based AGC algorithms are presented. The first algorithm uses the traditional control objective of limiting area control error (ACE) excursions, where as, in the second algorithm, the controller can restore the load-generation balance by only monitoring deviation in tie line flows and system frequency and it does not need to know or estimate the composite ACE signal as is done by all current approaches. The effectiveness and versatility of the approaches has been demonstrated using a two area AGC model. (C) 2002 Elsevier Science B.V. All rights reserved.

Opportunistic splitting for scheduling via stochastic approximation

We consider the problem of scheduling a wireless channel among multiple users. A slot is given to a user with a highest metric (e.g., channel gain) in that slot. The scheduler may not know the channel states of all the users at the beginning of each slot. In this scenario opportunistic splitting is an attractive solution. However this algorithm requires that the metrics of different users form independent, identically distributed (iid) sequences with same distribution and that their distribution and number be known to the scheduler. This limits the usefulness of opportunistic splitting. In this paper we develop a parametric version of this algorithm. The optimal parameters of the algorithm are learnt online through a stochastic approximation scheme. Our algorithm does not require the metrics of different users to have the same distribution. The statistics of these metrics and the number of users can be unknown and also vary with time. We prove the convergence of the algorithm and show its utility by scheduling the channel to maximize its throughput while satisfying some fairness and/or quality of service constraints.