Risk-sensitive optimal control for Markov decision processes with monotone cost

The existence of an optimal feedback law is established for the risk-sensitive optimal control problem with denumerable state space. The main assumptions imposed are irreducibility and a near monotonicity condition on the one-step cost function. A solution can be found constructively using either value iteration or policy iteration under suitable conditions on initial feedback law.

Performance analysis of subband-level channel quality indicator feedback scheme of LTE

Frequency-domain scheduling and rate adaptation enable next generation wireless cellular systems such as Long Term Evolution (LTE) to achieve significantly higher downlink throughput. LTE assigns subcarriers in chunks, called physical resource blocks (PRBs), to users to reduce control signaling overhead. To reduce the enormous feedback overhead, the channel quality indicator (CQI) report that is used to feed back channel state information is averaged over a subband, which, in turn, is a group of multiple PRBs. In this paper, we develop closed-form expressions for the throughput achieved by the subband-level CQI feedback mechanism of LTE. We show that the coarse frequency resolution of the CQI incurs a significant loss in throughput and limits the multi-user gains achievable by the system. We then show that the performance can be improved by means of an offset mechanism that effectively makes the users more conservative in reporting their CQI.

Transmit power Control with ARQ in energy harvesting sensors: a decision-theoretic approach

This paper addresses the problem of finding optimal power control policies for wireless energy harvesting sensor (EHS) nodes with automatic repeat request (ARQ)-based packet transmissions. The EHS harvests energy from the environment according to a Bernoulli process; and it is required to operate within the constraint of energy neutrality. The EHS obtains partial channel state information (CSI) at the transmitter through the link-layer ARQ protocol, via the ACK/NACK feedback messages, and uses it to adapt the transmission power for the packet (re)transmission attempts. The underlying wireless fading channel is modeled as a finite state Markov chain with known transition probabilities. Thus, the goal of the power management policy is to determine the best power setting for the current packet transmission attempt, so as to maximize a long-run expected reward such as the expected outage probability. The problem is addressed in a decision-theoretic framework by casting it as a partially observable Markov decision process (POMDP). Due to the large size of the state-space, the exact solution to the POMDP is computationally expensive. Hence, two popular approximate solutions are considered, which yield good power management policies for the transmission attempts. Monte Carlo simulation results illustrate the efficacy of the approach and show that the approximate solutions significantly outperform conventional approaches.

Risk-Sensitive Ergodic Control of Continuous Time Markov Processes With Denumerable State Space

In this article, we study risk-sensitive control problem with controlled continuous time Markov chain state dynamics. Using multiplicative dynamic programming principle along with the atomic structure of the state dynamics, we prove the existence and a characterization of optimal risk-sensitive control under geometric ergodicity of the state dynamics along with a smallness condition on the running cost.

Nonlinear control of an air-breathing engine including its validation with vehicle guidance

An implementable nonlinear control design approach is presented for a supersonic air-breathing ramjet engine. The primary objective is to ensure that the thrust generated by the engine tracks the commanded thrust without violating the operational constraints. An important constraint is to manage the shock wave location in the intake so that it neither gets detached nor gets too much inside the intake. Both the objectives are achieved by regulating the fuel flow to the combustion chamber and by varying the throat area of the nozzle simultaneously. The design approach accounts for the nonlinear cross-coupling effects and nullifies those. Also, an extended Kalman filter has been used to filter out the sensor and process noises as well as to make the states available for feedback. Furthermore, independent control design has been carried out for the actuators. To test the performance of the engine for a realistic flight trajectory, a representative trajectory is generated through a trajectory optimization process, which is augmented with a newly-developed finite-time state dependent Riccati equation technique for nullifying the perturbations online. Satisfactory overall performance has been obtained during both climb and cruise phases. (C) 2015 Elsevier Masson SAS. All rights reserved.

Ergodic control of partially observed Markov chains

The ergodic or long-run average cost control problem for a partially observed finite-state Markov chain is studied via the associated fully observed separated control problem for the nonlinear filter. Dynamic programming equations for the latter are derived, leading to existence and characterization of optimal stationary policies.

Supramolecular control of the magnetic anisotropy in two-dimensional high-spin Fe arrays at a metal interface

Magnetic atoms at surfaces are a rich model system for solid-state magnetic bits exhibiting either classical(1,2) or quantum(3,4) behaviour. Individual atoms, however, are difficult to arrange in regular patterns(1-5). Moreover, their magnetic properties are dominated by interaction with the substrate, which, as in the case of Kondo systems, often leads to a decrease or quench of their local magnetic moment(6,7). Here, we show that the supramolecular assembly of Fe and 1,4-benzenedicarboxylic acid molecules on a Cu surface results in ordered arrays of high-spin mononuclear Fe centres on a 1.5nm square grid. Lateral coordination with the molecular ligands yields unsaturated yet stable coordination bonds, which enable chemical modification of the electronic and magnetic properties of the Fe atoms independently from the substrate. The easy magnetization direction of the Fe centres can be switched by oxygen adsorption, thus opening a way to control the magnetic anisotropy in supramolecular layers akin to that used in metallic thin films.

State space formulation of ammonia reactor dynamics

The specific objective of this paper is to develop a state space model of a tubular ammonia reactor which is the heart of an ammonia plant in a fertiliser complex. A ninth order model with three control inputs and two disturbance inputs is generated from the nonlinear distributed model using linearization and lumping approximations. The lumped model is chosen such that the steady state temperature at the exit of the catalyst bed computed from the simplified state space model is close enough to the one computed from the nonlinear steady state model. The model developed in this paper is very useful for the design of continuous/discrete versions of single variable/multivariable control algorithms.

A DC-link Capacitor Voltage Balancing with CMV Elimination using only the Switching State Redundancies for a Reduced Switch Count Multi-Level Inverter fed IM Drive

Common mode voltage (CMV) variations in PWM inverter-fed drives generate unwanted shaft and bearing current resulting in early motor failure. Multilevel inverters reduce this problem to some extent, with higher number of levels. But the complexity of the power circuit increases with an increase in the number of inverter voltage levels. In this paper a five-level inverter structure is proposed for open-end winding induction motor (IM) drives, by cascading only two conventional two-level and three-level inverters, with the elimination of the common mode voltage over the entire modulation range. The DC link power supply requirement is also optimized by means of DC link capacitor voltage balancing, with PWM control., using only inverter switching state redundancies. The proposed power circuit gives a simple power bits structure.

Solid-State Acyl Migration In Salicylamides - An X-Ray Study Of The Ortho-Propionyl And Normal-Propionyl Derivatives

In recent years there has been an upsurge of interest in the study of organic reactions in the solid state. It is now realised that the crystalline matrix provides an extra-ordinary spatial control on the initiation and progress of these reactions. Electronic and dipolar effects which are important in solution are replaced by structural and geometric effects in solids. These 'spatial' or 'topochemical' aspects are important in understanding the mechanistic details of the reaction. In our laboratory, the thermally induced acyl migration in salicylamides from 0- to N- position in the solid state has been under study (Scheme 1). The structures of the acetyl and benzoyl derivatives (Ia,IIa, Ib and IIb) have been reported.

A Pulsewidth Modulated Control of Induction Motor Drive Using Multilevel 12-Sided Polygonal Voltage Space Vectors

In this paper, a novel 12-sided polygonal space vector structure is proposed for an induction motor drive. The space vector pattern presented in this paper consists of two 12-sided concentric polygons with the outer polygon having a radius double the inner one. As compared to previously reported 12-sided polygonal space vector structures, this paper subdivides the space vector plane into smaller sized triangles. This helps in reducing the switching frequency of the inverters without deteriorating the output voltage quality. It also reduces the device ratings and dv/dt stress on the devices to half. At the same time, other benefits obtained from the existing 12-sided space vector structure, such as increased linear modulation range and complete elimination of 5th and 7th order harmonics in the phase voltage, are also retained in this paper. The space vector structure is realized by feeding an open-end induction motor with two conventional three-level neutral point clamped (NPC) inverters with asymmetric isolated dc link voltage sources. The neutral point voltage fluctuations in the three-level NPC inverters are eliminated by utilizing the switching state multiplicities for a space vector point. The pulsewidth modulation timings are calculated using sampled reference waveform amplitudes and are explained in detail in this paper. Experimental verification on a laboratory prototype shows that this configuration may be considered suitable for high power drives.

On various types of decoupling for time-varying system

This paper deals with the problem of decoupling a class of linear time-varying multi-variable systems, based on the defining property that the impulse response matrix of a decoupled system is diagonal. Depending on the properties of the coefficient matrices of the vector differential equation of the open-loop system, the system may be uniformly or totally decoupled. The necessary and sufficient conditions that permit a system to be uniformly or totally decoupled by state variable feedback are given. The main contribution of this paper is the precise definition of these two classes of decoupling and a rigorous derivation of the necessary and sufficient conditions which show the necessity of requiring that the system be of constant ordered rank with respect to observability. A simple example illustrates the importance of having several definitions of decoupling. Finally, the results are specialized to the case of time invariant systems.

L2-stability of nonstationary feedback systems: Frequency-domain criteria

A frequency-domain positivity condition is derived for linear time-varying operators in2and is used to develop2stability criteria for linear and nonlinear feedback systems. These criteria permit the use of a very general class of operators in2with nonstationary kernels, as multipliers. More specific results are obtained by using a first-order differential operator with a time-varying coefficient as multiplier. Finally, by employing periodic multipliers, improved stability criteria are derived for the nonlinear damped Mathieu equation with a forcing function.