Optimal multi-layered congestion based pricing schemes for enhanced QoS

Pricing is an effective tool to control congestion and achieve quality of service (QoS) provisioning for multiple differentiated levels of service. In this paper, we consider the problem of pricing for congestion control in the case of a network of nodes with multiple queues and multiple grades of service. We present a closed-loop multi-layered pricing scheme and propose an algorithm for finding the optimal state dependent price levels for individual queues, at each node. This is different from most adaptive pricing schemes in the literature that do not obtain a closed-loop state dependent pricing policy. The method that we propose finds optimal price levels that are functions of the queue lengths at individual queues. Further, we also propose a variant of the above scheme that assigns prices to incoming packets at each node according to a weighted average queue length at that node. This is done to reduce frequent price variations and is in the spirit of the random early detection (RED) mechanism used in TCP/IP networks. We observe in our numerical results a considerable improvement in performance using both of our schemes over that of a recently proposed related scheme in terms of both throughput and delay performance. In particular, our first scheme exhibits a throughput improvement in the range of 67-82% among all routes over the above scheme. (C) 2011 Elsevier B.V. All rights reserved.

Stabilization of stochastic approximation by step size adaptation

A scheme for stabilizing stochastic approximation iterates by adaptively scaling the step sizes is proposed and analyzed. This scheme leads to the same limiting differential equation as the original scheme and therefore has the same limiting behavior, while avoiding the difficulties associated with projection schemes. The proof technique requires only that the limiting o.d.e. descend a certain Lyapunov function outside an arbitrarily large bounded set. (C) 2012 Elsevier B.V. All rights reserved.

Existence of Value in Stochastic Differential Games of Mixed Type

In this article, we address stochastic differential games of mixed type with both control and stopping times. Under standard assumptions, we show that the value of the game can be characterized as the unique viscosity solution of corresponding Hamilton-Jacobi-Isaacs (HJI) variational inequalities.

Non-Stationary Semi-Markov Decision Processes on a Finite Horizon

We introduce and study a class of non-stationary semi-Markov decision processes on a finite horizon. By constructing an equivalent Markov decision process, we establish the existence of a piecewise open loop relaxed control which is optimal for the finite horizon problem.

Optimal forwarding in delay tolerant networks with multiple destinations

We study the trade-off between delivery delay and energy consumption in a delay tolerant network in which a message (or a file) has to be delivered to each of several destinations by epidemic relaying. In addition to the destinations, there are several other nodes in the network that can assist in relaying the message. We first assume that, at every instant, all the nodes know the number of relays carrying the packet and the number of destinations that have received the packet. We formulate the problem as a controlled continuous time Markov chain and derive the optimal closed loop control (i.e., forwarding policy). However, in practice, the intermittent connectivity in the network implies that the nodes may not have the required perfect knowledge of the system state. To address this issue, we obtain an ODE (i.e., fluid) approximation for the optimally controlled Markov chain. This fluid approximation also yields an asymptotically optimal open loop policy. Finally, we evaluate the performance of the deterministic policy over finite networks. Numerical results show that this policy performs close to the optimal closed loop policy.

Reactive power and voltage control in grid connected wind farms

Wind power, as an alternative to fossil fuels, is plentiful, renewable, widely distributed, clean, produces no greenhouse gas emissions during operation, and uses little land. In operation, the overall cost per unit of energy produced is similar to the cost for new coal and natural gas installations. However, the stochastic behaviour of wind speeds leads to significant disharmony between wind energy production and electricity demand. Wind generation suffers from an intermittent characteristics due to the own diurnal and seasonal patterns of the wind behaviour. Both reactive power and voltage control are important under varying operating conditions of wind farm. To optimize reactive power flow and to keep voltages in limit, an optimization method is proposed in this paper. The objective proposed is minimization of the voltage deviations of the load buses (Vdesired). The approach considers the reactive power limits of wind generators and co-ordinates the transformer taps. This algorithm has been tested under practically varying conditions simulated on a test system. The results are obtained on a system of 50-bus real life equivalent power network. The result shows the efficiency of the proposed method.

Transmit Power Control Policies for Energy Harvesting Sensors With Retransmissions

This paper addresses the problem of finding outage-optimal power control policies for wireless energy harvesting sensor (EHS) nodes with automatic repeat request (ARQ)-based packet transmissions. The power control policy of the EHS specifies the transmission power for each packet transmission attempt, based on all the information available at the EHS. In particular, the acknowledgement (ACK) or negative acknowledgement (NACK) messages received provide the EHS with partial information about the channel state. We solve the problem of finding an optimal power control policy by casting it as a partially observable Markov decision process (POMDP). We study the structure of the optimal power policy in two ways. First, for the special case of binary power levels at the EHS, we show that the optimal policy for the underlying Markov decision process (MDP) when the channel state is observable is a threshold policy in the battery state. Second, we benchmark the performance of the EHS by rigorously analyzing the outage probability of a general fixed-power transmission scheme, where the EHS uses a predetermined power level at each slot within the frame. Monte Carlo simulation results illustrate the performance of the POMDP approach and verify the accuracy of the analysis. They also show that the POMDP solutions can significantly outperform conventional ad hoc approaches.

Confinement induced stochastic sensing of charged coronene and perylene aggregates in alpha-hemolysin nanochannels

Biological nanopores provide optimum dimensions and an optimal environment to study early aggregation kinetics of charged polyaromatic molecules in the nano-confined regime. It is expected that probing early stages of nucleation will enable us to design a strategy for supramolecular assembly and biocrystallization processes. Specifically, we have studied translocation dynamics of coronene and perylene based salts, through the alpha-hemolysin (alpha-HL) protein nanopore. The characteristic blocking events in the time-series signal are a function of concentration and bias voltage. We argue that different blocking events arise due to different aggregation processes as captured by all atomistic molecular dynamics (MD) simulations. These confinement induced aggregations of polyaromatic chromophores during the different stages of translocation are correlated with the spatial symmetry and charge distribution of the molecules.

Transmit power Control with ARQ in energy harvesting sensors: a decision-theoretic approach

This paper addresses the problem of finding optimal power control policies for wireless energy harvesting sensor (EHS) nodes with automatic repeat request (ARQ)-based packet transmissions. The EHS harvests energy from the environment according to a Bernoulli process; and it is required to operate within the constraint of energy neutrality. The EHS obtains partial channel state information (CSI) at the transmitter through the link-layer ARQ protocol, via the ACK/NACK feedback messages, and uses it to adapt the transmission power for the packet (re)transmission attempts. The underlying wireless fading channel is modeled as a finite state Markov chain with known transition probabilities. Thus, the goal of the power management policy is to determine the best power setting for the current packet transmission attempt, so as to maximize a long-run expected reward such as the expected outage probability. The problem is addressed in a decision-theoretic framework by casting it as a partially observable Markov decision process (POMDP). Due to the large size of the state-space, the exact solution to the POMDP is computationally expensive. Hence, two popular approximate solutions are considered, which yield good power management policies for the transmission attempts. Monte Carlo simulation results illustrate the efficacy of the approach and show that the approximate solutions significantly outperform conventional approaches.

Achievable secrecy sum-rate in a fading MAC-WT with power control and without CSI of eavesdropper

We consider a two user fading Multiple Access Channel with a wire-tapper (MAC-WT) where the transmitter has the channel state information (CSI) to the intended receiver but not to the eavesdropper (eve). We provide an achievable secrecy sum-rate with optimal power control. We next provide a secrecy sum-rate with optimal power control and cooperative jamming (CJ). We then study an achievable secrecy sum rate by employing an ON/OFF power control scheme which is more easily computable. We also employ CJ over this power control scheme. Results show that CJ boosts the secrecy sum-rate significantly even if we do not know the CSI of the eve's channel. At high SNR, the secrecy sum-rate (with CJ) without CSI of the eve exceeds the secrecy sum-rate (without CJ) with full CSI of the eve.

Power and delay optimal policies for wireless systems

In this paper we consider a single discrete time queue with infinite buffer. The channel may experience fading. The transmission rate is a linear function of power used for transmission. In this scenario we explicitly obtain power control policies which minimize mean power and/or mean delay. There may also be peak power constraint.

Subchannel allocation and power control in femtocells to provide quality of service

Femtocells are a new concept which improves the coverage and capacity of a cellular system. We consider the problem of channel allocation and power control to different users within a Femtocell. Knowing the channels available, the channel states and the rate requirements of different users the Femtocell base station (FBS), allocates the channels to different users to satisfy their requirements. Also, the Femtocell should use minimal power so as to cause least interference to its neighboring Femtocells and outside users. We develop efficient, low complexity algorithms which can be used online by the Femtocell. The users may want to transmit data or voice. We compare our algorithms with the optimal solutions.

Optimal Forwarding in Delay-Tolerant Networks With Multiple Destinations

We study the tradeoff between delivery delay and energy consumption in a delay-tolerant network in which a message (or a file) has to be delivered to each of several destinations by epidemic relaying. In addition to the destinations, there are several other nodes in the network that can assist in relaying the message. We first assume that, at every instant, all the nodes know the number of relays carrying the message and the number of destinations that have received the message. We formulate the problem as a controlled continuous-time Markov chain and derive the optimal closed-loop control (i.e., forwarding policy). However, in practice, the intermittent connectivity in the network implies that the nodes may not have the required perfect knowledge of the system state. To address this issue, we obtain an ordinary differential equation (ODE) (i.e., a deterministic fluid) approximation for the optimally controlled Markov chain. This fluid approximation also yields an asymptotically optimal open-loop policy. Finally, we evaluate the performance of the deterministic policy over finite networks. Numerical results show that this policy performs close to the optimal closed-loop policy.

Zero-Sum Stochastic Games with Partial Information and Average Payoff

We consider a discrete time partially observable zero-sum stochastic game with average payoff criterion. We study the game using an equivalent completely observable game. We show that the game has a value and also we present a pair of optimal strategies for both the players.

Partial Integrated Guidance and Control of Interceptors for High-Speed Ballistic Targets

A new partial integrated guidance and control design approach is proposed in this paper, which combines the benefits of both integrated guidance and control as well as the conventional guidance and control design philosophies. The proposed technique essentially operates in a two-loop structure. In the outer loop, an optimal guidance problem is formulated considering the nonlinear six degrees-of-freedom equation of motion of the interceptor. From this loop, the required pitch and yaw rates are generated by solving a nonlinear suboptimal guidance formulation in a computationally efficient manner while simultaneously assuring roll stabilization. Next, the inner loop tracks these outer loop body rate commands. This manipulation of the six degrees-of-freedom dynamics in both loops preserves the inherent time scale separation property between the translational and rotational dynamics, while retaining the philosophy of integrated guidance and control design as well. Because of this, the tuning process is quite straightforward and nontedious as well. Extensive six degrees-of-freedom simulations studies have been carried out, considering three-dimensional engagement geometry, to demonstrate the effectiveness of the proposed new design approach engaging high-speed ballistic targets. A variety of comparison studies have also been carried out to demonstrate the effectiveness of the proposed approach.