A Simultaneous Deterministic Perturbation Actor-Critic Algorithm with an Application to Optimal Mortgage Refinancing

We develop a simulation-based, two-timescale actor-critic algorithm for infinite horizon Markov decision processes with finite state and action spaces, with a discounted reward criterion. The algorithm is of the gradient ascent type and performs a search in the space of stationary randomized policies. The algorithm uses certain simultaneous deterministic perturbation stochastic approximation (SDPSA) gradient estimates for enhanced performance. We show an application of our algorithm on a problem of mortgage refinancing. Our algorithm obtains the optimal refinancing strategies in a computationally efficient manner

Optimal Parameterized Policies for Resource Allocation in Communication Networks

The problem of finding optimal parameterized feedback policies for dynamic bandwidth allocation in communication networks is studied. We consider a queueing model with two queues to which traffic from different competing flows arrive. The queue length at the buffers is observed every T instants of time, on the basis of which a decision on the amount of bandwidth to be allocated to each buffer for the next T instants is made. We consider two different classes of multilevel closed-loop feedback policies for the system and use a two-timescale simultaneous perturbation stochastic approximation (SPSA) algorithm to find optimal policies within each prescribed class. We study the performance of the proposed algorithm on a numerical setting and show performance comparisons of the two optimal multilevel closedloop policies with optimal open loop policies. We observe that closed loop policies of Class B that tune parameters for both the queues and do not have the constraint that the entire bandwidth be used at each instant exhibit the best results overall as they offer greater flexibility in parameter tuning. Index Terms — Resource allocation, dynamic bandwidth allocation in communication networks, two-timescale SPSA algorithm, optimal parameterized policies. I.

Optimal association of mobile wireless devices with a WLAN-3G access network

In this paper, we consider the problem of association of wireless stations (STAs) with an access network served by a wireless local area network (WLAN) and a 3G cellular network. There is a set of WLAN Access Points (APs) and a set of 3G Base Stations (BSs) and a number of STAs each of which needs to be associated with one of the APs or one of the BSs. We concentrate on downlink bulk elastic transfers. Each association provides each ST with a certain transfer rate. We evaluate an association on the basis of the sum log utility of the transfer rates and seek the utility maximizing association. We also obtain the optimal time scheduling of service from a 3G BS to the associated STAs. We propose a fast iterative heuristic algorithm to compute an association. Numerical results show that our algorithm converges in a few steps yielding an association that is within 1% (in objective value) of the optimal (obtained through exhaustive search); in most cases the algorithm yields an optimal solution.

Asymptotically optimal cooperative wireless networks without constellation expansion

In this work, we construct a unified family of cooperative diversity coding schemes for implementing the orthogonal amplify-and-forward and the orthogonal selection-decode-and-forward strategies in cooperative wireless networks. We show that, as the number of users increases, these schemes meet the corresponding optimal high-SNR outage region, and do so with minimal order of signaling complexity. This is an improvement over all outage-optimal schemes which impose exponential increases in signaling complexity for every new network user. Our schemes, which are based on commutative algebras of normal matrices, satisfy the outage-related information theoretic criteria, the duplex-related coding criteria, and maintain reduced signaling, encoding and decoding complexities

Optimal location of centre of gravity for swashplateless helicopter UAV and MAV

To investigate the use of centre of gravity location on reducing cyclic pitch control for helicopter UAV's (unmanned air vehicles) and MAV's (micro air vehicles). Low cyclic pitch is a necessity to implement the swashplateless rotor concept using trailing edge flaps or active twist using current generation low authority piezoceramic actuators. Design/methodology/approach – An aeroelastic analysis of the helicopter rotor with elastic blades is used to perform parametric and sensitivity studies of the effects of longitudinal and lateral center of gravity (cg) movements on the main rotor cyclic pitch. An optimization approach is then used to find cg locations which reduce the cyclic pitch at a given forward speed. Findings – It is found that the longitudinal cyclic pitch and lateral cyclic pitch can be driven to zero at a given forward speed by shifting the cg forward and to the port side, respectively. There also exist pairs of numbers for the longitudinal and lateral cg locations which drive both the cyclic pitch components to zero at a given forward speed. Based on these results, a compromise optimal cg location is obtained such that the cyclic pitch is bounded within ±5° for a BO105 helicopter rotor. Originality/value – The reduction in the cyclic pitch due to helicopter cg location is found to significantly reduce the maximum magnitudes of the control angles in flight, facilitating the swashplateless rotor concept. In addition, the existence of cg locations which drive the cyclic pitches to zero allows for the use of active cg movement as a way to replace the cyclic pitch control for helicopter MAV's.

Information-theoretic and communication-theoretic optimal power allocation for fading channels

In this paper we review the most peculiar and interesting information-theoretic and communications features of fading channels. We first describe the statistical models of fading channels which are frequently used in the analysis and design of communication systems. Next, we focus on the information theory of fading channels, by emphasizing capacity as the most important performance measure. Both single-user and multiuser transmission are examined. Further, we describe how the structure of fading channels impacts code design, and finally overview equalization of fading multipath channels.

Optimal maintenance policies for machines subject to deterioration and intermittent breakdowns

Optimal preventive maintenance policies, for a machine subject to deterioration with age and intermittent breakdowns and repairs, are derived using optimal control theory. The optimal policies are shown to be of bang-bang nature. The extension to the case when there are a large number of identical machines and several repairmen in the system is considered next. This model takes into account the waiting line formed at the repair facility and establishes a link between this problem and the classical ``repairmen problem.''

Scattering theory and linear optimal control: Regulator and servo problems

In this paper, several known computational solutions are readily obtained in a very natural way for the linear regulator, fixed end-point and servo-mechanism problems using a certain frame-work from scattering theory. The relationships between the solutions to the linear regulator problem with different terminal costs and the interplay between the forward and backward equations have enabled a concise derivation of the partitioned equations, the forward-backward equations, and Chandrasekhar equations for the problem. These methods have been extended to the fixed end-point, servo, and tracking problems.

Optimal maintenance policy of a machine subject to failure

Optimal maintenance policies for a machine with degradation in performance with age and subject to failure are derived using optimal control theory. The optimal policies are shown to be, normally, of bang-coast nature, except in the case when probability of machine failure is a function of maintenance. It is also shown, in the deterministic case that a higher depreciation rate tends to reverse this policy to coast-bang. When the probability of failure is a function of maintenance, considerable computational effort is needed to obtain an optimal policy and the resulting policy is not easily implementable. For this case also, an optimal policy in the class of bang-coast policies is derived, using a semi-Markov decision model. A simple procedure for modifying the probability of machine failure with maintenance is employed. The results obtained extend and unify the recent results for this problem along both theoretical and practical lines. Numerical examples are presented to illustrate the results obtained.

Optimal control for the rolling pullout maneuver of a modern fighter aircraft

Optimal control laws are obtained for the elevator and the ailerons for a modern fighter aircraft in a rolling pullout maneuver. The problem is solved for three flight conditions using the conjugate gradient method.

Asymptotically-Optimal, Fast-Decodable, Full-Diversity STBCs

For a family/sequence of Space-Time Block Codes (STBCs) C1, C2,⋯, with increasing number of transmit antennas Ni, with rates Ri complex symbols per channel use (cspcu), i = 1,2,⋯, the asymptotic normalized rate is defined as limi→∞ Ri/Ni. A family of STBCs is said to be asymptotically-good if the asymptotic normalized rate is non-zero, i.e., when the rate scales as a non-zero fraction of the number of transmit antennas, and the family of STBCs is said to be asymptotically-optimal if the asymptotic normalized rate is 1, which is the maximum possible value. In this paper, we construct a new class of full-diversity STBCs that have the least maximum-likelihood (ML) decoding complexity among all known codes for any number of transmit antennas N>;1 and rates R>;1 cspcu. For a large set of (R,N) pairs, the new codes have lower ML decoding complexity than the codes already available in the literature. Among the new codes, the class of full-rate codes (R=N) are asymptotically-optimal and fast-decodable, and for N>;5 have lower ML decoding complexity than all other families of asymptotically-optimal, fast-decodable, full-diversity STBCs available in the literature. The construction of the new STBCs is facilitated by the following further contributions of this paper: (i) Construction of a new class of asymptotically-good, full-diversity multigroup ML decodable codes, that not only includes STBCs for a larger set of antennas, but also either matches in rate or contains as a proper subset all other high-rate or asymptotically-good, delay-optimal, multigroup ML decodable codes available in the literature. (ii) Construction of a new class of fast-group-decodable codes (codes that combine the low ML decoding complexity properties of multigroup ML decodable codes and fast-decodable codes) for all even number of transmit antennas and rates 1 <; R ≤ 5/4.- - (iii) Given a design with full-rank linear dispersion matrices, we show that a full-diversity STBC can be constructed from this design by encoding the real symbols independently using only regular PAM constellations.

Optimal energy management policies for energy harvesting sensor nodes

We study a sensor node with an energy harvesting source. The generated energy can be stored in a buffer. The sensor node periodically senses a random field and generates a packet. These packets are stored in a queue and transmitted using the energy available at that time. We obtain energy management policies that are throughput optimal, i.e., the data queue stays stable for the largest possible data rate. Next we obtain energy management policies which minimize the mean delay in the queue. We also compare performance of several easily implementable sub-optimal energy management policies. A greedy policy is identified which, in low SNR regime, is throughput optimal and also minimizes mean delay.

A compliant mechanism kit with flexible beams and connectors along with analysis and optimal synthesis procedures

We present a compliant mechanism kit as a parallel to the kits available for rigid-body mechanisms. The kit consists of flexible beams and connectors that can be easily hand-assembled using snap fits. The mechanisms assembled using the kit accurately capture the aspects of the topology, shape, and size of joint-free compliant mechanisms. Thus, the kit enables designers to conceive and design new, practicable, single-piece compliant mechanisms that do not require assembly. The concept of the kit also resolves a discrepancy in the finite element (FE) modeling of beam-based compliant mechanisms. The discrepancy arises when two or more beams are joined at one point and thus leading to increased stiffness. After resolving this discrepancy, this work extends the topology optimization to automatically generate designs that can be assembled with the kit for quick and easy validation instead of time-consuming prototyping. Thus, the kit and the accompanying analysis and optimal synthesis procedures comprise a self-contained educational as well as a research and practice toolset for compliant mechanisms. The paper also illustrates how human creativity finds new ways of using the kit beyond the original intended use and how it enables even a novice to design compliant mechanisms. (C) 2011 Elsevier Ltd. All rights reserved.

The beta-Glucoside (bgl) Operon of Escherichia coli Is Involved in the Regulation of oppA, Encoding an Oligopeptide Transporter

We report that the bgl operon of Escherichia coli, encoding the functions necessary for the uptake and metabolism of aryl-beta-glucosides, is involved in the regulation of oligopeptide transport during stationary phase. Global analysis of intracellular proteins from Bgl-positive (Bgl(+)) and Bgl-negative (Bgl(-)) strains revealed that the operon exerts regulation on at least 12 downstream target genes. Of these, oppA, which encodes an oligopeptide transporter, was confirmed to be upregulated in the Bgl(+) strain. Loss of oppA function results in a partial loss of the growth advantage in stationary-phase (GASP) phenotype of Bgl(+) cells. The regulatory effect of the bgl operon on oppA expression is indirect and is mediated via gcvA, the activator of the glycine cleavage system, and gcvB, which regulates oppA at the posttranscriptional level. We show that BglG destabilizes the gcvA mRNA in vivo, leading to reduced expression of gcvA in the stationary phase. Deletion of gcvA results in the downregulation of gcvB and upregulation of oppA and can partially rescue the loss of the GASP phenotype seen in Delta bglG strains. A possible mechanism by which oppA confers a competitive advantage to Bgl(+) cells relative to Bgl(-) cells is discussed.