A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems

Even though dynamic programming offers an optimal control solution in a state feedback form, the method is overwhelmed by computational and storage requirements. Approximate dynamic programming implemented with an Adaptive Critic (AC) neural network structure has evolved as a powerful alternative technique that obviates the need for excessive computations and storage requirements in solving optimal control problems. In this paper, an improvement to the AC architecture, called the �Single Network Adaptive Critic (SNAC)� is presented. This approach is applicable to a wide class of nonlinear systems where the optimal control (stationary) equation can be explicitly expressed in terms of the state and costate variables. The selection of this terminology is guided by the fact that it eliminates the use of one neural network (namely the action network) that is part of a typical dual network AC setup. As a consequence, the SNAC architecture offers three potential advantages: a simpler architecture, lesser computational load and elimination of the approximation error associated with the eliminated network. In order to demonstrate these benefits and the control synthesis technique using SNAC, two problems have been solved with the AC and SNAC approaches and their computational performances are compared. One of these problems is a real-life Micro-Electro-Mechanical-system (MEMS) problem, which demonstrates that the SNAC technique is applicable to complex engineering systems.

Reduced Order Suboptimal Control Design for a Class of Nonlinear Distributed Parameter Systems Using POD and /spl thetas/-D Techniques

A new computational tool is presented in this paper for suboptimal control design of a class of nonlinear distributed parameter systems. First proper orthogonal decomposition based problem-oriented basis functions are designed, which are then used in a Galerkin projection to come up with a low-order lumped parameter approximation. Next, a suboptimal controller is designed using the emerging /spl thetas/-D technique for lumped parameter systems. This time domain sub-optimal control solution is then mapped back to the distributed domain using the same basis functions, which essentially leads to a closed form solution for the controller in a state feedback form. Numerical results for a real-life nonlinear temperature control problem indicate that the proposed method holds promise as a good suboptimal control design technique for distributed parameter systems.

Risk-sensitive optimal control for Markov decision processes with monotone cost

The existence of an optimal feedback law is established for the risk-sensitive optimal control problem with denumerable state space. The main assumptions imposed are irreducibility and a near monotonicity condition on the one-step cost function. A solution can be found constructively using either value iteration or policy iteration under suitable conditions on initial feedback law.

Benchmark Control Problem for Highway Bridge based on FLC

Fuzzy logic control (FLC) systems have been applied as an effective control system in various fields, including vibration control of structures. The advantage of this approach is its inherent robustness and ability to handle non‐linearities and uncertainties in structural behavior and loading. The study evaluates the three‐dimensional benchmark control problem for a seismically excited highway bridge using an ANFIS driven hydraulic actuators. An ANN based training strategy that considers both velocity and acceleration feedback together with a fuzzy logic rule base is developed. Present study needs only 4 accelerometers and 4 fuzzy rule bases to determine the control force, instead of 8 accelerometers and 4 displacement transducers used in the benchmark study problem. The results obtained are better than that obtained from the benchmark control algorithm.

Average variation L2-stability criteria for time-varying feedback systems-A unified approach

The problem of developing L2-stability criteria for feedback systems with a single time-varying gain, which impose average variation constraints on the gain is treated. A unified approach is presented which facilitates the development of such average variation criteria for both linear and nonlinear systems. The stability criteria derived here are shown to be more general than the existing results.

Performance analysis of subband-level channel quality indicator feedback scheme of LTE

Frequency-domain scheduling and rate adaptation enable next generation wireless cellular systems such as Long Term Evolution (LTE) to achieve significantly higher downlink throughput. LTE assigns subcarriers in chunks, called physical resource blocks (PRBs), to users to reduce control signaling overhead. To reduce the enormous feedback overhead, the channel quality indicator (CQI) report that is used to feed back channel state information is averaged over a subband, which, in turn, is a group of multiple PRBs. In this paper, we develop closed-form expressions for the throughput achieved by the subband-level CQI feedback mechanism of LTE. We show that the coarse frequency resolution of the CQI incurs a significant loss in throughput and limits the multi-user gains achievable by the system. We then show that the performance can be improved by means of an offset mechanism that effectively makes the users more conservative in reporting their CQI.

Control technique for non-linear and unbalance compensation of an integrated magnetics-based three-phase series-shunt-compensated uninterruptible power supply

Handling unbalanced and non-linear loads in a three-phase AC power supply has always been a difficult issue. This has been addressed in the literature by either using fast controllers in the fundamental rotating reference frame or using separate controllers in reference frames specific to the harmonics. In the former case, the controller needs to be fast and in the latter case, besides the need for many controllers, negative-sequence components need to be extracted from the measured signal. This study proposes a control scheme for harmonic and unbalance compensation of a three-phase uninterruptible power supply wherein the problems mentioned above are addressed. The control takes place in the fundamental positive-sequence reference frame using only a set of feedback and feed-forward compensators. The harmonic components are extracted by a process of frame transformations and used as feed-forward compensation terms in the positive-sequence fundamental reference frame. This study uses a method wherein the measured signal itself is used for fundamental negative-sequence compensation. As the feed-forward compensator handles the high-bandwidth components, the feedback compensator can be a simple low-bandwidth one. This control algorithm is explained and validated experimentally.

Control technique for non-linear and unbalance compensation of an integrated magnetics-based three-phase series-shunt-compensated uninterruptible power supply

Handling unbalanced and non-linear loads in a three-phase AC power supply has always been a difficult issue. This has been addressed in the literature by either using fast controllers in the fundamental rotating reference frame or using separate controllers in reference frames specific to the harmonics. In the former case, the controller needs to be fast and in the latter case, besides the need for many controllers, negative-sequence components need to be extracted from the measured signal. This study proposes a control scheme for harmonic and unbalance compensation of a three-phase uninterruptible power supply wherein the problems mentioned above are addressed. The control takes place in the fundamental positive-sequence reference frame using only a set of feedback and feed-forward compensators. The harmonic components are extracted by a process of frame transformations and used as feed-forward compensation terms in the positive-sequence fundamental reference frame. This study uses a method wherein the measured signal itself is used for fundamental negative-sequence compensation. As the feed-forward compensator handles the high-bandwidth components, the feedback compensator can be a simple low-bandwidth one. This control algorithm is explained and validated experimentally.

Control technique for non-linear and unbalance compensation of an integrated magnetics-based three-phase series-shunt-compensated uninterruptible power supply

Handling unbalanced and non-linear loads in a three-phase AC power supply has always been a difficult issue. This has been addressed in the literature by either using fast controllers in the fundamental rotating reference frame or using separate controllers in reference frames specific to the harmonics. In the former case, the controller needs to be fast and in the lattercase, besides the need for many controllers, negative-sequence components need to be extracted from the measured signal.This study proposes a control scheme for harmonic and unbalance compensation of a three-phase uninterruptible power supply wherein the problems mentioned above are addressed. The control takes place in the fundamental positive-sequence reference frame using only a set of feedback and feed-forward compensators. The harmonic components are extracted by process of frame transformations and used as feed-forward compensation terms in the positive-sequence fundamental reference frame. This study uses a method wherein the measured signal itself is used for fundamental negative-sequence compensation. As the feed-forward compensator handles the high-bandwidth components, the feedback compensator can be a simple low-bandwidth one. This control algorithm is explained and validated experimentally.

Transmit power Control with ARQ in energy harvesting sensors: a decision-theoretic approach

This paper addresses the problem of finding optimal power control policies for wireless energy harvesting sensor (EHS) nodes with automatic repeat request (ARQ)-based packet transmissions. The EHS harvests energy from the environment according to a Bernoulli process; and it is required to operate within the constraint of energy neutrality. The EHS obtains partial channel state information (CSI) at the transmitter through the link-layer ARQ protocol, via the ACK/NACK feedback messages, and uses it to adapt the transmission power for the packet (re)transmission attempts. The underlying wireless fading channel is modeled as a finite state Markov chain with known transition probabilities. Thus, the goal of the power management policy is to determine the best power setting for the current packet transmission attempt, so as to maximize a long-run expected reward such as the expected outage probability. The problem is addressed in a decision-theoretic framework by casting it as a partially observable Markov decision process (POMDP). Due to the large size of the state-space, the exact solution to the POMDP is computationally expensive. Hence, two popular approximate solutions are considered, which yield good power management policies for the transmission attempts. Monte Carlo simulation results illustrate the efficacy of the approach and show that the approximate solutions significantly outperform conventional approaches.

A media access and feedback protocol for reliable multicast over wireless channel

A link level reliable multicast requires a channel access protocol to resolve the collision of feedback messages sent by multicast data receivers. Several deterministic media access control protocols have been proposed to attain high reliability, but with large delay. Besides, there are also protocols which can only give probabilistic guarantee about reliability, but have the least delay. In this paper, we propose a virtual token-based channel access and feedback protocol (VTCAF) for link level reliable multicasting. The VTCAF protocol introduces a virtual (implicit) token passing mechanism based on carrier sensing to avoid the collision between feedback messages. The delay performance is improved in VTCAF protocol by reducing the number of feedback messages. Besides, the VTCAF protocol is parametric in nature and can easily trade off reliability with the delay as per the requirement of the underlying application. Such a cross layer design approach would be useful for a variety of multicast applications which require reliable communication with different levels of reliability and delay performance. We have analyzed our protocol to evaluate various performance parameters at different packet loss rate and compared its performance with those of others. Our protocol has also been simulated using Castalia network simulator to evaluate the same performance parameters. Simulation and analytical results together show that the VTCAF protocol is able to considerably reduce average access delay while ensuring very high reliability at the same time.