Output feedback control of discrete-time systems in networked environments

This correspondence paper addresses the problem of output feedback stabilization of control systems in networked environments with quality-of-service (QoS) constraints. The problem is investigated in discrete-time state space using Lyapunov’s stability theory and the linear inequality matrix technique. A new discrete-time modeling approach is developed to describe a networked control system (NCS) with parameter uncertainties and nonideal network QoS. It integrates a network-induced delay, packet dropout, and other network behaviors into a unified framework. With this modeling, an improved stability condition, which is dependent on the lower and upper bounds of the equivalent network-induced delay, is established for the NCS with norm-bounded parameter uncertainties. It is further extended for the output feedback stabilization of the NCS with nonideal QoS. Numerical examples are given to demonstrate the main results of the theoretical development.

Infinite-horizon policy-gradient estimation

Gradient-based approaches to direct policy search in reinforcement learning have received much recent attention as a means to solve problems of partial observability and to avoid some of the problems associated with policy degradation in value-function methods. In this paper we introduce GPOMDP, a simulation-based algorithm for generating a biased estimate of the gradient of the average reward in Partially Observable Markov Decision Processes (POMDPs) controlled by parameterized stochastic policies. A similar algorithm was proposed by Kimura, Yamamura, and Kobayashi (1995). The algorithm's chief advantages are that it requires storage of only twice the number of policy parameters, uses one free parameter β ∈ [0,1) (which has a natural interpretation in terms of bias-variance trade-off), and requires no knowledge of the underlying state. We prove convergence of GPOMDP, and show how the correct choice of the parameter β is related to the mixing time of the controlled POMDP. We briefly describe extensions of GPOMDP to controlled Markov chains, continuous state, observation and control spaces, multiple-agents, higher-order derivatives, and a version for training stochastic policies with internal states. In a companion paper (Baxter, Bartlett, & Weaver, 2001) we show how the gradient estimates generated by GPOMDP can be used in both a traditional stochastic gradient algorithm and a conjugate-gradient procedure to find local optima of the average reward. ©2001 AI Access Foundation and Morgan Kaufmann Publishers. All rights reserved.

Optimal strategies and minimax lower bounds for online convex games

A number of learning problems can be cast as an Online Convex Game: on each round, a learner makes a prediction x from a convex set, the environment plays a loss function f, and the learner’s long-term goal is to minimize regret. Algorithms have been proposed by Zinkevich, when f is assumed to be convex, and Hazan et al., when f is assumed to be strongly convex, that have provably low regret. We consider these two settings and analyze such games from a minimax perspective, proving minimax strategies and lower bounds in each case. These results prove that the existing algorithms are essentially optimal.

An infinite-horizon robust filter for uncertain hidden Markov Models with conditional relative entropy constraints

We consider a robust filtering problem for uncertain discrete-time, homogeneous, first-order, finite-state hidden Markov models (HMMs). The class of uncertain HMMs considered is described by a conditional relative entropy constraint on measures perturbed from a nominal regular conditional probability distribution given the previous posterior state distribution and the latest measurement. Under this class of perturbations, a robust infinite horizon filtering problem is first formulated as a constrained optimization problem before being transformed via variational results into an unconstrained optimization problem; the latter can be elegantly solved using a risk-sensitive information-state based filtering.

PAC-Bayes-empirical-Bernstein inequality

We present PAC-Bayes-Empirical-Bernstein inequality. The inequality is based on combination of PAC-Bayesian bounding technique with Empirical Bernstein bound. It allows to take advantage of small empirical variance and is especially useful in regression. We show that when the empirical variance is significantly smaller than the empirical loss PAC-Bayes-Empirical-Bernstein inequality is significantly tighter than PAC-Bayes-kl inequality of Seeger (2002) and otherwise it is comparable. PAC-Bayes-Empirical-Bernstein inequality is an interesting example of application of PAC-Bayesian bounding technique to self-bounding functions. We provide empirical comparison of PAC-Bayes-Empirical-Bernstein inequality with PAC-Bayes-kl inequality on a synthetic example and several UCI datasets.

Infinite Dimensional Slow Modulations In A Delayed Model For Orthogonal Cutting Vibrations

We apply the method of multiple scales (MMS) to a well known model of regenerative cutting vibrations in the large delay regime. By ``large'' we mean the delay is much larger than the time scale of typical cutting tool oscillations. The MMS upto second order for such systems has been developed recently, and is applied here to study tool dynamics in the large delay regime. The second order analysis is found to be much more accurate than first order analysis. Numerical integration of the MMS slow flow is much faster than for the original equation, yet shows excellent accuracy. The main advantage of the present analysis is that infinite dimensional dynamics is retained in the slow flow, while the more usual center manifold reduction gives a planar phase space. Lower-dimensional dynamical features, such as Hopf bifurcations and families of periodic solutions, are also captured by the MMS. Finally, the strong sensitivity of the dynamics to small changes in parameter values is seen clearly.

Scheme of squares: Part I. Systems formalism for potentiostatic studies

The so-called “Scheme of Squares”, displaying an interconnectivity of heterogeneous electron transfer and homogeneous (e.g., proton transfer) reactions, is analysed. Explicit expressions for the various partial currents under potentiostatic conditions are given. The formalism is applicable to several electrode geometries and models (e.g., semi-infinite linear diffusion, rotating disk electrodes, spherical or cylindrical systems) and the analysis is exact. The steady-state (t→∞) expressions for the current are directly given in terms of constant matrices whereas the transients are obtained as Laplace transforms that need to be inverted by approximation of numerical methods. The methodology employs a systems approach which replaces a system of partial differential equations (governing the concentrations of the several electroactive species) by an equivalent set of difference equations obeyed by the various partial currents.

Optimal hydrothermal load flow: Formulation and a successive approximation solution for fixed head systems

This paper deals with the optimal load flow problem in a fixed-head hydrothermal electric power system. Equality constraints on the volume of water available for active power generation at the hydro plants as well as inequality constraints on the reactive power generation at the voltage controlled buses are imposed. Conditions for optimal load flow are derived and a successive approximation algorithm for solving the optimal generation schedule is developed. Computer implementation of the algorithm is discussed, and the results obtained from the computer solution of test systems are presented.

On the growth of the Bergman kernel near an infinite-type point

We study diagonal estimates for the Bergman kernels of certain model domains in C-2 near boundary points that are of infinite type. To do so, we need a mild structural condition on the defining functions of interest that facilitates optimal upper and lower bounds. This is a mild condition; unlike earlier studies of this sort, we are able to make estimates for non-convex pseudoconvex domains as well. Thisn condition quantifies, in some sense, how flat a domain is at an infinite-type boundary point. In this scheme of quantification, the model domains considered below range-roughly speaking-from being mildly infinite-type'' to very flat at the infinite-type points.

Stability of systems with power-law non-linearities

It is shown that a sufficient condition for the asymptotic stability-in-the-large of an autonomous system containing a linear part with transfer function G(jω) and a non-linearity belonging to a class of power-law non-linearities with slope restriction [0, K] in cascade in a negative feedback loop is ReZ(jω)[G(jω) + 1 K] ≥ 0 for all ω where the multiplier is given by, Z(jω) = 1 + αjω + Y(jω) - Y(-jω) with a real, y(t) = 0 for t < 0 and ∫ 0 ∞ |y(t)|dt < 1 2c2, c2 being a constant associated with the class of non-linearity. Any allowable multiplier can be converted to the above form and this form leads to lesser restrictions on the parameters in many cases. Criteria for the case of odd monotonic non-linearities and of linear gains are obtained as limiting cases of the criterion developed. A striking feature of the present result is that in the linear case it reduces to the necessary and sufficient conditions corresponding to the Nyquist criterion. An inequality of the type |R(T) - R(- T)| ≤ 2c2R(0) where R(T) is the input-output cross-correlation function of the non-linearity, is used in deriving the results.

L2-Stability Of A Class Of Nonlinear-Systems

Sufficient conditions are given for the L2-stability of a class of feedback systems consisting of a linear operator G and a nonlinear gain function, either odd monotone or restricted by a power-law, in cascade, in a negative feedback loop. The criterion takes the form of a frequency-domain inequality, Re[1 + Z(jω)] G(jω) δ > 0 ω ε (−∞, +∞), where Z(jω) is given by, Z(jω) = β[Y1(jω) + Y2(jω)] + (1 − β)[Y3(jω) − Y3(−jω)], with 0 β 1 and the functions y1(·), y2(·) and y3(·) satisfying the time-domain inequalities, ∝−∞+∞¦y1(t) + y2(t)¦ dt 1 − ε, y1(·) = 0, t < 0, y2(·) = 0, t > 0 and ε > 0, and , c2 being a constant depending on the order of the power-law restricting the nonlinear function. The criterion is derived using Zames' passive operator theory and is shown to be more general than the existing criteria

Intermolecular interactions in pi -conjugated systems: Application to polyenes

A computational scheme has been developed for strongly interacting systems wherein the intermolecular interaction is introduced as a charge-induced-dipole term. Within this approximation, the model Hamiltonian is exactly solved using a valence-bond basis. The validity of the scheme has been checked by use of exact calculations on small model systems. The method has been applied to finite polyenes to study the shifts in the ground-state energies and dipole-allowed excited-state energies in the presence of neighbors. Our calculations show a red shift in the optical gap of the infinite polyene by 0.124 eV, which is rather small compared to the experimental red shift. This is traced to the larger inaccuracy in the calculated shift in the excited state. The calculated shift in the ground-state energies are more accurate and hence the method is better suited for studying the effect of intermolecular interactions on the properties of the ground state.

Infinite dimensional slow modulations in a well known delayed model for cutting tool vibrations

We apply the method of multiple scales (MMS) to a well-known model of regenerative cutting vibrations in the large delay regime. By ``large'' we mean the delay is much larger than the timescale of typical cutting tool oscillations. The MMS up to second order, recently developed for such systems, is applied here to study tool dynamics in the large delay regime. The second order analysis is found to be much more accurate than the first order analysis. Numerical integration of the MMS slow flow is much faster than for the original equation, yet shows excellent accuracy in that plotted solutions of moderate amplitudes are visually near-indistinguishable. The advantages of the present analysis are that infinite dimensional dynamics is retained in the slow flow, while the more usual center manifold reduction gives a planar phase space; lower-dimensional dynamical features, such as Hopf bifurcations and families of periodic solutions, are also captured by the MMS; the strong sensitivity of the slow modulation dynamics to small changes in parameter values, peculiar to such systems with large delays, is seen clearly; and though certain parameters are treated as small (or, reciprocally, large), the analysis is not restricted to infinitesimal distances from the Hopf bifurcation.

Power System Stabilizers Design for Interconnected Power Systems

This paper proposes a method of designing fixed parameter decentralized power system stabilizers (PSS) for interconnected multi-machine power systems. Conventional design technique using a single machine infinite bus approximation involves the frequency response estimation called the GEP(s) between the AVR input and the resultant electrical torque. This requires the knowledge of equivalent external reactance and infinite bus voltage or their estimated values at each machine. Other design techniques using P-Vr characteristics or residues are based on complete system information. In the proposed method, information available at the high voltage bus of the step-up transformer is used to set up a modified Heffron-Phillip's model. With this model it is possible to decide the structure of the PSS compensator and tune its parameters at each machine in the multi-machine environment, using only those signals that are available at the generating station. The efficacy of the proposed design technique has been evaluated on three of the most widely used test systems. The simulation results have shown that the performance of the proposed stabilizer is comparable to that which could be obtained by conventional design but without the need for the estimation and computation of external system parameters.