SINGULARLY PERTURBED DISCOUNTED MARKOV CONTROL PROCESSES IN A GENERAL STATE SPACE

This paper studies the asymptotic optimality of discrete-time Markov decision processes (MDPs) with general state space and action space and having weak and strong interactions. By using a similar approach as developed by Liu, Zhang, and Yin [Appl. Math. Optim., 44 (2001), pp. 105-129], the idea in this paper is to consider an MDP with general state and action spaces and to reduce the dimension of the state space by considering an averaged model. This formulation is often described by introducing a small parameter epsilon > 0 in the definition of the transition kernel, leading to a singularly perturbed Markov model with two time scales. Our objective is twofold. First it is shown that the value function of the control problem for the perturbed system converges to the value function of a limit averaged control problem as epsilon goes to zero. In the second part of the paper, it is proved that a feedback control policy for the original control problem defined by using an optimal feedback policy for the limit problem is asymptotically optimal. Our work extends existing results of the literature in the following two directions: the underlying MDP is defined on general state and action spaces and we do not impose strong conditions on the recurrence structure of the MDP such as Doeblin's condition.

Characteristic study, its identification and self-tuned approach to control hydro-power plants

The water time constant and mechanical time constant greatly influences the power and speed oscillations of hydro-turbine-generator unit. This paper discusses the turbine power transients in response to different nature and changes in the gate position. The work presented here analyses the characteristics of hydraulic system with an emphasis on changes in the above time constants. The simulation study is based on mathematical first-, second-, third- and fourth-order transfer function models. The study is further extended to identify discrete time-domain models and their characteristic representation without noise and with noise content of 10 & 20 dB signal-to-noise ratio (SNR). The use of self-tuned control approach in minimising the speed deviation under plant parameter changes and disturbances is also discussed.

Characterization of tomato Cycling Dof Factors reveals conserved and new functions in the control of flowering time and abiotic stress responses.

DNA binding with One Finger (DOF) transcription factors are involved in multiple aspects of plant growth and development but their precise roles in abiotic stress tolerance are largely unknown. Here we report a group of five tomato DOF genes, homologous to Arabidopsis Cycling DOF Factors (CDFs), that function as transcriptional regulators involved in responses to drought and salt stress and flowering-time control in a gene-specific manner. SlCDF1?5 are nuclear proteins that display specific binding with different affinities to canonical DNA target sequences and present diverse transcriptional activation capacities in vivo. SlCDF1?5 genes exhibited distinct diurnal expression patterns and were differentially induced in response to osmotic, salt, heat, and low-temperature stresses. Arabidopsis plants overexpressing SlCDF1 or SlCDF3 showed increased drought and salt tolerance. In addition, the expression of various stress-responsive genes, such as COR15, RD29A, and RD10, were differentially activated in the overexpressing lines. Interestingly, overexpression in Arabidopsis of SlCDF3 but not SlCDF1 promotes late flowering through modulation of the expression of flowering control genes such as CO and FT. Overall, our data connect SlCDFs to undescribed functions related to abiotic stress tolerance and flowering time through the regulation of specific target genes and an increase in particular metabolites

A control theoretic approach to solve a constrained uplink power dynamic game

This paper addresses an uplink power control dynamic game where we assume that each user battery represents the system state that changes with time following a discrete-time version of a differential game. To overcome the complexity of the analysis of a dynamic game approach we focus on the concept of Dynamic Potential Games showing that the game can be solved as an equivalent Multivariate Optimum Control Problem. The solution of this problem is quite interesting because different users split the activity in time, avoiding higher interferences and providing a long term fairness.

Análisis y Diseño de Algoritmos de Control Discreto de Sistemas MIMO Lineales y No Lineales Aplicando Técnicas de Control de Estructura Variable

Dentro de las técnicas de control de procesos no lineales, los controladores de estructura variable con modos deslizantes (VSC-SM en sus siglas en inglés) han demostrado ser una solución robusta, por lo cual han sido ampliamente estudiados en las cuatro últimas décadas. Desde los años ochenta se han presentado varios trabajos enfocados a especificar controladores VSC aplicados a sistemas de tiempo discreto (DVSC), siendo uno de los mayores intereses de análisis obtener las mismas prestaciones de robustez e invarianza de los controladores VSC-SM. El objetivo principal del trabajo de Tesis Doctoral consiste en estudiar, analizar y proponer unos esquemas de diseño de controladores DVSC en procesos multivariable tanto lineales como no lineales. De dicho estudio se propone una nueva filosofía de diseño de superficies deslizantes estables donde se han considerado aspectos hasta ahora no estudiados en el uso de DVSC-SM como son las limitaciones físicas de los actuadores y la dinámica deslizante no ideal. Lo más novedoso es 1) la propuesta de una nueva metodología de diseño de superficies deslizantes aplicadas a sistemas MIMO lineales y la extensión del mismo al caso de sistemas multivariables no lineales y 2) la definición de una nueva ley de alcance y de una ley de control robusta aplicada a sistemas MIMO, tanto lineales como no lineales, incluyendo un esquema de reducción de chattering. Finalmente, con el fin de ilustrar la eficiencia de los esquemas presentados, se incluyen ejemplos numéricos relacionados con el tema tratado en cada uno de los capítulos de la memoria. ABSTRACT Over the last four decades, variable structure controllers with sliding mode (VSC-SM) have been extensively studied, demonstrating to be a robust solution among robust nonlinear processes control techniques. Since the late 80s, several research works have been focused on the application of VSC controllers applied to discrete time or sampled data systems, which are known as DVSC-SM, where the most extensive source of analysis has been devoted to the robustness and invariance properties of VSC-SM controllers when applied to discrete systems. The main aim of this doctoral thesis work is to study, analyze and propose a design scheme of DVSC-SM controllers for lineal and nonlinear multivariable discrete time processes. For this purpose, a new design philosophy is proposed, where various design features have been considered that have not been analyzed in DVSC design approaches. Among them, the physical limitations and the nonideal dynamic sliding mode dynamics. The most innovative aspect is the inclusion of a new design methodology applied to lineal sliding surfaces MIMO systems and the extension to nonlinear multivariable systems, in addition to a new robust control law applied to lineal and nonlinear MIMO systems, including a chattering reduction scheme. Finally, to illustrate the efficiency of the proposed schemes, several numerical examples applied to lineal and nonlinear systems are included.

Functional analysis of a real-time protocol for networked control systems

Traditional real-time control systems are tightly integrated into the industrial processes they govern. Now, however, there is increasing interest in networked control systems. These provide greater flexibility and cost savings by allowing real-time controllers to interact with industrial processes over existing communications networks. New data packet queuing protocols are currently being developed to enable precise real-time control over a network with variable propagation delays. We show how one such protocol was formally modelled using timed automata, and how model checking was used to reveal subtle aspects of the control system's dynamic behaviour.

A structural method for the design of fault tolerant distributed control systems

Distributed digital control systems provide alternatives to conventional, centralised digital control systems. Typically, a modern distributed control system will comprise a multi-processor or network of processors, a communications network, an associated set of sensors and actuators, and the systems and applications software. This thesis addresses the problem of how to design robust decentralised control systems, such as those used to control event-driven, real-time processes in time-critical environments. Emphasis is placed on studying the dynamical behaviour of a system and identifying ways of partitioning the system so that it may be controlled in a distributed manner. A structural partitioning technique is adopted which makes use of natural physical sub-processes in the system, which are then mapped into the software processes to control the system. However, communications are required between the processes because of the disjoint nature of the distributed (i.e. partitioned) state of the physical system. The structural partitioning technique, and recent developments in the theory of potential controllability and observability of a system, are the basis for the design of controllers. In particular, the method is used to derive a decentralised estimate of the state vector for a continuous-time system. The work is also extended to derive a distributed estimate for a discrete-time system. Emphasis is also given to the role of communications in the distributed control of processes and to the partitioning technique necessary to design distributed and decentralised systems with resilient structures. A method is presented for the systematic identification of necessary communications for distributed control. It is also shwon that the structural partitions can be used directly in the design of software fault tolerant concurrent controllers. In particular, the structural partition can be used to identify the boundary of the conversation which can be used to protect a specific part of the system. In addition, for certain classes of system, the partitions can be used to identify processes which may be dynamically reconfigured in the event of a fault. These methods should be of use in the design of robust distributed systems.

Neural network enhanced self tuning adaptive control application for non-linear control of dynamic systems

The main theme of research of this project concerns the study of neutral networks to control uncertain and non-linear control systems. This involves the control of continuous time, discrete time, hybrid and stochastic systems with input, state or output constraints by ensuring good performances. A great part of this project is devoted to the opening of frontiers between several mathematical and engineering approaches in order to tackle complex but very common non-linear control problems. The objectives are: 1. Design and develop procedures for neutral network enhanced self-tuning adaptive non-linear control systems; 2. To design, as a general procedure, neural network generalised minimum variance self-tuning controller for non-linear dynamic plants (Integration of neural network mapping with generalised minimum variance self-tuning controller strategies); 3. To develop a software package to evaluate control system performances using Matlab, Simulink and Neural Network toolbox. An adaptive control algorithm utilising a recurrent network as a model of a partial unknown non-linear plant with unmeasurable state is proposed. Appropriately, it appears that structured recurrent neural networks can provide conveniently parameterised dynamic models for many non-linear systems for use in adaptive control. Properties of static neural networks, which enabled successful design of stable adaptive control in the state feedback case, are also identified. A survey of the existing results is presented which puts them in a systematic framework showing their relation to classical self-tuning adaptive control application of neural control to a SISO/MIMO control. Simulation results demonstrate that the self-tuning design methods may be practically applicable to a reasonably large class of unknown linear and non-linear dynamic control systems.

A probabilistic indirect adaptive control for systems with input-dependent noise

A probabilistic indirect adaptive controller is proposed for the general nonlinear multivariate class of discrete time system. The proposed probabilistic framework incorporates input–dependent noise prediction parameters in the derivation of the optimal control law. Moreover, because noise can be nonstationary in practice, the proposed adaptive control algorithm provides an elegant method for estimating and tracking the noise. For illustration purposes, the developed method is applied to the affine class of nonlinear multivariate discrete time systems and the desired result is obtained: the optimal control law is determined by solving a cubic equation and the distribution of the tracking error is shown to be Gaussian with zero mean. The efficiency of the proposed scheme is demonstrated numerically through the simulation of an affine nonlinear system.

Probabilistic DHP adaptive critic for nonlinear stochastic control systems

Following the recently developed algorithms for fully probabilistic control design for general dynamic stochastic systems (Herzallah & Káarnáy, 2011; Kárný, 1996), this paper presents the solution to the probabilistic dual heuristic programming (DHP) adaptive critic method (Herzallah & Káarnáy, 2011) and randomized control algorithm for stochastic nonlinear dynamical systems. The purpose of the randomized control input design is to make the joint probability density function of the closed loop system as close as possible to a predetermined ideal joint probability density function. This paper completes the previous work (Herzallah & Kárnáy, 2011; Kárný, 1996) by formulating and solving the fully probabilistic control design problem on the more general case of nonlinear stochastic discrete time systems. A simulated example is used to demonstrate the use of the algorithm and encouraging results have been obtained.

Distributed Extremum-Seeking Control with Applications to High-Altitude Balloons

The real-time optimization of large-scale systems is a difficult problem due to the need for complex models involving uncertain parameters and the high computational cost of solving such problems by a decentralized approach. Extremum-seeking control (ESC) is a model-free real-time optimization technique which can estimate unknown parameters and can optimize nonlinear time-varying systems using only a measurement of the cost function to be minimized. In this thesis, we develop a distributed version of extremum-seeking control which allows large-scale systems to be optimized without models and with minimal computing power. First, we develop a continuous-time distributed extremum-seeking controller. It has three main components: consensus, parameter estimation, and optimization. The consensus provides each local controller with an estimate of the cost to be minimized, allowing them to coordinate their actions. Using this cost estimate, parameters for a local input-output model are estimated, and the cost is minimized by following a gradient descent based on the estimate of the gradient. Next, a similar distributed extremum-seeking controller is developed in discrete-time. Finally, we consider an interesting application of distributed ESC: formation control of high-altitude balloons for high-speed wireless internet. These balloons must be steered into a favourable formation where they are spread out over the Earth and provide coverage to the entire planet. Distributed ESC is applied to this problem, and is shown to be effective for a system of 1200 ballons subjected to realistic wind currents. The approach does not require a wind model and uses a cost function based on a Voronoi partition of the sphere. Distributed ESC is able to steer balloons from a few initial launch sites into a formation which provides coverage to the entire Earth and can maintain a similar formation as the balloons move with the wind around the Earth.

Real-time load-side control of electric power systems

Two trends are emerging from modern electric power systems: the growth of renewable (e.g., solar and wind) generation, and the integration of information technologies and advanced power electronics. The former introduces large, rapid, and random fluctuations in power supply, demand, frequency, and voltage, which become a major challenge for real-time operation of power systems. The latter creates a tremendous number of controllable intelligent endpoints such as smart buildings and appliances, electric vehicles, energy storage devices, and power electronic devices that can sense, compute, communicate, and actuate. Most of these endpoints are distributed on the load side of power systems, in contrast to traditional control resources such as centralized bulk generators. This thesis focuses on controlling power systems in real time, using these load side resources. Specifically, it studies two problems.

(1) Distributed load-side frequency control: We establish a mathematical framework to design distributed frequency control algorithms for flexible electric loads. In this framework, we formulate a category of optimization problems, called optimal load control (OLC), to incorporate the goals of frequency control, such as balancing power supply and demand, restoring frequency to its nominal value, restoring inter-area power flows, etc., in a way that minimizes total disutility for the loads to participate in frequency control by deviating from their nominal power usage. By exploiting distributed algorithms to solve OLC and analyzing convergence of these algorithms, we design distributed load-side controllers and prove stability of closed-loop power systems governed by these controllers. This general framework is adapted and applied to different types of power systems described by different models, or to achieve different levels of control goals under different operation scenarios. We first consider a dynamically coherent power system which can be equivalently modeled with a single synchronous machine. We then extend our framework to a multi-machine power network, where we consider primary and secondary frequency controls, linear and nonlinear power flow models, and the interactions between generator dynamics and load control.

(2) Two-timescale voltage control: The voltage of a power distribution system must be maintained closely around its nominal value in real time, even in the presence of highly volatile power supply or demand. For this purpose, we jointly control two types of reactive power sources: a capacitor operating at a slow timescale, and a power electronic device, such as a smart inverter or a D-STATCOM, operating at a fast timescale. Their control actions are solved from optimal power flow problems at two timescales. Specifically, the slow-timescale problem is a chance-constrained optimization, which minimizes power loss and regulates the voltage at the current time instant while limiting the probability of future voltage violations due to stochastic changes in power supply or demand. This control framework forms the basis of an optimal sizing problem, which determines the installation capacities of the control devices by minimizing the sum of power loss and capital cost. We develop computationally efficient heuristics to solve the optimal sizing problem and implement real-time control. Numerical experiments show that the proposed sizing and control schemes significantly improve the reliability of voltage control with a moderate increase in cost.

AN ENERGY BASED MINIMUM-TIME OPTIMAL CONTROL OF DC-DC CONVERTERS

Time-optimal response is an important and sometimes necessary characteristic of dynamic systems for specific applications. Power converters are widely used in different electrical systems and their dynamic response will affect the whole system. In many electrical systems like microgrids or voltage regulators which supplies sensitive loads fast dynamic response is a must. Minimum time is the fastest converter to compensate the step output reference or load change. Boost converters as one of the wildly used power converters in the electrical systems are aimed to be controlled in optimal time in this study. Linear controllers are not able to provide the optimal response for a boost converter however they are still useful and functional for other applications like reference tracking or stabilization. To obtain the fastest possible response from boost converters, a nonlinear control approach based on the total energy of the system is studied in this research. Total energy of the system considers as the basis for developing the presented method, since it is easy and accurate to measure besides that the total energy of the system represents the actual operating condition of the boost converter. The detailed model of a boost converter is simulated in MATLAB/Simulink to achieve the time optimal response of the boost converter by applying the developed method. The simulation results confirmed the ability of the presented method to secure the time optimal response of the boost converter under four different scenarios.

Stability and ergodicity of piecewise deterministic Markov processes

The main goal of this paper is to establish some equivalence results on stability, recurrence, and ergodicity between a piecewise deterministic Markov process ( PDMP) {X( t)} and an embedded discrete-time Markov chain {Theta(n)} generated by a Markov kernel G that can be explicitly characterized in terms of the three local characteristics of the PDMP, leading to tractable criterion results. First we establish some important results characterizing {Theta(n)} as a sampling of the PDMP {X( t)} and deriving a connection between the probability of the first return time to a set for the discrete-time Markov chains generated by G and the resolvent kernel R of the PDMP. From these results we obtain equivalence results regarding irreducibility, existence of sigma-finite invariant measures, and ( positive) recurrence and ( positive) Harris recurrence between {X( t)} and {Theta(n)}, generalizing the results of [ F. Dufour and O. L. V. Costa, SIAM J. Control Optim., 37 ( 1999), pp. 1483-1502] in several directions. Sufficient conditions in terms of a modified Foster-Lyapunov criterion are also presented to ensure positive Harris recurrence and ergodicity of the PDMP. We illustrate the use of these conditions by showing the ergodicity of a capacity expansion model.

PARTIAL STABILITY FOR A CLASS OF NONLINEAR SYSTEMS

This paper studies a nonlinear, discrete-time matrix system arising in the stability analysis of Kalman filters. These systems present an internal coupling between the state components that gives rise to complex dynamic behavior. The problem of partial stability, which requires that a specific component of the state of the system converge exponentially, is studied and solved. The convergent state component is strongly linked with the behavior of Kalman filters, since it can be used to provide bounds for the error covariance matrix under uncertainties in the noise measurements. We exploit the special features of the system-mainly the connections with linear systems-to obtain an algebraic test for partial stability. Finally, motivated by applications in which polynomial divergence of the estimates is acceptable, we study and solve a partial semistability problem.