Solution of MDPS using simulation-based value iteration

This article proposes a three-timescale simulation based algorithm for solution of infinite horizon Markov Decision Processes (MDPs). We assume a finite state space and discounted cost criterion and adopt the value iteration approach. An approximation of the Dynamic Programming operator T is applied to the value function iterates. This 'approximate' operator is implemented using three timescales, the slowest of which updates the value function iterates. On the middle timescale we perform a gradient search over the feasible action set of each state using Simultaneous Perturbation Stochastic Approximation (SPSA) gradient estimates, thus finding the minimizing action in T. On the fastest timescale, the 'critic' estimates, over which the gradient search is performed, are obtained. A sketch of convergence explaining the dynamics of the algorithm using associated ODEs is also presented. Numerical experiments on rate based flow control on a bottleneck node using a continuous-time queueing model are performed using the proposed algorithm. The results obtained are verified against classical value iteration where the feasible set is suitably discretized. Over such a discretized setting, a variant of the algorithm of [12] is compared and the proposed algorithm is found to converge faster.

A unified model for auxiliary switch commutated DC-DC converters

A novel ZVS auxiliary switch commutated variation for all DGDC converter topologies has been proposed in 2006. With proper designation of the circuit variables (throw current I and the pole voltage V), all these converters are seen to be governed by an identical set of equations. With idealized switches, the steady-state performance is obtainable in an analytical form. The conversion ratio of the converter topologies is obtained. A generalized equivalent circuit emerges for all these converters from the steady-state conversion ratio. It also provides a dynamic model as well. With these generalized steady-state equivalent circuits, small signal analysis of these converters may be carried out readily. It enables one to use the familiar state space averaged results of the standard PWM DGDC converters for the resonant counterparts. Th dc and ac models reveals that dc and low frequency behaviour of the proposed family of converters is similiar to that of its PWM parent

Robust multivariable controllers for a tublar ammonia reactor

The specific objective of this paper is to develop multivariable controllers that would achieve asymptotic regulation in the presence of parameter variations and disturbance inputs for a tubular reactor used in ammonia synthesis. A ninth order state space model with three control inputs and two disturbance inputs is generated from the nonlinear distributed model using linearization and lumping approximations. Using this model, an approach for control system design is developed keeping in view the imperfections of the model and the measurability of the state variables. Specifically, the design of feedforward and robust integral controllers using state and output feedback is considered. Also, the design of robust multiloop proportional integral controllers is presented. Finally the performance of these controllers is evaluated through simulation.

Analysis Of The Hvdc Turbine Generator Torsional Interactions

In the recent past it has been found that HVDC transmission systems and turbine-generator shaft torsional dynamics can interact in an unfavourable manner. This paper presents a detailed linearised state space model of AC/DC system to study this torsional interaction. The model developed is used to study the effect of various system parameters, such as, dc line loading, converter firing angle, the firing scheme employed. The results obtained are compared with those given in[3].

Multicriterion differential games with applications to combat problems

In this paper a theory for two-person zero sum multicriterion differential games is presented. Various solution concepts based upon the notions of Pareto optimality (efficiency), security and equilibrium are defined. These are shown to have interesting applications in the formulation and analysis of two target or combat differential games. The methods for obtaining outcome regions in the state space, feedback strategies for the players and the mode of play has been discussed in the framework of bicriterion zero sum differential games. The treatment is conceptual rather than rigorous.

Free vibration of circular plates of arbitrary thickness

Free vibration of circular plates of arbitrary thickness is investigated using the method of initial functions. State-space approach is used to derive the governing equations of the above method. The formulation is such that theories of any desired order can be obtained by deleting higher terms in the infinite-order differential equations. Numerical results are obtained for flexural and extensional vibration of circular plates. Results are also computed using Mindlin's theory and they are in agreement with the present analysis.

Free vibration of rectangular beams of arbitrary depth

The state space approach is extended to the two dimensional elastodynamic problems. The formulation is in a form particularly amenable to consistent reduction to obtain approximate theories of any desired order. Free vibration of rectangular beams of arbitrary depth is investigated using this approach. The method does not involve the concept of the shear coefficientk. It takes into account the vertical normal stress and the transverse shear stress. The frequency values are calculated using the Timoshenko beam theory and the present analysis for different values of Poisson's ratio and they are in good agreement. Four cases of beams with different end conditions are considered.Die Zustandsraum-Technik wird auf zweidimensionale elastodynamische Probleme ausgedehnt. Die Formulierung ist besonders geeignet für die Aufstellung von Näherungstheorien beliebigen Grades. Freie Schwingungen von Rechteckbalken beliebiger Höhe wurden mit Hilfe dieser Technik untersucht. Das Verfahren umgeht den Begriff des Schubbeiwertsk. Es berücksichtigt die senkrechte Normalbeanspruchung und die Querkraft. Die Frequenzwerte werden mit Hilfe der Balkentheorie von Timoshenko und der vorliegenden Analyse berechnet, und zwar für verschiedene Werte der Querdehnzahl. Die berechneten Werte befinden sich in guter Übereinstimmung. Vier Fälle von Balken mit verschiedenen Endbedingungen werden untersucht.

Analysis of Torsional Interactions with Power System Stabilizer

Torsional interactions can occur due to the speed input Power System Stabilizer (PSS) that are primarily used to damp low frequency oscillations. The solution to this problem can be either in the form of providing a torsional filter or developing an alternate signal for the PSS. This paper deals with the formulation of a linearized state space model of the system and study of the interactions using eigenvalue analysis. The effects of the parameters of PSS and control signals on the damping of torsional modes are investigated.

On the Milito-Cruz adaptive control scheme for Markov chains

Milito and Cruz have introduced a novel adaptive control scheme for finite Markov chains when a finite parametrized family of possible transition matrices is available. The scheme involves the minimization of a composite functional of the observed history of the process incorporating both control and estimation aspects. We prove the a.s. optimality of a similar scheme when the state space is countable and the parameter space a compact subset ofR.

Weighted subspace methods and spatial smoothing: analysis and comparison

The effect of using a spatially smoothed forward-backward covariance matrix on the performance of weighted eigen-based state space methods/ESPRIT, and weighted MUSIC for direction-of-arrival (DOA) estimation is analyzed. Expressions for the mean-squared error in the estimates of the signal zeros and the DOA estimates, along with some general properties of the estimates and optimal weighting matrices, are derived. A key result is that optimally weighted MUSIC and weighted state-space methods/ESPRIT have identical asymptotic performance. Moreover, by properly choosing the number of subarrays, the performance of unweighted state space methods can be significantly improved. It is also shown that the mean-squared error in the DOA estimates is independent of the exact distribution of the source amplitudes. This results in a unified framework for dealing with DOA estimation using a uniformly spaced linear sensor array and the time series frequency estimation problems.

Risk-sensitive optimal control for Markov decision processes with monotone cost

The existence of an optimal feedback law is established for the risk-sensitive optimal control problem with denumerable state space. The main assumptions imposed are irreducibility and a near monotonicity condition on the one-step cost function. A solution can be found constructively using either value iteration or policy iteration under suitable conditions on initial feedback law.

Discrete control algorithms for a tubular ammonia reactor

The specific objective of this paper is to develop direct digital control strategies for an ammonia reactor using quadratic regulator theory and compare the performance of the resultant control system with that under conventional PID regulators. The controller design studies are based on a ninth order state-space model obtained from the exact nonlinear distributed model using linearization and lumping approximations. The evaluation of these controllers with reference to their disturbance rejection capabilities and transient response characteristics, is carried out using hybrid computer simulation.

Decision tree design and applications in speech processing

The design and operation of the minimum cost classifier, where the total cost is the sum of the measurement cost and the classification cost, is computationally complex. Noting the difficulties associated with this approach, decision tree design directly from a set of labelled samples is proposed in this paper. The feature space is first partitioned to transform the problem to one of discrete features. The resulting problem is solved by a dynamic programming algorithm over an explicitly ordered state space of all outcomes of all feature subsets. The solution procedure is very general and is applicable to any minimum cost pattern classification problem in which each feature has a finite number of outcomes. These techniques are applied to (i) voiced, unvoiced, and silence classification of speech, and (ii) spoken vowel recognition. The resulting decision trees are operationally very efficient and yield attractive classification accuracies.

Modeling stochastic hybrid systems

Stochastic hybrid systems arise in numerous applications of systems with multiple models; e.g., air traffc management, flexible manufacturing systems, fault tolerant control systems etc. In a typical hybrid system, the state space is hybrid in the sense that some components take values in a Euclidean space, while some other components are discrete. In this paper we propose two stochastic hybrid models, both of which permit diffusion and hybrid jump. Such models are essential for studying air traffic management in a stochastic framework.

Dynamical behavior in the nonlinear rheology of surfactant solutions

Several surfactant molecules self-assemble in solution to form long, flexible wormlike micelles which get entangled with each other, leading to viscoelastic gel phases. We discuss our recent work on the rheology of such a gel formed in the dilute aqueous solutions of a surfactant CTAT. In the linear rheology regime, the storage modulus G′(ω) and loss modulus G″(ω) have been measured over a wide frequency range. In the nonlinear regime, the shear stress σ shows a plateau as a function of the shear rate math above a certain cutoff shear rate mathc. Under controlled shear rate conditions in the plateau regime, the shear stress and the first normal stress difference show oscillatory time-dependence. The analysis of the measured time series of shear stress and normal stress has been done using several methods incorporating state space reconstruction by embedding of time delay vectors. The analysis shows the existence of a finite correlation dimension and a positive Lyapunov exponent, unambiguously implying that the dynamics of the observed mechanical instability can be described by that of a dynamical system with a strange attractor of dimension varying from 2.4 to 2.9.