The Policy Iteration Algorithm for Average Continuous Control of Piecewise Deterministic Markov Processes

The main goal of this paper is to apply the so-called policy iteration algorithm (PIA) for the long run average continuous control problem of piecewise deterministic Markov processes (PDMP`s) taking values in a general Borel space and with compact action space depending on the state variable. In order to do that we first derive some important properties for a pseudo-Poisson equation associated to the problem. In the sequence it is shown that the convergence of the PIA to a solution satisfying the optimality equation holds under some classical hypotheses and that this optimal solution yields to an optimal control strategy for the average control problem for the continuous-time PDMP in a feedback form.

Galerkin Method and Weighting Functions Applied to Nonlinear H(infinity) Control with Output Feedback

This paper considers two aspects of the nonlinear H(infinity) control problem: the use of weighting functions for performance and robustness improvement, as in the linear case, and the development of a successive Galerkin approximation method for the solution of the Hamilton-Jacobi-Isaacs equation that arises in the output-feedback case. Design of nonlinear H(infinity) controllers obtained by the well-established Taylor approximation and by the proposed Galerkin approximation method applied to a magnetic levitation system are presented for comparison purposes.

Sampled Control for Mean-Variance Hedging in a Jump Diffusion Financial Market

In this technical note we consider the mean-variance hedging problem of a jump diffusion continuous state space financial model with the re-balancing strategies for the hedging portfolio taken at discrete times, a situation that more closely reflects real market conditions. A direct expression based on some change of measures, not depending on any recursions, is derived for the optimal hedging strategy as well as for the ""fair hedging price"" considering any given payoff. For the case of a European call option these expressions can be evaluated in a closed form.

On the convergence of successive Galerkin approximation for nonlinear output feedback H(infinity) control

Although the formulation of the nonlinear theory of H(infinity) control has been well developed, solving the Hamilton-Jacobi-Isaacs equation remains a challenge and is the major bottleneck for practical application of the theory. Several numerical methods have been proposed for its solution. In this paper, results on convergence and stability for a successive Galerkin approximation approach for nonlinear H(infinity) control via output feedback are presented. An example is presented illustrating the application of the algorithm.

Who should wear mask against airborne infections? Altering the contact network for controlling the spread of contagious diseases

There are several ways of controlling the propagation of a contagious disease. For instance, to reduce the spreading of an airborne infection, individuals can be encouraged to remain in their homes and/or to wear face masks outside their domiciles. However, when a limited amount of masks is available, who should use them: the susceptible subjects, the infective persons or both populations? Here we employ susceptible-infective-recovered (SIR) models described in terms of ordinary differential equations and probabilistic cellular automata in order to investigate how the deletion of links in the random complex network representing the social contacts among individuals affects the dynamics of a contagious disease. The inspiration for this study comes from recent discussions about the impact of measures usually recommended by health public organizations for preventing the propagation of the swine influenza A (H1N1) virus. Our answer to this question can be valid for other eco-epidemiological systems. (C) 2010 Elsevier BM. All rights reserved.

THE VANISHING DISCOUNT APPROACH FOR THE AVERAGE CONTINUOUS CONTROL OF PIECEWISE DETERMINISTIC MARKOV PROCESSES

This work is concerned with the existence of an optimal control strategy for the long-run average continuous control problem of piecewise-deterministic Markov processes (PDMPs). In Costa and Dufour (2008), sufficient conditions were derived to ensure the existence of an optimal control by using the vanishing discount approach. These conditions were mainly expressed in terms of the relative difference of the alpha-discount value functions. The main goal of this paper is to derive tractable conditions directly related to the primitive data of the PDMP to ensure the existence of an optimal control. The present work can be seen as a continuation of the results derived in Costa and Dufour (2008). Our main assumptions are written in terms of some integro-differential inequalities related to the so-called expected growth condition, and geometric convergence of the post-jump location kernel associated to the PDMP. An example based on the capacity expansion problem is presented, illustrating the possible applications of the results developed in the paper.

Recognition of noisy images by PLL networks

We use networks composed of three phase-locked loops (PLLs), where one of them is the master, for recognizing noisy images. The values of the coupling weights among the PLLs control the noise level which does not affect the successful identification of the input image. Analytical results and numerical tests are presented concerning the scheme performance. (c) 2008 Elsevier B.V. All rights reserved.

Optimized network structure for full-synchronization

A network of Kuramoto oscillators with different natural frequencies is optimized for enhanced synchronizability. All node inputs are normalized by the node connectivity and some important properties of the network Structure are determined in this case: (i) optimized networks present a strong anti-correlation between natural frequencies of adjacent nodes: (ii) this anti-correlation should be as high as possible since the average path length between nodes is maintained as small as in random networks: and (iii) high anti-correlation is obtained without any relation between nodes natural frequencies and the degree of connectivity. We also propose a network construction model with which it is shown that high anti-correlation and small average paths may be achieved by randomly rewiring a fraction of the links of a totally anti-correlated network, and that these networks present optimal synchronization properties. (C) 2008 Elsevier B.V. All rights reserved.

Using bifurcations in the determination of lock-in ranges for third-order phase-locked loops

Transmission and switching in digital telecommunication networks require distribution of precise time signals among the nodes. Commercial systems usually adopt a master-slave (MS) clock distribution strategy building slave nodes with phase-locked loop (PLL) circuits. PLLs are responsible for synchronizing their local oscillations with signals from master nodes, providing reliable clocks in all nodes. The dynamics of a PLL is described by an ordinary nonlinear differential equation, with order one plus the order of its internal linear low-pass filter. Second-order loops are commonly used because their synchronous state is asymptotically stable and the lock-in range and design parameters are expressed by a linear equivalent system [Gardner FM. Phaselock techniques. New York: John Wiley & Sons: 1979]. In spite of being simple and robust, second-order PLLs frequently present double-frequency terms in PD output and it is very difficult to adapt a first-order filter in order to cut off these components [Piqueira JRC, Monteiro LHA. Considering second-harmonic terms in the operation of the phase detector for second order phase-locked loop. IEEE Trans Circuits Syst [2003;50(6):805-9; Piqueira JRC, Monteiro LHA. All-pole phase-locked loops: calculating lock-in range by using Evan`s root-locus. Int J Control 2006;79(7):822-9]. Consequently, higher-order filters are used, resulting in nonlinear loops with order greater than 2. Such systems, due to high order and nonlinear terms, depending on parameters combinations, can present some undesirable behaviors, resulting from bifurcations, as error oscillation and chaos, decreasing synchronization ranges. In this work, we consider a second-order Sallen-Key loop filter [van Valkenburg ME. Analog filter design. New York: Holt, Rinehart & Winston; 1982] implying a third order PLL The resulting lock-in range of the third-order PLL is determined by two bifurcation conditions: a saddle-node and a Hopf. (C) 2008 Elsevier B.V. All rights reserved.

Discrete-time mean variance optimal control of linear systems with Markovian jumps and multiplicative noise

In this article, we consider the stochastic optimal control problem of discrete-time linear systems subject to Markov jumps and multiplicative noise under three kinds of performance criterions related to the final value of the expectation and variance of the output. In the first problem it is desired to minimise the final variance of the output subject to a restriction on its final expectation, in the second one it is desired to maximise the final expectation of the output subject to a restriction on its final variance, and in the third one it is considered a performance criterion composed by a linear combination of the final variance and expectation of the output of the system. We present explicit sufficient conditions for the existence of an optimal control strategy for these problems, generalising previous results in the literature. We conclude this article presenting a numerical example of an asset liabilities management model for pension funds with regime switching.

Comparison of friction models applied to a control valve

Eight different models to represent the effect of friction in control valves are presented: four models based on physical principles and four empirical ones. The physical models, both static and dynamic, have the same structure. The models are implemented in Simulink/Matlab (R) and compared, using different friction coefficients and input signals. Three of the models were able to reproduce the stick-slip phenomenon and passed all the tests, which were applied following ISA standards. (C) 2008 Elsevier Ltd. All rights reserved.

Synaptic compensation on Hopfield network: implications for memory rehabilitation

The discrete-time neural network proposed by Hopfield can be used for storing and recognizing binary patterns. Here, we investigate how the performance of this network on pattern recognition task is altered when neurons are removed and the weights of the synapses corresponding to these deleted neurons are divided among the remaining synapses. Five distinct ways of distributing such weights are evaluated. We speculate how this numerical work about synaptic compensation may help to guide experimental studies on memory rehabilitation interventions.

Postural control during kneeling

Postural control was studied when the subject was kneeling with erect trunk in a quiet posture and compared to that obtained during quiet standing. The analysis was based on the center of pressure motion in the sagittal plane (CPx), both in the time and in the frequency domains. One could assume that postural control during kneeling would be poorer than in standing because it is a less natural posture. This could cause a higher CPx variability. The power spectral density (PSD) of the CPx obtained from the experimental data in the kneeling position (KN) showed a significant decrease at frequencies below 0.3 Hz compared to upright (UP) (P < 0.01), which indicates less sway in KN. Conversely, there was an increase in fast postural oscillations (above 0.7 Hz) during KN compared to UP (P < 0.05). The root mean square (RMS) of the CPx was higher for UP (P < 0.01) while the mean velocity (MV) was higher during KN (P < 0.05). Lack of vision had a significant effect on the PSD and the parameters estimated from the CPx in both positions. We also sought to verify whether the changes in the PSD of the CPx found between the UP and KN positions were exclusively due to biomechanical factors (e.g., lowered center of gravity), or also reflected changes in the neural processes involved in the control of balance. To reach this goal, besides the experimental approach, a simple feedback model (a PID neural system, with added neural noise and controlling an inverted pendulum) was used to simulate postural sway in both conditions (in KN the pendulum was shortened, the mass and the moment of inertia were decreased). A parameter optimization method was used to fit the CPx power spectrum given by the model to that obtained experimentally. The results indicated that the changed anthropometric parameters in KN would indeed cause a large decrease in the power spectrum at low frequencies. However, the model fitting also showed that there were considerable changes also in the neural subsystem when the subject went from standing to kneeling. There was a lowering of the proportional and derivative gains and an increase in the neural noise power. Additional increases in the neural noise power were found also when the subject closed his eyes.

Flatness-based control of a single qubit gate

This work considers the open-loop control problem of steering a two-level quantum system from any initial to any final condition. The model of this system evolves on the state space X = SU(2), having two inputs that correspond to the complex amplitude of a resonant laser field. A symmetry preserving flat output is constructed using a fully geometric construction and quaternion computations. Simulation results of this flatness-based open-loop control are provided.

Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes

This paper deals with the expected discounted continuous control of piecewise deterministic Markov processes (PDMP`s) using a singular perturbation approach for dealing with rapidly oscillating parameters. The state space of the PDMP is written as the product of a finite set and a subset of the Euclidean space a""e (n) . The discrete part of the state, called the regime, characterizes the mode of operation of the physical system under consideration, and is supposed to have a fast (associated to a small parameter epsilon > 0) and a slow behavior. By using a similar approach as developed in Yin and Zhang (Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach, Applications of Mathematics, vol. 37, Springer, New York, 1998, Chaps. 1 and 3) the idea in this paper is to reduce the number of regimes by considering an averaged model in which the regimes within the same class are aggregated through the quasi-stationary distribution so that the different states in this class are replaced by a single one. The main goal is to show that the value function of the control problem for the system driven by the perturbed Markov chain converges to the value function of this limit control problem as epsilon goes to zero. This convergence is obtained by, roughly speaking, showing that the infimum and supremum limits of the value functions satisfy two optimality inequalities as epsilon goes to zero. This enables us to show the result by invoking a uniqueness argument, without needing any kind of Lipschitz continuity condition.