SINGULARLY PERTURBED DISCOUNTED MARKOV CONTROL PROCESSES IN A GENERAL STATE SPACE

This paper studies the asymptotic optimality of discrete-time Markov decision processes (MDPs) with general state space and action space and having weak and strong interactions. By using a similar approach as developed by Liu, Zhang, and Yin [Appl. Math. Optim., 44 (2001), pp. 105-129], the idea in this paper is to consider an MDP with general state and action spaces and to reduce the dimension of the state space by considering an averaged model. This formulation is often described by introducing a small parameter epsilon > 0 in the definition of the transition kernel, leading to a singularly perturbed Markov model with two time scales. Our objective is twofold. First it is shown that the value function of the control problem for the perturbed system converges to the value function of a limit averaged control problem as epsilon goes to zero. In the second part of the paper, it is proved that a feedback control policy for the original control problem defined by using an optimal feedback policy for the limit problem is asymptotically optimal. Our work extends existing results of the literature in the following two directions: the underlying MDP is defined on general state and action spaces and we do not impose strong conditions on the recurrence structure of the MDP such as Doeblin's condition.

Path planning for image based visual servoing

This thesis deals with Visual Servoing and its strictly connected disciplines like projective geometry, image processing, robotics and non-linear control. More specifically the work addresses the problem to control a robotic manipulator through one of the largely used Visual Servoing techniques: the Image Based Visual Servoing (IBVS). In Image Based Visual Servoing the robot is driven by on-line performing a feedback control loop that is closed directly in the 2D space of the camera sensor. The work considers the case of a monocular system with the only camera mounted on the robot end effector (eye in hand configuration). Through IBVS the system can be positioned with respect to a 3D fixed target by minimizing the differences between its initial view and its goal view, corresponding respectively to the initial and the goal system configurations: the robot Cartesian Motion is thus generated only by means of visual informations. However, the execution of a positioning control task by IBVS is not straightforward because singularity problems may occur and local minima may be reached where the reached image is very close to the target one but the 3D positioning task is far from being fulfilled: this happens in particular for large camera displacements, when the the initial and the goal target views are noticeably different. To overcame singularity and local minima drawbacks, maintaining the good properties of IBVS robustness with respect to modeling and camera calibration errors, an opportune image path planning can be exploited. This work deals with the problem of generating opportune image plane trajectories for tracked points of the servoing control scheme (a trajectory is made of a path plus a time law). The generated image plane paths must be feasible i.e. they must be compliant with rigid body motion of the camera with respect to the object so as to avoid image jacobian singularities and local minima problems. In addition, the image planned trajectories must generate camera velocity screws which are smooth and within the allowed bounds of the robot. We will show that a scaled 3D motion planning algorithm can be devised in order to generate feasible image plane trajectories. Since the paths in the image are off-line generated it is also possible to tune the planning parameters so as to maintain the target inside the camera field of view even if, in some unfortunate cases, the feature target points would leave the camera images due to 3D robot motions. To test the validity of the proposed approach some both experiments and simulations results have been reported taking also into account the influence of noise in the path planning strategy. The experiments have been realized with a 6DOF anthropomorphic manipulator with a fire-wire camera installed on its end effector: the results demonstrate the good performances and the feasibility of the proposed approach.

SPACECRAFT ATTITUDE CONTROL USING MAGNETIC ACTUATORS

This report presents a study on the problem of spacecraft attitude control using magnetic actuators. Several existing approaches are reviewed and one control strategy is implemented and simulated. A time-varying feedback control law achieving inertial pointing for magnetically actuated spacecraft is implemented. The report explains the modeling of the spacecraft rigid body dynamics, kinematics and attitude control in detail. Besides the fact that control laws have been established for stabilization around local equilibrium, this report presents the results of a control law that yields a generic, global solution for attitude stabilization of a magnetically actuated spacecraft. The report also involves the use MATLAB as a tool for both modeling and simulation of the spacecraft and controller. In conclusion, the simulation outlines the performance of the controller in independently stabilizing the spacecraft in three mutually perpendicular directions.

STRUCTURAL CONTROL OF A SMALL-SCALE TEST-BED SHAKER STRUCTURE USING A SPONGE-TYPE MAGNETO-RHEOLOGICAL FLUID DAMPER

Semi-active damping devices have been shown to be effective in mitigating unwanted vibrations in civil structures. These devices impart force indirectly through real-time alterations to structural properties. Simulating the complex behavior of these devices for laboratory-scale experiments is a major challenge. Commercial devices for seismic applications typically operate in the 2-10 kN range; this force is too high for small-scale testing applications where requirements typically range from 0-10 N. Several challenges must be overcome to produce damping forces at this level. In this study, a small-scale magneto-rheological (MR) damper utilizing a fluid absorbent metal foam matrix is developed and tested to accomplish this goal. This matrix allows magneto-rheological (MR) fluid to be extracted upon magnetic excitation in order to produce MR-fluid shear stresses and viscosity effects between an electromagnetic piston, the foam, and the damper housing. Dampers for uniaxial seismic excitation are traditionally positioned in the horizontal orientation allowing MR-fluid to gather in the lower part of the damper housing when partially filled. Thus, the absorbent matrix is placed in the bottom of the housing relieving the need to fill the entire device with MR-fluid, a practice that requires seals that add significant unwanted friction to the desired low-force device. The damper, once constructed, can be used in feedback control applications to reduce seismic vibrations and to test structural control algorithms and wireless command devices. To validate this device, a parametric study was performed utilizing force and acceleration measurements to characterize damper performance and controllability for this actuator. A discussion of the results is presented to demonstrate the attainment of the damper design objectives.

Estimación del consumo de potencia en un terminal portátil multimedia basada en la PMU del procesador

Este proyecto se incluye en una línea de trabajo que tiene como objetivo final la optimización de la energía consumida por un dispositivo portátil multimedia mediante la aplicación de técnicas de control realimentado, a partir de una modificación dinámica de la frecuencia de trabajo del procesador y de su tensión de alimentación. La modificación de frecuencia y tensión se realiza a partir de la información de realimentación acerca de la potencia consumida por el dispositivo, lo que supone un problema ya que no suele ser posible la monitorización del consumo de potencia en dispositivos de estas características. Este es el motivo por el que se recurre a la estimación del consumo de potencia, utilizando para ello un modelo de predicción. A partir del número de veces que se producen ciertos eventos en el procesador del dispositivo, el modelo de predicción es capaz de obtener una estimación de la potencia consumida por dicho dispositivo. El trabajo llevado a cabo en este proyecto se centra en la implementación de un modelo de estimación de potencia en el kernel de Linux. La razón por la que la estimación se implementa en el sistema operativo es, en primer lugar para lograr un acceso directo a los contadores del procesador. En segundo lugar, para facilitar la modificación de frecuencia y tensión, una vez obtenida la estimación de potencia, ya que esta también se realiza desde el sistema operativo. Otro motivo para implementar la estimación en el sistema operativo, es que la estimación debe ser independiente de las aplicaciones de usuario. Además, el proceso de estimación se realiza de forma periódica, lo que sería difícil de lograr si no se trabajase desde el sistema operativo. Es imprescindible que la estimación se haga de forma periódica ya que al ser dinámica la modificación de frecuencia y tensión que se pretende implementar, se necesita conocer el consumo de potencia del dispositivo en todo momento. Cabe destacar también, que los algoritmos de control se tienen que diseñar sobre un patrón periódico de actuación. El modelo de estimación de potencia funciona de manera específica para el perfil de consumo generado por una única aplicación determinada, que en este caso es un decodificador de vídeo. Sin embargo, es necesario que funcione de la forma más precisa posible para cada una de las frecuencias de trabajo del procesador, y para el mayor número posible de secuencias de vídeo. Esto es debido a que las sucesivas estimaciones de potencia se pretenden utilizar para llevar a cabo la modificación dinámica de frecuencia, por lo que el modelo debe ser capaz de continuar realizando las estimaciones independientemente de la frecuencia con la que esté trabajando el dispositivo. Para valorar la precisión del modelo de estimación se toman medidas de la potencia consumida por el dispositivo a las distintas frecuencias de trabajo durante la ejecución del decodificador de vídeo. Estas medidas se comparan con las estimaciones de potencia obtenidas durante esas mismas ejecuciones, obteniendo de esta forma el error de predicción cometido por el modelo y realizando las modificaciones y ajustes oportunos en el mismo. ABSTRACT. This project is included in a work line which tries to optimize consumption of handheld multimedia devices by the application of feedback control techniques, from a dynamic modification of the processor work frequency and its voltage. The frequency and voltage modification is performed depending on the feedback information about the device power consumption. This is a problem because normally it is not possible to monitor the power consumption on this kind of devices. This is the reason why a power consumption estimation is used instead, which is obtained from a prediction model. Using the number of times some events occur on the device processor, the prediction model is able to obtain a power consumption estimation of this device. The work done in this project focuses on the implementation of a power estimation model in the Linux kernel. The main reason to implement the estimation in the operating system is to achieve a direct access to the processor counters. The second reason is to facilitate the frequency and voltage modification, because this modification is also done from the operating system. Another reason to implement the estimation in the operating system is because the estimation must be done apart of the user applications. Moreover, the estimation process is done periodically, what is difficult to obtain outside the operating system. It is necessary to make the estimation in a periodic way because the frequency and voltage modification is going to be dynamic, so it needs to know the device power consumption at every time. Also, it is important to say that the control algorithms have to be designed over a periodic pattern of action. The power estimation model works specifically for the consumption profile generated by a single application, which in this case is a video decoder. Nevertheless, it is necessary that the model works as accurate as possible for each frequency available on the processor, and for the greatest number of video sequences. This is because the power estimations are going to be used to modify dynamically the frequency, so the model must be able to work independently of the device frequency. To value the estimation model precision, some measurements of the device power consumption are taken at different frequencies during the video decoder execution. These measurements are compared with the power estimations obtained during that execution, getting the prediction error committed by the model, and if it is necessary, making modifications and settings on this model.

Nedd4 mediates control of an epithelial Na+ channel in salivary duct cells by cytosolic Na+

Epithelial Na+ channels are expressed widely in absorptive epithelia such as the renal collecting duct and the colon and play a critical role in fluid and electrolyte homeostasis. Recent studies have shown that these channels interact via PY motifs in the C terminals of their α, β, and γ subunits with the WW domains of the ubiquitin-protein ligase Nedd4. Mutation or deletion of these PY motifs (as occurs, for example, in the heritable form of hypertension known as Liddle’s syndrome) leads to increased Na+ channel activity. Thus, binding of Nedd4 by the PY motifs would appear to be part of a physiological control system for down-regulation of Na+ channel activity. The nature of this control system is, however, unknown. In the present paper, we show that Nedd4 mediates the ubiquitin-dependent down-regulation of Na+ channel activity in response to increased intracellular Na+. We further show that Nedd4 operates downstream of Go in this feedback pathway. We find, however, that Nedd4 is not involved in the feedback control of Na+ channels by intracellular anions. Finally, we show that Nedd4 has no influence on Na+ channel activity when the Na+ and anion feedback systems are inactive. We conclude that Nedd4 normally mediates feedback control of epithelial Na+ channels by intracellular Na+, and we suggest that the increased Na+ channel activity observed in Liddle’s syndrome is attributable to the loss of this regulatory feedback system.

Suppression of tumor necrosis factor-induced cell death by inhibitor of apoptosis c-IAP2 is under NF-κB control

Members of the NF-κB/Rel and inhibitor of apoptosis (IAP) protein families have been implicated in signal transduction programs that prevent cell death elicited by the cytokine tumor necrosis factor α (TNF). Although NF-κB appears to stimulate the expression of specific protective genes, neither the identities of these genes nor the precise role of IAP proteins in this anti-apoptotic process are known. We demonstrate here that NF-κB is required for TNF-mediated induction of the gene encoding human c-IAP2. When overexpressed in mammalian cells, c-IAP2 activates NF-κB and suppresses TNF cytotoxicity. Both of these c-IAP2 activities are blocked in vivo by coexpressing a dominant form of IκB that is resistant to TNF-induced degradation. In contrast to wild-type c-IAP2, a mutant lacking the C-terminal RING domain inhibits NF-κB induction by TNF and enhances TNF killing. These findings suggest that c-IAP2 is critically involved in TNF signaling and exerts positive feedback control on NF-κB via an IκB targeting mechanism. Functional coupling of NF-κB and c-IAP2 during the TNF response may provide a signal amplification loop that promotes cell survival rather than death.

Diazepam binding inhibitor is a potent cholecystokinin-releasing peptide in the intestine.

Pancreatic proteases in the duodenum inhibit the release of cholecystokinin (CCK) and thus exert feedback control of pancreatic exocrine secretion. Exclusion of proteases from the duodenum either by the diversion of bile-pancreatic juice or by the addition of protease inhibitors stimulates exocrine pancreatic secretion. The mechanism by which pancreatic proteases in the duodenum regulate CCK secretion is unknown. In this study, we isolated a trypsin-sensitive peptide that is secreted intraduodenally, releases CCK, and stimulates pancreatic enzyme secretion in rats. This peptide was found to be identical to the porcine diazepam binding inhibitor by peptide sequencing and mass spectrometry analysis. Intraduodenal infusion of 200 ng of synthetic porcine diazepam binding inhibitor1-86 in rats significantly stimulated pancreatic amylase output. Infusion of the CCK antagonist MK-329 completely blocked the diazepam binding inhibitor-stimulated amylase secretion. Similarly, diazepam binding inhibitor33-52 [corrected] also stimulated CCK release and pancreatic secretion in a dose-dependent manner although it was 100 times less potent than the whole peptide. Using a perfusion system containing isolated mucosal cells from the proximal intestine of rats, porcine diazepam binding inhibitor 10(-12) M) dose dependently stimulated CCK secretion. In separate studies, it was demonstrated that luminal secretion of the diazepam binding inhibitor immunoreactivity (7.5 X 10(11) M) could be detected in rat's intestinal washing following the diversion of bile-pancreatic juice. The secretion of this peptide was inhibited by atropine. In conclusion, we have isolated and characterized a CCK-releasing peptide that has a sequence identical to the porcine diazepam binding inhibitor from pig intestinal mucosa and that stimulates CCK release when administered intraduodenally in rat. This peptide may mediate feedback regulation of pancreatic enzyme secretion.

Nonlinear control of heart rate variability in human infants.

Nonlinear analyses of infant heart rhythms reveal a marked rise in the complexity of the electrocardiogram with maturation. We find that normal mature infants (gestation greater than or equal to 35 weeks) have complex and distinctly nonlinear heart rhythms (consistent with recent reports for healthy adults) but that such nonlinearity is lacking in preterm infants (gestation > or = to 27 weeks) where parasympathetic-sympathetic interaction and function are presumed to be less well developed. Our study further shows that infants with clinical brain death and those treated with atropine exhibit a similar lack of nonlinear feedback control. These three lines of evidence support the hypothesis championed by Goldberger et al. [Goldberger, A.L., Rigney, D.R. & West, B.J. (1990) Sci. Am. 262, 43-49] that autonomic nervous system control underlies the nonlinearity and possible chaos of normal heart rhythms. This report demonstrates the acquisition of nonlinear heart rate dynamics and possible chaos in developing human infants and its loss in brain death and with the administration of atropine. It parallels earlier work documenting changes in the variability of heart rhythms in each of these cases and suggests that nonlinearity may provide additional power in characterizing physiological states.

The analysis and compensation of nonlinear sampled-data feedback systems.

"Research was supported by the United States Air Force through the Air Force Office of Scientific Research, Air Research and Development Command."

On adaptive control systems,

At head of title: Microwave Research Institute, Polytechnic Institute of Brooklyn, Systems and Control Group, R-735, PIB-663, contract no. DA-30-069-ORD-1560.

Sampled-data control systems /

Includes bibliography.

Suppressing chaos via Lyapunov-Krasovskii's method

An algorithm for suppressing the chaotic oscillations in non-linear dynamical systems with singular Jacobian matrices is developed using a linear feedback control law based upon the Lyapunov-Krasovskii (LK) method. It appears that the LK method can serve effectively as a generalised method for the suppression of chaotic oscillations for a wide range of systems. Based on this method, the resulting conditions for undisturbed motions to be locally or globally stable are sufficient and conservative. The generalized Lorenz system and disturbed gyrostat equations are exemplified for the validation of the proposed feedback control rule. (c) 2005 Elsevier Ltd. All rights reserved.

Biologically inspired joint control for a humanoid robot

The GuRm is a 1.2m tall, 23 degree of freedom humanoid consuucted at the University of Queensland for research into humanoid robotics. The key challenge being addressed by the GuRw projcct is the development of appropriate learning strategies for control and coodinadon of the robot’s many joints. The development of learning strategies is Seen as a way to sidestep the inherent intricacy of modeling a multi-DOP biped robot. This paper outlines the approach taken to generate an appmpria*e control scheme for the joinis of the GuRoo. The paper demonsrrates the determination of local feedback control parameters using a genetic algorithm. The feedback loop is then augmented by a predictive modulator that learns a form of feed-fonward control to overcome the irregular loads experienced at each joint during the gait cycle. The predictive modulator is based on thc CMAC architecture. Results from tats on the GuRoo platform show that both systems provide improvements in stability and tracking of joint control.

A model for motion sharpening: contrast gain control precedes compressive nonlinearity

Blurred edges appear sharper in motion than when they are stationary. We have previously shown how such distortions in perceived edge blur may be explained by a model which assumes that luminance contrast is encoded by a local contrast transducer whose response becomes progressively more compressive as speed increases. To test this model further, we measured the sharpening of drifting, periodic patterns over a large range of contrasts, blur widths, and speeds Human Vision. The results indicate that, while sharpening increased with speed, it was practically invariant with contrast. This contrast invariance cannot be explained by a fixed compressive nonlinearity since that predicts almost no sharpening at low contrasts.We show by computational modelling of spatiotemporal responses that, if a dynamic contrast gain control precedes the static nonlinear transducer, then motion sharpening, its speed dependence, and its invariance with contrast can be predicted with reasonable accuracy.