Average control of Markov decision processes with Feller transition probabilities and general action spaces

This paper studies the average control problem of discrete-time Markov Decision Processes (MDPs for short) with general state space, Feller transition probabilities, and possibly non-compact control constraint sets A(x). Two hypotheses are considered: either the cost function c is strictly unbounded or the multifunctions A(r)(x) = {a is an element of A(x) : c(x, a) <= r} are upper-semicontinuous and compact-valued for each real r. For these two cases we provide new results for the existence of a solution to the average-cost optimality equality and inequality using the vanishing discount approach. We also study the convergence of the policy iteration approach under these conditions. It should be pointed out that we do not make any assumptions regarding the convergence and the continuity of the limit function generated by the sequence of relative difference of the alpha-discounted value functions and the Poisson equations as often encountered in the literature. (C) 2012 Elsevier Inc. All rights reserved.

Linear matrix inequality-based robust model predictive control for time-delayed systems

This work addresses the solution to the problem of robust model predictive control (MPC) of systems with model uncertainty. The case of zone control of multi-variable stable systems with multiple time delays is considered. The usual approach of dealing with this kind of problem is through the inclusion of non-linear cost constraint in the control problem. The control action is then obtained at each sampling time as the solution to a non-linear programming (NLP) problem that for high-order systems can be computationally expensive. Here, the robust MPC problem is formulated as a linear matrix inequality problem that can be solved in real time with a fraction of the computer effort. The proposed approach is compared with the conventional robust MPC and tested through the simulation of a reactor system of the process industry.

Modified constraint-induced movement therapy and modified forced-use therapy for stroke patients are both effective to promote balance and gait improvements

Background: Previous studies show that chronic hemiparetic patients after stroke, presents inabilities to perform movements in paretic hemibody. This inability is induced by positive reinforcement of unsuccessful attempts, a concept called learned non-use. Forced use therapy (FUT) and constraint induced movement therapy (CIMT) were developed with the goal of reversing the learned non-use. These approaches have been proposed for the rehabilitation of the paretic upper limb (PUL). It is unknown what would be the possible effects of these approaches in the rehabilitation of gait and balance. Objectives: To evaluate the effect of Modified FUT (mFUT) and Modified CIMT (mCIMT) on the gait and balance during four weeks of treatment and 3 months follow-up. Methods: This study included thirty-seven hemiparetic post-stroke subjects that were randomly allocated into two groups based on the treatment protocol. The non-paretic UL was immobilized for a period of 23 hours per day, five days a week. Participants were evaluated at Baseline, 1st, 2nd, 3rd and 4th weeks, and three months after randomization. For the evaluation we used: The Stroke Impact Scale (SIS), Berg Balance Scale (BBS) and Fugl-Meyer Motor Assessment (FM). Gait was analyzed by the 10-meter walk test (T10) and Timed Up & Go test (TUG). Results: Both groups revealed a better health status (SIS), better balance, better use of lower limb (BBS and FM) and greater speed in gait (T10 and TUG), during the weeks of treatment and months of follow-up, compared to the baseline. Conclusion: The results show mFUT and mCIMT are effective in the rehabilitation of balance and gait. Trial Registration ACTRN12611000411943.