SINGULARLY PERTURBED DISCOUNTED MARKOV CONTROL PROCESSES IN A GENERAL STATE SPACE

This paper studies the asymptotic optimality of discrete-time Markov decision processes (MDPs) with general state space and action space and having weak and strong interactions. By using a similar approach as developed by Liu, Zhang, and Yin [Appl. Math. Optim., 44 (2001), pp. 105-129], the idea in this paper is to consider an MDP with general state and action spaces and to reduce the dimension of the state space by considering an averaged model. This formulation is often described by introducing a small parameter epsilon > 0 in the definition of the transition kernel, leading to a singularly perturbed Markov model with two time scales. Our objective is twofold. First it is shown that the value function of the control problem for the perturbed system converges to the value function of a limit averaged control problem as epsilon goes to zero. In the second part of the paper, it is proved that a feedback control policy for the original control problem defined by using an optimal feedback policy for the limit problem is asymptotically optimal. Our work extends existing results of the literature in the following two directions: the underlying MDP is defined on general state and action spaces and we do not impose strong conditions on the recurrence structure of the MDP such as Doeblin's condition.

Continuity of Dynamical Structures for Nonautonomous Evolution Equations Under Singular Perturbations

In this paper we study the continuity of invariant sets for nonautonomous infinite-dimensional dynamical systems under singular perturbations. We extend the existing results on lower-semicontinuity of attractors of autonomous and nonautonomous dynamical systems. This is accomplished through a detailed analysis of the structure of the invariant sets and its behavior under perturbation. We prove that a bounded hyperbolic global solutions persists under singular perturbations and that their nonlinear unstable manifold behave continuously. To accomplish this, we need to establish results on roughness of exponential dichotomies under these singular perturbations. Our results imply that, if the limiting pullback attractor of a nonautonomous dynamical system is the closure of a countable union of unstable manifolds of global bounded hyperbolic solutions, then it behaves continuously (upper and lower) under singular perturbations.

Analytic perturbation theory for bound states in the transfer matrix spectrum of weakly correlated lattice ferromagnetic spin systems

We consider general d-dimensional lattice ferromagnetic spin systems with nearest neighbor interactions in the high temperature region ('beta' << 1). Each model is characterized by a single site apriori spin distribution taken to be even. We also take the parameter 'alfa' = ('S POT.4') - 3 '(S POT.2') POT.2' > 0, i.e. in the region which we call Gaussian subjugation, where ('S POT.K') denotes the kth moment of the apriori distribution. Associated with the model is a lattice quantum field theory known to contain a particle of asymptotic mass -ln 'beta' and a bound state below the two-particle threshold. We develop a 'beta' analytic perturbation theory for the binding energy of this bound state. As a key ingredient in obtaining our result we show that the Fourier transform of the two-point function is a meromorphic function, with a simple pole, in a suitable complex spectral parameter and the coefficients of its Laurent expansion are analytic in 'beta'.

Postural adjustment after an unexpected perturbation in children with haemophilia

. Children with haemophilia often bleed inside joints and muscles, which may impair postural adjustments. These postural adjustments are necessary to control postural balance during daily activities. The inability to quickly recover postural balance could elevate the risk of bleeding. To determine whether children with haemophilia have impaired postural adjustment after an unexpected perturbation compared with healthy children. Twenty children with haemophilia comprised the haemophilic group (HG), and 20 healthy, age-paired children comprised the control group (CG). Subjects stood on a force plate, and 4% of the subjects body weight was applied via a pulley system to a belt around the subjects trunks. The centre of pressure (COP) displacement was measured after the weight was unexpectedly released to produce a controlled postural perturbation followed by postural adjustment to recover balance. The subjects postural adjustments in eight subsequent intervals of 1 s (t1t8), beginning with the moment of weight removal, were compared among intervals and between groups. The applied perturbation magnitudes were the same for both groups, and no difference was observed between the groups in t1. However, the COP displacement in t2 in the HG was significantly higher than in the CG. No differences were observed between the groups in the other intervals. Within-group analysis showed that the COP was higher in t2 than in t4 (P = 0.016), t5 (P = 0.001) and t8 (P = 0.050) in the HG. No differences were observed among intervals in the CG. Children with haemophilia demonstrated differences in postural adjustment while undergoing unexpected balance perturbations when compared with healthily children.

Robustness of the non-Markovian Alzheimer walk under stochastic perturbation

The elephant walk model originally proposed by Schutz and Trimper to investigate non-Markovian processes led to the investigation of a series of other random-walk models. Of these, the best known is the Alzheimer walk model, because it was the first model shown to have amnestically induced persistence-i.e. superdiffusion caused by loss of memory. Here we study the robustness of the Alzheimer walk by adding a memoryless stochastic perturbation. Surprisingly, the solution of the perturbed model can be formally reduced to the solutions of the unperturbed model. Specifically, we give an exact solution of the perturbed model by finding a surjective mapping to the unperturbed model. Copyright (C) EPLA, 2012