SINGULARLY PERTURBED DISCOUNTED MARKOV CONTROL PROCESSES IN A GENERAL STATE SPACE

This paper studies the asymptotic optimality of discrete-time Markov decision processes (MDPs) with general state space and action space and having weak and strong interactions. By using a similar approach as developed by Liu, Zhang, and Yin [Appl. Math. Optim., 44 (2001), pp. 105-129], the idea in this paper is to consider an MDP with general state and action spaces and to reduce the dimension of the state space by considering an averaged model. This formulation is often described by introducing a small parameter epsilon > 0 in the definition of the transition kernel, leading to a singularly perturbed Markov model with two time scales. Our objective is twofold. First it is shown that the value function of the control problem for the perturbed system converges to the value function of a limit averaged control problem as epsilon goes to zero. In the second part of the paper, it is proved that a feedback control policy for the original control problem defined by using an optimal feedback policy for the limit problem is asymptotically optimal. Our work extends existing results of the literature in the following two directions: the underlying MDP is defined on general state and action spaces and we do not impose strong conditions on the recurrence structure of the MDP such as Doeblin's condition.

Continuity of Dynamical Structures for Nonautonomous Evolution Equations Under Singular Perturbations

In this paper we study the continuity of invariant sets for nonautonomous infinite-dimensional dynamical systems under singular perturbations. We extend the existing results on lower-semicontinuity of attractors of autonomous and nonautonomous dynamical systems. This is accomplished through a detailed analysis of the structure of the invariant sets and its behavior under perturbation. We prove that a bounded hyperbolic global solutions persists under singular perturbations and that their nonlinear unstable manifold behave continuously. To accomplish this, we need to establish results on roughness of exponential dichotomies under these singular perturbations. Our results imply that, if the limiting pullback attractor of a nonautonomous dynamical system is the closure of a countable union of unstable manifolds of global bounded hyperbolic solutions, then it behaves continuously (upper and lower) under singular perturbations.

A two-layer approximation to the Brazil Current-Intermediate Western Boundary Current System between 20 degrees S and 28 degrees S

It has been shown that the vertical structure of the Brazil Current (BC)-Intermediate Western Boundary Current (IWBC) System is dominated by the first baroclinic mode at 22 degrees S-23 degrees S. In this work, we employed the Miami Isopycnic Coordinate Ocean Model to investigate whether the rich mesoscale activity of this current system, between 20 degrees S and 28 degrees S, is reproduced by a two-layer approximation of its vertical structure. The model results showed cyclonic and anticyclonic meanders propagating southwestward along the current axis, resembling the dynamical pattern of Rossby waves superposed on a mean flow. Analysis of the upper layer zonal velocity component, using a space-time diagram, revealed a dominant wavelength of about 450 km and phase velocity of about 0.20 ms(-1) southwestward. The results also showed that the eddy-like structures slowly grew in amplitude as they moved downstream. Despite the simplified design of the numerical experiments conducted here, these results compared favorably with observations and seem to indicate that weakly unstable long baroclinic waves are responsible for most of the variability observed in the BC-IWBC system. (C) 2009 Elsevier Ltd. All rights reserved.

Analytic perturbation theory for bound states in the transfer matrix spectrum of weakly correlated lattice ferromagnetic spin systems

We consider general d-dimensional lattice ferromagnetic spin systems with nearest neighbor interactions in the high temperature region ('beta' << 1). Each model is characterized by a single site apriori spin distribution taken to be even. We also take the parameter 'alfa' = ('S POT.4') - 3 '(S POT.2') POT.2' > 0, i.e. in the region which we call Gaussian subjugation, where ('S POT.K') denotes the kth moment of the apriori distribution. Associated with the model is a lattice quantum field theory known to contain a particle of asymptotic mass -ln 'beta' and a bound state below the two-particle threshold. We develop a 'beta' analytic perturbation theory for the binding energy of this bound state. As a key ingredient in obtaining our result we show that the Fourier transform of the two-point function is a meromorphic function, with a simple pole, in a suitable complex spectral parameter and the coefficients of its Laurent expansion are analytic in 'beta'.

Postural adjustment after an unexpected perturbation in children with haemophilia

. Children with haemophilia often bleed inside joints and muscles, which may impair postural adjustments. These postural adjustments are necessary to control postural balance during daily activities. The inability to quickly recover postural balance could elevate the risk of bleeding. To determine whether children with haemophilia have impaired postural adjustment after an unexpected perturbation compared with healthy children. Twenty children with haemophilia comprised the haemophilic group (HG), and 20 healthy, age-paired children comprised the control group (CG). Subjects stood on a force plate, and 4% of the subjects body weight was applied via a pulley system to a belt around the subjects trunks. The centre of pressure (COP) displacement was measured after the weight was unexpectedly released to produce a controlled postural perturbation followed by postural adjustment to recover balance. The subjects postural adjustments in eight subsequent intervals of 1 s (t1t8), beginning with the moment of weight removal, were compared among intervals and between groups. The applied perturbation magnitudes were the same for both groups, and no difference was observed between the groups in t1. However, the COP displacement in t2 in the HG was significantly higher than in the CG. No differences were observed between the groups in the other intervals. Within-group analysis showed that the COP was higher in t2 than in t4 (P = 0.016), t5 (P = 0.001) and t8 (P = 0.050) in the HG. No differences were observed among intervals in the CG. Children with haemophilia demonstrated differences in postural adjustment while undergoing unexpected balance perturbations when compared with healthily children.

Structure and electronic properties of hydrated mesityl oxide: a sequential quantum mechanics/molecular mechanics approach

The hydration of mesityl oxide (MOx) was investigated through a sequential quantum mechanics/molecular mechanics approach. Emphasis was placed on the analysis of the role played by water in the MOx syn-anti equilibrium and the electronic absorption spectrum. Results for the structure of the MOx-water solution, free energy of solvation and polarization effects are also reported. Our main conclusion was that in gas-phase and in low-polarity solvents, the MOx exists dominantly in syn-form and in aqueous solution in anti-form. This conclusion was supported by Gibbs free energy calculations in gas phase and in-water by quantum mechanical calculations with polarizable continuum model and thermodynamic perturbation theory in Monte Carlo simulations using a polarized MOx model. The consideration of the in-water polarization of the MOx is very important to correctly describe the solute-solvent electrostatic interaction. Our best estimate for the shift of the pi-pi* transition energy of MOx, when it changes from gas-phase to water solvent, shows a red-shift of -2,520 +/- 90 cm(-1), which is only 110 cm(-1) (0.014 eV) below the experimental extrapolation of -2,410 +/- 90 cm(-1). This red-shift of around -2,500 cm(-1) can be divided in two distinct and opposite contributions. One contribution is related to the syn -> anti conformational change leading to a blue-shift of similar to 1,700 cm(-1). Other contribution is the solvent effect on the electronic structure of the MOx leading to a red-shift of around -4,200 cm(-1). Additionally, this red-shift caused by the solvent effect on the electronic structure can by composed by approximately 60 % due to the electrostatic bulk effect, 10 % due to the explicit inclusion of the hydrogen-bonded water molecules and 30 % due to the explicit inclusion of the nearest water molecules.

Statistical physics approach to dendritic computation: The excitable-wave mean-field approximation

We analytically study the input-output properties of a neuron whose active dendritic tree, modeled as a Cayley tree of excitable elements, is subjected to Poisson stimulus. Both single-site and two-site mean-field approximations incorrectly predict a nonequilibrium phase transition which is not allowed in the model. We propose an excitable-wave mean-field approximation which shows good agreement with previously published simulation results [Gollo et al., PLoS Comput. Biol. 5, e1000402 (2009)] and accounts for finite-size effects. We also discuss the relevance of our results to experiments in neuroscience, emphasizing the role of active dendrites in the enhancement of dynamic range and in gain control modulation.

Robustness of the non-Markovian Alzheimer walk under stochastic perturbation

The elephant walk model originally proposed by Schutz and Trimper to investigate non-Markovian processes led to the investigation of a series of other random-walk models. Of these, the best known is the Alzheimer walk model, because it was the first model shown to have amnestically induced persistence-i.e. superdiffusion caused by loss of memory. Here we study the robustness of the Alzheimer walk by adding a memoryless stochastic perturbation. Surprisingly, the solution of the perturbed model can be formally reduced to the solutions of the unperturbed model. Specifically, we give an exact solution of the perturbed model by finding a surjective mapping to the unperturbed model. Copyright (C) EPLA, 2012

On the optical properties of copper nanocubes as a function of the edge length as modeled by the discrete dipole approximation

This Letter reports an investigation on the optical properties of copper nanocubes as a function of size as modeled by the discrete dipole approximation. In the far-field, our results showed that the extinction resonances shifted from 595 to 670 nm as the size increased from 20 to 100 nm. Also, the highest optical efficiencies for absorption and scattering were obtained for nanocubes that were 60 and 100 nm in size, respectively. In the near-field, the electric-field amplitudes were investigated considering 514, 633 and 785 nm as the excitation wavelengths. The E-fields increased with size, being the highest at 633 nm. (c) 2012 Elsevier B.V. All rights reserved.

Pair approximation for a model of vertical and horizontal transmission of parasites.

We apply Stochastic Dynamics method for a differential equations model, proposed by Marc Lipsitch and collaborators (Proc. R. Soc. Lond. B 260, 321, 1995), for which the transmission dynamics of parasites occurs from a parent to its offspring (vertical transmission), and by contact with infected host (horizontal transmission). Herpes, Hepatitis and AIDS are examples of diseases for which both horizontal and vertical transmission occur simultaneously during the virus spreading. Understanding the role of each type of transmission in the infection prevalence on a susceptible host population may provide some information about the factors that contribute for the eradication and/or control of those diseases. We present a pair mean-field approximation obtained from the master equation of the model. The pair approximation is formed by the differential equations of the susceptible and infected population densities and the differential equations of pairs that contribute to the former ones. In terms of the model parameters, we obtain the conditions that lead to the disease eradication, and set up the phase diagram based on the local stability analysis of fixed points. We also perform Monte Carlo simulations of the model on complete graphs and Erdös-Rényi graphs in order to investigate the influence of population size and neighborhood on the previous mean-field results; by this way, we also expect to evaluate the contribution of vertical and horizontal transmission on the elimination of parasite. Pair Approximation for a Model of Vertical and Horizontal Transmission of Parasites.