Average control of Markov decision processes with Feller transition probabilities and general action spaces

This paper studies the average control problem of discrete-time Markov Decision Processes (MDPs for short) with general state space, Feller transition probabilities, and possibly non-compact control constraint sets A(x). Two hypotheses are considered: either the cost function c is strictly unbounded or the multifunctions A(r)(x) = {a is an element of A(x) : c(x, a) <= r} are upper-semicontinuous and compact-valued for each real r. For these two cases we provide new results for the existence of a solution to the average-cost optimality equality and inequality using the vanishing discount approach. We also study the convergence of the policy iteration approach under these conditions. It should be pointed out that we do not make any assumptions regarding the convergence and the continuity of the limit function generated by the sequence of relative difference of the alpha-discounted value functions and the Poisson equations as often encountered in the literature. (C) 2012 Elsevier Inc. All rights reserved.

A Simple Artificial Life Model Explains Irrational Behavior in Human Decision-Making

Although praised for their rationality, humans often make poor decisions, even in simple situations. In the repeated binary choice experiment, an individual has to choose repeatedly between the same two alternatives, where a reward is assigned to one of them with fixed probability. The optimal strategy is to perseverate with choosing the alternative with the best expected return. Whereas many species perseverate, humans tend to match the frequencies of their choices to the frequencies of the alternatives, a sub-optimal strategy known as probability matching. Our goal was to find the primary cognitive constraints under which a set of simple evolutionary rules can lead to such contrasting behaviors. We simulated the evolution of artificial populations, wherein the fitness of each animat (artificial animal) depended on its ability to predict the next element of a sequence made up of a repeating binary string of varying size. When the string was short relative to the animats' neural capacity, they could learn it and correctly predict the next element of the sequence. When it was long, they could not learn it, turning to the next best option: to perseverate. Animats from the last generation then performed the task of predicting the next element of a non-periodical binary sequence. We found that, whereas animats with smaller neural capacity kept perseverating with the best alternative as before, animats with larger neural capacity, which had previously been able to learn the pattern of repeating strings, adopted probability matching, being outperformed by the perseverating animats. Our results demonstrate how the ability to make predictions in an environment endowed with regular patterns may lead to probability matching under less structured conditions. They point to probability matching as a likely by-product of adaptive cognitive strategies that were crucial in human evolution, but may lead to sub-optimal performances in other environments.

Automatic design of decision-tree induction algorithms tailored to flexible-receptor docking data

Background: This paper addresses the prediction of the free energy of binding of a drug candidate with enzyme InhA associated with Mycobacterium tuberculosis. This problem is found within rational drug design, where interactions between drug candidates and target proteins are verified through molecular docking simulations. In this application, it is important not only to correctly predict the free energy of binding, but also to provide a comprehensible model that could be validated by a domain specialist. Decision-tree induction algorithms have been successfully used in drug-design related applications, specially considering that decision trees are simple to understand, interpret, and validate. There are several decision-tree induction algorithms available for general-use, but each one has a bias that makes it more suitable for a particular data distribution. In this article, we propose and investigate the automatic design of decision-tree induction algorithms tailored to particular drug-enzyme binding data sets. We investigate the performance of our new method for evaluating binding conformations of different drug candidates to InhA, and we analyze our findings with respect to decision tree accuracy, comprehensibility, and biological relevance. Results: The empirical analysis indicates that our method is capable of automatically generating decision-tree induction algorithms that significantly outperform the traditional C4.5 algorithm with respect to both accuracy and comprehensibility. In addition, we provide the biological interpretation of the rules generated by our approach, reinforcing the importance of comprehensible predictive models in this particular bioinformatics application. Conclusions: We conclude that automatically designing a decision-tree algorithm tailored to molecular docking data is a promising alternative for the prediction of the free energy from the binding of a drug candidate with a flexible-receptor.

Validation of the Short-Version of Rose Angina Questionnaire in Brazil

Background: Stable angina pectoris is a serious condition with few epidemiological studies in Brazil. Objective: To validate the short-version of the Rose angina questionnaire in Brazilian Portuguese for its implementation in surveys and longitudinal studies. Methods: A total of 116 consecutive patients from an outpatient clinic without prior myocardial infarction and/or coronary revascularization were enrolled for application of three questions of the Rose angina questionnaire addressing chest pain after exertion. We used the treadmill test as the gold standard with the Ellestad protocol. Results: The short-version of the Rose angina questionnaire of the 116 subjects submitted to the exercise treadmill test disclosed 89.7% of accuracy, 25% of sensitivity, 92.0% of specificity, 10.0% of positive predictive value, 97.2% of negative predictive value, and 3.1 of positive likelihood ratio and 0.82 of negative likelihood ratio. Conclusion: The Portuguese version with three items of the Rose angina questionnaire is suitable for epidemiological purposes. (Arq Bras Cardiol 2012; 99(5): 1056-1059)

Sets of probability distributions, independence, and convexity

This paper analyzes concepts of independence and assumptions of convexity in the theory of sets of probability distributions. The starting point is Kyburg and Pittarelli's discussion of "convex Bayesianism" (in particular their proposals concerning E-admissibility, independence, and convexity). The paper offers an organized review of the literature on independence for sets of probability distributions; new results on graphoid properties and on the justification of "strong independence" (using exchangeability) are presented. Finally, the connection between Kyburg and Pittarelli's results and recent developments on the axiomatization of non-binary preferences, and its impact on "complete" independence, are described.

Motor learning, retention and transfer after virtual-reality-based training in Parkinson's disease - effect of motor and cognitive demands of games: a longitudinal, controlled clinical study

Objectives To evaluate the learning, retention and transfer of performance improvements after Nintendo Wii Fit (TM) training in patients with Parkinson's disease and healthy elderly people. Design Longitudinal, controlled clinical study. Participants Sixteen patients with early-stage Parkinson's disease and 11 healthy elderly people. Interventions Warm-up exercises and Wii Fit training that involved training motor (shifts centre of gravity and step alternation) and cognitive skills. A follow-up evaluative Wii Fit session was held 60 days after the end of training. Participants performed a functional reach test before and after training as a measure of learning transfer. Main outcome measures Learning and retention were determined based on the scores of 10 Wii Fit games over eight sessions. Transfer of learning was assessed after training using the functional reach test. Results Patients with Parkinson's disease showed no deficit in learning or retention on seven of the 10 games, despite showing poorer performance on five games compared with the healthy elderly group. Patients with Parkinson's disease showed marked learning deficits on three other games, independent of poorer initial performance. This deficit appears to be associated with cognitive demands of the games which require decision-making, response inhibition, divided attention and working memory. Finally, patients with Parkinson's disease were able to transfer motor ability trained on the games to a similar untrained task. Conclusions The ability of patients with Parkinson's disease to learn, retain and transfer performance improvements after training on the Nintendo Wii Fit depends largely on the demands, particularly cognitive demands, of the games involved, reiterating the importance of game selection for rehabilitation purposes. (C) 2012 Chartered Society of Physiotherapy. Published by Elsevier Ltd. All rights reserved.

Longitudinal brain volumetric changes during one year in non-elderly healthy adults: a voxel-based morphometry study

Previous cross-sectional magnetic resonance imaging (MRI) studies of healthy aging in young adults have indicated the presence of significant inverse correlations between age and gray matter volumes, although not homogeneously across all brain regions. However, such cross-sectional studies have important limitations and there is a scarcity of detailed longitudinal MRI studies with repeated measures obtained in the same individuals in order to investigate regional gray matter changes during short periods of time in non-elderly healthy adults. In the present study, 52 healthy young adults aged 18 to 50 years (27 males and 25 females) were followed with repeated MRI acquisitions over approximately 15 months. Gray matter volumes were compared between the two times using voxel-based morphometry, with the prediction that volume changes would be detectable in the frontal lobe, temporal neocortex and hippocampus. Voxel-wise analyses showed significant (P < 0.05, family-wise error corrected) relative volume reductions of gray matter in two small foci located in the right orbitofrontal cortex and left hippocampus. Separate comparisons for males and females showed bilateral gray matter relative reductions in the orbitofrontal cortex over time only in males. We conclude that, in non-elderly healthy adults, subtle gray matter volume alterations are detectable after short periods of time. This underscores the dynamic nature of gray matter changes in the brain during adult life, with regional volume reductions being detectable in brain regions that are relevant to cognitive and emotional processes.