REGAL : a regularization based algorithm for reinforcement learning in weakly communicating MDPs

We provide an algorithm that achieves the optimal regret rate in an unknown weakly communicating Markov Decision Process (MDP). The algorithm proceeds in episodes where, in each episode, it picks a policy using regularization based on the span of the optimal bias vector. For an MDP with S states and A actions whose optimal bias vector has span bounded by H, we show a regret bound of ~ O(HS p AT ). We also relate the span to various diameter-like quantities associated with the MDP, demonstrating how our results improve on previous regret bounds.

Optimal allocation strategies for the dark pool problem

We study the problem of allocating stocks to dark pools. We propose and analyze an optimal approach for allocations, if continuous-valued allocations are allowed. We also propose a modification for the case when only integer-valued allocations are possible. We extend the previous work on this problem to adversarial scenarios, while also improving on their results in the iid setup. The resulting algorithms are efficient, and perform well in simulations under stochastic and adversarial inputs.

Stability and change

This special feature section of Journal of Management & Organization (Volume 17/1 - March 2011) sets out to widen understanding of the processes of stability and change in today's organizations, with a particular emphasis on the contribution of institutional approaches to organizational studies. Institutional perspectives on organization theory assume that rational, economic calculations, such as the maximization of profits or the optimization of resource allocation, are not sufficient to understand the behavior of organizations and their strategic choices. Institutionalists acknowledge the great uncertainty associated with the conduct of organizations and suggest that taken-for-granted values, beliefs and meanings within and outside organizations also play an important role in the determination of legitimate action.

A Regularization Approach to Metrical Task Systems

We address the problem of constructing randomized online algorithms for the Metrical Task Systems (MTS) problem on a metric δ against an oblivious adversary. Restricting our attention to the class of “work-based” algorithms, we provide a framework for designing algorithms that uses the technique of regularization. For the case when δ is a uniform metric, we exhibit two algorithms that arise from this framework, and we prove a bound on the competitive ratio of each. We show that the second of these algorithms is ln n + O(loglogn) competitive, which is the current state-of-the art for the uniform MTS problem.

Evidence for CRHR1 in multiple sclerosis using supervised machine learning and meta-analysis in 12 566 individuals

The primary genetic risk factor in multiple sclerosis (MS) is the HLA-DRB1*1501 allele; however, much of the remaining genetic contribution to MS has yet to be elucidated. Several lines of evidence support a role for neuroendocrine system involvement in autoimmunity which may, in part, be genetically determined. Here, we comprehensively investigated variation within eight candidate hypothalamic-pituitary-adrenal (HPA) axis genes and susceptibility to MS. A total of 326 SNPs were investigated in a discovery dataset of 1343 MS cases and 1379 healthy controls of European ancestry using a multi-analytical strategy. Random Forests, a supervised machine-learning algorithm, identified eight intronic SNPs within the corticotrophin-releasing hormone receptor 1 or CRHR1 locus on 17q21.31 as important predictors of MS. On the basis of univariate analyses, six CRHR1 variants were associated with decreased risk for disease following a conservative correction for multiple tests. Independent replication was observed for CRHR1 in a large meta-analysis comprising 2624 MS cases and 7220 healthy controls of European ancestry. Results from a combined meta-analysis of all 3967 MS cases and 8599 controls provide strong evidence for the involvement of CRHR1 in MS. The strongest association was observed for rs242936 (OR = 0.82, 95% CI = 0.74-0.90, P = 9.7 × 10-5). Replicated CRHR1 variants appear to exist on a single associated haplotype. Further investigation of mechanisms involved in HPA axis regulation and response to stress in MS pathogenesis is warranted. © The Author 2010. Published by Oxford University Press. All rights reserved.

Cabergoline Decreases Alcohol Drinking and Seeking Behaviors Via Glial Cell Line-Derived Neurotrophic Factor

Background: Cabergoline is an ergotamine derivative that increases the expression of glial cell line-derived neurotrophic factor (GDNF) in vitro. We recently showed that GDNF in the ventral tegmental area (VTA) reduces the motivation to consume alcohol. We therefore set out to determine whether cabergoline administration decreases alcohol-drinking and -seeking behaviors via GDNF. Methods: Reverse transcription polymerase chain reaction (RT-PCR) and Enzyme-Linked ImmunoSorbent Assay (ELISA) were used to measure GDNF levels. Western blot analysis was used for phosphorylation experiments. Operant self-administration in rats and a two-bottle choice procedure in mice were used to assess alcohol-drinking behaviors. Instrumental performance tested during extinction was used to measure alcohol-seeking behavior. The [35S]GTPγS binding assay was used to assess the expression and function of the dopamine D2 receptor (D2R). Results: We found that treatment of the dopaminergic-like cell line SH-SY5Y with cabergoline and systemic administration of cabergoline in rats resulted in an increase in GDNF level and in the activation of the GDNF pathway. Cabergoline treatment decreased alcohol-drinking and -seeking behaviors including relapse, and its action to reduce alcohol consumption was localized to the VTA. Finally, the increase in GDNF expression and the decrease in alcohol consumption by cabergoline were abolished in GDNF heterozygous knockout mice. Conclusions: Together, these findings suggest that cabergoline-mediated upregulation of the GDNF pathway attenuates alcohol-drinking behaviors and relapse. Alcohol abuse and addiction are devastating and costly problems worldwide. This study puts forward the possibility that cabergoline might be an effective treatment for these disorders. © 2009 Society of Biological Psychiatry.

The ghrelin signalling system is involved in the consumption of sweets

The gastric-derived orexigenic peptide ghrelin affects brain circuits involved in energy balance as well as in reward. Indeed, ghrelin activates an important reward circuit involved in natural- as well as drug-induced reward, the cholinergic-dopaminergic reward link. It has been hypothesized that there is a common reward mechanism for alcohol and sweet substances in both animals and humans. Alcohol dependent individuals have higher craving for sweets than do healthy controls and the hedonic response to sweet taste may, at least in part, depend on genetic factors. Rat selectively bred for high sucrose intake have higher alcohol consumption than non-sucrose preferring rats and vice versa. In the present study a group of alcohol-consuming individuals selected from a population cohort was investigated for genetic variants of the ghrelin signalling system in relation to both their alcohol and sucrose consumption. Moreover, the effects of GHS-R1A antagonism on voluntary sucrose- intake and operant self-administration, as well as saccharin intake were investigated in preclinical studies using rodents. The effects of peripheral grelin administration on sucrose intake were also examined. Here we found associations with the ghrelin gene haplotypes and increased sucrose consumption, and a trend for the same association was seen in the high alcohol consumers. The preclinical data show that a GHS-R1A antagonist reduces the intake and self-administration of sucrose in rats as well as saccharin intake in mice. Further, ghrelin increases the intake of sucrose in rats. Collectively, our data provide a clear indication that the GHS-R1A antagonists reduces and ghrelin increases the intake of rewarding substances and hence, the central ghrelin signalling system provides a novel target for the development of drug strategies to treat addictive behaviours. © 2011 Landgren et al.

Pharmacological consequence of the A118G μ opioid receptor polymorphism on morphine-and fentanyl-mediated modulation of Ca2+ channels in humanized mouse sensory neurons

Background: The most common functional single nucleotide polymorphism of the human OPRM1 gene, A118G, has been shown to be associated with interindividual differences in opioid analgesic requirements, particularly with morphine, in patients with acute postoperative pain. The purpose of this study was to examine whether this polymorphism would modulate the morphine and fentanyl pharmacological profile of sensory neurons isolated from a humanized mouse model homozygous for either the 118A or 118G allele. Methods: The coupling of wild-type and mutant μ opioid receptors to voltage-gated Ca channels after exposure to either ligand was examined by employing the whole cell variant of the patch-clamp technique in acutely dissociated trigeminal ganglion neurons. Morphine-mediated antinociception was measured in mice carrying either the 118AA or 118GG allele. RESULTS:: The biophysical parameters (cell size, current density, and peak current amplitude potential) measured from both groups of sensory neurons were not significantly different. In 118GG neurons, morphine was approximately fivefold less potent and 26% less efficacious than that observed in 118AA neurons. On the other hand, the potency and efficacy of fentanyl were similar for both groups of neurons. Morphine-mediated analgesia in 118GG mice was significantly reduced compared with the 118AA mice. Conclusions: This study provides evidence to suggest that the diminished clinical effect observed with morphine in 118G carriers results from an alteration of the receptor's pharmacology in sensory neurons. In addition, the impaired analgesic response with morphine may explain why carriers of this receptor variant have an increased susceptibility to become addicted to opioids. © 2011 the American Society of Anesthesiologists, Inc. Lippincott Williams & Wilkins. Anesthesiology.

Long-evans rats acquire operant self-administration of 20% ethanol without sucrose fading

A major obstacle in the development of new medications for the treatment of alcohol use disorders (AUDs) has been the lack of preclinical, oral ethanol consumption paradigms that elicit high consumption. We have previously shown that rats exposed to 20% ethanol intermittently in a two-bottle choice paradigm will consume two times more ethanol than those given continuous access without the use of water deprivation or sucrose fading (5-6 g/kg every 24 h vs 2-3 g/kg every 24 h, respectively). In this study, we have adapted the model to an operant self-administration paradigm. Long-Evans rats were given access to 20% ethanol in overnight sessions on one of two schedules: (1) intermittent (Monday, Wednesday, and Friday) or (2) daily (Monday through Friday). With the progression of the overnight sessions, both groups showed a steady escalation in drinking (3-6 g/kg every 14 h) without the use of a sucrose-fading procedure. Following the acquisition phase, the 20% ethanol groups consumed significantly more ethanol than did animals trained to consume 10% ethanol with a sucrose fade (1.5 vs 0.7 g/kg every 30 min) and reached significantly higher blood ethanol concentrations. In addition, training history (20% ethanol vs 10% ethanol with sucrose fade) had a significant effect on the subsequent self-administration of higher concentrations of ethanol. Administration of the pharmacological stressor yohimbine following extinction caused a significant reinstatement of ethanol-seeking behavior. Both 20% ethanol models show promise and are amenable to the study of maintenance, motivation, and reinstatement. Furthermore, training animals to lever press for ethanol without the use of sucrose fading removes a potential confound from self-administration studies. © 2010 Nature Publishing Group All rights reserved.

State convergence in the initialisation of the Sfinks stream cipher

Sfinks is a shift register based stream cipher designed for hardware implementation. The initialisation state update function is different from the state update function used for keystream generation. We demonstrate state convergence during the initialisation process, even though the individual components used in the initialisation are one-to-one. However, the combination of these components is not one-to-one.

Algebraic analysis of the SSS stream cipher

Both the SSS and SOBER-t32 stream cipher designs use a single word-based shift register and a nonlinear filter function to produce keystream. In this paper we show that the algebraic attack method previously applied to SOBER-t32 is prevented from succeeding on SSS by the use of the key dependent substitution box (SBox) in the nonlinear filter of SSS. Additional assumptions and modifications to the SSS cipher in an attempt to enable algebraic analysis result in other difficulties that also render the algebraic attack infeasible. Based on these results, we conclude that a well chosen key-dependent substitution box used in the nonlinear filter of the stream cipher provides resistance against such algebraic attacks.

Children physical activity correlates and parent physical activity do not have a strong association with physical activity amongst adolescents

Background: Physical activity is a key modifiable behavior impacting a number of important health outcomes. The path to developing chronic diseases commonly commences with lifestyle patterns developed during childhood and adolescence. This study examined whether parent physical activity and other factors correlated with physical activity amongst children are associated with self-reported physical activity in adolescents. Methods: A total of 115 adolescents (aged 12-14) and their parents completed questionnaire assessments. Self-reported physical activity was measured amongst adolescents and their parents using the International Physical Activity Questionnaire for Adolescents (IPAQ-A), and the International Physical Activity Questionnaire (IPAQ) respectively. Adolescents also completed the Children’s Physical Activity Correlates (CPAC), which measured factors that have previously demonstrated association with physical activity amongst children. To examine whether parent physical activity or items from the CPAC were associated with self-reported adolescent physical activity, backward step-wise regression was undertaken. One item was removed at each step in descending order of significance (until two tailed item alpha=0.05 was achieved). Results: A total of 93 (80.9%) adolescents and their parents had complete data sets and were included in the analysis. Independent variables were removed in the order: perceptions of parental role modeling; importance of exercise; perceptions of parental encouragement; peer acceptance; fun of physical exertion; perceived competence; parent physical activity; self-esteem; liking of exercise; and parental influence. The only variable remaining in the model was ‘liking of games and sport’ (p=0.003, adjusted r-squared=0.085). Discussion: These findings indicate that factors associated with self-reported physical activity in adolescents are not necessarily the same as younger children (aged 8-11). While ‘liking of games and sport’ was included in the final model, the r-squared value did not indicate a strong association. Interestingly, parent self-reported physical activity was not included in the final model. It is likely that adolescent physical activity may be influenced by a variety of direct and indirect forms of socialization. These findings do support the view that intrinsically motivated themes such as the liking of games and sport take precedence over outside influences, like those presented by parents, in determining youth physical activity behaviors. These findings do not suggest that parents have no influence on adolescent physical activity patterns, but rather, the influence is likely to be more complex than physical activity behavior modeling perceived by the adolescent. Further research in this field is warranted in order to better understand potential contributors to successful physical activity promotion interventions amongst young adolescents.

The applicability of aesthetic knowledge in organizational decision making processes

We argue that aesthetic knowledge, which is a form of tacit knowledge of beauty and related concepts, is an important, yet under-researched, topic in the study of organizational decision making processes. The significance of aesthetic knowledge for decision making processes is derived from its universal application by humans to commonplace practices; its use as the basis of decision criteria in complex situations to which the effective application of logic and reason is difficult; and its role both in assisting cognition in general and in enabling the choice of solutions generated from rational decision making processes. Despite its importance, the empirical research examining the application of aesthetic knowledge in organizational decision making processes is limited. Further detailed study of aesthetic knowledge in the context of organizational decision making processes is required to extend the recent movement in the field aimed at examining the role that extrarational, human-centered factors play in organizational decisions.