Dopamine and performance in a reinforcement learning task: Evidence from Parkinson's disease

The role dopamine plays in decision-making has important theoretical, empirical and clinical implications. Here, we examined its precise contribution by exploiting the lesion deficit model afforded by Parkinson's disease. We studied patients in a two-stage reinforcement learning task, while they were ON and OFF dopamine replacement medication. Contrary to expectation, we found that dopaminergic drug state (ON or OFF) did not impact learning. Instead, the critical factor was drug state during the performance phase, with patients ON medication choosing correctly significantly more frequently than those OFF medication. This effect was independent of drug state during initial learning and appears to reflect a facilitation of generalization for learnt information. This inference is bolstered by our observation that neural activity in nucleus accumbens and ventromedial prefrontal cortex, measured during simultaneously acquired functional magnetic resonance imaging, represented learnt stimulus values during performance. This effect was expressed solely during the ON state with activity in these regions correlating with better performance. Our data indicate that dopamine modulation of nucleus accumbens and ventromedial prefrontal cortex exerts a specific effect on choice behaviour distinct from pure learning. The findings are in keeping with the substantial other evidence that certain aspects of learning are unaffected by dopamine lesions or depletion, and that dopamine plays a key role in performance that may be distinct from its role in learning. © 2012 The Author.

Reinforcement learning using a continuous time actor-critic framework with spiking neurons.

Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modulated spike-timing-dependent plasticity have made first steps towards bridging the gap between the two approaches, but faced two problems. First, reinforcement learning is typically formulated in a discrete framework, ill-adapted to the description of natural situations. Second, biologically plausible models of reward-modulated spike-timing-dependent plasticity require precise calculation of the reward prediction error, yet it remains to be shown how this can be computed by neurons. Here we propose a solution to these problems by extending the continuous temporal difference (TD) learning of Doya (2000) to the case of spiking neurons in an actor-critic network operating in continuous time, and with continuous state and action representations. In our model, the critic learns to predict expected future rewards in real time. Its activity, together with actual rewards, conditions the delivery of a neuromodulatory TD signal to itself and to the actor, which is responsible for action choice. In simulations, we show that such an architecture can solve a Morris water-maze-like navigation task, in a number of trials consistent with reported animal performance. We also use our model to solve the acrobot and the cartpole problems, two complex motor control tasks. Our model provides a plausible way of computing reward prediction error in the brain. Moreover, the analytically derived learning rule is consistent with experimental evidence for dopamine-modulated spike-timing-dependent plasticity.

4-channel polymeric optical bus module for board-level optical interconnections

A 4-channel polymeric optical bus module suitable for use in board-levelinterconnections is presented. Low-loss and low-crosstalk module performance is achieved, while-1 dB alignment tolerances better than ± 8 μm are demonstrated. © OSA 2012.

A polymeric regenerative optical bus for board-level optical interconnections

A scalable polymer waveguide-based regenerative optical bus architecture for use in board-level communications is presented. As a proof-of-principle demonstration, a 4-channel polymer bus formed on a FR4 substrate providing 10 Gb/s/channel data transmission is reported. © 2012 OSA.

Evaluation of the functionality of biodegradable polymeric platforms for drug delivery systems

We present the development of a drug-loaded triple-layer platform consisting of thin film biodegradable polymers, in a properly designed form for the desired gradual degradation. Poly(dl-lactide-co-glycolide) (PLGA (65:35), PLGA (75:25)) and polycaprolactone (PCL) were grown by spin coating technique, to synthesize the platforms with the order PCL/PLGA (75:25)/PLGA (65:35) that determine their degradation rates. The outer PLGA (65:35) layer was loaded with dipyridamole, an antiplatelet drug. Spectroscopic ellipsometry (SE) in the Vis-far UV range was used to determine the nanostructure, as well as the content of the incorporated drug in the as-grown platforms. In situ and real-time SE measurements were carried out using a liquid cell for the dynamic evaluation of the fibrinogen and albumin protein adsorption processes. Atomic force microscopy studies justified the SE results concerning the nanopores formation in the polymeric platforms, and the dominant adsorption mechanisms of the proteins, which were defined by the drug incorporation in the platforms. © 2013 Elsevier B.V. All rights reserved.

Does temporal discounting explain unhealthy behavior? A systematic review and reinforcement learning perspective.

The tendency to make unhealthy choices is hypothesized to be related to an individual's temporal discount rate, the theoretical rate at which they devalue delayed rewards. Furthermore, a particular form of temporal discounting, hyperbolic discounting, has been proposed to explain why unhealthy behavior can occur despite healthy intentions. We examine these two hypotheses in turn. We first systematically review studies which investigate whether discount rates can predict unhealthy behavior. These studies reveal that high discount rates for money (and in some instances food or drug rewards) are associated with several unhealthy behaviors and markers of health status, establishing discounting as a promising predictive measure. We secondly examine whether intention-incongruent unhealthy actions are consistent with hyperbolic discounting. We conclude that intention-incongruent actions are often triggered by environmental cues or changes in motivational state, whose effects are not parameterized by hyperbolic discounting. We propose a framework for understanding these state-based effects in terms of the interplay of two distinct reinforcement learning mechanisms: a "model-based" (or goal-directed) system and a "model-free" (or habitual) system. Under this framework, while discounting of delayed health may contribute to the initiation of unhealthy behavior, with repetition, many unhealthy behaviors become habitual; if health goals then change, habitual behavior can still arise in response to environmental cues. We propose that the burgeoning development of computational models of these processes will permit further identification of health decision-making phenotypes.

Optical properties of NAEC-PMMA nonlinear polymeric thin film

Novel guest nonlinear optical (NLO) chromophore molecules (4-nitrobenzene)-3-azo-9-ethylcarbazole (NAEC) were doped in poly (methyl methacrylate) (PMMA) host with a concentration of approximately 15% by weight. For a useful macroscopic electro-optic (EO) effect, these NLO molecules NAEC were arranged in a noncentrosymmetric structure in the host polymer by corona-onset poling at elevated temperature (COPET). For applying NAEC-PMMA polymer in optical devices such as EO switch, its optical properties have been investigated. The UV/Visible absorption spectra for the unpoled and poled polymer film were determined. The refractive index of the film was also determined from measurements of the coupling angles with the reflective intensity at 632.8 nm wavelength. Using the simple reflection technique, the EO coefficient 33 value was measured as 60 pm/V at 632.8 nm wavelength. The second-order nonlinear coefficient d(33) was characterized by the second-harmonic-generation (SHG) experimental setup and the calculated d(33) value reached 18.4 pm/V at 1064 nm wavelength. The relation between the second-order nonlinear coefficients d(33) and d(13) for the poled polymer film was also discussed in detail and the ratio d(33)/d(13) value was obtained as 3.3. (C) 2002 Kluwer Academic Publishers.

Polymeric fullerene oxide films produced by decomposition of hexanitro[60]fullerene

A new poly(fullerene oxide) thin film material has been fabricated by thermal activation and electron bombardment on hexanitro[60]fullerene (HNF) film deposited on a An substrate, all under vacuum conditions. The reaction products in the polymerization process are analyzed by XPS, UPS, IR, TGA-MS and LDI-MS techniques. It is found that the main effect of thermal and radiation treatments is to induce cleavage of -NO bonds from HNF molecules resulted in the release of nitric oxide gas and the formation of fullerene-bound oxyradicals, C-60-C-6. Spectroscopic evidence strongly suggests that rearrangement of fullerenic nitro moieties into nitrito groups is involved in the HNF decomposition process prior to the generation of reactive oxyradical intermediates. Consequently, the intermolecular coupling reaction of these oxyradicals leads to carbon polymer networks containing oxygen-bridged fullerenes. The thermally generated polymeric thin film is stable up to 900 K. Electron bombardment is also effective in both the decomposition of -NO2 groups and the removal of -OH groups present in HNF films. UV irradiation at 365 nm alone is shown to be not as efficient for the polymer formation. (C) 2003 Elsevier Ltd. All rights reserved.

Highly enantioselective resolution of terminal epoxides with cross-linked polymeric salen-Co(III) complexes

Crosslinked polymeric salen-Co(III) complexes derived from a novel dialdehyde and a trialdehyde were synthesized and employed in the hydrolytic kinetic resolution (HKR) of terminal epoxides. Up to 99%, ee were obtained with only 0.16-0.02 mol%,, of catalyst (based on catalytic unit). (C) 2003 Elsevier Ltd. All rights reserved.

Synthesis and structure of new low dimensional polymeric haloplumbate(II) complexes [Pb4Br12(C40H40N8)] and [Pb4Cl12(C40H40N8)]center dot H2O

New low dimensional polymeric haloplumbate(II) complexes of the dication of (4,4'-bis(imidazolyl-ylmethyl)biphenyl) were synthesised and their crystal structures determined. Complex 1, [Pb4Br12(C40H40N8)], has cis-edge-shared, octahedral, lead bromide double chains. In compound 2 [Pb4Cl12(C40H40N8)]center dot H2O, the inorganic chains are corner-shared, square pyramidal chains of lead chloride. In both compounds the organic ammoniums form regular layers that alternate with the inorganic chains.