Dopamine and performance in a reinforcement learning task: evidence from Parkinson's disease.

The role dopamine plays in decision-making has important theoretical, empirical and clinical implications. Here, we examined its precise contribution by exploiting the lesion deficit model afforded by Parkinson's disease. We studied patients in a two-stage reinforcement learning task, while they were ON and OFF dopamine replacement medication. Contrary to expectation, we found that dopaminergic drug state (ON or OFF) did not impact learning. Instead, the critical factor was drug state during the performance phase, with patients ON medication choosing correctly significantly more frequently than those OFF medication. This effect was independent of drug state during initial learning and appears to reflect a facilitation of generalization for learnt information. This inference is bolstered by our observation that neural activity in nucleus accumbens and ventromedial prefrontal cortex, measured during simultaneously acquired functional magnetic resonance imaging, represented learnt stimulus values during performance. This effect was expressed solely during the ON state with activity in these regions correlating with better performance. Our data indicate that dopamine modulation of nucleus accumbens and ventromedial prefrontal cortex exerts a specific effect on choice behaviour distinct from pure learning. The findings are in keeping with the substantial other evidence that certain aspects of learning are unaffected by dopamine lesions or depletion, and that dopamine plays a key role in performance that may be distinct from its role in learning.

Dopamine and performance in a reinforcement learning task: Evidence from Parkinson's disease

The role dopamine plays in decision-making has important theoretical, empirical and clinical implications. Here, we examined its precise contribution by exploiting the lesion deficit model afforded by Parkinson's disease. We studied patients in a two-stage reinforcement learning task, while they were ON and OFF dopamine replacement medication. Contrary to expectation, we found that dopaminergic drug state (ON or OFF) did not impact learning. Instead, the critical factor was drug state during the performance phase, with patients ON medication choosing correctly significantly more frequently than those OFF medication. This effect was independent of drug state during initial learning and appears to reflect a facilitation of generalization for learnt information. This inference is bolstered by our observation that neural activity in nucleus accumbens and ventromedial prefrontal cortex, measured during simultaneously acquired functional magnetic resonance imaging, represented learnt stimulus values during performance. This effect was expressed solely during the ON state with activity in these regions correlating with better performance. Our data indicate that dopamine modulation of nucleus accumbens and ventromedial prefrontal cortex exerts a specific effect on choice behaviour distinct from pure learning. The findings are in keeping with the substantial other evidence that certain aspects of learning are unaffected by dopamine lesions or depletion, and that dopamine plays a key role in performance that may be distinct from its role in learning. © 2012 The Author.

Reinforcement learning using a continuous time actor-critic framework with spiking neurons.

Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modulated spike-timing-dependent plasticity have made first steps towards bridging the gap between the two approaches, but faced two problems. First, reinforcement learning is typically formulated in a discrete framework, ill-adapted to the description of natural situations. Second, biologically plausible models of reward-modulated spike-timing-dependent plasticity require precise calculation of the reward prediction error, yet it remains to be shown how this can be computed by neurons. Here we propose a solution to these problems by extending the continuous temporal difference (TD) learning of Doya (2000) to the case of spiking neurons in an actor-critic network operating in continuous time, and with continuous state and action representations. In our model, the critic learns to predict expected future rewards in real time. Its activity, together with actual rewards, conditions the delivery of a neuromodulatory TD signal to itself and to the actor, which is responsible for action choice. In simulations, we show that such an architecture can solve a Morris water-maze-like navigation task, in a number of trials consistent with reported animal performance. We also use our model to solve the acrobot and the cartpole problems, two complex motor control tasks. Our model provides a plausible way of computing reward prediction error in the brain. Moreover, the analytically derived learning rule is consistent with experimental evidence for dopamine-modulated spike-timing-dependent plasticity.

Electrospun fiber - Hydrogel composites for nucleus pulposus tissue engineering

New materials are needed to replace degenerated intervertebral disc tissue and to provide longer-term solutions for chronic back-pain. Replacement tissue potentially could be engineered by seeding cells into a scaffold that mimics the architecture of natural tissue. Many natural tissues, including the nucleus pulposus (the central region of the intervertebral disc) consist of collagen nanofibers embedded in a gel-like matrix. Recently it was shown that electrospun micro- or nano-fiber structures of considerable thickness can be produced by collecting fibers in an ethanol bath. Here, randomly aligned polycaprolactone electrospun fiber structures up to 50 mm thick are backfilled with alginate hydrogels to form novel composite materials that mimic the fiber-reinforced structure of the nucleus pulposus. The composites are characterized using both indentation and tensile testing. The composites are mechanically robust, exhibiting substantial strain-to-failure. The method presented here provides a way to create large biomimetic scaffolds that more closely mimic the composite structure of natural tissue. © 2012 Materials Research Society.

3 Gbit/s LED-based step index plastic optical fiber link using multilevel pulse amplitude modulation

Multilevel PAM is investigated for a LED-based SI-POF link. Using PAM-8, transmission at a record 3 Gbit/s is demonstrated for a maximum length of 25 m step index POF with offline post-receiver processing. © 2013 OSA.

Compressive response of glass fiber composite sandwich structures

Sandwich panels with crushable foam cores have attracted significant interest for impulsive load mitigation. We describe a method for making a lightweight, energy absorbing, glass fiber composite sandwich structure and explore it is through thickness (out-of-plane) compressive response. The sandwich structure utilized corrugated composite cores constructed from delamination resistant 3D woven E-glass fiber textiles folded over triangular cross section prismatic closed cell, PVC foam inserts. The corrugated structure was stitched to 3D woven S2-glass fiber face sheets and infiltrated with a rubber toughened, impact resistant epoxy. The quasi-static compressive stress-strain response of the panels was experimentally investigated as a function of the strut width to length ratio and compared to micromechanical predictions. Slender struts failed by elastic (Euler) buckling which transitioned to plastic microbuckling as the strut aspect ratio increased. Good agreement was observed between experimental results and micromechanical predictions over the wide range of core densities investigated in the study.

Precracked reinforced concrete t-beams repaired in shear with prestressed carbon fiber-reinforced polymer straps

The results of an experimental and numerical investigation involving unstrengthened reinforced concrete (RC) T-beams and precracked RC T-beams strengthened in shear with prestressed carbon fiber-reinforced polymer (CFRP) straps are presented and discussed. The results provide insights into the influence of load history and beam depth on the structural behavior of both unstrengthened and strengthened beams. The strengthened beams exhibited capacity enhancements of 21.6 to 46% compared to the equivalent unstrengthened beams, demonstrating the potential effectiveness of the prestressed CFRP strap system. Nonlinear finite element (FE) predictions, which incorporated the load history, reproduced the observed experimental behavior but either underestimated or overestimated the post-cracking stiffness of the beams and strap strain at higher load levels. These limitations were attributed to the concrete shear models used in the FE analyses.

3 Gbit/s LED-based step index plastic optical fiber link using multilevel pulse amplitude modulation

Multilevel PAM is investigated for a LED-based SI-POF link. Using PAM-8, transmission at a record 3 Gbit/s is demonstrated for a maximum length of 25 m step index POF with offline post-receiver processing. © 2013 OSA.

Bio-inspired hierarchical polymer fiber-carbon nanotube adhesives.

Hierarchical pillar arrays consisting of micrometer-sized polymer setae covered by carbon nanotubes are engineered to deliver the role of spatulae, mimicking the fibrillar adhesive surfaces of geckos. These biomimetic structures conform well and achieve better attachment to rough surfaces, providing a new platform for a variety of applications.

Mode-locking by nanotubes of a raman laser based on a highly doped GeO2 fiber

A mode-locked Raman laser, using 25 m of a GeO2 doped fiber as the gain medium, is reported employing carbon nanotubes. The oscillator generates 850 ps chirped pulses, which are externally compressed to 185 ps. © OSA 2012.

Double-wall carbon nanotube Q-switched and mode-locked two-micron fiber lasers

We fabricate double-wall carbon nanotube polymer composite saturable absorbers and demonstrate stable Q-switched and Mode-locked Thulium fiber lasers in a linear cavity and a ring cavity respectively. © 2011 Optical Society of America.

Graphene passively Q-switched two-micron fiber lasers

We demonstrate a passively Q-switched thulium fiber laser, using a graphene-based saturable absorber. The laser is based on an all-fiber ring cavity and produces ~2.3 μs pulses at 1884nm, with a maximum pulse energy of 70 nJ. © 2011 Optical Society of America.

Graphene passively Q-switched two-micron fiber lasers

We demonstrate a passively Q-switched thulium fiber laser, using a graphene-based saturable absorber. The laser is based on an all-fiber ring cavity and produces ~2.3 μs pulses at 1884nm, with a maximum pulse energy of 70 nJ. © 2011 Optical Society of America.