Reinforcement learning using a continuous time actor-critic framework with spiking neurons.

Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modulated spike-timing-dependent plasticity have made first steps towards bridging the gap between the two approaches, but faced two problems. First, reinforcement learning is typically formulated in a discrete framework, ill-adapted to the description of natural situations. Second, biologically plausible models of reward-modulated spike-timing-dependent plasticity require precise calculation of the reward prediction error, yet it remains to be shown how this can be computed by neurons. Here we propose a solution to these problems by extending the continuous temporal difference (TD) learning of Doya (2000) to the case of spiking neurons in an actor-critic network operating in continuous time, and with continuous state and action representations. In our model, the critic learns to predict expected future rewards in real time. Its activity, together with actual rewards, conditions the delivery of a neuromodulatory TD signal to itself and to the actor, which is responsible for action choice. In simulations, we show that such an architecture can solve a Morris water-maze-like navigation task, in a number of trials consistent with reported animal performance. We also use our model to solve the acrobot and the cartpole problems, two complex motor control tasks. Our model provides a plausible way of computing reward prediction error in the brain. Moreover, the analytically derived learning rule is consistent with experimental evidence for dopamine-modulated spike-timing-dependent plasticity.

Electrospun fiber - Hydrogel composites for nucleus pulposus tissue engineering

New materials are needed to replace degenerated intervertebral disc tissue and to provide longer-term solutions for chronic back-pain. Replacement tissue potentially could be engineered by seeding cells into a scaffold that mimics the architecture of natural tissue. Many natural tissues, including the nucleus pulposus (the central region of the intervertebral disc) consist of collagen nanofibers embedded in a gel-like matrix. Recently it was shown that electrospun micro- or nano-fiber structures of considerable thickness can be produced by collecting fibers in an ethanol bath. Here, randomly aligned polycaprolactone electrospun fiber structures up to 50 mm thick are backfilled with alginate hydrogels to form novel composite materials that mimic the fiber-reinforced structure of the nucleus pulposus. The composites are characterized using both indentation and tensile testing. The composites are mechanically robust, exhibiting substantial strain-to-failure. The method presented here provides a way to create large biomimetic scaffolds that more closely mimic the composite structure of natural tissue. © 2012 Materials Research Society.

3 Gbit/s LED-based step index plastic optical fiber link using multilevel pulse amplitude modulation

Multilevel PAM is investigated for a LED-based SI-POF link. Using PAM-8, transmission at a record 3 Gbit/s is demonstrated for a maximum length of 25 m step index POF with offline post-receiver processing. © 2013 OSA.

Compressive response of glass fiber composite sandwich structures

Sandwich panels with crushable foam cores have attracted significant interest for impulsive load mitigation. We describe a method for making a lightweight, energy absorbing, glass fiber composite sandwich structure and explore it is through thickness (out-of-plane) compressive response. The sandwich structure utilized corrugated composite cores constructed from delamination resistant 3D woven E-glass fiber textiles folded over triangular cross section prismatic closed cell, PVC foam inserts. The corrugated structure was stitched to 3D woven S2-glass fiber face sheets and infiltrated with a rubber toughened, impact resistant epoxy. The quasi-static compressive stress-strain response of the panels was experimentally investigated as a function of the strut width to length ratio and compared to micromechanical predictions. Slender struts failed by elastic (Euler) buckling which transitioned to plastic microbuckling as the strut aspect ratio increased. Good agreement was observed between experimental results and micromechanical predictions over the wide range of core densities investigated in the study.

Precracked reinforced concrete t-beams repaired in shear with prestressed carbon fiber-reinforced polymer straps

The results of an experimental and numerical investigation involving unstrengthened reinforced concrete (RC) T-beams and precracked RC T-beams strengthened in shear with prestressed carbon fiber-reinforced polymer (CFRP) straps are presented and discussed. The results provide insights into the influence of load history and beam depth on the structural behavior of both unstrengthened and strengthened beams. The strengthened beams exhibited capacity enhancements of 21.6 to 46% compared to the equivalent unstrengthened beams, demonstrating the potential effectiveness of the prestressed CFRP strap system. Nonlinear finite element (FE) predictions, which incorporated the load history, reproduced the observed experimental behavior but either underestimated or overestimated the post-cracking stiffness of the beams and strap strain at higher load levels. These limitations were attributed to the concrete shear models used in the FE analyses.

3 Gbit/s LED-based step index plastic optical fiber link using multilevel pulse amplitude modulation

Multilevel PAM is investigated for a LED-based SI-POF link. Using PAM-8, transmission at a record 3 Gbit/s is demonstrated for a maximum length of 25 m step index POF with offline post-receiver processing. © 2013 OSA.

Bio-inspired hierarchical polymer fiber-carbon nanotube adhesives.

Hierarchical pillar arrays consisting of micrometer-sized polymer setae covered by carbon nanotubes are engineered to deliver the role of spatulae, mimicking the fibrillar adhesive surfaces of geckos. These biomimetic structures conform well and achieve better attachment to rough surfaces, providing a new platform for a variety of applications.

Mode-locking by nanotubes of a raman laser based on a highly doped GeO2 fiber

A mode-locked Raman laser, using 25 m of a GeO2 doped fiber as the gain medium, is reported employing carbon nanotubes. The oscillator generates 850 ps chirped pulses, which are externally compressed to 185 ps. © OSA 2012.

Double-wall carbon nanotube Q-switched and mode-locked two-micron fiber lasers

We fabricate double-wall carbon nanotube polymer composite saturable absorbers and demonstrate stable Q-switched and Mode-locked Thulium fiber lasers in a linear cavity and a ring cavity respectively. © 2011 Optical Society of America.

Graphene passively Q-switched two-micron fiber lasers

We demonstrate a passively Q-switched thulium fiber laser, using a graphene-based saturable absorber. The laser is based on an all-fiber ring cavity and produces ~2.3 μs pulses at 1884nm, with a maximum pulse energy of 70 nJ. © 2011 Optical Society of America.

Graphene passively Q-switched two-micron fiber lasers

We demonstrate a passively Q-switched thulium fiber laser, using a graphene-based saturable absorber. The laser is based on an all-fiber ring cavity and produces ~2.3 μs pulses at 1884nm, with a maximum pulse energy of 70 nJ. © 2011 Optical Society of America.

Behavior of RC beams retrofitted with CARDIFRC after thermal cycling

This study investigates the effect of thermal cycling on the performance of concrete beams retrofitted with CARDIFRC, a new class of high performance fiber-reinforced cement-based material that is compatible with concrete. Twenty four beams were subjected to 24 h thermal cycles between 25 and 90°C. One third of the beams were reinforced either in flexure only or in flexure and shear with conventional steel reinforcement and used as control specimens. The remaining sixteen beams were retrofitted with CARDIFRC strips to provide external flexural and/or shear strengthening. All beams were exposed to a varied number of 24 h thermal cycles ranging from 0 to 90 and were tested in four-point bending at room temperature. The tests indicated that the retrofitted members were stronger and stiffer than control beams, and more importantly, that their failure initiated in flexure without any signs of interfacial delamination cracking. The results of these tests are presented and compared to analytical predictions. The predictions show good correlation with the experimental results. © 2010 ASCE.

Hybrid core carbon fiber composite sandwich panels: Fabrication and mechanical response

Carbon fiber reinforced polymer (CFRP) composite sandwich panels with hybrid foam filled CFRP pyramidal lattice cores have been assembled from a carbon fiber braided net, 3D woven face sheets and various polymeric foams, and infused with an epoxy resin using a vacuum assisted resin transfer process. Sandwich panels with a fixed CFRP truss mass have been fabricated using a variety of closed cell polymer and syntactic foams, resulting in core densities ranging from 44-482kgm-3. The through thickness and in-plane shear modulus and strength of the cores increased with increasing foam density. The use of low compressive strength foams within the core was found to result in a significant reduction in the compressive strength contributed by the CFRP trusses. X-ray tomography led to the discovery that the trusses develop an elliptical cross-section shape during pressure assisted resin transfer. The ellipticity of the truss cross-sections increased, and the lattice contribution to the core strength decreased as the foam density was reduced. Micromechanical modeling was used to investigate the relationships between the mechanical properties and volume fractions of the core materials and truss topology of the hybrid core. The specific strength and moduli of the hybrid cores lay between those of the CFRP lattices and foams used to fabricate them. However, their volumetric and gravimetric energy absorptions significantly exceeded those of the materials from which they were fabricated. They compare favorably with other lightweight energy absorbing materials and structures. © 2013.