Reinforcement learning using a continuous time actor-critic framework with spiking neurons.

Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modulated spike-timing-dependent plasticity have made first steps towards bridging the gap between the two approaches, but faced two problems. First, reinforcement learning is typically formulated in a discrete framework, ill-adapted to the description of natural situations. Second, biologically plausible models of reward-modulated spike-timing-dependent plasticity require precise calculation of the reward prediction error, yet it remains to be shown how this can be computed by neurons. Here we propose a solution to these problems by extending the continuous temporal difference (TD) learning of Doya (2000) to the case of spiking neurons in an actor-critic network operating in continuous time, and with continuous state and action representations. In our model, the critic learns to predict expected future rewards in real time. Its activity, together with actual rewards, conditions the delivery of a neuromodulatory TD signal to itself and to the actor, which is responsible for action choice. In simulations, we show that such an architecture can solve a Morris water-maze-like navigation task, in a number of trials consistent with reported animal performance. We also use our model to solve the acrobot and the cartpole problems, two complex motor control tasks. Our model provides a plausible way of computing reward prediction error in the brain. Moreover, the analytically derived learning rule is consistent with experimental evidence for dopamine-modulated spike-timing-dependent plasticity.

100 Gb/s uncooled WDM system using conventional WDM components and advanced receiver signal processing

We demonstrate an uncooled WDM system using standard WDM components and receiver signal processing, with a different number of receivers to transmitters, to allow wide temperature drift of the transmitter lasers. A 100 Gb/s 8-wavelength demonstrator has been developed, which proves the feasibility of the approach over 25 km of SMF. © 2012 OSA.

An integrated representation of the services provided by global water resources

Water is essential not only to maintain the livelihoods of human beings but also to sustain ecosystems. Over the last few decades several global assessments have reviewed current and future uses of water, and have offered potential solutions to a possible water crisis. However, these have tended to focus on water supply rather than on the range of demands for all water services (including those of ecosystems). In this paper, a holistic global view of water resources and the services they provide is presented, using Sankey diagrams as a visualisation tool. These diagrams provide a valuable addition to the spatial maps of other global assessments, as they track the sources, uses, services and sinks of water resources. They facilitate comparison of different water services, and highlight trade-offs amongst them. For example, they reveal how increasing the supply of water resources to one service (crop production) can generate a reduction in provision of other water services (e.g., to ecosystem maintenance). The potential impacts of efficiency improvements in the use of water are also highlighted; for example, reduction in soil evaporation from crop production through better farming practices, or the results of improved treatment and re-use of return flows leading to reduction of delivery to final sinks. This paper also outlines the measures needed to ensure sustainable water resource use and supply for multiple competing services in the future, and emphasises that integrated management of land and water resources is essential to achieve this goal. © 2013 Elsevier Ltd.

Detecting bipedal motion from correlated probabilistic trajectories

This paper is about detecting bipedal motion in video sequences by using point trajectories in a framework of classification. Given a number of point trajectories, we find a subset of points which are arising from feet in bipedal motion by analysing their spatio-temporal correlation in a pairwise fashion. To this end, we introduce probabilistic trajectories as our new features which associate each point over a sufficiently long time period in the presence of noise. They are extracted from directed acyclic graphs whose edges represent temporal point correspondences and are weighted with their matching probability in terms of appearance and location. The benefit of the new representation is that it practically tolerates inherent ambiguity for example due to occlusions. We then learn the correlation between the motion of two feet using the probabilistic trajectories in a decision forest classifier. The effectiveness of the algorithm is demonstrated in experiments on image sequences captured with a static camera, and extensions to deal with a moving camera are discussed. © 2013 Elsevier B.V. All rights reserved.

A fracture criterion for the notch strength of high strength steels in the presence of hydrogen

High strength steels can suffer from a loss of ductility when exposed to hydrogen, and this may lead to sudden failure. The hydrogen is either accommodated in the lattice or is trapped at defects, such as dislocations, grain boundaries and carbides. The challenge is to identify the effect of hydrogen located at different sites upon the drop in tensile strength of a high strength steel. For this purpose, literature data on the failure stress of notched and un-notched steel bars are re-analysed; the bars were tested over a wide range of strain rates and hydrogen concentrations. The local stress state at failure has been determined by the finite element (FE) method, and the concentration of both lattice and trapped hydrogen is predicted using Oriani's theory along with the stress-driven diffusion equation. The experimental data are rationalised in terms of a postulated failure locus of peak maximum principal stress versus lattice hydrogen concentration. This failure locus is treated as a unique material property for the given steel and heat treatment condition. We conclude that the presence of lattice hydrogen increases the susceptibility to hydrogen embrittlement whereas trapped hydrogen has only a negligible effect. It is also found that the observed failure strength of hydrogen charged un-notched bars is less than the peak local stress within the notched geometries. Weakest link statistics are used to account for this stressed volume effect. © 2013 Elsevier Ltd.

Static analysis of Euler-Bernoulli beams with multiple unilateral cracks under combined axial and transverse loads

The paper deals with the static analysis of pre-damaged Euler-Bernoulli beams with any number of unilateral cracks and subjected to tensile or compression forces combined with arbitrary transverse loads. The mathematical representation of cracks with a bilateral behaviour (i.e. always open) via Dirac delta functions is extended by introducing a convenient switching variable, which allows each crack to be open or closed depending on the sign of the axial strain at the crack centre. The proposed model leads to analytical solutions, which depend on four integration constants (to be computed by enforcing the boundary conditions) along with the Boolean switching variables associated with the cracks (whose role is to turn on and off the additional flexibility due to the presence of the cracks). An efficient computational procedure is also presented and numerically validated. For this purpose, the proposed approach is applied to two pre-damaged beams, with different damage and loading conditions, and the results so obtained are compared against those given by a standard finite element code (in which the correct opening of the cracks is pre-assigned), always showing a perfect agreement. © 2013 Elsevier Ltd. All rights reserved.

100 Gb/s uncooled WDM system using conventional WDM components and advanced receiver signal processing

We demonstrate an uncooled WDM system using standard WDM components and receiver signal processing, with a different number of receivers to transmitters, to allow wide temperature drift of the transmitter lasers. A 100 Gb/s 8-wavelength demonstrator has been developed, which proves the feasibility of the approach over 25 km of SMF. © 2012 Optical Society of America.

CO2-gasification of a lignite coal in the presence of an iron-based oxygen carrier for chemical-looping combustion

Chemical-looping combustion (CLC) has the inherent property of separating the product CO2 from flue gases. Instead of air, it uses an oxygen carrier, usually in the form of a metal oxide, to provide oxygen for combustion. All techniques so far proposed for chemical looping with solid fuels involve initially the gasification of the solid fuel in order for the gaseous products to react with the oxygen carrier. Here, the rates of gasification of coal were compared when gasification was undertaken in a fluidised bed of either (i) an active Fe-based oxygen carrier used for chemical looping or (ii) inert sand. This enabled an examination of the ability of chemical looping materials to enhance the rate of gasification of solid fuels. Batch gasification and chemical-looping combustion experiments with a German lignite and its char are reported, using an electrically-heated fluidised bed reactor at temperatures from 1073 to 1223 K. The fluidising gas was CO2 in nitrogen. The kinetics of the gasification were found to be significantly faster in the presence of the oxygen carrier, especially at temperatures above 1123 K. A numerical model was developed to account for external and internal mass transfer and for the effect of the looping agent. The model also included the effects of the evolution of the pore structure at different conversions. The presence of Fe2O3 led to an increase in the rate of gasification because of the rapid oxidation of CO by the oxygen carrier to CO2. This resulted in the removal of CO and maintained a higher mole fraction of CO2 in the mixture of gas around the particle of char, i.e. within the mass transfer boundary layer surrounding the particle. This effect was most prominent at about 20% conversion when (i) the surface area for reaction was at its maximum and (ii) because of the accompanying increase in porosity and pore size, intraparticle resistance to gas mass transfer within the particle of char had fallen, compared with that in the initial particle. Excellent agreement was observed between the rates predicted by the numerical model and those observed experimentally. ©2013 Elsevier Ltd. All rights reserved.

High performance optical data links using hybrid CAP/QAM transmitter/receiver scheme

We experimentally demonstrate the first optical data link at 20Gb/s using hybrid CAP- 4/QAM-4 with transmission over 4.3km SSMF and a power penalty ~1.5dBo at BER=10-9. The hybrid CAP-4/QAM-4 link significantly outperforms a reference PAM-4 link. © OSA 2013.