Reinforcement learning using a continuous time actor-critic framework with spiking neurons.

Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modulated spike-timing-dependent plasticity have made first steps towards bridging the gap between the two approaches, but faced two problems. First, reinforcement learning is typically formulated in a discrete framework, ill-adapted to the description of natural situations. Second, biologically plausible models of reward-modulated spike-timing-dependent plasticity require precise calculation of the reward prediction error, yet it remains to be shown how this can be computed by neurons. Here we propose a solution to these problems by extending the continuous temporal difference (TD) learning of Doya (2000) to the case of spiking neurons in an actor-critic network operating in continuous time, and with continuous state and action representations. In our model, the critic learns to predict expected future rewards in real time. Its activity, together with actual rewards, conditions the delivery of a neuromodulatory TD signal to itself and to the actor, which is responsible for action choice. In simulations, we show that such an architecture can solve a Morris water-maze-like navigation task, in a number of trials consistent with reported animal performance. We also use our model to solve the acrobot and the cartpole problems, two complex motor control tasks. Our model provides a plausible way of computing reward prediction error in the brain. Moreover, the analytically derived learning rule is consistent with experimental evidence for dopamine-modulated spike-timing-dependent plasticity.

Novel experiment on in-cylinder desorption of fuel from the oil layer

A technique has been developed to measure the desorption and subsequent oxidation of fuel in the oil layer by spiking the oil with liquid fuel and firing the engine on gaseous fuel or motoring with air. Experiments suggest that fuel desorption is not diffusion limited above 50°C and indicated that approximately two to four percent of the cylinder oil layer is fresh oil from the sump. The increase in hydrocarbon emissions is of the order of 100 ppmC1 per 1% liquid fuel introduced into the fresh oil in a methane fired engine at mid-speed and light load conditions. Calculations indicate that fuel desorbing from oil is much more likely to produce hydrocarbon emissions than fuel emerging from crevices. © Copyright 1994 Society of Automotive Engineers, Inc.

Continuous dynamical systems that realize discrete optimization on the hypercube

We study the problem of finding a local minimum of a multilinear function E over the discrete set {0,1}n. The search is achieved by a gradient-like system in [0,1]n with cost function E. Under mild restrictions on the metric, the stable attractors of the gradient-like system are shown to produce solutions of the problem, even when they are not in the vicinity of the discrete set {0,1}n. Moreover, the gradient-like system connects with interior point methods for linear programming and with the analog neural network studied by Vidyasagar (IEEE Trans. Automat. Control 40 (8) (1995) 1359), in the same context. © 2004 Elsevier B.V. All rights reserved.

Plane-strain discrete dislocation plasticity with climb-assisted glide motion of dislocations

A small-strain two-dimensional discrete dislocation plasticity (DDP) framework is developed wherein dislocation motion is caused by climb-assisted glide. The climb motion of the dislocations is assumed to be governed by a drag-type relation similar to the glide-only motion of dislocations: such a relation is valid when vacancy kinetics is either diffusion limited or sink limited. The DDP framework is employed to predict the effect of dislocation climb on the uniaxial tensile and pure bending response of single crystals. Under uniaxial tensile loading conditions, the ability of dislocations to bypass obstacles by climb results in a reduced dislocation density over a wide range of specimen sizes in the climb-assisted glide case compared to when dislocation motion is only by glide. A consequence is that, at least in a single slip situation, size effects due to dislocation starvation are reduced. By contrast, under pure bending loading conditions, the dislocation density is unaffected by dislocation climb as geometrically necessary dislocations (GNDs) dominate. However, climb enables the dislocations to arrange themselves into lower energy configurations which significantly reduces the predicted bending size effect as well as the amount of reverse plasticity observed during unloading. The results indicate that the intrinsic plasticity material length scale associated with GNDs is strongly affected by thermally activated processes and will be a function of temperature. © 2013 IOP Publishing Ltd.

The impact of sand slugs against beams and plates: Coupled discrete particle/finite element simulations

The impact of a slug of dry sand particles against a metallic sandwich beam or circular sandwich plate is analysed in order to aid the design of sandwich panels for shock mitigation. The sand particles interact via a combined linear-spring-and-dashpot law whereas the face sheets and compressible core of the sandwich beam and plate are treated as rate-sensitive, elastic-plastic solids. The majority of the calculations are performed in two dimensions and entail the transverse impact of end-clamped monolithic and sandwich beams, with plane strain conditions imposed. The sand slug is of rectangular shape and comprises a random loose packing of identical, circular cylindrical particles. These calculations reveal that loading due to the sand is primarily inertial in nature with negligible fluid-structure interaction: the momentum transmitted to the beam is approximately equal to that of the incoming sand slug. For a slug of given incoming momentum, the dynamic deflection of the beam increases with decreasing duration of sand-loading until the impulsive limit is attained. Sandwich beams with thick, strong cores significantly outperform monolithic beams of equal areal mass. This performance enhancement is traced to the "sandwich effect" whereby the sandwich beams have a higher bending strength than that of the monolithic beams. Three-dimensional (3D) calculations are also performed such that the sand slug has the shape of a circular cylindrical column of finite height, and contains spherical sand particles. The 3D slug impacts a circular monolithic plate or sandwich plate and we show that sandwich plates with thick strong cores again outperform monolithic plates of equal areal mass. Finally, we demonstrate that impact by sand particles is equivalent to impact by a crushable foam projectile. The calculations on the equivalent projectile are significantly less intensive computationally, yet give predictions to within 5% of the full discrete particle calculations for the monolithic and sandwich beams and plates. These foam projectile calculations suggest that metallic foam projectiles can be used to simulate the loading by sand particles within a laboratory setting. © 2013 Elsevier Ltd.

Study on small-strain behaviours of methane hydrate sandy sediments using discrete element method

Methane hydrate bearing soil has attracted increasing interest as a potential energy resource where methane gas can be extracted from dissociating hydrate-bearing sediments. Seismic testing techniques have been applied extensively and in various ways, to detect the presence of hydrates, due to the fact that hydrates increase the stiffness of hydrate-bearing sediments. With the recognition of the limitations of laboratory and field tests, wave propagation modelling using Discrete Element Method (DEM) was conducted in this study in order to provide some particle-scale insights on the hydrate-bearing sandy sediment models with pore-filling and cementation hydrate distributions. The relationship between shear wave velocity and hydrate saturation was established by both DEM simulations and analytical solutions. Obvious differences were observed in the dependence of wave velocity on hydrate saturation for these two cases. From the shear wave velocity measurement and particle-scale analysis, it was found that the small-strain mechanical properties of hydrate-bearing sandy sediments are governed by both the hydrate distribution patterns and hydrate saturation. © 2013 AIP Publishing LLC.

What kind of friction model is needed to predict brake squeal?

Predictive models of friction-induced vibration have proved elusive despite decades of research. There are many mechanisms that can cause brake squeal; friction coupled systems can be highly sensitive to small perturbations; and the dynamic properties of friction at the contact zone seem to be poorly understood. This paper describes experimental and theoretical work aimed at identifying the key ingredients of a predictive model. A large-scale experiment was carried out to identify squeal initiations using a pin-on-disc test rig: approximately 30,000 squeal initiations were recorded, covering a very wide range of frequencies. The theoretical model allows for completely general linear systems coupled at a single sliding point by friction: squeal is predicted using a linearised stability analysis. Results will be presented that show that almost all observed squeal events can be predicted within this model framework, but that some subsets require innovative friction modelling: predictions are highly dependent on the particular choice of friction model and its associated parameters. Copyright © 2012 by ASME.

What kind of friction model is needed to predict brake squeal?

Predictive models of friction-induced vibration have proved elusive despite decades of research. There are many mechanisms that can cause brake squeal; friction coupled systems can be highly sensitive to small perturbations; and the dynamic properties of friction at the contact zone seem to be poorly understood. This paper describes experimental and theoretical work aimed at identifying the key ingredients of a predictive model. A large-scale experiment was carried out to identify squeal initiations using a pin-on-disc test rig: approximately 30,000 squeal initiations were recorded, covering a very wide range of frequencies. The theoretical model allows for completely general linear systems coupled at a single sliding point by friction: squeal is predicted using a linearised stability analysis. Results will be presented that show that almost all observed squeal events can be predicted within this model framework, but that some subsets require innovative friction modelling: predictions are highly dependent on the particular choice of friction model and its associated parameters. Copyright © 2012 by ASME.

Climb-enabled discrete dislocation plasticity

A small strain two-dimensional discrete dislocation plasticity framework coupled to vacancy diffusion is developed wherein the motion of edge dislocations is by a combination of glide and climb. The dislocations are modelled as line defects in a linear elastic medium and the mechanical boundary value problem is solved by the superposition of the infinite medium elastic fields of the dislocations and a complimentary non-singular solution that enforces the boundary conditions. Similarly, the climbing dislocations are modelled as line sources/sinks of vacancies and the vacancy diffusion boundary value problem is also solved by a superposition of the fields of the line sources/sinks in an infinite medium and a complementary non-singular solution that enforces the boundary conditions. The vacancy concentration field along with the stress field provides the climb rate of the dislocations. Other short-range interactions of the dislocations are incorporated via a set of constitutive rules. We first employ this formulation to investigate the climb of a single edge dislocation in an infinite medium and illustrate the existence of diffusion-limited and sink-limited climb regimes. Next, results are presented for the pure bending and uniaxial tension of single crystals oriented for single slip. These calculations show that plasticity size effects are reduced when dislocation climb is permitted. Finally, we contrast predictions of this coupled framework with an ad hoc model in which dislocation climb is modelled by a drag-type relation based on a quasi steady-state solution. © 2013 Elsevier Ltd. All rights reserved.

The role of cell-surface-bound phosphatases in species competition within natural phytoplankton assemblage: an in situ experiment

Despite it is widely acknowledged that the ability to hydrolyze dissolved organic matter using extracellular phosphatases is diverse in fresh water phytoplankton, the competition within single species related to presence and quantity of cell-surface-bound phosphatases has not been examined in natural conditions yet. Here, we studied phytoplankton species competition in a freshwater reservoir during an in situ experiment. A natural plankton community, with the exclusion of large zooplankton, was enclosed in permeable dialysis bags inside two large containers of different bioavailable phosphate concentrations. Phytoplankton species biomass and the abundance of bacteria were determined in purpose to compare the development of enclosed microbial communities. Total and cell-surface-bound phosphatase activities in the phytoplankton were investigated using the Fluorescently Labelled Enzyme Activity (FLEA) technique that allows for direct microscopic detection of phosphatase-positive cells and, with image cytometry, enables quantification of phosphatase hydrolytic capacity. Production of extracellular phosphatases was not completely inhibited or stopped in the phosphate-enriched environment, phytoplankton cells only showed the activity less often. Under the phosphate-nonenriched conditions, the production of phosphatases was enhanced, but active species did not proliferate amongst phytoplankton assemblage. Further, specific growth rates of the phosphatase-positive species in the non-enriched environment were lower than the same phosphatase-positive species in phosphate-enriched environment. Interestingly, the phosphatase-positive cells of Ankyra ancora increased their size in both treatments equally, although the population in phosphate-enriched environment grew much faster and the cell-specific phosphatase activity was lower. We hypothesize that brand new daughter cells had sufficient phosphorus reserves and therefore did not employ extracellular phosphatases until they matured and needed extra bioavailable phosphorus to support their metabolism before cell division. Based on presented in situ experiment, we propose that the ability to hydrolyze organic polymers and particles with cell-surface-hound phosphatases is advantageous for longer persistence of given population in a phosphate-scarce environment; although phosphatase-positive species cannot dominate the reservoir phytoplankton solely because of specific phosphorus-scavenging strategy.

Non linear constitutive models for lattice materials

We use a computational homogenisation approach to derive a non linear constitutive model for lattice materials. A representative volume element (RVE) of the lattice is modelled by means of discrete structural elements, and macroscopic stress-strain relationships are numerically evaluated after applying appropriate periodic boundary conditions to the RVE. The influence of the choice of the RVE on the predictions of the model is discussed. The model has been used for the analysis of the hexagonal and the triangulated lattices subjected to large strains. The fidelity of the model has been demonstrated by analysing a plate with a central hole under prescribed in plane compressive and tensile loads, and then comparing the results from the discrete and the homogenised models. © 2013 Elsevier Ltd.

Electrical addressing of polymer stabilized hyper-twisted chiral nematic liquid crystals with interdigitated electrodes: Experiment and model

Electro-optic switching in short-pitch polymer stabilized chiral nematic liquid crystals was studied and the relative contributions of flexoelectric and dielectric coupling were investigated: polymer stabilization was found to effectively suppress unwanted textural transitions of the chiral nematic liquid crystal and thereby enhance the electro-optical performance (high optical contrast for visible light, a near ideal optical hysteresis, fast electro-optic response). Test cells were studied that possessed interdigitated electrodes to electrically address the liquid crystal. Based on simulations, a well-fitted phenomenological description of the electro-optic response was derived considering both flexoelectro-optic and Kerr-effect based electro-optic response. © 2014 AIP Publishing LLC.