Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Fine structure of hippocampal mossy fiber synapses following rapid high-pressure freezing

Synapses of hippocampal neurons play important roles in learning and memory processes and are involved in aberrant hippocampal function in temporal lobe epilepsy. Major neuronal types in the hippocampus as well as their input and output synapses are well known, but it has remained an open question to what extent conventional electron microscopy (EM) has provided us with the real appearance of synaptic fine structure under in vivo conditions. There is reason to assume that conventional aldehyde fixation and dehydration lead to protein denaturation and tissue shrinkage, likely associated with the occurrence of artifacts. However, realistic fine-structural data of synapses are required for our understanding of the transmission process and for its simulation. Here, we used high-pressure freezing and cryosubstitution of hippocampal tissue that was not subjected to aldehyde fixation and dehydration in ethanol to monitor the fine structure of an identified synapse in the hippocampal CA3 region, that is, the synapse between granule cell axons, the mossy fibers, and the proximal dendrites of CA3 pyramidal neurons. Our results showed that high-pressure freezing nicely preserved ultrastructural detail of this particular synapse and allowed us to study rapid structural changes associated with synaptic plasticity.

Closed-loop control of mean arterial blood pressure during surgery with alfentanil: clinical evaluation of a novel model-based predictive controller

BACKGROUND: In contrast to hypnosis, there is no surrogate parameter for analgesia in anesthetized patients. Opioids are titrated to suppress blood pressure response to noxious stimulation. The authors evaluated a novel model predictive controller for closed-loop administration of alfentanil using mean arterial blood pressure and predicted plasma alfentanil concentration (Cp Alf) as input parameters. METHODS: The authors studied 13 healthy patients scheduled to undergo minor lumbar and cervical spine surgery. After induction with propofol, alfentanil, and mivacurium and tracheal intubation, isoflurane was titrated to maintain the Bispectral Index at 55 (+/- 5), and the alfentanil administration was switched from manual to closed-loop control. The controller adjusted the alfentanil infusion rate to maintain the mean arterial blood pressure near the set-point (70 mmHg) while minimizing the Cp Alf toward the set-point plasma alfentanil concentration (Cp Alfref) (100 ng/ml). RESULTS: Two patients were excluded because of loss of arterial pressure signal and protocol violation. The alfentanil infusion was closed-loop controlled for a mean (SD) of 98.9 (1.5)% of presurgery time and 95.5 (4.3)% of surgery time. The mean (SD) end-tidal isoflurane concentrations were 0.78 (0.1) and 0.86 (0.1) vol%, the Cp Alf values were 122 (35) and 181 (58) ng/ml, and the Bispectral Index values were 51 (9) and 52 (4) before surgery and during surgery, respectively. The mean (SD) absolute deviations of mean arterial blood pressure were 7.6 (2.6) and 10.0 (4.2) mmHg (P = 0.262), and the median performance error, median absolute performance error, and wobble were 4.2 (6.2) and 8.8 (9.4)% (P = 0.002), 7.9 (3.8) and 11.8 (6.3)% (P = 0.129), and 14.5 (8.4) and 5.7 (1.2)% (P = 0.002) before surgery and during surgery, respectively. A post hoc simulation showed that the Cp Alfref decreased the predicted Cp Alf compared with mean arterial blood pressure alone. CONCLUSION: The authors' controller has a similar set-point precision as previous hypnotic controllers and provides adequate alfentanil dosing during surgery. It may help to standardize opioid dosing in research and may be a further step toward a multiple input-multiple output controller.

The World Trade Organization’s Legitimacy Crisis: What Does The Beast Look Like?

The concept of legitimacy has many facets. The article reviews from a politics and law perspective the diagnosis of an ``institution in crisis''. This article is divided into three sections. It starts with a cautionary note on existing fallacies about assessing multilateral intergovernmental institutions and discusses competing schools of thought that approach the World Trade Organization (WTO) with varying perceptions of democracy and legitimacy. Section II takes up the actual debate on redesigning the WTO and directs attention to the question of balancing input and output legitimacy. Section III sketches potential avenues of research that have been neglected in the past.

Business and Public–Private Partnerships for Sustainability: Beyond Corporate Social Responsibility?

This article analyses public–private partnerships (PPPs) in the field of sustainable development from an international relations perspective with insights from the business and ethics literature. We argue that the role of business in these types of arrangements has not been sufficiently explored. After presenting three ways of approaching PPPs to stress the many facets of partnerships across the public–private divide, we discuss the emergence of these novel forms of governance from a demand side and contrast such a functionalist reading with the supply side. Then we look at the micro-economic incentives for corporations to engage in such endeavours. We develop arguments derived from the dominant literature to provide an analytical framework for explaining business participation. Finally, we discuss the role of PPPs in light of input and output legitimacy. We conclude by alluding to the emergence of an expectation-capacity gap and normative issues related to the global PPP architecture.

Minimum carbon payment along an aridity gradient for dryland afforestation in non annex I countries with stochastic weather and prices

Recent findings demonstrate that trees in deserts are efficient carbon sinks. It remains however unknown whether the Clean Development Mechanism will accelerate the planting of trees in Non Annex I dryland countries. We estimated the price of carbon at which a farmer would be indifferent between his customary activity and the planting of trees to trade carbon credits, along an aridity gradient. Carbon yields were simulated by means of the CO2FIX v3.1 model for Pinus halepensis with its respective yield classes along the gradient (Arid – 100mm to Dry Sub Humid conditions – 900mm). Wheat and pasture yields were predicted on somewhat similar nitrogen-based quadratic models, using 30 years of weather data to simulate moisture stress. Stochastic production, input and output prices were afterwards simulated on a Monte Carlo matrix. Results show that, despite the high levels of carbon uptake, carbon trading by afforesting is unprofitable anywhere along the gradient. Indeed, the price of carbon would have to raise unrealistically high, and the certification costs would have to drop significantly, to make the Clean Development Mechanism worthwhile for non annex I dryland countries farmers. From a government agency's point of view the Clean Development Mechanism is attractive. However, such agencies will find it difficult to demonstrate “additionality”, even if the rule may be somewhat flexible. Based on these findings, we will further discuss why the Clean Development Mechanism, a supposedly pro-poor instrument, fails to assist farmers in Non Annex I dryland countries living at minimum subsistence level.

Influence of directionality and maximal power output on speech understanding with bone anchored hearing implants in single sided deafness

Bone-anchored hearing implants (BAHI) are routinely used to alleviate the effects of the acoustic head shadow in single-sided sensorineural deafness (SSD). In this study, the influence of the directional microphone setting and the maximum power output of the BAHI sound processor on speech understanding in noise in a laboratory setting were investigated. Eight adult BAHI users with SSD participated in this pilot study. Speech understanding in noise was measured using a new Slovak speech-in-noise test in two different spatial settings, either with noise coming from the front and noise from the side of the BAHI (S90N0) or vice versa (S0N90). In both spatial settings, speech understanding was measured without a BAHI, with a Baha BP100 in omnidirectional mode, with a BP100 in directional mode, with a BP110 power in omnidirectional and with a BP110 power in directional mode. In spatial setting S90N0, speech understanding in noise with either sound processor and in either directional mode was improved by 2.2-2.8 dB (p = 0.004-0.016). In spatial setting S0N90, speech understanding in noise was reduced by either BAHI, but was significantly better by 1.0-1.8 dB, if the directional microphone system was activated (p = 0.046), when compared to the omnidirectional setting. With the limited number of subjects in this study, no statistically significant differences were found between the two sound processors.

Simultaneous registration of ECG and cardiac motion by a single esophageal probe

Long-term surface ECG is routinely used to diagnose paroxysmal arrhythmias. However, this method only provides information about the heart's electrical activity. To this end, we investigated a novel esophageal catheter that features synchronous esophageal ECG and acceleration measurements, the latter being a record of the heart's mechanical activity. The acceleration data were quantified in a small study and successfully linked to the activity sequences of the heart in all subjects. The acceleration signals were additionally transformed into motion. The extracted cardiac motion was proved to be a valid reference input for an adaptive filter capable of removing relevant baseline wandering in the recorded esophageal ECGs. Taking both capabilities into account, the proposed recorder might be a promising tool for future long-term heart monitoring.