Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Gradient estimation in dendritic reinforcement learning

We study synaptic plasticity in a complex neuronal cell model where NMDA-spikes can arise in certain dendritic zones. In the context of reinforcement learning, two kinds of plasticity rules are derived, zone reinforcement (ZR) and cell reinforcement (CR), which both optimize the expected reward by stochastic gradient ascent. For ZR, the synaptic plasticity response to the external reward signal is modulated exclusively by quantities which are local to the NMDA-spike initiation zone in which the synapse is situated. CR, in addition, uses nonlocal feedback from the soma of the cell, provided by mechanisms such as the backpropagating action potential. Simulation results show that, compared to ZR, the use of nonlocal feedback in CR can drastically enhance learning performance. We suggest that the availability of nonlocal feedback for learning is a key advantage of complex neurons over networks of simple point neurons, which have previously been found to be largely equivalent with regard to computational capability.

Minimum ten year results of total hip arthroplasty with the acetabular reinforcement ring in avascular osteonecrosis

Total hip arthroplasty (THA) still carries a higher failure rate in patients with avascular necrosis of the femoral head (AVN) than in a similar patient population with THA for other reasons. This is particularly true for the acetabular component. One of the major factors accounting for this is the compromised acetabular bone quality with structural defects subsequent to collapsing of the femoral head in high-grade AVN. In this study we implanted an acetabular reinforcement ring with hook (ARRH), which had been used successfully for other indications with acetabular bone stock deficiency, in 32 consecutive THA's in 29 patients with AVN. Five patients died during the observation period of causes unrelated to the surgery, one patient was lost to follow-up and one patient could not be followed up due to chronic illness, leaving 25 hips (23 patients) with a minimum follow-up of ten years (mean: 11.8; range: 10-15). The mean Merle d'Aubigne score increased significantly from 7.7 preoperatively to 16.6 postoperatively (p < 0.001). One revision was performed for aseptic stem loosening. Of the unrevised hips, one acetabular component was classified as definitively loose. The cumulative 12-year survivorship for THA with ARRH in AVN was 95.2% (confidence interval: 86.1-100%) for both components, 100% for the cup and 95.2% for the stem (86.1-100%).

Preliminary results following reinforcement of the pulmonary autograft to prevent dilatation after the Ross procedure

OBJECTIVE: The Ross operation remains a controversially discussed procedure, because concern exists regarding late dilatation of the neoaortic root and progressive regurgitation of the autograft valve. We present our early experience with an external reinforcement of the autograft, which is inserted into a prosthetic Dacron graft with an artificial aortic root configuration. This detail should help to prevent neoaortic root dilatation. PATIENTS AND METHODS: Between 2006 and 2007, 12 patients (mean age 16 +/- 38 years; range 15-38 years) underwent a Ross procedure by this technique. Indications were aortic regurgitation (n = 2), aortic stenosis (n = 5), and combined aortic stenosis and insufficiency (n = 5). A bicuspid aortic valve was present in 9 patients. Balloon valvuloplasty had been performed in 7 patients. Follow-up was performed by clinical and echocardiographic examinations. RESULTS: No early or late deaths occurred in this small series, and freedom from reoperation is 100%. Echocardiographic follow-up confirmed absence of aortic insufficiency in 11 patients after a mean of 11 months (range 2-30 months). In 1 patient, a small asymmetric regurgitation jet was already observed at discharge echocardiography. As expected, no neoaortic root dilatation was observed during follow-up. All patients are in New York Heart Association class I. CONCLUSIONS: The present technique is a simple and reproducible technical step that does not require significant additional time. Inclusion of the autograft within a root prosthesis may be especially indicated in situations known for late autograft dilatation, namely, bicuspid aortic valve, predominant aortic insufficiency, and ascending aortic enlargement.

Acetabular reinforcement ring in primary total hip arthroplasty: a minimum 10-year follow-up

INTRODUCTION: We report the results of a titanium acetabular reinforcement ring with a hook (ARRH) in primary total hip arthroplasty (THA), which was introduced in 1987 and continues to be used routinely in our center. The favorable results of this device in arthroplasty for developmental dysplasia and difficult revisions motivated its use in primary THA. With this implant only minimal acetabular reaming is necessary, anatomic positioning is achieved by placing the hook around the teardrop and a homogenous base for cementing the polyethylene cup is provided. MATERIALS AND METHODS: Between April 1987 and December 1991, 241 THAs with insertion of an ARRH were performed in 178 unselected, consecutive patients (average age 58 years; range 30-84 years) with a secondary osteoarthrosis in 41% of the cases. RESULTS: At the time of the latest follow-up, 33 patients (39 hips) had died and 17 cases had been lost to follow-up. The median follow-up was 122 months with a minimum of 10 years. Eight hips had been revised, leaving 177 hips in 120 living patients without revision. Six cups were revised because of aseptic loosening. Two hips were revised for sepsis. The mean Merle d'Aubigné score for the remaining hips was 16 (range 7-18) at the latest follow-up. For aseptic loosening, the probability of survival of the cup was 0.97 (95% confidence interval, 0.94-0.99). However, analysis of radiographs implied loosening in seven other cups without clinical symptoms. CONCLUSIONS: The results of primary THA using an acetabular reinforcement ring parallel the excellent results of these implants often observed in difficult primary and revision arthroplasty at a minimum of 10 years. Survivorship is comparable to modern cementless implants. Medial migration that occurs with loosening of the acetabular component seems to be prevented with this implant. Radiographic loosening signs can exist without clinical symptoms.