Gradient estimation in dendritic reinforcement learning

We study synaptic plasticity in a complex neuronal cell model where NMDA-spikes can arise in certain dendritic zones. In the context of reinforcement learning, two kinds of plasticity rules are derived, zone reinforcement (ZR) and cell reinforcement (CR), which both optimize the expected reward by stochastic gradient ascent. For ZR, the synaptic plasticity response to the external reward signal is modulated exclusively by quantities which are local to the NMDA-spike initiation zone in which the synapse is situated. CR, in addition, uses nonlocal feedback from the soma of the cell, provided by mechanisms such as the backpropagating action potential. Simulation results show that, compared to ZR, the use of nonlocal feedback in CR can drastically enhance learning performance. We suggest that the availability of nonlocal feedback for learning is a key advantage of complex neurons over networks of simple point neurons, which have previously been found to be largely equivalent with regard to computational capability.

Minimum ten year results of total hip arthroplasty with the acetabular reinforcement ring in avascular osteonecrosis

Total hip arthroplasty (THA) still carries a higher failure rate in patients with avascular necrosis of the femoral head (AVN) than in a similar patient population with THA for other reasons. This is particularly true for the acetabular component. One of the major factors accounting for this is the compromised acetabular bone quality with structural defects subsequent to collapsing of the femoral head in high-grade AVN. In this study we implanted an acetabular reinforcement ring with hook (ARRH), which had been used successfully for other indications with acetabular bone stock deficiency, in 32 consecutive THA's in 29 patients with AVN. Five patients died during the observation period of causes unrelated to the surgery, one patient was lost to follow-up and one patient could not be followed up due to chronic illness, leaving 25 hips (23 patients) with a minimum follow-up of ten years (mean: 11.8; range: 10-15). The mean Merle d'Aubigne score increased significantly from 7.7 preoperatively to 16.6 postoperatively (p < 0.001). One revision was performed for aseptic stem loosening. Of the unrevised hips, one acetabular component was classified as definitively loose. The cumulative 12-year survivorship for THA with ARRH in AVN was 95.2% (confidence interval: 86.1-100%) for both components, 100% for the cup and 95.2% for the stem (86.1-100%).

Preliminary results following reinforcement of the pulmonary autograft to prevent dilatation after the Ross procedure

OBJECTIVE: The Ross operation remains a controversially discussed procedure, because concern exists regarding late dilatation of the neoaortic root and progressive regurgitation of the autograft valve. We present our early experience with an external reinforcement of the autograft, which is inserted into a prosthetic Dacron graft with an artificial aortic root configuration. This detail should help to prevent neoaortic root dilatation. PATIENTS AND METHODS: Between 2006 and 2007, 12 patients (mean age 16 +/- 38 years; range 15-38 years) underwent a Ross procedure by this technique. Indications were aortic regurgitation (n = 2), aortic stenosis (n = 5), and combined aortic stenosis and insufficiency (n = 5). A bicuspid aortic valve was present in 9 patients. Balloon valvuloplasty had been performed in 7 patients. Follow-up was performed by clinical and echocardiographic examinations. RESULTS: No early or late deaths occurred in this small series, and freedom from reoperation is 100%. Echocardiographic follow-up confirmed absence of aortic insufficiency in 11 patients after a mean of 11 months (range 2-30 months). In 1 patient, a small asymmetric regurgitation jet was already observed at discharge echocardiography. As expected, no neoaortic root dilatation was observed during follow-up. All patients are in New York Heart Association class I. CONCLUSIONS: The present technique is a simple and reproducible technical step that does not require significant additional time. Inclusion of the autograft within a root prosthesis may be especially indicated in situations known for late autograft dilatation, namely, bicuspid aortic valve, predominant aortic insufficiency, and ascending aortic enlargement.

Acetabular reinforcement ring in primary total hip arthroplasty: a minimum 10-year follow-up

INTRODUCTION: We report the results of a titanium acetabular reinforcement ring with a hook (ARRH) in primary total hip arthroplasty (THA), which was introduced in 1987 and continues to be used routinely in our center. The favorable results of this device in arthroplasty for developmental dysplasia and difficult revisions motivated its use in primary THA. With this implant only minimal acetabular reaming is necessary, anatomic positioning is achieved by placing the hook around the teardrop and a homogenous base for cementing the polyethylene cup is provided. MATERIALS AND METHODS: Between April 1987 and December 1991, 241 THAs with insertion of an ARRH were performed in 178 unselected, consecutive patients (average age 58 years; range 30-84 years) with a secondary osteoarthrosis in 41% of the cases. RESULTS: At the time of the latest follow-up, 33 patients (39 hips) had died and 17 cases had been lost to follow-up. The median follow-up was 122 months with a minimum of 10 years. Eight hips had been revised, leaving 177 hips in 120 living patients without revision. Six cups were revised because of aseptic loosening. Two hips were revised for sepsis. The mean Merle d'Aubigné score for the remaining hips was 16 (range 7-18) at the latest follow-up. For aseptic loosening, the probability of survival of the cup was 0.97 (95% confidence interval, 0.94-0.99). However, analysis of radiographs implied loosening in seven other cups without clinical symptoms. CONCLUSIONS: The results of primary THA using an acetabular reinforcement ring parallel the excellent results of these implants often observed in difficult primary and revision arthroplasty at a minimum of 10 years. Survivorship is comparable to modern cementless implants. Medial migration that occurs with loosening of the acetabular component seems to be prevented with this implant. Radiographic loosening signs can exist without clinical symptoms.

Kindlin-3 regulates integrin activation and adhesion reinforcement of effector T cells

Activated T cells use very late antigen-4/α4β1 integrin for capture, rolling on, and firm adhesion to endothelial cells, and use leukocyte function-associated antigen-1/αLβ2 integrin for subsequent crawling and extravasation. Inhibition of α4β1 is sufficient to prevent extravasation of activated T cells and is successfully used to combat autoimmune diseases, such as multiple sclerosis. Here we show that effector T cells lacking the integrin activator Kindlin-3 extravasate and induce experimental autoimmune encephalomyelitis in mice immunized with autoantigen. In sharp contrast, adoptively transferred autoreactive T cells from Kindlin-3-deficient mice fail to extravasate into the naïve CNS. Mechanistically, autoreactive Kindlin-3-null T cells extravasate when the CNS is inflamed and the brain microvasculature expresses high levels of integrin ligands. Flow chamber assays under physiological shear conditions confirmed that Kindlin-3-null effector T cells adhere to high concentrations of vascular cell adhesion molecule-1 and intercellular adhesion molecule-1, albeit less efficiently than WT T cells. Although these arrested T cells polarize and start crawling, only few remain firmly adherent over time. Our data demonstrate that the requirement of Kindlin-3 for effector T cells to induce α4β1 and αLβ2 integrin ligand binding and stabilization of integrin-ligand bonds is critical when integrin ligand levels are low, but of less importance when integrin ligand levels are high.

Personalized Tuning of a Reinforcement Learning Control Algorithm for Glucose Regulation

Artificial pancreas is in the forefront of research towards the automatic insulin infusion for patients with type 1 diabetes. Due to the high inter- and intra-variability of the diabetic population, the need for personalized approaches has been raised. This study presents an adaptive, patient-specific control strategy for glucose regulation based on reinforcement learning and more specifically on the Actor-Critic (AC) learning approach. The control algorithm provides daily updates of the basal rate and insulin-to-carbohydrate (IC) ratio in order to optimize glucose regulation. A method for the automatic and personalized initialization of the control algorithm is designed based on the estimation of the transfer entropy (TE) between insulin and glucose signals. The algorithm has been evaluated in silico in adults, adolescents and children for 10 days. Three scenarios of initialization to i) zero values, ii) random values and iii) TE-based values have been comparatively assessed. The results have shown that when the TE-based initialization is used, the algorithm achieves faster learning with 98%, 90% and 73% in the A+B zones of the Control Variability Grid Analysis for adults, adolescents and children respectively after five days compared to 95%, 78%, 41% for random initialization and 93%, 88%, 41% for zero initial values. Furthermore, in the case of children, the daily Low Blood Glucose Index reduces much faster when the TE-based tuning is applied. The results imply that automatic and personalized tuning based on TE reduces the learning period and improves the overall performance of the AC algorithm.

Tracing reinforcement through asymmetrical partner preference in the European common vole Microtus arvalis

Background The mechanistic basis of speciation and in particular the contribution of behaviour to the completion of the speciation process is often contentious. Contact zones between related taxa provide a situation where selection against hybridization might reinforce separation by behavioural mechanisms, which could ultimately fully isolate the taxa. One of the most abundant European mammals, the common vole Microtus arvalis, forms multiple natural hybrid zones where rapidly diverging evolutionary lineages meet in secondary contact. Very narrow zones of hybridization spanning only a few kilometres and sex-specific gene flow patterns indicate reduced fitness of natural hybrids and incipient speciation between some of the evolutionary lineages. In this study, we examined the contribution of behavioural mechanisms to the speciation process in these rodents by fine-mapping allopatric and parapatric populations in the hybrid zone between the Western and Central lineages and experimental testing of the partner preferences of wild, pure-bred and hybrid female common voles. Results Genetic analysis based on microsatellite markers revealed the presence of multiple parapatric and largely non-admixed populations at distances of about 10 km at the edge of the area of natural hybridization between the Western and Central lineages. Wild females from Western parapatric populations and lab-born F1 hybrids preferred males from the Western lineage whereas wild females of Central parapatric origin showed no measurable preference. Furthermore, wild and lab-born females from allopatric populations of the Western or Central lineages showed no detectable preference for males from either lineage. Conclusions The detected partner preferences are consistent with asymmetrical reinforcement of pre-mating reproductive isolation mechanisms in the European common vole and with earlier results suggesting that hybridization is more detrimental to the Western lineage. As a consequence, these differences in behaviour might contribute to a further geographical stabilization of this moving hybrid zone. Such behavioural processes could also provide a mechanistic perspective for frequently-detected asymmetrical introgression patterns in the largely allopatrically diversifying Microtus genus and other rapidly speciating rodents.

The Effects Of Reinforcement Inclusions On Wear Tolerance, Playing Quality And Physical Properties In A Silt Loam And Sand Rootzone Matrix

Reinforcement inclusions have been advocated to alleviate wear, compaction, and unstable surfaces in sports fields, but little research on the effects of these materials has been conducted in the USA. Experiments were established on a native silt loam and a sand rootzone matrix, seeded with a Kentucky bluegrass (Poa pratensis L.) blend, at the Joseph Troll Turf Research Center, University of Massachusetts, Amherst, USA to determine the effects of reinforcement inclusions on wear, surface hardness, traction, ball roll, ball bounce resilience, water infiltration rate, soil bulk density, air porosity, total porosity, and root weights. Three types of reinforcement inclusions (Sportgrass, Netlon, Turfgrids) were tested along with a non-reinforced control in a three year study. The treatments were set out in a randomized complete block design with four replications in both soils. No inclusion provided less wear or greater infiltration or air-filled porosity relative to the control. Reinforcement inclusions showed significant differences, however, in surface hardness, traction, and ball roll relative to the control, although this varied with the time of year. Infiltration rates, airfilled porosity, total pore space, bulk density, hardness, traction, ball roll, and ball rebound were greater on the sand rootzone than on the silt loam. Significant correlations were present between soil bulk density, surface hardness, traction, and ball roll. Based on our study, the use of reinforcement inclusions to provide better wear tolerance for sand or native soil athletic fields is not warranted. Certain playing surface characteristics, however, may be slightly improved with the use of reinforcement inclusions. The use of sands for sports surfaces is justified based upon the improvement in playing quality characteristics and soil physical properties important to a good playing surface.

Variación de las características mecánicas de armaduras de alta ductilidad B500SD en función de su grado de corrosión / Mechanical properties variation of B500SD high ductility reinforcement regarding its corrosion degree

Los efectos de la corrosión sobre las armaduras se manifiestan por la pérdida de sección y la variación de las propiedades mecánicas relacionadas con la ductilidad. En este trabajo se han ensayado a tracción 96 barras de acero B500SD que previamente se han sometido a niveles variables de corrosión. Los resultados muestran que los alargamientos de las barras disminuyen y el cociente entre la tensión máxima y el límite elástico aumenta conforme el nivel de corrosión avanza. A partir del estudio del efecto de entalla y de la distinta constitución metalográfica del acero a nivel de sección debido a su procedimiento de fabricación se pueden explicar los fenómenos anteriores.

Co-evolutionary and Reinforcement Learning Techniques Applied to Computer Go players

The objective of this thesis is model some processes from the nature as evolution and co-evolution, and proposing some techniques that can ensure that these learning process really happens and useful to solve some complex problems as Go game. The Go game is ancient and very complex game with simple rules which still is a challenge for the Artificial Intelligence. This dissertation cover some approaches that were applied to solve this problem, proposing solve this problem using competitive and cooperative co-evolutionary learning methods and other techniques proposed by the author. To study, implement and prove these methods were used some neural networks structures, a framework free available and coded many programs. The techniques proposed were coded by the author, performed many experiments to find the best configuration to ensure that co-evolution is progressing and discussed the results. Using co-evolutionary learning processes can be observed some pathologies which could impact co-evolution progress. In this dissertation is introduced some techniques to solve pathologies as loss of gradients, cycling dynamics and forgetting. According to some authors, one solution to solve these co-evolution pathologies is introduce more diversity in populations that are evolving. In this thesis is proposed some techniques to introduce more diversity and some diversity measurements for neural networks structures to monitor diversity during co-evolution. The genotype diversity evolved were analyzed in terms of its impact to global fitness of the strategies evolved and their generalization. Additionally, it was introduced a memory mechanism in the network neural structures to reinforce some strategies in the genes of the neurons evolved with the intention that some good strategies learned are not forgotten. In this dissertation is presented some works from other authors in which cooperative and competitive co-evolution has been applied. The Go board size used in this thesis was 9x9, but can be easily escalated to more bigger boards.The author believe that programs coded and techniques introduced in this dissertation can be used for other domains.