Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Promotion of health in older people: a randomised controlled trial of health risk appraisal in British general practice

BACKGROUND: there is inadequate evidence to support currently formulated NHS strategies to achieve health promotion and preventative care in older people through broad-based screening and assessment in primary care. The most extensively evaluated delivery instrument for this purpose is Health Risk Appraisal (HRA). This article describes a trial using HRA to evaluate the effect on health behaviour and preventative-care uptake in older people in NHS primary care. METHODS: a randomised controlled trial was undertaken in three London primary care group practices. Functionally independent community-dwelling patients older than 65 years (n = 2,503) received a self-administered Health Risk Appraisal for Older Persons (HRA-O) questionnaire leading to computer-generated individualised written feedback to participants and general practitioners (GPs), integrated into practice information-technology (IT) systems. All primary care staff received training in preventative health in older people. The main outcome measures were self-reported health behaviour and preventative care uptake at 1-year follow-up. RESULTS: of 2,503 individuals randomised, 2,006 respondents (80.1%) (intervention, n = 940, control n = 1,066) were available for analysis. Intervention group respondents reported slightly higher pneumococcal vaccination uptake and equivocal improvement in physical activity levels compared with controls. No significant differences were observed for any other categories of health behaviour or preventative care measures at 1-year follow-up. CONCLUSIONS: HRA-O implemented in this way resulted in minimal improvement of health behaviour or uptake of preventative care measures in older people. Supplementary reinforcement involving contact by health professionals with patients over and above routine clinical encounters may be a prerequisite to the effectiveness of IT-based delivery systems for health promotion in older people.

Impacts of a word-picture training on reading in youth with mixed intellectual disabilities. A waiting-list control group comparison

Individuals with intellectual disabilities (ID) often struggle with learning how to read. Reading difficulties seem to be the most common secondary condition of ID. Only one in five children with mild or moderate ID achieves even minimal literacy skills. However, literacy education for children and adolescents with ID has been largely overlooked by researchers and educators. While there is little research on reading of children with ID, many training studies have been conducted with other populations with reading difficulties. The most common approach of acquiring literacy skills consists of sophisticated programs that train phonological skills and auditory perception. Only few studies investigated the influence of implicit learning on literacy skills. Implicit learning processes seem to be largely independent of age and IQ. Children are sensitive to the statistics of their learning environment. By frequent word reading they acquire implicit knowledge about the frequency of single letters and letter patterns in written words. Additionally, semantic connections not only improve the word understanding, but also facilitate storage of words in memory. Advances in communication technology have introduced new possibilities for remediating literacy skills. Computers can provide training material in attractive ways, for example through animations and immediate feedback .These opportunities can scaffold and support attention processes central to learning. Thus, the aim of this intervention study was to develop and implement a computer based word-picture training, which is based on statistical and semantic learning, and to examine the training effects on reading, spelling and attention in children and adolescents (9-16 years) diagnosed with mental retardation (general IQ  74). Fifty children participated in four to five weekly training sessions of 15-20 minutes over 4 weeks, and completed assessments of attention, reading, spelling, short-term memory and fluid intelligence before and after training. After a first assessment (T1), the entire sample was divided in a training group (group A) and a waiting control group (group B). After 4 weeks of training with group A, a second assessment (T2) was administered with both training groups. Afterwards, group B was trained for 4 weeks, before a last assessment (T3) was carried out in both groups. Overall, the results showed that the word-picture training led to substantial gains on word decoding and attention for both training groups. These effects were preserved six weeks later (group A). There was also a clear tendency of improvement in spelling after training for both groups, although the effect did not reach significance. These findings highlight the fact that an implicit statistical learning training in a playful way by motivating computer programs can not only promote reading development, but also attention in children with intellectual disabilities.

Sustained impact of a short small group course with systematic feedback in addition to regular clinical clerkship activities on musculoskeletal examination skills-a controlled study.

BACKGROUND The discrepancy between the extensive impact of musculoskeletal complaints and the common deficiencies in musculoskeletal examination skills lead to increased emphasis on structured teaching and assessment. However, studies of single interventions are scarce and little is known about the time-dependent effect of assisted learning in addition to a standard curriculum. We therefore evaluated the immediate and long-term impact of a small group course on musculoskeletal examination skills. METHODS All 48 Year 4 medical students of a 6 year curriculum, attending their 8 week clerkship of internal medicine at one University department in Berne, participated in this controlled study. Twenty-seven students were assigned to the intervention of a 6×1 h practical course (4-7 students, interactive hands-on examination of real patients; systematic, detailed feedback to each student by teacher, peers and patients). Twenty-one students took part in the regular clerkship activities only and served as controls. In all students clinical skills (CS, 9 items) were assessed in an Objective Structured Clinical Examination (OSCE) station, including specific musculoskeletal examination skills (MSES, 7 items) and interpersonal skills (IPS, 2 items). Two raters assessed the skills on a 4-point Likert scale at the beginning (T0), the end (T1) and 4-12 months after (T2) the clerkship. Statistical analyses included Friedman test, Wilcoxon rank sum test and Mann-Whitney U test. RESULTS At T0 there were no significant differences between the intervention and control group. At T1 and T2 the control group showed no significant changes of CS, MSES and IPS compared to T0. In contrast, the intervention group significantly improved CS, MSES and IPS at T1 (p < 0.001). This enhancement was sustained for CS and MSES (p < 0.05), but not for IPS at T2. CONCLUSIONS Year 4 medical students were incapable of improving their musculoskeletal examination skills during regular clinical clerkship activities. However, an additional small group, interactive clinical skills course with feedback from various sources, improved these essential examination skills immediately after the teaching and several months later. We conclude that supplementary specific teaching activities are needed. Even a single, short-lasting targeted module can have a long lasting effect and is worth the additional effort.