Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.

Spatio-temporal credit assignment in population learning

We present a model for plasticity induction in reinforcement learning which is based on a cascade of synaptic memory traces. In the cascade of these so called eligibility traces presynaptic input is first corre lated with postsynaptic events, next with the behavioral decisions and finally with the external reinforcement. A population of leaky integrate and fire neurons endowed with this plasticity scheme is studied by simulation on different tasks. For operant co nditioning with delayed reinforcement, learning succeeds even when the delay is so large that the delivered reward reflects the appropriateness, not of the immediately preceeding response, but of a decision made earlier on in the stimulus - decision sequence . So the proposed model does not rely on the temporal contiguity between decision and pertinent reward and thus provides a viable means of addressing the temporal credit assignment problem. In the same task, learning speeds up with increasing population si ze, showing that the plasticity cascade simultaneously addresses the spatial problem of assigning credit to the different population neurons. Simulations on other task such as sequential decision making serve to highlight the robustness of the proposed sch eme and, further, contrast its performance to that of temporal difference based approaches to reinforcement learning.

Minimization of Intra-Operative Shaping of Orthopaedic Fixation Plates: A Population-Based Design

In this paper we present a new population-based method for the design of bone fixation plates. Standard pre-contoured plates are designed based on the mean shape of a certain population. We propose a computational process to design implants while reducing the amount of required intra-operative shaping, thus reducing the mechanical stresses applied to the plate. A bending and torsion model was used to measure and minimize the necessary intra-operative deformation. The method was applied and validated on a population of 200 femurs that was further augmented with a statistical shape model. The obtained results showed substantial reduction in the bending and torsion needed to shape the new design into any bone in the population when compared to the standard mean-based plates.

Irradiated disks and planet population synthesis

Recent planet population synthesis models (Alibert et al. 2010, submitted) have emphasized the key role played by the proto-planetary disk properties in determining the overall planet population characteristics. We present a disk model that takes into account viscous heating and irradiation by a central star. We consider the case of an equilibrium flaring angle. We illustrate the consequences of the resulting changes in the disk structure on the planet population by the synthetic populations corresponding to each of the different structures.

Habitat-specific demography and source-sink dynamics in a population of Siberian jays

P>1. There are a number of models describing population structure, many of which have the capacity to incorporate spatial habitat effects. One such model is the source-sink model, that describes a system where some habitats have a natality that is higher than mortality (source) and others have a mortality that exceeds natality (sink). A source can be maintained in the absence of migration, whereas a sink will go extinct. 2. However, the interaction between population dynamics and habitat quality is complex, and concerns have been raised about the validity of published empirical studies addressing source-sink dynamics. In particular, some of these studies fail to provide data on survival, a significant component in disentangling a sink from a low quality source. Moreover, failing to account for a density-dependent increase in mortality, or decrease in fecundity, can result in a territory being falsely assigned as a sink, when in fact, this density-dependent suppression only decreases the population size to a lower level, hence indicating a 'pseudo-sink'. 3. In this study, we investigate a long-term data set for key components of territory-specific demography (mortality and reproduction) and their relationship to habitat characteristics in the territorial, group-living Siberian jay (Perisoreus infaustus). We also assess territory-specific population growth rates (r), to test whether spatial population dynamics are consistent with the ideas of source-sink dynamics. 4. Although average mortality did not differ between sexes, habitat-specific mortality did. Female mortality was higher in older forests, a pattern not observed in males. Male mortality only increased with an increasing amount of open areas. Moreover, reproductive success was higher further away from human settlement, indicating a strong effect of human-associated nest predators. 5. Averaged over all years, 76% of the territories were sources. These territories generally consisted of less open areas, and were located further away from human settlement. 6. The source-sink model provides a tool for modelling demography in distinct habitat patches of different quality, which can aid in identifying key habitats within the landscape, and thus, reduce the risk of implementing unsound management decisions.

A Method for Detecting Population Genetic Structure in Diverse, High Gene-Flow Species

Detecting small amounts of genetic subdivision across geographic space remains a persistent challenge. Often a failure to detect genetic structure is mistaken for evidence of panmixia, when more powerful statistical tests may uncover evidence for subtle geographic differentiation. Such slight subdivision can be demographically and evolutionarily important as well as being critical for management decisions. We introduce here a method, called spatial analysis of shared alleles (SAShA), that detects geographically restricted alleles by comparing the spatial arrangement of allelic co-occurrences with the expectation under panmixia. The approach is allele-based and spatially explicit, eliminating the loss of statistical power that can occur with user-defined populations and statistical averaging within populations. Using simulated data sets generated under a stepping-stone model of gene flow, we show that this method outperforms spatial autocorrelation (SA) and UST under common real-world conditions: at relatively high migration rates when diversity is moderate or high, especially when sampling is poor. We then use this method to show clear differences in the genetic patterns of 2 nearshore Pacific mollusks, Tegula funebralis (5 Chlorostoma funebralis) and Katharina tunicata, whose overall patterns of within-species differentiation are similar according to traditional population genetics analyses. SAShA meaningfully complements UST/FST, SA, and other existing geographic genetic analyses and is especially appropriate for evaluating species with high gene flow and subtle genetic differentiation.

TILLING for Mutations in Model Plants and Crops

A growing world population, changing climate and limiting fossil fuels will provide new pressures on human production of food, medicine, fuels and feed stock in the twenty-first century. Enhanced crop production promises to ameliorate these pressures. Crops can be bred for increased yields of calories, starch, nutrients, natural medicinal compounds, and other important products. Enhanced resistance to biotic and abiotic stresses can be introduced, toxins removed, and industrial qualities such as fibre strength and biofuel per mass can be increased. Induced and natural mutations provide a powerful method for the generation of heritable enhanced traits. While mainly exploited in forward, phenotype driven, approaches, the rapid accumulation of plant genomic sequence information and hypotheses regarding gene function allows the use of mutations in reverse genetic approaches to identify lesions in specific target genes. Such gene-driven approaches promise to speed up the process of creating novel phenotypes, and can enable the generation of phenotypes unobtainable by traditional forward methods. TILLING (Targeting Induced Local Lesions IN Genome) is a high-throughput and low cost reverse genetic method for the discovery of induced mutations. The method has been modified for the identification of natural nucleotide polymorphisms, a process called Ecotilling. The methods are general and have been applied to many species, including a variety of different crops. In this chapter the current status of the TILLING and Ecotilling methods and provide an overview of progress in applying these methods to different plant species, with a focus on work related to food production for developing nations.

Spatio-Temporal Credit Assignment in Neuronal Population Learning

n learning from trial and error, animals need to relate behavioral decisions to environmental reinforcement even though it may be difficult to assign credit to a particular decision when outcomes are uncertain or subject to delays. When considering the biophysical basis of learning, the credit-assignment problem is compounded because the behavioral decisions themselves result from the spatio-temporal aggregation of many synaptic releases. We present a model of plasticity induction for reinforcement learning in a population of leaky integrate and fire neurons which is based on a cascade of synaptic memory traces. Each synaptic cascade correlates presynaptic input first with postsynaptic events, next with the behavioral decisions and finally with external reinforcement. For operant conditioning, learning succeeds even when reinforcement is delivered with a delay so large that temporal contiguity between decision and pertinent reward is lost due to intervening decisions which are themselves subject to delayed reinforcement. This shows that the model provides a viable mechanism for temporal credit assignment. Further, learning speeds up with increasing population size, so the plasticity cascade simultaneously addresses the spatial problem of assigning credit to synapses in different population neurons. Simulations on other tasks, such as sequential decision making, serve to contrast the performance of the proposed scheme to that of temporal difference-based learning. We argue that, due to their comparative robustness, synaptic plasticity cascades are attractive basic models of reinforcement learning in the brain.

Can wild ungulate carcasses provide enough biomass to maintain avian scavenger populations? An empirical assessment using a bio-inspired computational model

Background The reduction in the amount of food available for European avian scavengers as a consequence of restrictive public health policies is a concern for managers and conservationists. Since 2002, the application of several sanitary regulations has limited the availability of feeding resources provided by domestic carcasses, but theoretical studies assessing whether the availability of food resources provided by wild ungulates are enough to cover energetic requirements are lacking. Methodology/Findings We assessed food provided by a wild ungulate population in two areas of NE Spain inhabited by three vulture species and developed a P System computational model to assess the effects of the carrion resources provided on their population dynamics. We compared the real population trend with to a hypothetical scenario in which only food provided by wild ungulates was available. Simulation testing of the model suggests that wild ungulates constitute an important food resource in the Pyrenees and the vulture population inhabiting this area could grow if only the food provided by wild ungulates would be available. On the contrary, in the Pre-Pyrenees there is insufficient food to cover the energy requirements of avian scavenger guilds, declining sharply if biomass from domestic animals would not be available. Conclusions/Significance Our results suggest that public health legislation can modify scavenger population trends if a large number of domestic ungulate carcasses disappear from the mountains. In this case, food provided by wild ungulates could be not enough and supplementary feeding could be necessary if other alternative food resources are not available (i.e. the reintroduction of wild ungulates), preferably in European Mediterranean scenarios sharing similar and socio-economic conditions where there are low densities of wild ungulates. Managers should anticipate the conservation actions required by assessing food availability and the possible scenarios in order to make the most suitable decisions.

Population genomic tests of models of adaptive radiation in Lake Victoria Region cichlid fish

Adaptive radiation is usually thought to be associated with speciation, but the evolution of intraspecific polymorphisms without speciation is also possible. The radiation of cichlid fish in Lake Victoria (LV) is perhaps the most impressive example of a recent rapid adaptive radiation, with 600+ very young species. Key questions about its origin remain poorly characterized, such as the importance of speciation versus polymorphism, whether species persist on evolutionary time scales, and if speciation happens more commonly in small isolated or in large connected populations. We used 320 individuals from 105 putative species from Lakes Victoria, Edward, Kivu, Albert, Nabugabo and Saka, in a radiation-wide amplified fragment length polymorphism (AFLP) genome scan to address some of these questions. We demonstrate pervasive signatures of speciation supporting the classical model of adaptive radiation associated with speciation. A positive relationship between the age of lakes and the average genomic differentiation of their species, and a significant fraction of molecular variance explained by above-species level taxonomy suggest the persistence of species on evolutionary time scales, with radiation through sequential speciation rather than a single starburst. Finally the large gene diversity retained from colonization to individual species in every radiation suggests large effective population sizes and makes speciation in small geographical isolates unlikely.

The SURTAVI model: proposal for a pragmatic risk stratification for patients with severe aortic stenosis

Transcatheter aortic valve implantation (TAVI) is a less invasive alternative to surgical aortic valve replacement (SAVR) for patients with symptomatic severe aortic stenosis (AS) and a high operative risk. Risk stratification plays a decisive role in the optimal selection of therapeutic strategies for AS patients. The accuracy of contemporary surgical risk algorithms for AS patients has spurred considerable debate especially in the higher risk patient population. Future trials will explore TAVI in patients at intermediate operative risk. During the design of the SURgical replacement and Transcatheter Aortic Valve Implantation (SURTAVI) trial, a novel concept of risk stratification was proposed based upon age in combination with a fixed number of predefined risk factors, which are relatively prevalent, easy to capture and with a reasonable impact on operative mortality. Retrospective application of this algorithm to a contemporary academic practice dealing with clinically significant AS patients allocates about one-fourth of these patients as being at intermediate operative risk. Further testing is required for validation of this new paradigm in risk stratification. Finally, the Heart Team, consisting of at least an interventional cardiologist and cardiothoracic surgeon, should have the decisive role in determining whether a patient could be treated with TAVI or SAVR.

Segregation analysis of overweight body condition in an experimental cat population

The goal of this study was to analyze the mode of inheritance of an overweight body condition in an experimental cat population. The cat population consisted of 95 cats of which 81 cats could be clearly classified into lean or overweight using the body condition scoring system according to Laflamme. The lean or overweight classification was then used for segregation analyses. Complex segregation analyses were employed to test for the significance of one environmental and 4 genetic models (general, mixed inheritance, major gene, and polygene). The general genetic model fit the data significantly better than the environmental model (P models employed, the major gene model explained the segregation of the overweight phenotype best. This is the first study in which a genetic component could be shown to be responsible for the development of overweight in cats.

Pathology hinting as the combination of automatic segmentation with a statistical shape model

With improvements in acquisition speed and quality, the amount of medical image data to be screened by clinicians is starting to become challenging in the daily clinical practice. To quickly visualize and find abnormalities in medical images, we propose a new method combining segmentation algorithms with statistical shape models. A statistical shape model built from a healthy population will have a close fit in healthy regions. The model will however not fit to morphological abnormalities often present in the areas of pathologies. Using the residual fitting error of the statistical shape model, pathologies can be visualized very quickly. This idea is applied to finding drusen in the retinal pigment epithelium (RPE) of optical coherence tomography (OCT) volumes. A segmentation technique able to accurately segment drusen in patients with age-related macular degeneration (AMD) is applied. The segmentation is then analyzed with a statistical shape model to visualize potentially pathological areas. An extensive evaluation is performed to validate the segmentation algorithm, as well as the quality and sensitivity of the hinting system. Most of the drusen with a height of 85.5 microm were detected, and all drusen at least 93.6 microm high were detected.

A prediction model for assessing residential radon concentration in Switzerland

Indoor radon is regularly measured in Switzerland. However, a nationwide model to predict residential radon levels has not been developed. The aim of this study was to develop a prediction model to assess indoor radon concentrations in Switzerland. The model was based on 44,631 measurements from the nationwide Swiss radon database collected between 1994 and 2004. Of these, 80% randomly selected measurements were used for model development and the remaining 20% for an independent model validation. A multivariable log-linear regression model was fitted and relevant predictors selected according to evidence from the literature, the adjusted R², the Akaike's information criterion (AIC), and the Bayesian information criterion (BIC). The prediction model was evaluated by calculating Spearman rank correlation between measured and predicted values. Additionally, the predicted values were categorised into three categories (50th, 50th-90th and 90th percentile) and compared with measured categories using a weighted Kappa statistic. The most relevant predictors for indoor radon levels were tectonic units and year of construction of the building, followed by soil texture, degree of urbanisation, floor of the building where the measurement was taken and housing type (P-values <0.001 for all). Mean predicted radon values (geometric mean) were 66 Bq/m³ (interquartile range 40-111 Bq/m³) in the lowest exposure category, 126 Bq/m³ (69-215 Bq/m³) in the medium category, and 219 Bq/m³ (108-427 Bq/m³) in the highest category. Spearman correlation between predictions and measurements was 0.45 (95%-CI: 0.44; 0.46) for the development dataset and 0.44 (95%-CI: 0.42; 0.46) for the validation dataset. Kappa coefficients were 0.31 for the development and 0.30 for the validation dataset, respectively. The model explained 20% overall variability (adjusted R²). In conclusion, this residential radon prediction model, based on a large number of measurements, was demonstrated to be robust through validation with an independent dataset. The model is appropriate for predicting radon level exposure of the Swiss population in epidemiological research. Nevertheless, some exposure misclassification and regression to the mean is unavoidable and should be taken into account in future applications of the model.

Neuronal expression of the ubiquitin ligase Nedd4-2 in rat dorsal root ganglia: modulation in the spared nerve injury model of neuropathic pain

Neuronal hyperexcitability following peripheral nerve lesions may stem from altered activity of voltage-gated sodium channels (VGSCs), which gives rise to allodynia or hyperalgesia. In vitro, the ubiquitin ligase Nedd4-2 is a negative regulator of VGSC α-subunits (Na(v)), in particular Na(v)1.7, a key actor in nociceptor excitability. We therefore studied Nedd4-2 in rat nociceptors, its co-expression with Na(v)1.7 and Na(v)1.8, and its regulation in pathology. Adult rats were submitted to the spared nerve injury (SNI) model of neuropathic pain or injected with complete Freund's adjuvant (CFA), a model of inflammatory pain. L4 dorsal root ganglia (DRG) were analyzed in sham-operated animals, seven days after SNI and 48 h after CFA with immunofluorescence and Western blot. We observed Nedd4-2 expression in almost 50% of DRG neurons, mostly small and medium-sized. A preponderant localization is found in the non-peptidergic sub-population. Additionally, 55.7 ± 2.7% and 55.0 ± 3.6% of Nedd4-2-positive cells are co-labeled with Na(v)1.7 and Na(v)1.8 respectively. SNI significantly decreases the proportion of Nedd4-2-positive neurons from 45.9 ± 1.9% to 33.5 ± 0.7% (p<0.01) and the total Nedd4-2 protein to 44% ± 0.13% of its basal level (p<0.01, n=4 animals in each group, mean ± SEM). In contrast, no change in Nedd4-2 was found after peripheral inflammation induced by CFA. These results indicate that Nedd4-2 is present in nociceptive neurons, is downregulated after peripheral nerve injury, and might therefore contribute to the dysregulation of Na(v)s involved in the hyperexcitability associated with peripheral nerve injuries.