A Neural Network of Adaptively Timed Reinforcement Learning and Hippocampal Dynamics

A neural model is described of how adaptively timed reinforcement learning occurs. The adaptive timing circuit is suggested to exist in the hippocampus, and to involve convergence of dentate granule cells on CA3 pyramidal cells, and NMDA receptors. This circuit forms part of a model neural system for the coordinated control of recognition learning, reinforcement learning, and motor learning, whose properties clarify how an animal can learn to acquire a delayed reward. Behavioral and neural data are summarized in support of each processing stage of the system. The relevant anatomical sites are in thalamus, neocortex, hippocampus, hypothalamus, amygdala, and cerebellum. Cerebellar influences on motor learning are distinguished from hippocampal influences on adaptive timing of reinforcement learning. The model simulates how damage to the hippocampal formation disrupts adaptive timing, eliminates attentional blocking, and causes symptoms of medial temporal amnesia. It suggests how normal acquisition of subcortical emotional conditioning can occur after cortical ablation, even though extinction of emotional conditioning is retarded by cortical ablation. The model simulates how increasing the duration of an unconditioned stimulus increases the amplitude of emotional conditioning, but does not change adaptive timing; and how an increase in the intensity of a conditioned stimulus "speeds up the clock", but an increase in the intensity of an unconditioned stimulus does not. Computer simulations of the model fit parametric conditioning data, including a Weber law property and an inverted U property. Both primary and secondary adaptively timed conditioning are simulated, as are data concerning conditioning using multiple interstimulus intervals (ISIs), gradually or abruptly changing ISis, partial reinforcement, and multiple stimuli that lead to time-averaging of responses. Neurobiologically testable predictions are made to facilitate further tests of the model.

Household Health and Cocoa Production: A Baseline Survey of Smallholder Farming Households in Western Region, Ghana

Background Chronic illness and premature mortality from malaria, water-borne diseases, and respiratory illnesses have long been known to diminish the welfare of individuals and households in developing countries. Previous research has also shown that chronic diseases among farming populations suppress labor productivity and agricultural output. As the illness and death toll from HIV/AIDS continues to climb in most of sub-Saharan Africa, concern has arisen that the loss of household labor it causes will reduce crop yields, impoverish farming households, intensify malnutrition, and suppress growth in the agricultural sector. If chronic morbidity and premature mortality among individuals in farming households have substantial impacts on household production, and if a large number of households are affected, it is possible that an increase in morbidity and mortality from HIV/AIDS or other diseases could affect national aggregate output and exports. If, on the other hand, the impact at the household farm level is modest, or if relatively few households are affected, there is likely to be little effect on aggregate production across an entire country. Which of these outcomes is more likely in West Africa is unknown. Little rigorous, quantitative research has been published on the impacts of AIDS on smallholder farm production, particularly in West Africa. The handful of studies that have been conducted have looked mainly at small populations in areas of very high HIV prevalence in southern and eastern Africa. Conclusions about how HIV/AIDS, and other causes of chronic morbidity and mortality, are affecting agriculture across the continent cannot be drawn from these studies. In view of the importance of agriculture, and particularly smallholder agriculture, in the economies of most African countries and the scarcity of resources for health interventions, it is valuable to identify, describe, and quantify the impact of chronic morbidity and mortality on smallholder production of important crops in West Africa. One such crop is cocoa. In Ghana, cocoa is a crop of national importance that is produced almost exclusively by smallholder households. In 2003, Ghana was the world’s second-largest producer of cocoa. Cocoa accounted for a quarter of Ghana’s export revenues that year and generated 15 percent of employment. The success and growth of the cocoa industry is thus vital to the country’s overall social and economic development. Study Objectives and Methods In February and March 2005, the Center for International Health and Development of Boston University (CIHD) and the Department of Agricultural Economics and Agribusiness (DAEA) of the University of Ghana, with financial support from the Africa Bureau of the U.S. Agency for International Development and from Mars, Inc., which is a major purchaser of West African cocoa, conducted a survey of a random sample of cocoa farming households in the Western Region of Ghana. The survey documented the extent of chronic morbidity and mortality in cocoa growing households in the Western Region of Ghana, the country’s largest cocoa growing region, and analyzed the impact of morbidity and mortality on cocoa production. It aimed to answer three specific research questions. (1) What is the baseline status of the study population in terms of household size and composition, acute and chronic morbidity, recent mortality, and cocoa production? (2) What is the relationship between household size and cocoa production, and how can this relationship be used to understand the impact of adult mortality and chronic morbidity on the production of cocoa at the household level? The study population was the approximately 42,000 cocoa farming households in the southern part of Ghana’s Western Region. A random sample of households was selected from a roster of eligible households developed from existing administrative information. Under the supervision of the University of Ghana field team, enumerators were graduate students of the Department of Agricultural Economics and Agribusiness or employees of the Cocoa Services Division. A total of 632 eligible farmers participated in the survey. Of these, 610 provided complete responses to all questions needed to complete the multivariate statistical analysis reported here.

Color Region Grouping and Shape Recognition with Deformable Models

A new deformable shape-based method for color region segmentation is described. The method includes two stages: over-segmentation using a traditional color region segmentation algorithm, followed by deformable model-based region merging via grouping and hypothesis selection. During the second stage, region merging and object identification are executed simultaneously. A statistical shape model is used to estimate the likelihood of region groupings and model hypotheses. The prior distribution on deformation parameters is precomputed using principal component analysis over a training set of region groupings. Once trained, the system autonomously segments deformed shapes from the background, while not merging them with similarly colored adjacent objects. Furthermore, the recovered parametric shape model can be used directly in object recognition and comparison. Experiments in segmentation and image retrieval are reported.

Deformable Shape Detection and Description via Model-Based Region Grouping

A method for deformable shape detection and recognition is described. Deformable shape templates are used to partition the image into a globally consistent interpretation, determined in part by the minimum description length principle. Statistical shape models enforce the prior probabilities on global, parametric deformations for each object class. Once trained, the system autonomously segments deformed shapes from the background, while not merging them with adjacent objects or shadows. The formulation can be used to group image regions based on any image homogeneity predicate; e.g., texture, color, or motion. The recovered shape models can be used directly in object recognition. Experiments with color imagery are reported.

Region Segmentation via Deformable Model-Guided Split and Merge

An improved method for deformable shape-based image segmentation is described. Image regions are merged together and/or split apart, based on their agreement with an a priori distribution on the global deformation parameters for a shape template. The quality of a candidate region merging is evaluated by a cost measure that includes: homogeneity of image properties within the combined region, degree of overlap with a deformed shape model, and a deformation likelihood term. Perceptually-motivated criteria are used to determine where/how to split regions, based on the local shape properties of the region group's bounding contour. A globally consistent interpretation is determined in part by the minimum description length principle. Experiments show that the model-based splitting strategy yields a significant improvement in segmention over a method that uses merging alone.

MosaicShape: Stochastic Region Grouping with Shape Prior

A novel method that combines shape-based object recognition and image segmentation is proposed for shape retrieval from images. Given a shape prior represented in a multi-scale curvature form, the proposed method identifies the target objects in images by grouping oversegmented image regions. The problem is formulated in a unified probabilistic framework and solved by a stochastic Markov Chain Monte Carlo (MCMC) mechanism. By this means, object segmentation and recognition are accomplished simultaneously. Within each sampling move during the simulation process,probabilistic region grouping operations are influenced by both the image information and the shape similarity constraint. The latter constraint is measured by a partial shape matching process. A generalized parallel algorithm by Barbu and Zhu,combined with a large sampling jump and other implementation improvements, greatly speeds up the overall stochastic process. The proposed method supports the segmentation and recognition of multiple occluded objects in images. Experimental results are provided for both synthetic and real images.

Hippocampal Modulation of Recognition, Conditioning, Timing, and Space: Why So Many Functions?

Advanced Research Projects Agency (ONR N00014-92-J-4015); National Science Foundation (IRI-90-24877); Office of Naval Research (N00014-91-J-1309)

Space, Time and Learning in the Hippocampus: How Fine Spatial and Temporal Scales Are Expanded into Population Codes for Behavioral Control

The hippocampus participates in multiple functions, including spatial navigation, adaptive timing, and declarative (notably, episodic) memory. How does it carry out these particular functions? The present article proposes that hippocampal spatial and temporal processing are carried out by parallel circuits within entorhinal cortex, dentate gyrus, and CA3 that are variations of the same circuit design. In particular, interactions between these brain regions transform fine spatial and temporal scales into population codes that are capable of representing the much larger spatial and temporal scales that are needed to control adaptive behaviors. Previous models of adaptively timed learning propose how a spectrum of cells tuned to brief but different delays are combined and modulated by learning to create a population code for controlling goal-oriented behaviors that span hundreds of milliseconds or even seconds. Here it is proposed how projections from entorhinal grid cells can undergo a similar learning process to create hippocampal place cells that can cover a space of many meters that are needed to control navigational behaviors. The suggested homology between spatial and temporal processing may clarify how spatial and temporal information may be integrated into an episodic memory.

Normal and Amnesis Learning, Recognition, and Memory by a Neural Model of Cortico-Hippocampal Interactions

The processes by which humans and other primates learn to recognize objects have been the subject of many models. Processes such as learning, categorization, attention, memory search, expectation, and novelty detection work together at different stages to realize object recognition. In this article, Gail Carpenter and Stephen Grossberg describe one such model class (Adaptive Resonance Theory, ART) and discuss how its structure and function might relate to known neurological learning and memory processes, such as how inferotemporal cortex can recognize both specialized and abstract information, and how medial temporal amnesia may be caused by lesions in the hippocampal formation. The model also suggests how hippocampal and inferotemporal processing may be linked during recognition learning.