Metacognition in monkeys during an oculomotor task.

This study investigated whether rhesus monkeys show evidence of metacognition in a reduced, visual oculomotor task that is particularly suitable for use in fMRI and electrophysiology. The 2-stage task involved punctate visual stimulation and saccadic eye movement responses. In each trial, monkeys made a decision and then made a bet. To earn maximum reward, they had to monitor their decision and use that information to bet advantageously. Two monkeys learned to base their bets on their decisions within a few weeks. We implemented an operational definition of metacognitive behavior that relied on trial-by-trial analyses and signal detection theory. Both monkeys exhibited metacognition according to these quantitative criteria. Neither external visual cues nor potential reaction time cues explained the betting behavior; the animals seemed to rely exclusively on internal traces of their decisions. We documented the learning process of one monkey. During a 10-session transition phase, betting switched from random to a decision-based strategy. The results reinforce previous findings of metacognitive ability in monkeys and may facilitate the neurophysiological investigation of metacognitive functions.

Game-Theoretically Allocating Resources to Catch Evaders and Payments to Stabilize Teams

Allocating resources optimally is a nontrivial task, especially when multiple

self-interested agents with conflicting goals are involved. This dissertation

uses techniques from game theory to study two classes of such problems:

allocating resources to catch agents that attempt to evade them, and allocating

payments to agents in a team in order to stabilize it. Besides discussing what

allocations are optimal from various game-theoretic perspectives, we also study

how to efficiently compute them, and if no such algorithms are found, what

computational hardness results can be proved.

The first class of problems is inspired by real-world applications such as the

TOEFL iBT test, course final exams, driver's license tests, and airport security

patrols. We call them test games and security games. This dissertation first

studies test games separately, and then proposes a framework of Catcher-Evader

games (CE games) that generalizes both test games and security games. We show

that the optimal test strategy can be efficiently computed for scored test

games, but it is hard to compute for many binary test games. Optimal Stackelberg

strategies are hard to compute for CE games, but we give an empirically

efficient algorithm for computing their Nash equilibria. We also prove that the

Nash equilibria of a CE game are interchangeable.

The second class of problems involves how to split a reward that is collectively

obtained by a team. For example, how should a startup distribute its shares, and

what salary should an enterprise pay to its employees. Several stability-based

solution concepts in cooperative game theory, such as the core, the least core,

and the nucleolus, are well suited to this purpose when the goal is to avoid

coalitions of agents breaking off. We show that some of these solution concepts

can be justified as the most stable payments under noise. Moreover, by adjusting

the noise models (to be arguably more realistic), we obtain new solution

concepts including the partial nucleolus, the multiplicative least core, and the

multiplicative nucleolus. We then study the computational complexity of those

solution concepts under the constraint of superadditivity. Our result is based

on what we call Small-Issues-Large-Team games and it applies to popular

representation schemes such as MC-nets.

Stimulus Integration and Parsing in the Primate Auditory Midbrain

Integrating information from multiple sources is a crucial function of the brain. Examples of such integration include multiple stimuli of different modalties, such as visual and auditory, multiple stimuli of the same modality, such as auditory and auditory, and integrating stimuli from the sensory organs (i.e. ears) with stimuli delivered from brain-machine interfaces.

The overall aim of this body of work is to empirically examine stimulus integration in these three domains to inform our broader understanding of how and when the brain combines information from multiple sources.

First, I examine visually-guided auditory, a problem with implications for the general problem in learning of how the brain determines what lesson to learn (and what lessons not to learn). For example, sound localization is a behavior that is partially learned with the aid of vision. This process requires correctly matching a visual location to that of a sound. This is an intrinsically circular problem when sound location is itself uncertain and the visual scene is rife with possible visual matches. Here, we develop a simple paradigm using visual guidance of sound localization to gain insight into how the brain confronts this type of circularity. We tested two competing hypotheses. 1: The brain guides sound location learning based on the synchrony or simultaneity of auditory-visual stimuli, potentially involving a Hebbian associative mechanism. 2: The brain uses a ‘guess and check’ heuristic in which visual feedback that is obtained after an eye movement to a sound alters future performance, perhaps by recruiting the brain’s reward-related circuitry. We assessed the effects of exposure to visual stimuli spatially mismatched from sounds on performance of an interleaved auditory-only saccade task. We found that when humans and monkeys were provided the visual stimulus asynchronously with the sound but as feedback to an auditory-guided saccade, they shifted their subsequent auditory-only performance toward the direction of the visual cue by 1.3-1.7 degrees, or 22-28% of the original 6 degree visual-auditory mismatch. In contrast when the visual stimulus was presented synchronously with the sound but extinguished too quickly to provide this feedback, there was little change in subsequent auditory-only performance. Our results suggest that the outcome of our own actions is vital to localizing sounds correctly. Contrary to previous expectations, visual calibration of auditory space does not appear to require visual-auditory associations based on synchrony/simultaneity.

My next line of research examines how electrical stimulation of the inferior colliculus influences perception of sounds in a nonhuman primate. The central nucleus of the inferior colliculus is the major ascending relay of auditory information before it reaches the forebrain, and thus an ideal target for understanding low-level information processing prior to the forebrain, as almost all auditory signals pass through the central nucleus of the inferior colliculus before reaching the forebrain. Thus, the inferior colliculus is the ideal structure to examine to understand the format of the inputs into the forebrain and, by extension, the processing of auditory scenes that occurs in the brainstem. Therefore, the inferior colliculus was an attractive target for understanding stimulus integration in the ascending auditory pathway.

Moreover, understanding the relationship between the auditory selectivity of neurons and their contribution to perception is critical to the design of effective auditory brain prosthetics. These prosthetics seek to mimic natural activity patterns to achieve desired perceptual outcomes. We measured the contribution of inferior colliculus (IC) sites to perception using combined recording and electrical stimulation. Monkeys performed a frequency-based discrimination task, reporting whether a probe sound was higher or lower in frequency than a reference sound. Stimulation pulses were paired with the probe sound on 50% of trials (0.5-80 µA, 100-300 Hz, n=172 IC locations in 3 rhesus monkeys). Electrical stimulation tended to bias the animals’ judgments in a fashion that was coarsely but significantly correlated with the best frequency of the stimulation site in comparison to the reference frequency employed in the task. Although there was considerable variability in the effects of stimulation (including impairments in performance and shifts in performance away from the direction predicted based on the site’s response properties), the results indicate that stimulation of the IC can evoke percepts correlated with the frequency tuning properties of the IC. Consistent with the implications of recent human studies, the main avenue for improvement for the auditory midbrain implant suggested by our findings is to increase the number and spatial extent of electrodes, to increase the size of the region that can be electrically activated and provide a greater range of evoked percepts.

My next line of research employs a frequency-tagging approach to examine the extent to which multiple sound sources are combined (or segregated) in the nonhuman primate inferior colliculus. In the single-sound case, most inferior colliculus neurons respond and entrain to sounds in a very broad region of space, and many are entirely spatially insensitive, so it is unknown how the neurons will respond to a situation with more than one sound. I use multiple AM stimuli of different frequencies, which the inferior colliculus represents using a spike timing code. This allows me to measure spike timing in the inferior colliculus to determine which sound source is responsible for neural activity in an auditory scene containing multiple sounds. Using this approach, I find that the same neurons that are tuned to broad regions of space in the single sound condition become dramatically more selective in the dual sound condition, preferentially entraining spikes to stimuli from a smaller region of space. I will examine the possibility that there may be a conceptual linkage between this finding and the finding of receptive field shifts in the visual system.

In chapter 5, I will comment on these findings more generally, compare them to existing theoretical models, and discuss what these results tell us about processing in the central nervous system in a multi-stimulus situation. My results suggest that the brain is flexible in its processing and can adapt its integration schema to fit the available cues and the demands of the task.

Expectation Modulates Episodic Memory Formation via Dopaminergic Circuitry

Episodic memory formation is shaped by expectation. Events that generate expectations have the capacity to influence memory. Additionally, whether subsequent events meet or violate expectations has consequences for memory. However, clarification is still required to illuminate the circumstances and direction of memory modulation. In the brain, the mechanisms by which expectation modulates memory formation also require consideration. The dopamine system has been implicated in signaling events associated with different states of expectancy; it has also been shown to modulate episodic memory formation in the hippocampus. Thus, the studies included in this dissertation utilized both functional magnetic resonance imaging (fMRI) and behavioral testing to examine when and how the dopaminergic system supports the modulation of memory by expectation. The work aimed to characterize the activation of dopaminergic circuitry in response to cues that generate expectancy, during periods of anticipation, and in response to outcomes that resolve expectancy. The studies also examined how each of these event types influenced episodic memory formation. The present findings demonstrated that novelty and expectancy violation both drive dopaminergic circuitry capable of contributing to memory formation. Consistent with elevated dopaminergic midbrain and hippocampus activation for each, expected versus expectancy violating novelty did not differentially affect memory success. We also showed that high curiosity expectancy states drive memory formation. This was supported by activation in dopaminergic circuitry that was greater for subsequently remembered information only in the high curiosity state. Finally, we showed that cues that generate high expected reward value versus high reward uncertainty differentially modulate memory formation during reward anticipation. This behavioral result was consistent with distinct temporal profiles of dopaminergic action having differential downstream effects on episodic memory formation. Integrating the present studies with previous research suggests that dopaminergic circuitry signals events that are unpredicted, whether cuing or resolving expectations. It also suggests that contextual differences change the contribution of the dopaminergic system during anticipation, depending on the nature of the expectation. And finally, this work is consistent with a model in which dopamine elevation in response to expectancy events positively modulates episodic memory formation.

Neural Mechanisms of Young Adult Sexual Decision-Making and Risk Behavior

Sexual risk behavior among young adults is a serious public health concern; 50% will contract a sexually transmitted infection (STI) before the age of 25. The current study collected self-report personality and sexual history data, as well as neuroimaging, experimental behavioral (e.g., real-time hypothetical sexual decision making data), and self-report sexual arousal data from 120 heterosexual young adults ages 18-26. In addition, longitudinal changes in self-reported sexual behavior were collected from a subset (n = 70) of the participants. The primary aims of the study were (1) to predict differences in self-report sexual behavior and hypothetical sexual decision-making (in response to sexually explicit audio-visual cues) as a function of ventral striatum (VS) and amygdala activity, (2) test whether the association between sexual behavior/decision-making and brain function is moderated by gender, self-reported sexual arousal, and/or trait-level personality factors (i.e., self-control, impulsivity, and sensation seeking) and (3) to examine how the main effects of neural function and interaction effects predict sexual risk behavior over time. Our hypotheses were mostly supported across the sexual behavior and decision-making outcome variables, such that neural risk phenotypes (heightened reward-related ventral striatum activity coupled with decreased threat-related amygdala activity) were associated with greater lifetime sexual partners at baseline measured and over time (longitudinal analyses). Impulsivity moderated the relationship between neural function and self-reported number of sexual partners at baseline and follow up measures, as well as experimental condom use decision-making. Sexual arousal and sensation seeking moderated the relationship between neural function and baseline and follow up self-reports of number of sexual partners. Finally, unique gender differences were observed in the relationship between threat and reward-related neural reactivity and self-reported sexual risk behavior. The results of this study provide initial evidence for the potential role for neurobiological approaches to understanding sexual decision-making and risk behavior. With continued research, establishing biomarkers for sexual risk behavior could help inform the development of novel and more effective individually tailored sexual health prevention and intervention efforts.

On the Regulation of Small Actors: Three Experimental Essays about Policies based on Voluntary Compliance and Decentralized Monitoring

Monitoring and enforcement are perhaps the biggest challenges in the design and implementation of environmental policies in developing countries where the actions of many small informal actors cause significant impacts on the ecosystem services and where the transaction costs for the state to regulate them could be enormous. This dissertation studies the potential of innovative institutions based on decentralized coordination and enforcement to induce better environmental outcomes. Such policies have in common that the state plays the role of providing the incentives for organization but the process of compliance happens through decentralized agreements, trust building, signaling and monitoring. I draw from the literatures in collective action, common-pool resources, game-theory and non-point source pollution to develop the instruments proposed here. To test the different conditions in which such policies could be implemented I designed two field-experiments that I conducted with small-scale gold miners in the Colombian Pacific and with users and providers of ecosystem services in the states of Veracruz, Quintana Roo and Yucatan in Mexico. This dissertation is organized in three essays.

The first essay, “Collective Incentives for Cleaner Small-Scale Gold Mining on the Frontier: Experimental Tests of Compliance with Group Incentives given Limited State Monitoring”, examines whether collective incentives, i.e. incentives provided to a group conditional on collective compliance, could “outsource” the required local monitoring, i.e. induce group interactions that extend the reach of the state that can observe only aggregate consequences in the context of small-scale gold mining. I employed a framed field-lab experiment in which the miners make decisions regarding mining intensity. The state sets a collective target for an environmental outcome, verifies compliance and provides a group reward for compliance which is split equally among members. Since the target set by the state transforms the situation into a coordination game, outcomes depend on expectations of what others will do. I conducted this experiment with 640 participants in a mining region of the Colombian Pacific and I examine different levels of policy severity and their ordering. The findings of the experiment suggest that such instruments can induce compliance but this regulation involves tradeoffs. For most severe targets – with rewards just above costs – raise gains if successful but can collapse rapidly and completely. In terms of group interactions, better outcomes are found when severity initially is lower suggesting learning.

The second essay, “Collective Compliance can be Efficient and Inequitable: Impacts of Leaders among Small-Scale Gold Miners in Colombia”, explores the channels through which communication help groups to coordinate in presence of collective incentives and whether the reached solutions are equitable or not. Also in the context of small-scale gold mining in the Colombian Pacific, I test the effect of communication in compliance with a collective environmental target. The results suggest that communication, as expected, helps to solve coordination challenges but still some groups reach agreements involving unequal outcomes. By examining the agreements that took place in each group, I observe that the main coordination mechanism was the presence of leaders that help other group members to clarify the situation. Interestingly, leaders not only helped groups to reach efficiency but also played a key role in equity by defining how the costs of compliance would be distributed among group members.

The third essay, “Creating Local PES Institutions and Increasing Impacts of PES in Mexico: A real-Time Watershed-Level Framed Field Experiment on Coordination and Conditionality”, considers the creation of a local payments for ecosystem services (PES) mechanism as an assurance game that requires the coordination between two groups of participants: upstream and downstream. Based on this assurance interaction, I explore the effect of allowing peer-sanctions on upstream behavior in the functioning of the mechanism. This field-lab experiment was implemented in three real cases of the Mexican Fondos Concurrentes (matching funds) program in the states of Veracruz, Quintana Roo and Yucatan, where 240 real users and 240 real providers of hydrological services were recruited and interacted with each other in real time. The experimental results suggest that initial trust-game behaviors align with participants’ perceptions and predicts baseline giving in assurance game. For upstream providers, i.e. those who get sanctioned, the threat and the use of sanctions increase contributions. Downstream users contribute less when offered the option to sanction – as if that option signal an uncooperative upstream – then the contributions rise in line with the complementarity in payments of the assurance game.

Neural mechanisms of negative reinforcement in children and adolescents with autism spectrum disorders.

BACKGROUND: Previous research has found accumulating evidence for atypical reward processing in autism spectrum disorders (ASD), particularly in the context of social rewards. Yet, this line of research has focused largely on positive social reinforcement, while little is known about the processing of negative reinforcement in individuals with ASD. METHODS: The present study examined neural responses to social negative reinforcement (a face displaying negative affect) and non-social negative reinforcement (monetary loss) in children with ASD relative to typically developing children, using functional magnetic resonance imaging (fMRI). RESULTS: We found that children with ASD demonstrated hypoactivation of the right caudate nucleus while anticipating non-social negative reinforcement and hypoactivation of a network of frontostriatal regions (including the nucleus accumbens, caudate nucleus, and putamen) while anticipating social negative reinforcement. In addition, activation of the right caudate nucleus during non-social negative reinforcement was associated with individual differences in social motivation. CONCLUSIONS: These results suggest that atypical responding to negative reinforcement in children with ASD may contribute to social motivational deficits in this population.

Association between the oxytocin receptor (OXTR) gene and mesolimbic responses to rewards.

BACKGROUND: There has been significant progress in identifying genes that confer risk for autism spectrum disorders (ASDs). However, the heterogeneity of symptom presentation in ASDs impedes the detection of ASD risk genes. One approach to understanding genetic influences on ASD symptom expression is to evaluate relations between variants of ASD candidate genes and neural endophenotypes in unaffected samples. Allelic variations in the oxytocin receptor (OXTR) gene confer small but significant risk for ASDs for which the underlying mechanisms may involve associations between variability in oxytocin signaling pathways and neural response to rewards. The purpose of this preliminary study was to investigate the influence of allelic variability in the OXTR gene on neural responses to monetary rewards in healthy adults using functional magnetic resonance imaging (fMRI). METHODS: The moderating effects of three single nucleotide polymorphisms (SNPs) (rs1042778, rs2268493 and rs237887) of the OXTR gene on mesolimbic responses to rewards were evaluated using a monetary incentive delay fMRI task. RESULTS: T homozygotes of the rs2268493 SNP demonstrated relatively decreased activation in mesolimbic reward circuitry (including the nucleus accumbens, amygdala, insula, thalamus and prefrontal cortical regions) during the anticipation of rewards but not during the outcome phase of the task. Allelic variation of the rs1042778 and rs237887 SNPs did not moderate mesolimbic activation during either reward anticipation or outcomes. CONCLUSIONS: This preliminary study suggests that the OXTR SNP rs2268493, which has been previously identified as an ASD risk gene, moderates mesolimbic responses during reward anticipation. Given previous findings of decreased mesolimbic activation during reward anticipation in ASD, the present results suggest that OXTR may confer ASD risk via influences on the neural systems that support reward anticipation.