Neural Network Evidence for the Coupling of Presaccadic Visual Remapping to Predictive Eye Position Updating.

As we look around a scene, we perceive it as continuous and stable even though each saccadic eye movement changes the visual input to the retinas. How the brain achieves this perceptual stabilization is unknown, but a major hypothesis is that it relies on presaccadic remapping, a process in which neurons shift their visual sensitivity to a new location in the scene just before each saccade. This hypothesis is difficult to test in vivo because complete, selective inactivation of remapping is currently intractable. We tested it in silico with a hierarchical, sheet-based neural network model of the visual and oculomotor system. The model generated saccadic commands to move a video camera abruptly. Visual input from the camera and internal copies of the saccadic movement commands, or corollary discharge, converged at a map-level simulation of the frontal eye field (FEF), a primate brain area known to receive such inputs. FEF output was combined with eye position signals to yield a suitable coordinate frame for guiding arm movements of a robot. Our operational definition of perceptual stability was "useful stability," quantified as continuously accurate pointing to a visual object despite camera saccades. During training, the emergence of useful stability was correlated tightly with the emergence of presaccadic remapping in the FEF. Remapping depended on corollary discharge but its timing was synchronized to the updating of eye position. When coupled to predictive eye position signals, remapping served to stabilize the target representation for continuously accurate pointing. Graded inactivations of pathways in the model replicated, and helped to interpret, previous in vivo experiments. The results support the hypothesis that visual stability requires presaccadic remapping, provide explanations for the function and timing of remapping, and offer testable hypotheses for in vivo studies. We conclude that remapping allows for seamless coordinate frame transformations and quick actions despite visual afferent lags. With visual remapping in place for behavior, it may be exploited for perceptual continuity.

Supplementary Eye Fields

The supplementary eye fields (SEFs) are located in dorsomedial frontal cortex and contribute to high-level control of eye movements. Recordings in the SEF reveal neural activity related to vision, saccades, and fixations, and electrical stimulation in the SEF evokes saccades and fixations. Inactivations and lesions of the SEF, however, cause minimal oculomotor deficits. The SEF thus processes information relevant to eye movements and influences critical oculomotor centers but seems unnecessary for generating action. Instead, the SEF has overarching, subtle functions that include limb-eye coordination, the timing and sequencing of actions, learning, monitoring conflict, prediction, supervising behavior, value-based decision making, and the monitoring of decisions.

Supplementary Eye Fields

The supplementary eye fields (SEFs) are located in dorsomedial frontal cortex and contribute to high-level control of eye movements. Recordings in the SEF reveal neural activity related to vision, saccades, and fixations, and electrical stimulation in the SEF evokes saccades and fixations. Inactivations and lesions of the SEF, however, cause minimal oculomotor deficits. The SEF thus processes information relevant to eye movements and influences critical oculomotor centers but seems unnecessary for generating action. Instead, the SEF has overarching, subtle functions that include limb-eye coordination, the timing and sequencing of actions, learning, monitoring conflict, prediction, supervising behavior, value-based decision making, and the monitoring of decisions.

Circuits for presaccadic remapping

Saccadic eye movements rapidly displace the image of the world that is projected onto the retinas. In anticipation of each saccade, many neurons in the visual system shift their receptive fields. This presaccadic change in visual sensitivity, known as remapping, was first documented in the parietal cortex and has been studied in many other brain regions. Remapping requires information about upcoming saccades via corollary discharge. Analyses of neurons in a corollary discharge pathway that targets the frontal eye field (FEF) suggest that remapping may be assembled in the FEF’s local microcircuitry. Complementary data from reversible inactivation, neural recording, and modeling studies provide evidence that remapping contributes to transsaccadic continuity of action and perception. Multiple forms of remapping have been reported in the FEF and other brain areas, however, and questions remain about reasons for these differences. In this review of recent progress, we identify three hypotheses that may help to guide further investigations into the structure and function of circuits for remapping.

Circuits for Presaccadic Visual Remapping.

Saccadic eye movements rapidly displace the image of the world that is projected onto the retinas. In anticipation of each saccade, many neurons in the visual system shift their receptive fields. This presaccadic change in visual sensitivity, known as remapping, was first documented in the parietal cortex and has been studied in many other brain regions. Remapping requires information about upcoming saccades via corollary discharge. Analyses of neurons in a corollary discharge pathway that targets the frontal eye field (FEF) suggest that remapping may be assembled in the FEF's local microcircuitry. Complementary data from reversible inactivation, neural recording, and modeling studies provide evidence that remapping contributes to transsaccadic continuity of action and perception. Multiple forms of remapping have been reported in the FEF and other brain areas, however, and questions remain about reasons for these differences. In this review of recent progress, we identify three hypotheses that may help to guide further investigations into the structure and function of circuits for remapping.

3D considerations for motion perception and visuomotor transformations

Moving through a stable, three-dimensional world is a hallmark of our motor and perceptual experience. This stability is constantly being challenged by movements of the eyes and head, inducing retinal blur and retino-spatial misalignments for which the brain must compensate. To do so, the brain must account for eye and head kinematics to transform two-dimensional retinal input into the reference frame necessary for movement or perception. The four studies in this thesis used both computational and psychophysical approaches to investigate several aspects of this reference frame transformation. In the first study, we examined the neural mechanism underlying the visuomotor transformation for smooth pursuit using a feedforward neural network model. After training, the model performed the general, three-dimensional transformation using gain modulation. This gave mechanistic significance to gain modulation observed in cortical pursuit areas while also providing several testable hypotheses for future electrophysiological work. In the second study, we asked how anticipatory pursuit, which is driven by memorized signals, accounts for eye and head geometry using a novel head-roll updating paradigm. We showed that the velocity memory driving anticipatory smooth pursuit relies on retinal signals, but is updated for the current head orientation. In the third study, we asked how forcing retinal motion to undergo a reference frame transformation influences perceptual decision making. We found that simply rolling one's head impairs perceptual decision making in a way captured by stochastic reference frame transformations. In the final study, we asked how torsional shifts of the retinal projection occurring with almost every eye movement influence orientation perception across saccades. We found a pre-saccadic, predictive remapping consistent with maintaining a purely retinal (but spatially inaccurate) orientation perception throughout the movement. Together these studies suggest that, despite their spatial inaccuracy, retinal signals play a surprisingly large role in our seamless visual experience. This work therefore represents a significant advance in our understanding of how the brain performs one of its most fundamental functions.

The neural network of saccadic foreknowledge.

Foreknowledge about upcoming events may be exploited to optimize behavioural responses. In a previous work, using an eye movement paradigm, we showed that different types of partial foreknowledge have different effects on saccadic efficiency. In the current study, we investigated the neural circuitry involved in processing of partial foreknowledge using functional magnetic resonance imaging. Fourteen subjects performed a mixed antisaccade, prosaccade paradigm with blocks of no foreknowledge, complete foreknowledge or partial foreknowledge about stimulus location, response direction or task. We found that saccadic foreknowledge is processed primarily within the well-known oculomotor network for saccades and antisaccades. Moreover, we found a consistent decrease in BOLD activity in the primary and secondary visual cortex in all foreknowledge conditions compared to the no-foreknowledge conditions. Furthermore we found that the different types of partial foreknowledge are processed in distinct brain areas: response foreknowledge is processed in the frontal eye field, while stimulus foreknowledge is processed in the frontal and parietal eye field. Task foreknowledge, however, revealed no positive BOLD correlate. Our results show different patterns of engagement in the saccade-related neural network depending upon precisely what type of information is known ahead.

Which way is up? Grounded mental representations of space

Processing language is postulated to involve a mental simulation, or re-enactment of perceptual, motor, and introspective states that were acquired experientially (Barsalou, 1999, 2008). One such aspect that is mentally simulated during processing of certain concepts is spatial location. For example, upon processing the word “moon” the prominent spatial location of the concept (e.g. ‘upward’) is mentally simulated. In six eye-tracking experiments, we investigate how mental simulations of spatial location affect processing. We first address a conflict in previous literature whereby processing is shown to be impacted in both a facilitatory and inhibitory way. Two of our experiments showed that mental simulations of spatial association facilitate saccades launched toward compatible locations; however, a third experiment showed an inhibitory effect on saccades launched towards incompatible locations. We investigated these differences with further experiments, which led us to conclude that the nature of the effect (facilitatory or inhibitory) is dependent on the demands of the task and, in fitting with the theory of Grounded Cognition (Barsalou, 2008), that mental simulations impact processing in a dynamic way. Three further experiments explored the nature of verticality – specifically, whether ‘up’ is perceived as away from gravity, or above our head. Using similar eye-tracking methods, and by manipulating the position of participants, we were able to dissociate these two possible standpoints. The results showed that mental simulations of spatial location facilitated saccades to compatible locations, but only when verticality was dissociated from gravity (i.e. ‘up’ was above the participant’s head). We conclude that this is not due to an ‘embodied’ mental simulation, but rather a result of heavily ingrained visuo-motor association between vertical space and eye movements.

Comparing reading processes on e-ink displays and print

E-book reading devices open new possibilities in the field of reading. More activities than just reading a book can be performed with a single electronic device. For a long time, electronic reading devices have not been favored because their active LCD displays used to have a relatively low contrast. The new generation of electronic reading devices differs from earlier ones in the nature of the display: active LCD displays have been replaced with displays based on e-ink technology, which has display properties closer to that of printed paper. Moreover, e-ink technology has higher power efficiency, thereby increasing battery life and reducing weight. At first sight, the display looks similar to paper print, but the question remains whether the reading behavior also is equal to that of reading a printed book. In the present study, we analyzed and compared reading behavior on e-reader displays and on printed paper. The results suggest that the reading behavior on e-readers is indeed very similar to the reading behavior on print. Participants shared similar proportions of regressive saccades while reading on e-readers and print. Significant differences in fixation duration suggest that e-readers, in some situations, may even provide better legibility.

ROLE OF THE FRONTAL EYE FIELD, SUPERIOR COLLICULUS, AND BASAL GANGLIA IN FLEXIBLE SACCADE BEHAVIOR

A distributed network of cortical and subcortical brain regions mediates the control of voluntary behavior, but it is unclear how this complex system may flexibly shift between different behavioral events. This thesis describes the neurophysiological changes in several key nuclei across the brain during flexible behavior, using saccadic eye movements in rhesus macaque monkeys. We examined five nuclei critical for saccade initiation and modulation: the frontal eye field (FEF) in the cerebral cortex, the subthalamic nucleus (STN), caudate nucleus (CD), and substantia nigra pars reticulata (SNr) in the basal ganglia (BG), and the superior colliculus (SC) in the midbrain. The first study tested whether a ‘threshold’ theory of how neuronal activity cues saccade initiation is consistent with the flexible control of behavior. The theory suggests there is a fixed level of FEF and SC neuronal activation at which saccades are initiated. Our results provide strong evidence against a fixed saccade threshold in either structure during flexible behavior, and indicate that threshold variability might depend on the level of inhibitory signals applied to the FEF or SC. The next two studies investigated the BG network as a likely candidate to modulate a saccade initiation mechanism, based on strong inhibitory output signals from the BG to the FEF and SC. We investigated the STN and CD (BG input), and the SNr (BG oculomotor output) to examine changes across the BG network. This revealed robust task-contingent shifts in BG signaling (Chapter 3), which uniquely impacted saccade initiation according to behavioral condition (Chapters 3 and 4). The thesis concludes with a published short review of the mechanistic effects of BG deep brain stimulation (Chapter 5), and a general discussion including proof of concept saccade behavioral changes in an MPTP-induced Parkinsonian model (Chapter 6). The studies presented here demonstrate that the conditions for saccade initiation by the FEF and SC vary according to behavioral condition, while simultaneously, large-scale task dependent shifts occur in BG signaling consistent with the observed modulation of FEF and SC activity. Taken together, these describe a mechanistic framework by which the cortico-BG loop may contribute to the flexible control of behavior.