Visual search and eye movements: Studies of perceptual span

In visual search one tries to find the currently relevant item among other, irrelevant items. In the present study, visual search performance for complex objects (characters, faces, computer icons and words) was investigated, and the contribution of different stimulus properties, such as luminance contrast between characters and background, set size, stimulus size, colour contrast, spatial frequency, and stimulus layout were investigated. Subjects were required to search for a target object among distracter objects in two-dimensional stimulus arrays. The outcome measure was threshold search time, that is, the presentation duration of the stimulus array required by the subject to find the target with a certain probability. It reflects the time used for visual processing separated from the time used for decision making and manual reactions. The duration of stimulus presentation was controlled by an adaptive staircase method. The number and duration of eye fixations, saccade amplitude, and perceptual span, i.e., the number of items that can be processed during a single fixation, were measured. It was found that search performance was correlated with the number of fixations needed to find the target. Search time and the number of fixations increased with increasing stimulus set size. On the other hand, several complex objects could be processed during a single fixation, i.e., within the perceptual span. Search time and the number of fixations depended on object type as well as luminance contrast. The size of the perceptual span was smaller for more complex objects, and decreased with decreasing luminance contrast within object type, especially for very low contrasts. In addition, the size and shape of perceptual span explained the changes in search performance for different stimulus layouts in word search. Perceptual span was scale invariant for a 16-fold range of stimulus sizes, i.e., the number of items processed during a single fixation was independent of retinal stimulus size or viewing distance. It is suggested that saccadic visual search consists of both serial (eye movements) and parallel (processing within perceptual span) components, and that the size of the perceptual span may explain the effectiveness of saccadic search in different stimulus conditions. Further, low-level visual factors, such as the anatomical structure of the retina, peripheral stimulus visibility and resolution requirements for the identification of different object types are proposed to constrain the size of the perceptual span, and thus, limit visual search performance. Similar methods were used in a clinical study to characterise the visual search performance and eye movements of neurological patients with chronic solvent-induced encephalopathy (CSE). In addition, the data about the effects of different stimulus properties on visual search in normal subjects were presented as simple practical guidelines, so that the limits of human visual perception could be taken into account in the design of user interfaces.

Human brain networks of auditory attention and working memory

This thesis examines brain networks involved in auditory attention and auditory working memory using measures of task performance, brain activity, and neuroanatomical connectivity. Auditory orienting and maintenance of attention were compared with visual orienting and maintenance of attention, and top-down controlled attention was compared to bottom-up triggered attention in audition. Moreover, the effects of cognitive load on performance and brain activity were studied using an auditory working memory task. Corbetta and Shulman s (2002) model of visual attention suggests that what is known as the dorsal attention system (intraparietal sulcus/superior parietal lobule, IPS/SPL and frontal eye field, FEF) is involved in the control of top-down controlled attention, whereas what is known as the ventral attention system (temporo-parietal junction, TPJ and areas of the inferior/middle frontal gyrus, IFG/MFG) is involved in bottom-up triggered attention. The present results show that top-down controlled auditory attention also activates IPS/SPL and FEF. Furthermore, in audition, TPJ and IFG/MFG were activated not only by bottom-up triggered attention, but also by top-down controlled attention. In addition, the posterior cerebellum and thalamus were activated by top-down controlled attention shifts and the ventromedial prefrontal cortex (VMPFC) was activated by to-be-ignored, but attention-catching salient changes in auditory input streams. VMPFC may be involved in the evaluation of environmental events causing the bottom-up triggered engagement of attention. Auditory working memory activated a brain network that largely overlapped with the one activated by top-down controlled attention. The present results also provide further evidence of the role of the cerebellum in cognitive processing: During auditory working memory tasks, both activity in the posterior cerebellum (the crus I/II) and reaction speed increased when the cognitive load increased. Based on the present results and earlier theories on the role of the cerebellum in cognitive processing, the function of the posterior cerebellum in cognitive tasks may be related to the optimization of response speed.

Functional imaging of peripheral vision and dorsal stream function in the human cerebral cortex

Visual information processing in brain proceeds in both serial and parallel fashion throughout various functionally distinct hierarchically organised cortical areas. Feedforward signals from retina and hierarchically lower cortical levels are the major activators of visual neurons, but top-down and feedback signals from higher level cortical areas have a modulating effect on neural processing. My work concentrates on visual encoding in hierarchically low level cortical visual areas in human brain and examines neural processing especially in cortical representation of visual field periphery. I use magnetoencephalography and functional magnetic resonance imaging to measure neuromagnetic and hemodynamic responses during visual stimulation and oculomotor and cognitive tasks from healthy volunteers. My thesis comprises six publications. Visual cortex forms a great challenge for modeling of neuromagnetic sources. My work shows that a priori information of source locations are needed for modeling of neuromagnetic sources in visual cortex. In addition, my work examines other potential confounding factors in vision studies such as light scatter inside the eye which may result in erroneous responses in cortex outside the representation of stimulated region, and eye movements and attention. I mapped cortical representations of peripheral visual field and identified a putative human homologue of functional area V6 of the macaque in the posterior bank of parieto-occipital sulcus. My work shows that human V6 activates during eye-movements and that it responds to visual motion at short latencies. These findings suggest that human V6, like its monkey homologue, is related to fast processing of visual stimuli and visually guided movements. I demonstrate that peripheral vision is functionally related to eye-movements and connected to rapid stream of functional areas that process visual motion. In addition, my work shows two different forms of top-down modulation of neural processing in the hierachically lowest cortical levels; one that is related to dorsal stream activation and may reflect motor processing or resetting signals that prepare visual cortex for change in the environment and another local signal enhancement at the attended region that reflects local feed-back signal and may perceptionally increase the stimulus saliency.

Absolute quantification of human tear lactoferrin using multiple reaction monitoring technique with stable-isotopic labeling

The mass spectrometry technique of multiple reaction monitoring (MRM) was used to quantify and compare the expression level of lactoferrin in tear films among control, prostate cancer (CaP), and benign prostate hyperplasia (BPH) groups. Tear samples from 14 men with CaP, 15 men with BPH, and 14 controls were analyzed in the study. Collected tears (2 μl) of each sample were digested with trypsin overnight at 37 °C without any pretreatment, and tear lactoferrin was quantified using a lactoferrin-specific peptide, VPSHAVVAR, both using natural/light and isotopic-labeled/heavy peptides with MRM. The average tear lactoferrin concentration was 1.01 ± 0.07 μg/μl in control samples, 0.96 ± 0.07 μg/μl in the BPH group, and 0.98 ± 0.07 μg/μl in the CaP group. Our study is the first to quantify tear proteins using a total of 43 individual (non-pooled) tear samples and showed that direct digestion of tear samples is suitable for MRM studies. The calculated average lactoferrin concentration in the control group matched that in the published range of human tear lactoferrin concentration measured by enzyme-linked immunosorbent assay (ELISA). Moreover, the lactoferrin was stably expressed across all of the samples, with no significant differences being observed among the control, BPH, and CaP groups.

Interrelations between Dry Eye Syndrome and Tear Fluid Phospholipid Transfer Protein

The simplified model of human tear fluid (TF) is a three-layered structure composed of a homogenous gel-like layer of hydrated mucins, an aqueous phase, and a lipid-rich outermost layer found in the tear-air interface. It is assumed that amphiphilic phospholipids are found adjacent to the aqueous-mucin layer and externally to this a layer composed of non-polar lipids face the tear-air interface. The lipid layer prevents evaporation of the TF and protects the eye, but excess accumulation of lipids may lead to drying of the corneal epithelium. Thus the lipid layer must be controlled and maintained by some molecular mechanisms. In the circulation, phospholipid transfer protein (PLTP) and cholesteryl ester transfer protein (CETP) mediate lipid transfers. The aim of this thesis was to investigate the presence and molecular mechanisms of lipid transfer proteins in human TF. The purpose was also to study the role of these proteins in the development of dry eye syndrome (DES). The presence of TF PLTP and CETP was studied by western blotting and mass spectrometry. The concentration of these proteins was determined by ELISA. The activities of the enzymes were determined by specific lipid transfer assays. To study the molecular mechanisms involved in PLTP mediated lipid transfer Langmuir monolayers and asymmetrical flow field-flow fractionation (AsFlFFF) was used. Ocular tissue samples were stained with monoclonal antibodies against PLTP to study the secretion route of PLTP. Heparin-Sepharose affinity chromatography was used for PLTP pull-down experiments and co-eluted proteins were identified with MALDI-TOF mass spectrometry or Western blot analysis. To study whether PLTP plays any functional role in TF PLTP-deficient mice were examined. The activity of PLTP was also studied in dry eye patients. PLTP is a component of normal human TF, whereas CETP is not. TF PLTP concentration was about 2-fold higher than that in human plasma. Inactivation of PLTP by heat treatment or immunoinhibition abolished the phospholipid transfer activity in tear fluid. PLTP was found to be secreted from lacrimal glands. PLTP seems to be surface active and is capable of accepting lipid molecules without the presence of lipid-protein complexes. The active movement of radioactively labeled lipids and high activity form of PLTP to acceptor particles suggested a shuttle model of PLTP-mediated lipid transfer. In this model, PLTP physically transports lipids between the donor and acceptor. Protein-protein interaction assays revealed ocular mucins as PLTP interaction partners in TF. In mice with a full deficiency of functional PLTP enhanced corneal epithelial damage, increased corneal permeability to carboxyfluorescein, and decreased corneal epithelial occludin expression was demonstrated. Increased tear fluid PLTP activity was observed among human DES patients. These results together suggest a scavenger property of TF PLTP: if the corneal epithelium is contaminated by hydrophobic material, PLTP could remove them and transport them to the superficial layer of the TF or, alternatively, transport them through the naso-lacrimal duct. Thus, PLTP might play an integral role in tear lipid trafficking and in the protection of the corneal epithelium. The increased PLTP activity in human DES patients suggests an ocular surface protective role for this lipid transfer protein.

Online Estimation of Evolving Human Visual Interest

Regions in video streams attracting human interest contribute significantly to human understanding of the video. Being able to predict salient and informative Regions of Interest (ROIs) through a sequence of eye movements is a challenging problem. Applications such as content-aware retargeting of videos to different aspect ratios while preserving informative regions and smart insertion of dialog (closed-caption text) into the video stream can significantly be improved using the predicted ROIs. We propose an interactive human-in-the-loop framework to model eye movements and predict visual saliency into yet-unseen frames. Eye tracking and video content are used to model visual attention in a manner that accounts for important eye-gaze characteristics such as temporal discontinuities due to sudden eye movements, noise, and behavioral artifacts. A novel statistical-and algorithm-based method gaze buffering is proposed for eye-gaze analysis and its fusion with content-based features. Our robust saliency prediction is instantiated for two challenging and exciting applications. The first application alters video aspect ratios on-the-fly using content-aware video retargeting, thus making them suitable for a variety of display sizes. The second application dynamically localizes active speakers and places dialog captions on-the-fly in the video stream. Our method ensures that dialogs are faithful to active speaker locations and do not interfere with salient content in the video stream. Our framework naturally accommodates personalisation of the application to suit biases and preferences of individual users.

Visual-Context Boosting for Eye Detection

Eye detection plays an important role in many practical applications. This paper presents a novel two-step scheme for eye detection. The first step models an eye by a newly defined visual-context pattern (VCP), and the second step applies semisupervised boosting for precise detection. VCP describes both the space and appearance relations between an eye region (region of eye) and a reference region (region of reference). The context feature of a VCP is extracted by using the integral image. Aiming to reduce the human labeling efforts, we apply semisupervised boosting, which integrates the context feature and the Haar-like features for precise eye detection. Experimental results on several standard face data sets demonstrate that the proposed approach is effective, robust, and efficient. We finally show that this approach is ready for practical applications.

Illness and elevated Human Mortality in Europe Coincident with the Laki Fissure eruption

Grattan, J. Durand, M. Taylor, S., Illness and elevated Human Mortality in Europe Coincident with the Laki fissure eruption. In: 'Volcanic Degassing: Geological Society, Special Publication 213', Oppenheimer, C., Pyle, D.M. and Barclay, J. (eds). Geological Society, London, Special Publications, 410-414, 2003.

Real Time Eye Tracking and Blink Detection with USB Cameras

A human-computer interface (HCI) system designed for use by people with severe disabilities is presented. People that are severely paralyzed or afflicted with diseases such as ALS (Lou Gehrig's disease) or multiple sclerosis are unable to move or control any parts of their bodies except for their eyes. The system presented here detects the user's eye blinks and analyzes the pattern and duration of the blinks, using them to provide input to the computer in the form of a mouse click. After the automatic initialization of the system occurs from the processing of the user's involuntary eye blinks in the first few seconds of use, the eye is tracked in real time using correlation with an online template. If the user's depth changes significantly or rapid head movement occurs, the system is automatically reinitialized. There are no lighting requirements nor offline templates needed for the proper functioning of the system. The system works with inexpensive USB cameras and runs at a frame rate of 30 frames per second. Extensive experiments were conducted to determine both the system's accuracy in classifying voluntary and involuntary blinks, as well as the system's fitness in varying environment conditions, such as alternative camera placements and different lighting conditions. These experiments on eight test subjects yielded an overall detection accuracy of 95.3%.

A Biomechanical Model of Human Oculomotor Plant Kinematics Based Upon Geometric Algebra

A biomechanical model of the human oculomotor plant kinematics in 3-D as a function of muscle length changes is presented. It can represent a range of alternative interpretations of the data as a function of one parameter. The model is free from such deficits as singularities and the nesting of axes found in alternative formulations such as the spherical wrist (Paul, l98l). The equations of motion are defined on a quaternion based representation of eye rotations and are compact atnd computationally efficient.

High-field FMRI reveals brain activation patterns underlying saccade execution in the human superior colliculus.

BACKGROUND: The superior colliculus (SC) has been shown to play a crucial role in the initiation and coordination of eye- and head-movements. The knowledge about the function of this structure is mainly based on single-unit recordings in animals with relatively few neuroimaging studies investigating eye-movement related brain activity in humans. METHODOLOGY/PRINCIPAL FINDINGS: The present study employed high-field (7 Tesla) functional magnetic resonance imaging (fMRI) to investigate SC responses during endogenously cued saccades in humans. In response to centrally presented instructional cues, subjects either performed saccades away from (centrifugal) or towards (centripetal) the center of straight gaze or maintained fixation at the center position. Compared to central fixation, the execution of saccades elicited hemodynamic activity within a network of cortical and subcortical areas that included the SC, lateral geniculate nucleus (LGN), occipital cortex, striatum, and the pulvinar. CONCLUSIONS/SIGNIFICANCE: Activity in the SC was enhanced contralateral to the direction of the saccade (i.e., greater activity in the right as compared to left SC during leftward saccades and vice versa) during both centrifugal and centripetal saccades, thereby demonstrating that the contralateral predominance for saccade execution that has been shown to exist in animals is also present in the human SC. In addition, centrifugal saccades elicited greater activity in the SC than did centripetal saccades, while also being accompanied by an enhanced deactivation within the prefrontal default-mode network. This pattern of brain activity might reflect the reduced processing effort required to move the eyes toward as compared to away from the center of straight gaze, a position that might serve as a spatial baseline in which the retinotopic and craniotopic reference frames are aligned.

Frontal eye field neurons assess visual stability across saccades.

The image on the retina may move because the eyes move, or because something in the visual scene moves. The brain is not fooled by this ambiguity. Even as we make saccades, we are able to detect whether visual objects remain stable or move. Here we test whether this ability to assess visual stability across saccades is present at the single-neuron level in the frontal eye field (FEF), an area that receives both visual input and information about imminent saccades. Our hypothesis was that neurons in the FEF report whether a visual stimulus remains stable or moves as a saccade is made. Monkeys made saccades in the presence of a visual stimulus outside of the receptive field. In some trials, the stimulus remained stable, but in other trials, it moved during the saccade. In every trial, the stimulus occupied the center of the receptive field after the saccade, thus evoking a reafferent visual response. We found that many FEF neurons signaled, in the strength and timing of their reafferent response, whether the stimulus had remained stable or moved. Reafferent responses were tuned for the amount of stimulus translation, and, in accordance with human psychophysics, tuning was better (more prevalent, stronger, and quicker) for stimuli that moved perpendicular, rather than parallel, to the saccade. Tuning was sometimes present as well for nonspatial transaccadic changes (in color, size, or both). Our results indicate that FEF neurons evaluate visual stability during saccades and may be general purpose detectors of transaccadic visual change.

Cochlin in the eye: Functional implications

Aqueous humor is actively produced in the ciliary epithelium of the anterior chamber and has important functions for the eye. Under normal physiological conditions, the inflow and outflow of the aqueous humor are tightly regulated, but in the pathologic state this balance is lost. Aqueous outflow involves structures of the anterior chamber and experiences most resistance at the level of the trabecular meshwork (TM) that acts as a filter. The modulation of the TM structure regulates the filter and its mechanism remains poorly understood. Proteomic analyses have identified cochlin, a protein of poorly understood function, in the glaucomatous TM but not in healthy control TM from human cadaver eyes. The presence of cochlin has subsequently been confirmed by Western and immunohistochemical analyses. Functionally, cochlin undergoes multimerization induced by shear stress and other changes in the microenvironment. Cochlin along with mucopolysaccharide deposits have been found in the TM of glaucoma patients and in the inner ear of subjects affected by the hearing disorder DNFA9, a late onset, progressive disease that also involves alterations in fluid shear regimes. In vitro, cochlin induces aggregation of primary TM cells suggesting a role in cell adhesion, possibly in mechanosensation, and in modulation of the TM filter.

Identification and Functional Analysis of a Novel Tryptophyllin Peptide from the Skin of the Red-eye Leaf Frog, Agalychnis callidryas

Amphibian skin has proved repeatedly to be a largely untapped source of bioactive peptides and this is especially true of members of the Phyllomedusinae subfamily of frogs native to South and Central America. Tryptophyllins are a group of peptides mainly found in the skin of members of this genus. In this study, a novel tryptophyllin (TPH) type 3 peptide, named AcT-3, has been isolated and structurally-characterised from the skin secretion and lyophilised skin extract of the red-eye leaf frog, Agalychnis callidryas. The peptide was identified in and purified from the skin secretion by reverse-phase HPLC. MALDI-TOF mass spectrometry and MS/MS fragmentation sequencing established its primary structure as: pGlu-Gly-Lys-Pro-Tyr-Trp-Pro-Pro-Pro-Phe-Leu-Pro-Glu, with a non-protonated molecular mass of 1538.19Da. The mature peptide possessed the canonical N-terminal pGlu residue that arises from post-translational modification of a Gln residue. The deduced open-reading frame consisted of 63 amino acid residues encoding a highly-conserved signal peptide of approximately 22 amino acid residues, an intervening acidic spacer peptide domain, a single AcT-3 encoding domain and a C terminal processing site. A synthetic replicate of AcT-3 was found to antagonise the effect of BK on rat tail artery smooth muscle and to contract the intestinal smooth muscle preparations. It was also found that AcT-3 could dose-dependently inhibit the proliferation of human prostate cancer cell lines after 72h incubation.

Eye tracking data representation and visualization : on information and communication studies at CETAC.MEDIA

Eye tracking has become a preponderant technique in the evaluation of user interaction and behaviour with study objects in defined contexts. Common eye tracking related data representation techniques offer valuable input regarding user interaction and eye gaze behaviour, namely through fixations and saccades measurement. However, these and other techniques may be insufficient for the representation of acquired data in specific studies, namely because of the complexity of the study object being analysed. This paper intends to contribute with a summary of data representation and information visualization techniques used in data analysis within different contexts (advertising, websites, television news and video games). Additionally, several methodological approaches are presented in this paper, which resulted from several studies developed and under development at CETAC.MEDIA - Communication Sciences and Technologies Research Centre. In the studies described, traditional data representation techniques were insufficient. As a result, new approaches were necessary and therefore, new forms of representing data, based on common techniques were developed with the objective of improving communication and information strategies. In each of these studies, a brief summary of the contribution to their respective area will be presented, as well as the data representation techniques used and some of the acquired results.