The Importance of Semantics: Visual World Studies on Drawing Inferences and Resolving Anaphors

The present thesis investigated the importance of semantics in generating inferences during discourse processing. Three aspects of semantics, gender stereotypes, implicit causality information and proto-role properties, were used to investigate whether semantics is activated elaboratively during discourse comprehension and what its relative importance is in backward inferencing compared to discourse/structural cues. Visual world eye-tracking studies revealed that semantics plays an important role in both backward and forward inferencing: Gender stereotypes and implicit causality information is activated elaboratively during online discourse comprehension. Moreover, gender stereotypes, implicit causality and proto-role properties of verbs are all used in backward inferencing. Importantly, the studies demonstrated that semantic cues are weighed against discourse/structural cues. When the structural cues consist of a combination of cues that have been independently shown to be important in backward inferencing, semantic effects may be masked, whereas when the structural cues consist of a combination of fewer prominent cues, semantics can have an earlier effect than structural factors in pronoun resolution. In addition, the type of inference matters, too: During anaphoric inferencing semantics has a prominent role, while discourse/structural salience attains more prominence during non-anaphoric inferencing. Finally, semantics exhibits a strong role in inviting new inferences to revise earlier made inferences even in the case the additional inference is not needed to establish coherence in discourse. The findings are generally in line with the Mental Model approaches. Two extended model versions are presented that incorporate the current findings into the earlier literature. These models allow both forward and backward inferencing to occur at any given moment during the course of processing; they also allow semantic and discourse/structural cues to contribute to both of these processes. However, while the Mental Model 1 does not assume interactions between semantic and discourse/structural factors in forward inferencing, the Mental Model 2 does assume such a link.

Young toddlers' word comprehension is flexible and efficient.

Much of what is known about word recognition in toddlers comes from eyetracking studies. Here we show that the speed and facility with which children recognize words, as revealed in such studies, cannot be attributed to a task-specific, closed-set strategy; rather, children's gaze to referents of spoken nouns reflects successful search of the lexicon. Toddlers' spoken word comprehension was examined in the context of pictures that had two possible names (such as a cup of juice which could be called "cup" or "juice") and pictures that had only one likely name for toddlers (such as "apple"), using a visual world eye-tracking task and a picture-labeling task (n = 77, mean age, 21 months). Toddlers were just as fast and accurate in fixating named pictures with two likely names as pictures with one. If toddlers do name pictures to themselves, the name provides no apparent benefit in word recognition, because there is no cost to understanding an alternative lexical construal of the picture. In toddlers, as in adults, spoken words rapidly evoke their referents.

Feasibility of using eye tracking to increase resolution for visual telepresence

Visual telepresence seeks to extend existing teleoperative capability by supplying the operator with a 3D interactive view of the remote environment. This is achieved through the use of a stereo camera platform which, through appropriate 3D display devices, provides a distinct image to each eye of the operator, and which is slaved directly from the operator's head and eye movements. However, the resolution within current head mounted displays remains poor, thereby reducing the operator's visual acuity. This paper reports on the feasibility of incorporation of eye tracking to increase resolution and investigates the stability and control issues for such a system. Continuous domain and discrete simulations are presented which indicates that eye tracking provides a stable feedback loop for tracking applications, though some empirical testing (currently being initiated) of such a system will be required to overcome indicated stability problems associated with micro saccades of the human operator.

The link between visual exploration and neuronal activity: A multi-modal study combining eye tracking, functional magnetic resonance imaging and transcranial magnetic stimulation

In the present multi-modal study we aimed to investigate the role of visual exploration in relation to the neuronal activity and performance during visuospatial processing. To this end, event related functional magnetic resonance imaging er-fMRI was combined with simultaneous eye tracking recording and transcranial magnetic stimulation (TMS). Two groups of twenty healthy subjects each performed an angle discrimination task with different levels of difficulty during er-fMRI. The number of fixations as a measure of visual exploration effort was chosen to predict blood oxygen level-dependent (BOLD) signal changes using the general linear model (GLM). Without TMS, a positive linear relationship between the visual exploration effort and the BOLD signal was found in a bilateral fronto-parietal cortical network, indicating that these regions reflect the increased number of fixations and the higher brain activity due to higher task demands. Furthermore, the relationship found between the number of fixations and the performance demonstrates the relevance of visual exploration for visuospatial task solving. In the TMS group, offline theta bursts TMS (TBS) was applied over the right posterior parietal cortex (PPC) before the fMRI experiment started. Compared to controls, TBS led to a reduced correlation between visual exploration and BOLD signal change in regions of the fronto-parietal network of the right hemisphere, indicating a disruption of the network. In contrast, an increased correlation was found in regions of the left hemisphere, suggesting an intent to compensate functionality of the disturbed areas. TBS led to fewer fixations and faster response time while keeping accuracy at the same level, indicating that subjects explored more than actually needed.

Investigating tiredness and ad complexity effects on visual processing of advertisements:an eye-tracking study

Although advertising is pervasive in our daily, it proves to be not necessarily efficient all the times due to bad conditions or bad contexts of reception. Indeed, the communication process might be jeopardized at its very last stage because of advertising exposure quality. However critical it may be, ad exposure quality is not very much examined by researchers or practitioners. In this paper, we investigate how tiredness combined with ad complexity might influence the way consumers extract and process ad elements. Investigating tiredness is useful because it is a common daily state experienced by everyone at various moments of the day. And although it might drastically alter ad reception, it has not been studied in advertising for the moment. In this regards, we observe eye movement patterns of consumers viewing simple or complex advertisements being tired or not. We surprisingly find that tired subjects viewing complex ads don’t adopt a lessening effort visual strategy. They rather use a resource demanding one. We assume that the Sustained Attention strategy occurring is a kind of adaptive strategy allowing to deal with an anticipated lack of resource.

Pinterest: um estudo de eye tracking

Dissertação de mestrado em Média Interativos

Cultural differences on seeking information: an eye tracking study

The main goal of this research is to investigate how people with different cultural background differ in their interaction style and visual behavior on search engine results pages (SERP), more specifically between groups from the Middle Eastern region vs. Western Europe. We conducted a controlled eye-tracking experiment to explore and evaluate the visual behavior of Arabs and Spaniardusers when scanning through the first page of the search results in Google. Big differences can be observed in the 4 aspects studied: U.A.E. participants stayed on the SERPs for longer, they read more results and they read each snippet in a more complete way than Spaniards. In Spain, people tended to scan the SERP, reading less text on each snippet, and choose a result among the first top rankedones without hardly seeing those in bottom positions.

A Tomada de Decisão (Bio)ética: Estudo Preliminar Utilizando o Mobile Eye Tracking

RESUMO A tomada de decisão é uma das dimensões essenciais da formação do profissional da saúde, como mencionado nas atuais Diretrizes Curriculares Nacionais do Curso de Graduação em Medicina. O processo decisório, no âmbito da saúde, envolve diferentes aspectos, incluindo os elementos (bio)éticos. Nesse sentido, pesquisas que investiguem a tomada de decisão em (bio)ética poderão elucidar passos ainda não completamente esclarecidos, permitindo uma construção mais efetiva das competências em (bio)ética, na graduação e na pós-graduação. Diante desta perspectiva, o objetivo deste estudo é explorar possibilidades de uso do Mobile Eye Tracking para o estudo do papel da atenção visual – durante a exibição de filmes de cinema – no processo decisório em (bio)ética.

Eye tracking and gaze based interaction within immersive virtual environments

Our eyes are input sensors which Provide our brains with streams of visual data. They have evolved to be extremely efficient, and they will constantly dart to-and-fro to rapidly build up a picture of the salient entities in a viewed scene. These actions are almost subconscious. However, they can provide telling signs of how the brain is decoding the visuals and call indicate emotional responses, prior to the viewer becoming aware of them. In this paper we discuss a method of tracking a user's eye movements, and Use these to calculate their gaze within an immersive virtual environment. We investigate how these gaze patterns can be captured and used to identify viewed virtual objects, and discuss how this can be used as a, natural method of interacting with the Virtual Environment. We describe a flexible tool that has been developed to achieve this, and detail initial validating applications that prove the concept.

Stars in their eyes: what eye-tracking reveal about multimedia perceptual quality

Perceptual multimedia quality is of paramount importance to the continued take-up and proliferation of multimedia applications: users will not use and pay for applications if they are perceived to be of low quality. Whilst traditionally distributed multimedia quality has been characterised by Quality of Service (QoS) parameters, these neglect the user perspective of the issue of quality. In order to redress this shortcoming, we characterise the user multimedia perspective using the Quality of Perception (QoP) metric, which encompasses not only a user’s satisfaction with the quality of a multimedia presentation, but also his/her ability to analyse, synthesise and assimilate informational content of multimedia. In recognition of the fact that monitoring eye movements offers insights into visual perception, as well as the associated attention mechanisms and cognitive processes, this paper reports on the results of a study investigating the impact of differing multimedia presentation frame rates on user QoP and eye path data. Our results show that provision of higher frame rates, usually assumed to provide better multimedia presentation quality, do not significantly impact upon the median coordinate value of eye path data. Moreover, higher frame rates do not significantly increase level of participant information assimilation, although they do significantly improve overall user enjoyment and quality perception of the multimedia content being shown.

Seeing Spaces: An Eye-Tracking Study Of Speech Segmentation

Speech is often a multimodal process, presented audiovisually through a talking face. One area of speech perception influenced by visual speech is speech segmentation, or the process of breaking a stream of speech into individual words. Mitchel and Weiss (2013) demonstrated that a talking face contains specific cues to word boundaries and that subjects can correctly segment a speech stream when given a silent video of a speaker. The current study expanded upon these results, using an eye tracker to identify highly attended facial features of the audiovisual display used in Mitchel and Weiss (2013). In Experiment 1, subjects were found to spend the most time watching the eyes and mouth, with a trend suggesting that the mouth was viewed more than the eyes. Although subjects displayed significant learning of word boundaries, performance was not correlated with gaze duration on any individual feature, nor was performance correlated with a behavioral measure of autistic-like traits. However, trends suggested that as autistic-like traits increased, gaze duration of the mouth increased and gaze duration of the eyes decreased, similar to significant trends seen in autistic populations (Boratston & Blakemore, 2007). In Experiment 2, the same video was modified so that a black bar covered the eyes or mouth. Both videos elicited learning of word boundaries that was equivalent to that seen in the first experiment. Again, no correlations were found between segmentation performance and SRS scores in either condition. These results, taken with those in Experiment, suggest that neither the eyes nor mouth are critical to speech segmentation and that perhaps more global head movements indicate word boundaries (see Graf, Cosatto, Strom, & Huang, 2002). Future work will elucidate the contribution of individual features relative to global head movements, as well as extend these results to additional types of speech tasks.

Integración de tecnologías eye-tracking con vídeos interactivos para neurorrehabilitación cognitiva

La integración de las nuevas tecnologías en el proceso de rehabilitación permite la generación de terapias personalizadas, ubicuas y basadas en la evidencia. Tecnologías como el vídeo interactivo son propicias para el desarrollo de entornos virtuales en los que el paciente se ve inmerso dentro de actividades de la vida diaria en los que tiene que lograr un objetivo ecológico en un contexto seguro, controlado y adaptado a su perfil disfuncional. Dentro de este marco de rehabilitación la interacción visual paciente-entorno virtual se entiende como el mecanismo de comunicación principal, siendo además la atención visual un reflejo del estado cognitivo del paciente. El trabajo presentado en este artículo permite la integración de un sistema de eye-tracking con un entorno de neurorrehabilitación basado en vídeo interactivo. El objetivo último del sistema es la monitorización en tiempo real de la atención visual del usuario durante el proceso de neurorrehabilitación. Esta monitorización permite no sólo reproducir la ejecución de la actividad junto con el foco de la mirada, sino también detectar faltas de atención por parte del usuario, que permiten al vídeo interactivo reaccionar y adaptar la presentación de estímulos para ayudar a centrar su atención y así completar el objetivo de la actividad.

Evaluación de un entorno de rehabilitación cognitiva basado en tecnologías de Vídeo Interactivo y Eye-Tracking

El Daño Cerebral Adquirido (DCA) se ha convertido en una de las principales causas de discapacidad neurológica de las sociedades desarrolladas. La alteración de las funciones cognitivas como consecuencia del DCA, limita no sólo la calidad de vida del paciente sino también la de las persona de su entorno. Aunque la neurorrehabilitación permite recuperar algunas de las funciones alteradas aprovechando la naturaleza plástica del sistema nervioso, su práctica siguiendo procesos tradicionales no permiten en muchos casos ajustarse a las necesidades de cada individuo ni, en general, cubrir todos los aspectos necesarios que conviertan al proceso rehabilitador en un tratamiento realmente efectivo. La incorporación al proceso de rehabilitación de las nuevas tecnologías ha permitido aumentar la intensidad del tratamiento, personalizando y prolongándolo en el tiempo de forma sostenible. Los entornos virtuales (EV) apoyados en esta tendencia permiten reproducir Actividades de Vida Diaria (AVD) controladas que incrementan el valor ecológico de las terapias. Este Trabajo Fin de Grado aborda el uso pionero de la tecnología de Vídeo Interactivo (VI) para el desarrollo de dichos entornos en el campo de la rehabilitación cognitiva. En concreto, el objetivo del TFG es la evaluación de un EV de rehabilitación desarrollado mediante tecnología de VI e integrado con un sistema de Eye-Tracking, capaz de capturar y analizar la información referente al comportamiento visual del paciente. Para este fin, se realiza el diseño, implementación y evaluación de un estudio experimental que registre el comportamiento de diferentes sujetos ante dos modalidades de AVD.