Reading hybrid texts : remarks on text/image transitions

The reading of printed materials implies the visual processing of information originated in two distinct semiotic systems. The rapid identification of redundancy, complementation or contradiction rhetoric strategies between the two information types may be crucial for an adequate interpretation of bimodal materials. Hybrid texts (verbal and visual) are particular instances of bimodal materials, where the redundant information is often neglected while the complementary and the contradictory ones are essential.Studies using the 504 ASL eye-tracking system while reading either additive or exhibiting captions (Baptista, 2009) revealed fixations on the verbal material and transitions between the written and the pictorial in a much higher number and duration than the initially foreseen as necessary to read the verbal text. We therefore hypothesized that confirmation strategies of the written information are taking place, by using information available in the other semiotic system.Such eye-gaze patterns obtained from denotative texts and pictures seem to contradict some of the scarce existing data on visual processing of texts and images, namely cartoons (Carroll, Young and Guertain, 1992), descriptive captions (Hegarty, 1992 a and b), and advertising images with descriptive and explanatory texts (cf. Rayner and Rotello, 2001, who refer to a previous reading of the whole text before looking at the image, or even Rayner, Miller and Rotello, 2008 who refer to an earlier and longer look at the picture) and seem to consolidate findings of Radach et al. (2003) on systematic transitions between text and image.By framing interest areas in the printed pictorial material of non redundant hybrid texts, we have identified the specific areas where transitions take place after fixations in the verbal text. The way those transitions are processed brings a new interest to further research.

Find the difference! : eye tracking study on information seeking behavior using an online game

With the advent of digital era web applications have become inevitable part of our lives. We are using the web to manage even the financially or ethically sensitive issues. For this reason exploration of information seeking behavior is an exciting area of research. Current study provides insight on information seeking behavior using a classic ‘Find the Difference’ game. 50 university students between the age of 19 and 26 participated in the study. Eye movement data were recorded with a Tobii T120 device. Participants carried out 4 continuous tasks. Each task included two pictures side by side with 7 hidden differences. After finishing the tasks, participants were asked to repeat the game with the same picture set. This data collection methodology allows the evaluation of learning curves. Additionally, participants were asked about their hand preference. For the purpose of analysis the following metrics were applied: task times (including saccades), fixation count and fixation duration (without saccades). The right- and left-hand side on each picture was selected as AOI (Area of Interest) to detect side preference in connection with hand preference. Results suggest a significant difference between male and female participants regarding aggregated task times (male 58.37s respectively female 68.37s), deviation in the number of fixations and fixation duration (apparently female have less but longer fixations) and also in the distribution of fixations between AOIs. Using eyetracking data current paper highlights the similarities and differences in information acquisition strategies respectively reveals gender and education (Arts vs. Sciences) dependent characteristics of interaction.

Emotion recognition in multiple sclerosis

The effect of multiple sclerosis (MS) on the ability to identify emotional expressions in faces was investigated, and possible associations with patients’ characteristics were explored. 56 non-demented MS patients and 56 healthy subjects (HS) with similar demographic characteristics performed an emotion recognition task (ERT), the Benton Facial Recognition Test (BFRT), and answered the Hospital Anxiety and Depression Scale (HADS). Additionally, MS patients underwent a neurological examination and a comprehensive neuropsychological evaluation. The ERT consisted of 42 pictures of faces (depicting anger, disgust, fear, happiness, sadness, surprise and neutral expressions) from the NimStim set. An iViewX high-speed eye tracker was used to record eye movements during ERT. The fixation times were calculated for two regions of interest (i.e., eyes and rest of the face). No significant differences were found between MS and HC on ERT’s behavioral and oculomotor measures. Bivariate and multiple regression analyses revealed significant associations between ERT’s behavioral performance and demographic, clinical, psychopathological, and cognitive measures.

Mosteiro de S. Salvador de Paço de Sousa Contributo para a sua reabilitação arquitetónica

Este trabalho pretende contribuir para a reabilitação do Mosteiro de S. Salvador de Paço de Sousa assente na interpretação espacial do complexo monástico desde a fundação à extinção da comunidade religiosa, com particular nota para a intervenção efetuada pela DGEMN - Direção Geral dos Edifícios e Monumentos Nacionais, no segundo quartel do século XX. Procura trazer pela primeira vez à luz uma representação dos seus espaços, usos e funções ao longo do tempo com recurso a desenhos, esquemas e imagens para permitir a apreensão do conjunto em determinadas fases da história e por consequência fundamentar uma eventual intervenção. O estudo fundamenta-se na recolhe de dados escritos que permitam aferir a vivência dos espaços, sua simbologia e utilidade, nomeadamente através de referências históricas e de carater simbólico e documentação gráfica que permita avaliar e comparar o assunto de estudo com outros exemplos e situações análogas. O cruzamento de dados analisados resulta na sua simbolização gráfica, de forma a veicular o entendimento espacial e simbólico, que remete o real conhecimento da arquitetura do Mosteiro de S. Salvador de Paço de Sousa e, por comparação, viabiliza um ponto de partida para futuras intervenções, seja deste mosteiro, seja de edifícios de características semelhantes.