Reading hybrid texts : remarks on text/image transitions

The reading of printed materials implies the visual processing of information originated in two distinct semiotic systems. The rapid identification of redundancy, complementation or contradiction rhetoric strategies between the two information types may be crucial for an adequate interpretation of bimodal materials. Hybrid texts (verbal and visual) are particular instances of bimodal materials, where the redundant information is often neglected while the complementary and the contradictory ones are essential.Studies using the 504 ASL eye-tracking system while reading either additive or exhibiting captions (Baptista, 2009) revealed fixations on the verbal material and transitions between the written and the pictorial in a much higher number and duration than the initially foreseen as necessary to read the verbal text. We therefore hypothesized that confirmation strategies of the written information are taking place, by using information available in the other semiotic system.Such eye-gaze patterns obtained from denotative texts and pictures seem to contradict some of the scarce existing data on visual processing of texts and images, namely cartoons (Carroll, Young and Guertain, 1992), descriptive captions (Hegarty, 1992 a and b), and advertising images with descriptive and explanatory texts (cf. Rayner and Rotello, 2001, who refer to a previous reading of the whole text before looking at the image, or even Rayner, Miller and Rotello, 2008 who refer to an earlier and longer look at the picture) and seem to consolidate findings of Radach et al. (2003) on systematic transitions between text and image.By framing interest areas in the printed pictorial material of non redundant hybrid texts, we have identified the specific areas where transitions take place after fixations in the verbal text. The way those transitions are processed brings a new interest to further research.

Find the difference! : eye tracking study on information seeking behavior using an online game

With the advent of digital era web applications have become inevitable part of our lives. We are using the web to manage even the financially or ethically sensitive issues. For this reason exploration of information seeking behavior is an exciting area of research. Current study provides insight on information seeking behavior using a classic ‘Find the Difference’ game. 50 university students between the age of 19 and 26 participated in the study. Eye movement data were recorded with a Tobii T120 device. Participants carried out 4 continuous tasks. Each task included two pictures side by side with 7 hidden differences. After finishing the tasks, participants were asked to repeat the game with the same picture set. This data collection methodology allows the evaluation of learning curves. Additionally, participants were asked about their hand preference. For the purpose of analysis the following metrics were applied: task times (including saccades), fixation count and fixation duration (without saccades). The right- and left-hand side on each picture was selected as AOI (Area of Interest) to detect side preference in connection with hand preference. Results suggest a significant difference between male and female participants regarding aggregated task times (male 58.37s respectively female 68.37s), deviation in the number of fixations and fixation duration (apparently female have less but longer fixations) and also in the distribution of fixations between AOIs. Using eyetracking data current paper highlights the similarities and differences in information acquisition strategies respectively reveals gender and education (Arts vs. Sciences) dependent characteristics of interaction.