Reading hybrid texts : remarks on text/image transitions

The reading of printed materials implies the visual processing of information originated in two distinct semiotic systems. The rapid identification of redundancy, complementation or contradiction rhetoric strategies between the two information types may be crucial for an adequate interpretation of bimodal materials. Hybrid texts (verbal and visual) are particular instances of bimodal materials, where the redundant information is often neglected while the complementary and the contradictory ones are essential.Studies using the 504 ASL eye-tracking system while reading either additive or exhibiting captions (Baptista, 2009) revealed fixations on the verbal material and transitions between the written and the pictorial in a much higher number and duration than the initially foreseen as necessary to read the verbal text. We therefore hypothesized that confirmation strategies of the written information are taking place, by using information available in the other semiotic system.Such eye-gaze patterns obtained from denotative texts and pictures seem to contradict some of the scarce existing data on visual processing of texts and images, namely cartoons (Carroll, Young and Guertain, 1992), descriptive captions (Hegarty, 1992 a and b), and advertising images with descriptive and explanatory texts (cf. Rayner and Rotello, 2001, who refer to a previous reading of the whole text before looking at the image, or even Rayner, Miller and Rotello, 2008 who refer to an earlier and longer look at the picture) and seem to consolidate findings of Radach et al. (2003) on systematic transitions between text and image.By framing interest areas in the printed pictorial material of non redundant hybrid texts, we have identified the specific areas where transitions take place after fixations in the verbal text. The way those transitions are processed brings a new interest to further research.

Universal Museum - universal language? Art historian - curator

Marshal McLuhan’s prophetic vision of the global village is about to be realized. If we are aware of the fact that mass communication reduces the dimensions of our world and makes it more unified and universal, we should take this into consideration when planning the Universal Museum and the language that should be used in it. As curators, educators and museum staff we should not ignore the fact that the spectator/viewer is drawn to the exhibits not only by their own merit, but also guided and assisted by verbal messages, i. e. Labels, brochures. Catalogues etc. Hence, the crucial question is what we, the museologists, use as a means of communication when preparing for a Universal Museum. Should we use pictorial semiotics? This may be a partial solution, which is mainly restricted to objects that can be manipulated and moved by the visitor, as is the case in most of the technological museums. But since the range of objects on display at museums is vast and varied - fine art, archaeological finds, ethnographic objects etc., it may not be the answer to the whole spectrum of exhibits. Dr. Ludwig Lazarus Zamenhof, being an optimist, hoped that by introducing Esperanto to the multi-lingual world population, humanity would be able to bridge and diminish the gap of linguistic differences, thus creating a better understanding between the international communities. Unfortunately this vision was not realized. Esperanto was and still is an utopian and esoteric phenomenon. The barriers between nations still exist although, as mentioned earlier, mass media do help, in some ways, to reduce them.