Historical timelines : analyzing multimodal text design

The recent focus on literacy in Social Studies has been on linguistic design, particularly that related to the grammar of written and spoken text. When students are expected to produce complex hybridized genres such as timelines, a focus on the teaching and learning of linguistic design is necessary but not sufficient to complete the task. Theorizations of new literacies identify five interrelated meaning making designs for text deconstruction and reproduction: linguistic, spatial, visual, gestural, and audio design. Honing in on the complexity of timelines, this paper casts a lens on the linguistic, visual, spatial, and gestural designs of three pairs of primary school aged Social Studies learners. Drawing on a functional metalanguage, we analyze the linguistic, visual, spatial, and gestural designs of their work. We also offer suggestions of their effect, and from there consider the importance of explicit instruction in text design choices for this Social Studies task. We conclude the analysis by suggesting the foci of explicit instruction for future lessons.

Reading in the Australian curriculum English : describing the effects of structure and organisation on multimodal texts

The recently introduced Australian Curriculum: English Version 3.0 (Australian Curriculum, Assessment and Reporting Authority [ACARA], 2012) requires students to ‘read’ multimodal text and describe the effects of structure and organisation. We begin this article by tracing the variable understandings of what reading multimodal text might entail through the Framing Paper (National Curriculum Board, 2008), the Framing Paper Consultation Report (National Curriculum Board, 2009a), the Shaping Paper (National Curriculum Board, 2009b) and Version 3.0 of the Australian Curriculum English (ACARA, 2012). Our findings show that the theoretical and descriptive framework for doing so is implicit. Drawing together multiple but internally coherent theories from the field of semiotics, we suggest one way to work towards three Year 5 learning outcomes from the reading/writing mode. The affordances of assembling a broad but explicit technical metalanguage for an informed reading of the integrated design elements of multimodal texts are noted.

Reterritorializing multimodal design in the classroom through a socially mediated Indigenous heritage

National or International Significance Flows of cultural heritage in textual practices are vital to sustaining Indigenous communities - a national and international priority (Commonwealth of Australia, 2011). Indigenous heritage, whether passed on by oral tradition or ubiquitous social media, can be seen as a "conversation between the past and the future" (Fairclough, 2012, p. xv). Indigenous heritage involves appropriating memories within a cultural flow to pass on a spiritual legacy. This presentation reports ethnographic research of social media practices in a small independent Aboriginal school in Southeast Queensland, Australia that is resided over by the Yuggera elders and an Aboriginal principal. Quality of Research The purpose was to rupture existing notions of white literacies in schools, and to deterritorialize the uses of digital media by dominant cultures in the public sphere. Examples of learning experiences included the following: i. Integrating Indigenous language and knowledge into media text production; ii. Classroom visits from Indigenous elders; and iii. Publishing oral histories through digital scrapbooking. The program aligned with the Australian National Curriculum English (ACARA, 2014), which mandates the teaching of multimodal text creation. Data sources included a class set of digital scrapbooks collaboratively created in a preparatory-one primary classroom. The digital scrapbooks combined digitally encoded words, images of material artifacts, and digital music files. A key feature of the writing and digital design task was to retell and digitally display and archive a cultural narrative of significance to the Indigenous Australian community and its memories and material traces of the past for the future. Data analysis of the students' digital stories involved the application of key themes of negotiated, material, and digitally mediated forms of heritage practice. It drew on Australian Indigenous research by Keddie et al. (2013) to guard against the homogenizing of culture that can arise from a focus on a static view of culture. The interpretation of findings located Indigenous appropriation of social media within broader racialized politics that enables Indigenous literacy to be understood as a dynamic, negotiated, and transgenerational flows of practice. It demonstrates that Indigenous children's use of media production reflects "shifting and negotiated identities" in response to changing media environments that can function to sustain Indigenous cultural heritages (Appadurai, 1696, p. xv). Impact on practice, policy or theory The findings are important for teachers at a time when Aboriginal and Torres Strait Islander Histories and Cultures is a cross-curricular policy priority in the Australian Curriculum (ACARA, 2014). The findings show how curriculum policies can be applied to classroom practice in ways that are epistemologically consistent with Indigenous ways of knowing and being. Theoretically, it demonstrates how the children's experiences of culture are layered over time, as successive generations inherit, interweave, and hear others' cultural stories or maps. Practically, recommendations are provided for an approach to appropriating social media in schools that explicitly attends to the dynamic nature of Indigenous practices, negotiated through intercultural constructions and flows, and opening space for a critical anti-racist approach to multimodal text production. Timeliness The research is timely in the context of the accessibility and role of digital and multimodal forms of communication, including for Aboriginal and Torres Strait Islander communities.

Scaffolding multimodal literacies: Learning in and out of the classroom

This chapter presents data produced by a research project that looked at pedagogy for print and digital literacies in a high poverty, high diversity primary school. The student population included refugee, immigrant and Aboriginal and Torres Strait Islander young people. In an environment in which schools, such as the study site, are under pressure to narrow the curriculum to ‘the basics’, the project sought to support teachers as they worked to create a rich curriculum for all students. The chapter will focus on pedagogy in an after-school media club. The aim of the club, which ran weekly for several years, was to build students’ media literacy skills. The data suggest that established ways of scaffolding linguistic texts cannot be simply transferred to multimodal text production. The chapter will also address implications from the research outcomes for other teachers working with At Risk EAL students.

Doing new things with texts : multimodality and Mao’s Last Dancer

This paper describes a senior, multimodal task developed by Shauna O’Connor and the English staff at Brigidine College after consultation in the form of media workshops with Anita Jetnikoff. Gunther Kress (2006) suggested recently that due to the affordances of media platforms such as Web 2.0, “we need to be doing new things with texts”. The year 11 unit’s Finding a Voice parent text was the memoir, Mao’s last Dancer. The summative assessment task morphed over time from an ‘identity portrait’, into ‘a multimodal, first person narrative’.

Cultural flows in an Aboriginal school : deterritorializing textual production through a socially mediated Indigenous heritage

Flows of cultural heritage in textual practices are vital to sustaining Indigenous communities. Indigenous heritage, whether passed on by oral tradition or ubiquitous social media, can be seen as a “conversation between the past and the future” (Fairclough, 2012, xv). Indigenous heritage involves appropriating memories within a cultural flow to pass on a spiritual legacy. This presentation reports ethnographic research of social media practices in a small independent Aboriginal school in Southeast Queensland, Australia that is resided over by the Yugambeh elders and an Aboriginal principal. The purpose was to rupture existing notions of white literacies in schools, and to deterritorialize the uses of digital media by dominant cultures in the public sphere. Examples of learning experiences included the following: i. Integrating Indigenous language and knowledge into media text production; ii. Using conversations with Indigenous elders and material artifacts as an entry point for storytelling; iii. Dadirri – spiritual listening in the yarning circle to develop storytelling (Ungunmerr-Baumann, 2002); and iv. Writing and publicly sharing oral histories through digital scrapbooking shared via social media. The program aligned with the Australian National Curriculum English (ACARA, 2012), which mandates the teaching of multimodal text creation. Data sources included a class set of digital scrapbooks collaboratively created in a multi-age primary classroom. The digital scrapbooks combined digitally encoded words, images of material artifacts, and digital music files. A key feature of the writing and digital design task was to retell and digitally display and archive a cultural narrative of significance to the Indigenous Australian community and its memories and material traces of the past for the future. Data analysis of the students’ digital stories involved the application of key themes of negotiated, material, and digitally mediated forms of heritage practice. It drew on Australian Indigenous research by Keddie et al. (2013) to guard against the homogenizing of culture that can arise from a focus on a static view of culture. The interpretation of findings located Indigenous appropriation of social media within broader racialized politics that enables Indigenous literacy to be understood as a dynamic, negotiated, and transgenerational flows of practice. The findings demonstrate that Indigenous children’s use of media production reflects “shifting and negotiated identities” in response to changing media environments that can function to sustain Indigenous cultural heritages (Appadurai, 1696, xv). It demonstrated how the children’s experiences of culture are layered over time, as successive generations inherit, interweave, and hear others’ cultural stories or maps. It also demonstrated how the children’s production of narratives through multimedia can provide a platform for the flow and reconstruction of performative collective memories and “lived traces of a common past” (Giaccardi, 2012). It disrupts notions of cultural reductionism and racial incommensurability that fix and homogenize Indigenous practices within and against a dominant White norm. Recommendations are provided for an approach to appropriating social media in schools that explicitly attends to the dynamic nature of Indigenous practices, negotiated through intercultural constructions and flows, and opening space for a critical anti-racist approach to multimodal text production.

Multiliteracies pedagogy : creating claymations with adolescent, post-beginner English language learners

In addressing literacy in high school education, it is important to foreground the particular issues faced by growing numbers of English Language Learners (ELLs). In our increasingly culturally and linguistically diverse classrooms, this is a matter for all literacy teachers, as well as ELL specialists. In Australia, teachers of ELLs are experimenting with Multiliteracies pedagogy which provides rich opportunities to explore language learning experiences and outcomes that stretch beyond exercises in reproduction in written and oral modes only. This paper documents the practice of a high school teacher who uses a claymation project, producing a movie by stop-motion filming of clay figures, with a class of low-level English literacy learners. Drawing on observations of three particular students, the paper outlines a number of possibilities of this approach for English language learners. These include increased individual agency; enhanced engagement through collaboration; and the opportunity to explore various elements of multimodal text design.

Doing Digital Composition on the Social Web: Knowledge Processes in Literacy Learning

The power to influence others in ever-expanding social networks in the new knowledge economy is tied to capabilities with digital media production. This chapter draws on research in elementary classrooms to examine the repertoires of cross-disciplinary knowledge that literacy learners need to produce innovative digital media via the “social web”. It focuses on the knowledge processes that occurred when elementary students engaged in multimodal text production with new digital media. It draws on Kalantzis and Cope’s (2008) heuristic for theorizing “Knowledge Processes” in the Learning by Design approach to pedagogy. Learners demonstrate eight “Knowledge Processes” across different subject domains, skills areas, and sensibilities. Drawing data from media-based lessons across several classroom and schools, this chapter examines what kinds of knowledge students utilize when they produce digital, multimodal texts in the classroom. The Learning by Design framework is used as an analytic tool to theorize how students learn when they engaged in a specific domain of learning – digital media production.

手持通信设备的多通道文本输入研究

Recently，Handheld Communication Devices is developing very fast, extending in users and spreading in application fields, and has an promising future. This study investigated the acceptance of the multimodal text entry method and the behavioral characteristics when using it. Based on the general information process model of a bimodal system and the human factor studies about the multimodal map system, the present study mainly focused on the hand-speech bimodal text entry method. For acceptance, the study investigated the subjective perception of the accuracy of speech recognition by Wizard of Oz (WOz) experiment and a questionnaire. Results showed that there was a linear relationship between the speech recognition accuracy and the subjective accuracy. Furthermore, as the familiarity increasing, the difference between the acceptable accuracy and the subjective accuracy gradually decreased. In addition, the similarity of meaning between the outcome of speech recognition and the correct sentences was an important referential criterion. The second study investigated three aspects of the bimodal text entry method, including input, error recovery and modal shifts. The first experiment aimed to find the behavioral characteristics of user when doing error recovery task. Results indicated that participants preferred to correct the error by handwriting, which had no relationship with the input modality. The second experiment aimed to discover the behavioral characteristics of users when doing text entry in various types of text. Results showed that users preferred to speech input in both words and sentences conditions, which was highly consistent among individuals, while no significant difference was found between handwriting and speech input in the character condition. Participants used more direct strategy than jumping strategy to deal with mixed text, especially for the Chinese-English mixed type. The third experiment examined the cognitive load in the different modal shifts, results suggesting that there were significant differences between different shifts. Moreover, relevant little time was needed in the Shift from speech input to hand input. Based on the main findings, implications were discussed as follows: Firstly, when evaluating a speech recognition system, attention should be paid to the fact that the speech recognition accuracy was not equal to the subjective accuracy. Secondly, in order to make a speech input system more acceptable, a good method is to train and supply the feedback for the accuracy in training, which improving the familiarity and sensitivity to the system. Thirdly, both the universal and individual behavioral patterns were taken into consideration to improve the error recovery method. Fourthly, easing the study and the use of speech input, the operations of speech input should be simpler. Fifthly, more convenient text input method for non-Chinese text entry should be provided. Finally, the shifting time between hand input and speech input provides an important parameter for the design of automatic-evoked speech recognition system.

Interpretazione per i media: il caso particolare delle interviste sul podio della Formula 1

This dissertation analyses the live simultaneous interpretation from English into Italian of six 2013 Formula 1 World Championship podium interviews and focuses on four main aspects: how the interpreter handled the décalage at the end of the interview and during the turn-taking; if he used any marker to indicate that he was starting to translate a new turn of the source text; what he did when overlapped speech in the source texts occurred; what happened when the Italian commentators talked during the interpreter’s translation. In the first chapter a description mainly of what a Formula 1 podium interview is and what an interpreter translates during the Formula 1 weekends is present. In the second chapter a literature review on media interpreting, with particular attention put on Straniero Sergio’s paper on translating Formula 1 press-conferences (2003), and turn-taking is provided. In the third chapter the methodology used to obtain and process the video and audio files of source and target texts and to transcribe them is described. We concentrated primarily on Thibault’s multimodal text transcription techniques (2000) and on how they were used and adapted to fit the purposes of this dissertation. In the fourth chapter the results obtained through the analysis of the source and target texts are shown and described, focusing only on the objectives of the dissertation, without aiming to provide a qualitative evaluation of the interpretations. In the fifth and last chapter the conclusions and some final remarks are made, based on the results obtained during the analysis and the hope for a more in depth knowledge of Italian Formula 1 interpreter’s working conditions.

iPed : pedagogy for digital text production

Reading and writing are being transformed by global changes in communication practices using new media technologies. This paper introduces iPed, a research-based pedagogy that enables teachers to navigate innovative digital text production in the literacy classroom. The pedagogy was generated in the context of a longitudinal digital literacy intervention in a school that services low-socioeconomic and ethnically diverse students. iPed synthesizes four key pedagogies that were salient in the analysis of over 180 hours of lesson observations – Link, Challenge, Co-Create, and Share. The strengths of the pedagogy include connecting to students’ home cultures, critical media literacy, collaborative and creative digital text production, and gaining cosmopolitan recognition within global communities.

Multimodal and monomodal discourses of marketization in higher education : power, ideology, and the absence of the image

The ubiquity of multimodality in hypermedia environments is undeniable. Bezemer and Kress (2008) have argued that writing has been displaced by image as the central mode for representation. Given the current technical affordances of digital technology and user-friendly interfaces that enable the ease of multimodal design, the conspicuous absence of images in certain domains of cyberspace is deserving of critical analysis. In this presentation, I examine the politics of discourses implicit within hypertextual spaces, drawing textual examples from a higher education website. I critically examine the role of writing and other modes of production used in what Fairclough (1993) refers to as discourses of marketisation in higher education, tracing four pervasive discourses of teaching and learning in the current economy: i) materialization, ii) personalization, iii) technologisation, and iv) commodification (Fairclough, 1999). Each of these arguments is supported by the critical analysis of multimodal texts. The first is a podcast highlighting the new architectonic features of a university learning space. The second is a podcast and transcript of a university Open Day interview with prospective students. The third is a time-lapse video showing the construction of a new science and engineering precinct. These three multimodal texts contrast a final web-based text that exhibits a predominance of writing and the powerful absence or silencing of the image. I connect the weightiness of words and the function of monomodality in the commodification of discourses, and its resistance to the multimodal affordances of web-based technologies, and how this is used to establish particular sets of subject positions and ideologies through which readers are constrained to occupy. Applying principles of critical language study by theorists that include Fairclough, Kress, Lemke, and others whose semiotic analysis of texts focuses on the connections between language, power, and ideology, I demonstrate how the denial of image and the privileging of written words in the multimodality of cyberspace is an ideological effect to accentuate the dominance of the institution.

1000 voices: reflective online multimodal narrative inquiry as a research methodology for disability research

This article outlines the research approach used in the international 1000 Voices Project. The 1000 Voices project is an interdisciplinary research and public awareness project that uses a customised online multimodal storytelling platform to explore the lives of people with disability internationally. Through the project, researchers and partners have encouraged diverse participants to select the modes of storytelling (e.g. images, text, videos and combinations thereof) that suit them best and to self-define what both ‘disability’ and ‘life story’ mean to them. The online reflective component of the approach encourages participants to organically and reflectively develop story events and revisions over time in ways that suit them and their emerging lives. This article provides a detailed summary of the project's theoretical and methodological development alongside suggestions for future development in social work and qualitative research.

Time, space, and text in the elementary school digital writing classroom

Theorists of multiliteracies, social semiotics, and the New Literacy Studies have drawn attention to the potential changing nature of writing and literacy in the context of networked communications. This article reports findings from a design-based research project in Year 4 classrooms (students aged 8.5-10 years) in a low socioeconomic status school. A new writing program taught students how to design multimodal and digital texts across a range of genres and text types, such as web pages, online comics, video documentaries, and blogs. The authors use Bernstein’s theory of the pedagogic device to theorize the pedagogic struggles and resolutions in remaking English through the specialization of time, space, and text. The changes created an ideological struggle as new writing practices were adapted from broader societal fields to meet the instructional and regulative discourses of a conventional writing curriculum.

Patrimonio e Identidad en la Investigación Educativa Basada en las Artes desde un Enfoque Multimodal

Este artículo describe dos experiencias de investigación de nuestro grupo interconectadas, la primera desarrollada durante el año 2007 a través del proyecto internacional CALVINO del Programa Cultura 2000 de la Unión Europea, y la segunda implementada durante el año 2014 en el marco del Proyecto Investigación e Innovación en Secundaria en Andalucía (PIIISA). Ambos proyectos tienen en común el eje temático de la identidad a partir de una idea de patrimonio y el hecho de haber puesto en práctica metodologías de investigación basadas en las artes visuales con un enfoque multimodal. Desde estos dos puntos de anclaje relativos a la temática (qué) y a la metodología (cómo) analizamos lo acontecido para obtener conclusiones relevantes que, por una parte, pongan en valor estas prácticas significativas y, por otra, aporten nuestra experiencia para futuras propuestas de investigación en el ámbito temático y/o metodológico.