Multimodal Education for Very Young Learners

Invited Plenary Speaker

Deep Head Pose: Gaze-Direction Estimation in Multimodal Video

In this paper we present a convolutional neuralnetwork (CNN)-based model for human head pose estimation inlow-resolution multi-modal RGB-D data. We pose the problemas one of classification of human gazing direction. We furtherfine-tune a regressor based on the learned deep classifier. Next wecombine the two models (classification and regression) to estimateapproximate regression confidence. We present state-of-the-artresults in datasets that span the range of high-resolution humanrobot interaction (close up faces plus depth information) data tochallenging low resolution outdoor surveillance data. We buildupon our robust head-pose estimation and further introduce anew visual attention model to recover interaction with theenvironment. Using this probabilistic model, we show thatmany higher level scene understanding like human-human/sceneinteraction detection can be achieved. Our solution runs inreal-time on commercial hardware

Identity Text: an educational intervention to foster cultural interaction

Background: Sociocultural theories state that learning results from people participating in contexts where social interaction is facilitated. There is a need to create such facilitated pedagogical spaces where participants share their ways of knowing and doing. The aim of this exploratory study was to introduce pedagogical space for sociocultural interaction using ‘Identity Text’.
Methods: Identity texts are sociocultural artifacts produced by participants, which can be written, spoken, visual, musical, or multimodal. In 2013, participants of an international medical education fellowship program were asked to create their own Identity Texts to promote discussion about participants’ cultural backgrounds. Thematic analysis was used to make the analysis relevant to studying the pedagogical utility of the intervention.
Result: The Identity Text intervention created two spaces: a ‘reflective space’ helped
participants reflect on sensitive topics like institutional environments, roles in
interdisciplinary teams, and gender discrimination. A ‘narrative space’ allowed
participants to tell powerful stories that provided cultural insights and challenged cultural hegemony; they described the conscious and subconscious transformation in identity that evolved secondary to struggles with local power dynamics and social demands involving the impact of family, peers and country of origin.
Conclusion: Whilst the impact of providing pedagogical space using Identity Text on
cognitive engagement and enhanced learning requires further research, the findings of
this study suggest that it is a useful pedagogical strategy to support cross-cultural
education.

Multimodal attention system for an interactive robot

[EN]Social robots are receiving much interest in the robotics community. The most important goal for such robots lies in their interaction capabilities. An attention system is crucial, both as a filter to center the robot’s perceptual resources and as a mean of letting the observer know that the robot has intentionality. In this paper a simple but flexible and functional attentional model is described. The model, which has been implemented in an interactive robot currently under development, fuses both visual and auditive information extracted from the robot’s environment, and can incorporate knowledge-based influences on attention.

Comparison of Authentic and Simplified Texts : A case study of Wuthering Heights

The aim of this essay is to explore in what way Graded Readers are different from authentic texts against the background of English as a Second Language (ESL) and the use of authentic and simplified text in ESL teaching. The material used for this purpose is the authentic text of Wuthering Heights by Emily Brontë and two upper-intermediate Graded Readers from two different publishers. The study uses the software readability-score and manual analysis to examine the texts with regards to lexical choice, language structure and story. The study showed that the Graded Readers are simplified in all aspects studied. Moreover, the Graded Readers differ from each other as well, most notably in the style of the text due to sentence structure and story simplification. This could imply that different authors of Graded Readers adopt different styles when simplifying text and that the grading levels are not comparable between different publishers.

Clinton Multimodal (U. S. 30 Mississippi River Bridge) Study, 2003

The purpose of this memorandum is to document the benefit-cost analysis of the river crossing concept alternatives described in the "Concept Alternatives Technical Memo." Benefit-cost studies are designed to measure, in dollars, the potential positive or negative impacts of large-scale construction projects. The concept alternatives analyzed include improvements to the Union Pacific Railroad (UPRR) River Crossing and the U.S. Highway 30 (U.S. 30) River Crossing.

Un-Editing Alfred: Rethinking Modern Editions of Pre-modern Texts from a Post-modern Sensibility

Thesis (Ph.D.)--University of Washington, 2016-06

La explotación de las artes en la promoción de la Marca Finlandia: análisis crítico del discurso multimodal

En este trabajo de fin de máster se investiga qué impacto tuvo la Marca Finlandia en el uso de las artes finlandesas en Focus – economía y tecnología, una revista publicada por el Ministerio de Asuntos Exteriores de Finlandia durante los años más profundos de la crisis económica 2008-2013. En 2008 el Ministro de Asuntos Exteriores de entonces, Alexander Stubb, designó una delegación para desarrollar la marca país de Finlandia, fruto de un proceso del que se presentó en 2010 un documento llamado Tehtävä Suomelle! (¡Misión para Finlandia!), cuyo objetivo fue concretar el diseño y las funciones de la Marca Finlandia. Este trabajo se enfoca en la utilización del arte y de artistas finlandeses en el material creado para apoyar la promoción de la imagen del país y, en última instancia, su competividad económica. Para tener en cuenta el contexto del material objeto de estudio, presentamos el concepto de la marca país y distintos planteamientos teóricos para su realización. Siendo una estrategia estatal polémica, recorremos los puntos de crítica más importantes sobre esta práctica promocional. También nos familiarizamos con la historia de la promoción nacional antes de introducir el presente proyecto, la Marca Finlandia, y sus objetivos. En la parte teórica se presentan los conceptos relevantes para el estudio: el discurso y su análisis crítico, enfatizando su capacidad ideológica para mantener y crear relaciones del poder muy a menudo desiguales. Ya que nuestro material de estudio empírico está compuesto de publicaciones físicas con dimensiones textuales, visuales y hápticas, nuestro método de investigación es observar críticamente estas relaciones desde la perspectiva multimodal. En la parte empírica del trabajo analizamos la composición de la comunicación multimodal del material imprimido, producido por el Ministerio de Asuntos Exteriores de Finlandia y sus socios de cooperación. Las revistas anuales Focus– Economía y tecnología se enfocan en noticias y artículos sobre los sectores de la economía y tecnología, pero también incluyen contenido sobre el diseño, la música, la moda, la arquitectura y la danza finlandeses. En nuestro análisis recorremos los campos verbales y visuales utilizados en las revistas para investigar de qué manera se presentaban las artes finlandesas en la operación de Marca Finlandia. Detectamos representaciones textuales, resaltaciones visuales y la combinación de ambos, lo que servía a las metas preterminadas y la imagen requerida por la delegación diseñadora de la marca país. Las revistas compartían el discurso en común que corría paralelo con los objetivos de la Marca Finlandia, que a su vez se puede ver como parte del discurso hegemónico neoliberal en general.

Interaction techniques with novel multimodal feedback for addressing gesture-sensing systems

Users need to be able to address in-air gesture systems, which means finding where to perform gestures and how to direct them towards the intended system. This is necessary for input to be sensed correctly and without unintentionally affecting other systems. This thesis investigates novel interaction techniques which allow users to address gesture systems properly, helping them find where and how to gesture. It also investigates audio, tactile and interactive light displays for multimodal gesture feedback; these can be used by gesture systems with limited output capabilities (like mobile phones and small household controls), allowing the interaction techniques to be used by a variety of device types. It investigates tactile and interactive light displays in greater detail, as these are not as well understood as audio displays. Experiments 1 and 2 explored tactile feedback for gesture systems, comparing an ultrasound haptic display to wearable tactile displays at different body locations and investigating feedback designs. These experiments found that tactile feedback improves the user experience of gesturing by reassuring users that their movements are being sensed. Experiment 3 investigated interactive light displays for gesture systems, finding this novel display type effective for giving feedback and presenting information. It also found that interactive light feedback is enhanced by audio and tactile feedback. These feedback modalities were then used alongside audio feedback in two interaction techniques for addressing gesture systems: sensor strength feedback and rhythmic gestures. Sensor strength feedback is multimodal feedback that tells users how well they can be sensed, encouraging them to find where to gesture through active exploration. Experiment 4 found that they can do this with 51mm accuracy, with combinations of audio and interactive light feedback leading to the best performance. Rhythmic gestures are continuously repeated gesture movements which can be used to direct input. Experiment 5 investigated the usability of this technique, finding that users can match rhythmic gestures well and with ease. Finally, these interaction techniques were combined, resulting in a new single interaction for addressing gesture systems. Using this interaction, users could direct their input with rhythmic gestures while using the sensor strength feedback to find a good location for addressing the system. Experiment 6 studied the effectiveness and usability of this technique, as well as the design space for combining the two types of feedback. It found that this interaction was successful, with users matching 99.9% of rhythmic gestures, with 80mm accuracy from target points. The findings show that gesture systems could successfully use this interaction technique to allow users to address them. Novel design recommendations for using rhythmic gestures and sensor strength feedback were created, informed by the experiment findings.

El papel de la gestualidad en la estructuraci?n multimodal del discurso [recurso electr?nico]

Durante los ?ltimos a?os el an?lisis del discurso oral ha tomado un lugar importante en los estudios de la ling??stica y sobretodo en investigaciones enfocadas en el discurso en el aula. Su naturaleza en el habla permite obtener un gran abanico de elementos por analizar que marca la diferencia con el an?lisis del discurso escrito. Por otro lado, en la comunicaci?n oral el hablante est? pendiente de lo que dice con las palabras, pero no controla de la misma manera los gestos. Aspectos como la expresi?n del rostro, la orientaci?n corporal, la direcci?n de la mirada, etc., enfatizan lo que se quiere expresar en el discurso verbal. As?, los enunciados son bimodales ya que utilizan tanto la modalidad auditivo-vocal como la viso-gestual. Sin embargo, este estudio caracteriza este tipo de comunicaci?n on l t rm no ?mult mo l? o l o qu n l s urso or l l orpus que se analiza hay un canal extra de comunicaci?n como lo es el uso del tablero. Esta exploraci?n le da valor a los gestos en el ?mbito pedag?gico, pues el lenguaje no verbal puede reforzar o sustituir la expresi?n verbal y puede llegar a tener una gran injerencia en la construcci?n del sentido que los alumnos elaboran en torno a la comunicaci?n transmitida por el profesor. Teniendo en cuenta que la lecci?n es una unidad textual, esta investigaci?n busca observar y analizar c?mo los gestos contribuyen a la estructuraci?n del discurso en el aula, es decir, cu?l ser?a la marcaci?n multimodal de la estructura de la lecci?n. Se exploran dos temas centrales: el an?lisis del discurso en el aula y la gestualidad que acompa?a al habla. Para ello se utiliz? el modelo de an?lisis del discurso oral de Sinclair y Coulthard (1992) que tiene como elementos principales la lecci?n, las transacciones y los intercambios, y el modelo de an?lisis de gesticulaciones de McNeill (1998, 2005) que propone cinco dimensiones de gesticulaciones: gestos r?tmicos, gestos de?cticos, gestos ic?nicos, gestos metaf?ricos y gestos cohesivos.Se busc? determinar el papel de los gestos como marcadores o reforzadores de la estructura del discurso oral de una clase de Matem?ticas en ingl?s, dada por un docente nativo para quinto grado en un contexto educativo biling?e. Se hizo una grabaci?n de esa clase de 45 minutos, se procedi? a transcribirla en un formato de dos columnas: Profesor (con los enunciados del profesor) y Estudiantes (con las intervenciones de los estudiantes). Se observ? que el estudio del discurso oral en el aula se enriquece cuando se le a?aden elementos de an?lisis gestual en el mismo. Esta investigaci?n invita a indagar m?s sobre el papel de estos dos componentes de la interacci?n (discurso oral y gestualidad) tanto en el aula como en otras situaciones comunicativas

Queering religious texts: An exploration of British non-heterosexual Christians’ and Muslims’ strategy of constructing sexuality-affirming hermeneutics.

Religious authority figures often use religious texts as the primary basis for censuring homosexuality. In recent years, however, non-heterosexual Christians and Muslims have begun to contest the discursively produced boundary of sexual morality.Drawing upon two research projects on non-heterosexual Christians and Muslims, this article explores the three approaches embedded in this strategy.While acknowledging that homosexuality is indeed portrayed negatively in some parts of religious texts, the participants critique traditional hermeneutics by highlighting its inaccuracy and socio-cultural specificity, and arguing for a contextualized and culturally relevant interpretation. They also critique the credibility of institutional interpretive authority by highlighting its inadequacy and ideology, and relocating authentic interpretive authority to personal experience. Finally, they recast religious texts to construct resources for their spiritual nourishment.This strategy generally reflects the contemporary western religious landscape that prioritizes the authority of the self over that of religious institution.

Russian social networks on the web: cohesion and coherence in Vkontakte

In this thesis connections between messages on the public wall of the Russian social network Vkontakte are analysed and classified. A total of 1818 messages from three different Vkontakte groups were collected and analysed according to a new framework based on Halliday and Hasan’s (1976) research into cohesion and Simmons’s (1981) adaptation of their classification for Russian. The two categories of textuality, cohesion and coherence, describe the linguistic connections between messages. The main aim was to find out how far the traditional categories of cohesion are applicable to an online social network including written text as well as multimedia-files. In addition to linguistic cohesion the pragmatic and topic coherence between Vkontakte messages was also analysed. The analysis of pragmatic coherence classifies the messages with acts according to their pragmatic function in relation to surrounding messages. Topic coherence analyses the content of the messages, describes where a topic begins, changes or is abandoned. Linguistic cohesion, topic coherence and pragmatic coherence enable three different types of connections between messages and these together form a coherent communication on the message wall. The cohesion devices identified by Halliday and Hasan and Simmons were found to occur in these texts, but additional devices were also identified: these are multimodal, graphical and grammatical cohesion.