Beat that Word : How Listeners Integrate Beat Gesture and Focus in Multimodal Speech Discourse

Peer reviewed

Correction to “Reminder and 2AFC tasks provide similar estimates of the difference limen: A reanalysis of data from Lapid, Ulrich, and Rammsayer (2008) and a discussion of Ulrich and Vorberg (2009)”

We recently published an article (García-Pérez & Alcalá- Quintana, 2010) reanalyzing data presented by Lapid, Ulrich, and Rammsayer (2008) and discussing a theoretical argument developed by Ulrich and Vorberg (2009). The purpose of this note is to correct an error in our study that has some theoretical importance, although it does not affect the conclusion that was raised. The error lies in that asymptote parameters reflecting lapses or finger errors should not enter the constraint relating the psychometric functions that describe performance when the comparison stimulus in a two-alternative forced choice (2AFC) discrimination task is presented in the first or second interval.

Introduction of novel video-based tasks to clinical fMRI: Comparing traditional fMRI tasks with novel video-based tasks for mapping brain language areas

Recently, blood oxygen level-dependent (BOLD) functional magnetic resonance imaging (fMRI) has become a routine clinical procedure for localization of language and motor brain regions and has been replacing more invasive preoperative procedures. However, the fMRI results from these tasks are not always reproducible even from the same patient. Evaluating the reproducibility of language and speech mapping is especially complicated due to the complex brain circuitry that may become activated during the functional task. Non-language areas such as sensory, attention, decision-making, and motor brain regions may also be activated in addition to the specific language regions during a traditional sentence-completion task. In this study, I test a new approach, which utilizes 4-minute video-based tasks, to map language and speech brain regions for patients undergoing brain surgery. Results from 35 subjects have shown that the video-based task activates Wernicke’s area, as well as Broca’s area in most subjects. The computed laterality indices, which indicate the dominant hemisphere from that functional task, have indicated left dominance from the video-based tasks. This study has shown that the video-based task may be an alternative method for localization of language and speech brain regions for patients who are unable to complete the sentence-completion task.

The Organizational Tasks of High-Performance Gymnastic Coaches

The purpose of this study was to report the knowledge used by expert high performance gymnastic coaches in the organization of training and competition. In-depth interviews were conducted with 9 coaches who worked with male gymnasts and 8 coaches who worked with female gymnasts. Qualitative analyses showed that coaches of males and coaches of females planned training similarly, except that coaches of females appeared to emphasize esthetic and nutritional issues to a greater extent. Coaches of males revealed more concerns about the organization of gymnasts' physical conditioning. Analysis indicated that expert gymnastic coaches of males and females are constantly involved in dynamic social interactions with gymnasts, parents, and assistant coaches. Many areas of coaches' organizational work, such as dealing with the athletes' personal concerns and working with parents, are not part of the structure of coaches' training programs and emerged as crucial tasks of expert gymnastic coaches for developing elite gymnasts.

The Material Dynamics of a London-French Blog: A Multimodal Reading of Migrant Habitus

Through a fine-grained reading of a London-French blog, this article aims to shed light on the lived experience of the French community in London. The ethnosemiotic conceptual framework brings together ethnographic and semiotic schools of thought, focusing in particular on Pierre Bourdieu’s concept of habitus and Gunther Kress’s multimodal social semiotic analytical model. Habitus is broken down into its material manifestations of habitat, habit and habituation, all displayed in the blog and revealing of the blogger’s identity and positioning within the migration setting. As all modes are considered to be of equal semiotic potential, equivalent emphasis is placed on the multiple modes of meaning-making present in the blog, such as layout, colour, typography and language. By examining the dynamic relationships between blogger and audience, subjectivity and objectivity, on-line and on-land habitus, and intermodal dynamics themselves, through the prism of multimodality, hidden facets of the blogger’s cultural identity and sense of community belonging within the diasporic context begin to materialise.

Peer Support to Facilitate Knowledge Sharing on Complex Tasks

This thesis explores how to design a peer support system to facilitate self-organized knowledge sharing in non-formal learning environments, in particular when learners work on complex tasks. The peer support system aims to replace two teacher-led didactic arrangements: selecting a tutor at the initial stage, and guidance during the interaction process (Dillenbourg, 1999; Topping, 1996). Such a system has previously been developed by Van Rosmalen (2008) and De Bakker (2010) and has been tentatively used to facilitate knowledge sharing on content-related questions. In this thesis, we would like to find out how to further improve the design of this peer support system, especially to facilitate knowledge sharing on complex tasks. Since little pedagogical theory is available to inform the design of our peer support system, this thesis attempts to apply cognitive load theory (Sweller, Van Merriënboer, & Paas, 1998; Van Merriënboer & Sweller, 2005) that informs instructional designs in classroom settings to the design of our peer support system in Learning Networks.

Multimodal Education for Very Young Learners

Invited Plenary Speaker

Deep Head Pose: Gaze-Direction Estimation in Multimodal Video

In this paper we present a convolutional neuralnetwork (CNN)-based model for human head pose estimation inlow-resolution multi-modal RGB-D data. We pose the problemas one of classification of human gazing direction. We furtherfine-tune a regressor based on the learned deep classifier. Next wecombine the two models (classification and regression) to estimateapproximate regression confidence. We present state-of-the-artresults in datasets that span the range of high-resolution humanrobot interaction (close up faces plus depth information) data tochallenging low resolution outdoor surveillance data. We buildupon our robust head-pose estimation and further introduce anew visual attention model to recover interaction with theenvironment. Using this probabilistic model, we show thatmany higher level scene understanding like human-human/sceneinteraction detection can be achieved. Our solution runs inreal-time on commercial hardware

Multimodal attention system for an interactive robot

[EN]Social robots are receiving much interest in the robotics community. The most important goal for such robots lies in their interaction capabilities. An attention system is crucial, both as a filter to center the robot’s perceptual resources and as a mean of letting the observer know that the robot has intentionality. In this paper a simple but flexible and functional attentional model is described. The model, which has been implemented in an interactive robot currently under development, fuses both visual and auditive information extracted from the robot’s environment, and can incorporate knowledge-based influences on attention.

Clinton Multimodal (U. S. 30 Mississippi River Bridge) Study, 2003

The purpose of this memorandum is to document the benefit-cost analysis of the river crossing concept alternatives described in the "Concept Alternatives Technical Memo." Benefit-cost studies are designed to measure, in dollars, the potential positive or negative impacts of large-scale construction projects. The concept alternatives analyzed include improvements to the Union Pacific Railroad (UPRR) River Crossing and the U.S. Highway 30 (U.S. 30) River Crossing.

La explotación de las artes en la promoción de la Marca Finlandia: análisis crítico del discurso multimodal

En este trabajo de fin de máster se investiga qué impacto tuvo la Marca Finlandia en el uso de las artes finlandesas en Focus – economía y tecnología, una revista publicada por el Ministerio de Asuntos Exteriores de Finlandia durante los años más profundos de la crisis económica 2008-2013. En 2008 el Ministro de Asuntos Exteriores de entonces, Alexander Stubb, designó una delegación para desarrollar la marca país de Finlandia, fruto de un proceso del que se presentó en 2010 un documento llamado Tehtävä Suomelle! (¡Misión para Finlandia!), cuyo objetivo fue concretar el diseño y las funciones de la Marca Finlandia. Este trabajo se enfoca en la utilización del arte y de artistas finlandeses en el material creado para apoyar la promoción de la imagen del país y, en última instancia, su competividad económica. Para tener en cuenta el contexto del material objeto de estudio, presentamos el concepto de la marca país y distintos planteamientos teóricos para su realización. Siendo una estrategia estatal polémica, recorremos los puntos de crítica más importantes sobre esta práctica promocional. También nos familiarizamos con la historia de la promoción nacional antes de introducir el presente proyecto, la Marca Finlandia, y sus objetivos. En la parte teórica se presentan los conceptos relevantes para el estudio: el discurso y su análisis crítico, enfatizando su capacidad ideológica para mantener y crear relaciones del poder muy a menudo desiguales. Ya que nuestro material de estudio empírico está compuesto de publicaciones físicas con dimensiones textuales, visuales y hápticas, nuestro método de investigación es observar críticamente estas relaciones desde la perspectiva multimodal. En la parte empírica del trabajo analizamos la composición de la comunicación multimodal del material imprimido, producido por el Ministerio de Asuntos Exteriores de Finlandia y sus socios de cooperación. Las revistas anuales Focus– Economía y tecnología se enfocan en noticias y artículos sobre los sectores de la economía y tecnología, pero también incluyen contenido sobre el diseño, la música, la moda, la arquitectura y la danza finlandeses. En nuestro análisis recorremos los campos verbales y visuales utilizados en las revistas para investigar de qué manera se presentaban las artes finlandesas en la operación de Marca Finlandia. Detectamos representaciones textuales, resaltaciones visuales y la combinación de ambos, lo que servía a las metas preterminadas y la imagen requerida por la delegación diseñadora de la marca país. Las revistas compartían el discurso en común que corría paralelo con los objetivos de la Marca Finlandia, que a su vez se puede ver como parte del discurso hegemónico neoliberal en general.