Auditory-visual speech integration by prelinguistic infants: Perception of an emergent consonant in the McGurk effect

The McGurk effect, in which auditory [ba] dubbed onto [go] lip movements is perceived as da or tha, was employed in a real-time task to investigate auditory-visual speech perception in prelingual infants. Experiments 1A and 1B established the validity of real-time dubbing for producing the effect. In Experiment 2, 4(1)/(2)-month-olds were tested in a habituation-test paradigm, in which 2 an auditory-visual stimulus was presented contingent upon visual fixation of a live face. The experimental group was habituated to a McGurk stimulus (auditory [ba] visual [ga]), and the control group to matching auditory-visual [ba]. Each group was then presented with three auditory-only test trials, [ba], [da], and [deltaa] (as in then). Visual-fixation durations in test trials showed that the experimental group treated the emergent percept in the McGurk effect, [da] or [deltaa], as familiar (even though they had not heard these sounds previously) and [ba] as novel. For control group infants [da] and [deltaa] were no more familiar than [ba]. These results are consistent with infants'perception of the McGurk effect, and support the conclusion that prelinguistic infants integrate auditory and visual speech information. (C) 2004 Wiley Periodicals, Inc.

Towards an Asynchronous Cinema: how can the asynchronous use of sound in artists' moving image underpin the creation of dialectic tension between the audio, the visual and the audience?

This PhD by publication examines selected practice-based audio-visual works made by the author over a ten-year period, placing them in a critical context. Central to the publications, and the focus of the thesis, is an exploration of the role of sound in the creation of dialectic tension between the audio, the visual and the audience. By first analysing a number of texts (films/videos and key writings) the thesis locates the principal issues and debates around the use of audio in artists’ moving image practice. From this it is argued that asynchronism, first advocated in 1929 by Pudovkin as a response to the advent of synchronised sound, can be used to articulate audio-visual relationships. Central to asynchronism’s application in this paper is a recognition of the propensity for sound and image to adhere, and in visual music for there to be a literal equation of audio with the visual, often married with a quest for the synaesthetic. These elements can either be used in an illusionist fashion, or employed as part of an anti-illusionist strategy for realising dialectic. Using this as a theoretical basis, the paper examines how the publications implement asynchronism, including digital mapping to facilitate innovative reciprocal sound and image combinations, and the asynchronous use of ‘found sound’ from a range of online sources to reframe the moving image. The synthesis of publications and practice demonstrates that asynchronism can both underpin the creation of dialectic, and be an integral component in an audio-visual anti-illusionist methodology.

Preperceptual and stimulus-selective enhancement of low-level human visual cortex excitability by sounds.

Evidence of multisensory interactions within low-level cortices and at early post-stimulus latencies has prompted a paradigm shift in conceptualizations of sensory organization. However, the mechanisms of these interactions and their link to behavior remain largely unknown. One behaviorally salient stimulus is a rapidly approaching (looming) object, which can indicate potential threats. Based on findings from humans and nonhuman primates suggesting there to be selective multisensory (auditory-visual) integration of looming signals, we tested whether looming sounds would selectively modulate the excitability of visual cortex. We combined transcranial magnetic stimulation (TMS) over the occipital pole and psychophysics for "neurometric" and psychometric assays of changes in low-level visual cortex excitability (i.e., phosphene induction) and perception, respectively. Across three experiments we show that structured looming sounds considerably enhance visual cortex excitability relative to other sound categories and white-noise controls. The time course of this effect showed that modulation of visual cortex excitability started to differ between looming and stationary sounds for sound portions of very short duration (80 ms) that were significantly below (by 35 ms) perceptual discrimination threshold. Visual perceptions are thus rapidly and efficiently boosted by sounds through early, preperceptual and stimulus-selective modulation of neuronal excitability within low-level visual cortex.

Introducción a la conversión de audio a texto

Este documento es una introducción a las herramientas Dragon Naturally Speaking y Audacity, especializadas en optimizar la transcripción de archivos sonoros.

Réadaptation et performance visuelle chez la personne hémianopsique : une étude de cas portant sur les saccades oculaires et le blindsight.

But: La perte unilatérale du cortex visuel postérieur engendre une cécité corticale controlatérale à la lésion, qu’on appelle hémianopsie homonyme (HH). Celle-ci est notamment accompagnée de problèmes d’exploration visuelle dans l’hémichamp aveugle dus à des stratégies oculaires déficitaires, qui ont été la cible des thérapies de compensation. Or, cette perte de vision peut s’accompagner d’une perception visuelle inconsciente, appelée blindsight. Notre hypothèse propose que le blindsight soit médié par la voie rétino-colliculaire extrastriée, recrutant le colliculus supérieur (CS), une structure multisensorielle. Notre programme a pour objectif d’évaluer l’impact d’un entraînement multisensoriel (audiovisuel) sur la performance visuelle inconsciente des personnes hémianopsiques et les stratégies oculaires. Nous essayons, ainsi, de démontrer l’implication du CS dans le phénomène de blindsight et la pertinence de la technique de compensation multisensorielle comme thérapie de réadaptation. Méthode: Notre participante, ML, atteinte d’une HH droite a effectué un entraînement d’intégration audiovisuel pour une période de 10 jours. Nous avons évalué la performance visuelle en localisation et en détection ainsi que les stratégies oculaires selon trois comparaisons principales : (1) entre l’hémichamp normal et l’hémichamp aveugle; (2) entre la condition visuelle et les conditions audiovisuelles; (3) entre les sessions de pré-entraînement, post-entraînement et 3 mois post-entraînement. Résultats: Nous avons démontré que (1) les caractéristiques des saccades et des fixations sont déficitaires dans l’hémichamp aveugle; (2) les stratégies saccadiques diffèrent selon les excentricités et les conditions de stimulations; (3) une adaptation saccadique à long terme est possible dans l’hémichamp aveugle si l’on considère le bon cadre de référence; (4) l’amélioration des mouvements oculaires est liée au blindsight. Conclusion(s): L’entraînement multisensoriel conduit à une amélioration de la performance visuelle pour des cibles non perçues, tant en localisation qu’en détection, ce qui est possiblement induit par le développement de la performance oculomotrice.

Geografia humana, recerca de suports audio-visuals : la ciutat. 'Geografía humana, búsqueda de soportes audio-visuales : la ciudad'.

Crear un material audio-visual. Mejorar la calidad de la enseñanza. Estudiar la aplicación de programas audio-visuales en el aula. Buscar una metodología adecuada a la utilización didáctica de los medios audio-visuales. Comprobar las diferencias que pueden existir entre diferentes medios audio-visuales, diapositivas-vídeo. La muestra está formada por los niños de tres aulas de segundo de BUP del Colegio Escoles Pies de Sarrià (Barcelona). En total 102 sujetos que han estudiado primero de BUP en el mismo centro. Se expone el marco teórico. Se describen las variables (medios audio-visuales, rendimiento escolar, rendimiento escolar anterior, metodología, inteligencia, clase social, profesor y edad). Se describe la muestra. División de la muestra en tres clases (sin medio audio-visual, con vídeo, con diapositivas). Realización del material audio-visual. Se realizan las sesiones pertinentes en cada clase. Aplicación de la prueba objetiva. Se analizan los datos. Se ofrecen conclusiones y alternativas. Prueba objetiva de rendimiento. Test d'aptituds diferencials. Baremo de puntuaciones anteriores. Diferencia de medias, estadística descriptiva, análisis de varianza, prueba de Scheffe, para establecer si hay diferencias entre el grupo que ha trabajado con medio audio-visual, visual y sin medio audiovisual. La metodología experimental aplicada no ha producido los resultados esperados, hay razones para afirmar que han intervenido factores no controlados, ajenos a la experimentación. Se constata un gran interés de los alumnos por el uso del vídeo como elemento de motivación. Se señala la importancia de incidir en este campo creando metodologías activas adecuadas y series de programas válidos. Hace falta una intensa investigación en las posibilidades y efectos de dichas metodologías.

Auditory, visual, and auditory-visual recognition of vowels by hearing-impaired children

This paper studies the auditory, visual and combined audio-visual recognition of vowels by severely and profoundly hearing impaired children.

Filmljudets funktioner i krigsfilm : En audio-visuell analys av Apocalypse Now och Saving Private Ryan

I denna uppsats har filmljudet i krigsfilmerna Apocalypse Now och Saving Private Ryan undersökts. Detta har gjorts för att försöka bidra med ökad förståelse för filmljudets användningsområde och funktioner, främst för filmerna i fråga, men även för krigsfilm rent generellt. Filmljud i denna kontext omfattar allt det ljud som finns i film, men utesluter dock all ickediegetisk musik. Båda filmerna har undersökts genom en audio-visuell analys. En sådan analys görs genom att detaljgranska båda filmernas ljud- och bildinnehåll var för sig, för att slutligen undersöka samma filmsekvens som helhet då ljudet och bilden satts ihop igen. Den audio-visuella analysmetod som nyttjats i uppsatsen är Michel Chions metod, Masking. De 30 minuter film som analyserades placerades sedan i olika filmljudzoner, där respektive filmljudzons ljudinnehåll bland annat visade vilka främsta huvudfunktioner somfilmljudet hade i dessa filmer. Dessa funktioner är till för att bibehålla åskådarens fokus och intresse, att skapa närhet till rollkaraktärerna, samt att tillföra en hög känsla av realism och närvaro. Intentionerna med filmljudet verkade vara att flytta åskådaren in i filmens verklighet, att låta åskådaren bli ett med filmen. Att återspegla denna känsla av realism, närvaro, fokus samt intresse, visade sig också vara de intentioner som funnits redan i de båda filmernas förproduktionsstadier. Detta bevisar att de lyckats åstadkomma det de eftersträvat. Men om filmljudet använts på samma sätt eller innehar samma funktioner i krigsfilm rent genrellt går inte att säga.I have for this bachelor’s thesis examined the movie sound of the classic warfare movies Apocalypse Now and Saving Private Ryan. This is an attempt to contribute to a more profound comprehension of the appliance and importance of movie sound. In this context movie sound implies all kinds of sounds within the movies, accept from non-diegetic music. These two movies have been examined by an audio-visual analysis. It's done by auditing the sound and picture content separately, and then combined to audit the same sequence as a whole. Michel Chion, which is the founder of this analysis, calls this method Masking. The sound in this 30 minute sequence was then divided into different zones, where every zone represented a certain main function. These functions are provided to create a stronger connection to the characters, sustain the viewers interest and bring a sense of realism and presence. It seems though the intention with the movies sound is to bring the viewers to the scene in hand, and let it become their reality. To mirror this sense of realism, presence, focus and interest, proves to be the intention from an early stage of the production. This bachelor’s thesis demonstrates a success in their endeavours. Although it can’t confirm whether the movie sound have been utilized in the same manner or if they posess the same functions to warefare movies in general.

A comunicação como barreira à inclusão de alunos com deficiência visual em aulas de óptica

O presente artigo representa uma continuidade dos resultados apresentados em Camargo e Nardi (Revista Brasileira de Ensino de Física 29, 117 (2007)). Encontra-se inserido dentro de um estudo que busca compreender as principais barreiras para a inclusão de alunos com deficiência visual no contexto do ensino de física. Focalizando aulas de óptica, analisa as dificuldades comunicacionais entre licenciandos e discentes com deficiência visual. Para tal, enfatiza as estruturas empírica e semântico-sensorial das linguagens utilizadas, indicando fatores geradores de dificuldades de acessibilidade nas informações veiculadas. Recomenda, ainda, alternativas que visam dar condições à participação efetiva do discente com deficiência visual no processo comunicativo, das quais destacam-se: a identificação da estrutura semântico-sensorial dos significados veiculados, o conhecimento da história visual do aluno, a destituição da estrutura empírica audiovisual interdependente e a exploração das potencialidades comunicacionais das linguagens constituídas de estruturas empíricas de acesso visualmente independente. Conclui afirmando que a comunicação representa a principal barreira à participação efetiva de alunos com deficiência visual em aulas de óptica e enfatiza a importância da criação de canais comunicacionais adequados como condição básica à inclusão desses alunos.

Eficácia do programa de remediação auditivo-visual computadorizado em escolares com dislexia

TEMA: programa de remediação auditivo-visual computadorizado em escolares com dislexia do desenvolvimento. OBJETIVOS: verificar a eficácia de um programa de remediação auditivo-visual computadorizado em escolares com dislexia do desenvolvimento. Dentre os objetivos específicos, o estudo teve como finalidade comparar o desempenho cognitivo-lingüístico de escolares com dislexia do desenvolvimento com escolares bons leitores; comparar os achados dos procedimentos de avaliação de pré e pós testagem em escolares com dislexia submetidos e não submetidos ao programa; e, por fim, comparar os achados do programa de remediação em escolares com dislexia e escolares bons leitores submetidos ao programa de remediação. MÉTODO: participaram deste estudo 20 escolares, sendo o grupo I (GI) subdivido em: GIe, composto de cinco escolares com dislexia do desenvolvimento submetidos ao programa, e GIc, composto de cinco escolares com dislexia do desenvolvimento não submetidos ao programa. O grupo II (GII) foi subdividido em GIIe, composto de cinco escolares bons leitores submetidos à remediação, e GIIc, composto de cinco escolares bons leitores não submetidos à remediação. Foi realizado o programa de remediação auditivo-visual computadorizado Play-on. RESULTADOS: os resultados deste estudo revelaram que o GI apresentou desempenho inferior em habilidade de processamento auditivo e de consciência fonológica em comparação com o GII em situação de pré-testagem. Entretanto, o GIe apresentou desempenho semelhante ao GII em situação de pós-testagem, evidenciando a eficácia da remediação auditivo-visual em escolares com dislexia do desenvolvimento. CONCLUSÃO: o estudo evidenciou a eficácia do programa de remediação auditivo-visual em escolares com dislexia do desenvolvimento.

TV digital sob a perspectiva publicitária: uma análise do advertainment na narrativa audiovisual

Pós-graduação em Televisão Digital: Informação e Conhecimento - FAAC

Contextos comunicacionais adequados e inadequados à inclusão de alunos com deficiência visual em aulas de mecânica

This article is inserted in a study aimed at the identification of the main barriers for the inclusion of visually-impaired students in Physics classes. It focuses on the understanding of the communication context which facilitates or hardens the effective participation of students with visual impairment in Mechanics activities. To do so, the research defines, from empirical - sensory and semantic structures, the language to be applied in the activities, as well as, the moment and the speech pattern in which the languages have been used. As a result, it identifies the rela tion between the uses of the interdependent audio-visual empirical lan guage structure in the non-interactive episodes of authority; the decrease in the use of this structure in interactive episodes; the creation of educa tional segregation environments within the classroom and the frequent use of the interdependent tactile-hearing empirical language structure in such environments.

Contextos comunicacionais adequados e inadequados à inclusão de alunos com deficiência visual em aulas de termologia

This article represents a continuation of the results of a research presented in Camargo and Nardi (2007). It is inserted in the study that seeks to understand the main student’s inclusion barriers with visual impairment in the Physics classes. It aims to understand which communication context shows kindness or unkindness to the impairment visual student’s real participation in thermology activities. For this, the research defines, from the empirical - sensory and semantics structures, the used languages in the activities, as well, the moment and the speech pattern in which the languages have been used. As result, identifies a strong relation between the uses of the interdependent empirical structure audio-visual language in the non-interactive episodes of authority; a decrease of this structure use in the interactive episodes and the creation of education segregation environments within the classroom.

Contextos comunicacionales adecuados e inadecuados para la inclusión de alumnos con discapacidad visual en clases de física moderna

This article is inserted in a wider study that seeks to understand the main inclusion barriers in Physics classes for students with visual impairment It aims to understand which communication context favors or impedes the visually impaired student participation to the impairment visual student’s real participation in Modern Physics activities. The research defines, from the empirical-sensory and semantics structures, the languages used in the activities, as well as, the moment and the speech pattern in which those languages have been used. As a result, this study identifies a strong relation between the uses of the interdependent empirical structure audio-visual language in the non-interactive episodes of authority; a decrease of this structure use in the interactive episodes; the creation of education segregation environments within the clasroom and the frequent use of empirical tactile-hearing interdependent language structure in these environments. Moreover, the concept of «special educational need» is discussed and its inadequate use is analyzed. Suggestions are given for its correct use of «special educational need,» its inadequate use, giving suggestions for its correct use.

Ensino de Física para alunos com deficiência visual: o processo de ensino-aprendizagem nos ambientes escolares das salas de aula regular e de recursos

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)