A Cognitive Science Reasoning in Recognition of Emotions in Audio-Visual Speech

In this report we summarize the state-of-the-art of speech emotion recognition from the signal processing point of view. On the bases of multi-corporal experiments with machine-learning classifiers, the observation is made that existing approaches for supervised machine learning lead to database dependent classifiers which can not be applied for multi-language speech emotion recognition without additional training because they discriminate the emotion classes following the used training language. As there are experimental results showing that Humans can perform language independent categorisation, we made a parallel between machine recognition and the cognitive process and tried to discover the sources of these divergent results. The analysis suggests that the main difference is that the speech perception allows extraction of language independent features although language dependent features are incorporated in all levels of the speech signal and play as a strong discriminative function in human perception. Based on several results in related domains, we have suggested that in addition, the cognitive process of emotion-recognition is based on categorisation, assisted by some hierarchical structure of the emotional categories, existing in the cognitive space of all humans. We propose a strategy for developing language independent machine emotion recognition, related to the identification of language independent speech features and the use of additional information from visual (expression) features.

Re-visualising international relations:audio-visual projects and direct encounters with the political in security studies

In this paper we discuss how an innovative audio-visual project was adopted to foster active, rather than declarative learning, in critical International Relations (IR). First, we explore the aesthetic turn in IR, to contrast this with forms of representation that have dominated IR scholarship. Second, we describe how students were asked to record short audio or video projects to explore their own insights through aesthetic and non-written formats. Third, we explain how these projects are understood to be deeply embedded in social science methodologies. We cite our inspiration from applying a personal sociological imagination, as a way to counterbalance a ‘marketised’ slant in higher education, in a global economy where students are often encouraged to consume, rather than produce knowledge. Finally, we draw conclusions in terms of deeper forms of student engagement leading to new ways of thinking and presenting new skills and new connections between theory and practice.

Audio-Visual Information Processing across different age groups

People possess different sensory modalities to detect, interpret, and efficiently act upon various events in a complex and dynamic environment (Fetsch, DeAngelis, & Angelaki, 2013). Much empirical work has been done to understand the interplay of modalities (e.g. audio-visual interactions, see Calvert, Spence, & Stein, 2004). On the one hand, integration of multimodal input as a functional principle of the brain enables the versatile and coherent perception of the environment (Lewkowicz & Ghazanfar, 2009). On the other hand, sensory integration does not necessarily mean that input from modalities is always weighted equally (Ernst, 2008). Rather, when two or more modalities are stimulated concurrently, one often finds one modality dominating over another. Study 1 and 2 of the dissertation addressed the developmental trajectory of sensory dominance. In both studies, 6-year-olds, 9-year-olds, and adults were tested in order to examine sensory (audio-visual) dominance across different age groups. In Study 3, sensory dominance was put into an applied context by examining verbal and visual overshadowing effects among 4- to 6-year olds performing a face recognition task. The results of Study 1 and Study 2 support default auditory dominance in young children as proposed by Napolitano and Sloutsky (2004) that persists up to 6 years of age. For 9-year-olds, results on privileged modality processing were inconsistent. Whereas visual dominance was revealed in Study 1, privileged auditory processing was revealed in Study 2. Among adults, a visual dominance was observed in Study 1, which has also been demonstrated in preceding studies (see Spence, Parise, & Chen, 2012). No sensory dominance was revealed in Study 2 for adults. Potential explanations are discussed. Study 3 referred to verbal and visual overshadowing effects in 4- to 6-year-olds. The aim was to examine whether verbalization (i.e., verbally describing a previously seen face), or visualization (i.e., drawing the seen face) might affect later face recognition. No effect of visualization on recognition accuracy was revealed. As opposed to a verbal overshadowing effect, a verbal facilitation effect occurred. Moreover, verbal intelligence was a significant predictor for recognition accuracy in the verbalization group but not in the control group. This suggests that strengthening verbal intelligence in children can pay off in non-verbal domains as well, which might have educational implications.

Technology enhanced learning and augmented reality : an application on multimedia interactive books

Multimedia Interactive Book (miBook) reflects the development of a new concept of virtual interpretation of traditional text books and audio-visual content. By encompassing new technological approaches, using augmented reality technology, allows the final user to experience a variety of sensorial stimuli while enjoying and interacting with the content; therefore enhancing the learning process. miBook stands for a global educational intention to enable people not only to access but also to appropriate intellectually valuable contents coming from different linguistic and cultural contexts.

Televisão digital móvel e portátil: estudo de caso da televisão japonesa NHK

Pós-graduação em Televisão Digital: Informação e Conhecimento - FAAC

A estereoscopia: investigação de processos de aquisição, edição e exibição de imagens estereoscópicas em movimento

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Effects of visual and auditory stimuli in a choice reaction time task

The effect produced by a warning stimulus(i) (WS) in reaction time (RT) tasks is commonly attributed to a facilitation of sensorimotor mechanisms by alertness. Recently, evidence was presented that this effect is also related to a proactive inhibition of motor control mechanisms. This inhibition would hinder responding to the WS instead of the target stimulus (TS). Some studies have shown that auditory WS produce a stronger facilitatory effect than visual WS. The present study investigated whether the former WS also produces a stronger inhibitory effect than the latter WS. In one session, the RTs to a visual target in two groups of volunteers were evaluated. In a second session, subjects reacted to the visual target both with (50% of the trials) and without (50% of the trials) a WS. During trials, when subjects received a WS, one group received a visual WS and the other group was presented with an auditory WS. In the first session, the mean RTs of the two groups did not differ significantly. In the second session, the mean RT of the two groups in the presence of the WS was shorter than in their absence. The mean RT in the absence of the auditory WS was significantly longer than the mean RT in the absence of the visual WS. Mean RTs did not differ significantly between the present conditions of the visual and auditory WS. The longer RTs of the auditory WS group as opposed to the visual WS group in the WS-absent trials suggest that auditory WS exert a stronger inhibitory influence on responsivity than visual WS.

Die Überarbeitung der Fernsehrichtlinie unter Berücksichtigung der EuGH-Entscheidungen zur bisherigen Fassung

Gegenstand der vorliegenden Arbeit ist die Überarbeitung der Richtlinie 89/552/EWG des Rates zur Koordinierung bestimmter Rechts- und Verwaltungsvorschriften der Mitgliedstaaten über die Ausübung der Fernsehtätigkeit, welche aus praktikablen Gründen meist als „(EG-)Fernsehrichtlinie“ bezeichnet wird. Sie bildet den Eckpfeiler der audiovisuellen Politik der EU. Seit Erlass der Fernsehrichtlinie im Jahre 1989 bewirkt der technologische Fortschritt jedoch zunehmend enorme Veränderungen nicht nur im Bereich des klassischen Fernsehens, sondern auch und vor allem im Bereich der neuen Medien. Ausgangspunkt hierfür ist die Verbesserung der Digitaltechnologie, die ihrerseits wiederum technische Konvergenzprozesse begünstigt. Diese Entwicklungen führen nicht nur zu einer Vervielfachung von Übertragungskapazitäten und –techniken, sondern ermöglichen neben neuen Formen audiovisueller Angebote auch die Entstehung neuer Dienste. Unsere Medienlandschaft steht vor „epochalen Umbrüchen“. Im Hinblick auf diese Vorgänge wird seit geraumer Zeit eine Überarbeitung der EG-Fernsehrichtlinie angestrebt, um dem technologischen Fortschritt auch „regulatorisch“ gerecht werden zu können. Diesem Überarbeitungsprozess möchte sich die vorliegende Arbeit widmen, indem sie die Fernsehrichtlinie in einem ersten Teil sowohl inhaltlich wie auch hinsichtlich ihrer Entstehungsgeschichte und der zu ihr ergangenen EuGH-Entscheidungen erläutert. Anschließend werden alle Überarbeitungsvorgänge der Fernsehrichtlinie seit 1997 dargestellt, um sodann die aktuellen Reformansätze analysieren und bewerten zu können. Aus zeitlichen Gründen (der neue Richtlinienvorschlag der Kommission vom 13. Dezember 2005 wurde ca. 2 Wochen vor dem Abgabetermin der Arbeit verabschiedet) sind die Ausführungen zum Entwurf der neuen „Richtlinie über audiovisuelle Mediendienste“ allerdings relativ knapp gehalten.

Book Review: Regulating Code : Good Governance and Better Regulation in the Information Age

Digital technologies have profoundly changed not only the ways we create, distribute, access, use and re-use information but also many of the governance structures we had in place. Overall, "older" institutions at all governance levels have grappled and often failed to master the multi-faceted and multi-directional issues of the Internet. Regulatory entrepreneurs have yet to discover and fully mobilize the potential of digital technologies as an influential factor impacting upon the regulability of the environment and as a potential regulatory tool in themselves. At the same time, we have seen a deterioration of some public spaces and lower prioritization of public objectives, when strong private commercial interests are at play, such as most tellingly in the field of copyright. Less tangibly, private ordering has taken hold and captured through contracts spaces, previously regulated by public law. Code embedded in technology often replaces law. Non-state action has in general proliferated and put serious pressure upon conventional state-centered, command-and-control models. Under the conditions of this "messy" governance, the provision of key public goods, such as freedom of information, has been made difficult or is indeed jeopardized.The grand question is how can we navigate this complex multi-actor, multi-issue space and secure the attainment of fundamental public interest objectives. This is also the question that Ian Brown and Chris Marsden seek to answer with their book, Regulating Code, as recently published under the "Information Revolution and Global Politics" series of MIT Press. This book review critically assesses the bold effort by Brown and Marsden.

Short structured feedback training is equivalent to a mechanical feedback device in two-rescuer BLS: a randomised simulation study.

BACKGROUND Resuscitation guidelines encourage the use of cardiopulmonary resuscitation (CPR) feedback devices implying better outcomes after sudden cardiac arrest. Whether effective continuous feedback could also be given verbally by a second rescuer ("human feedback") has not been investigated yet. We, therefore, compared the effect of human feedback to a CPR feedback device. METHODS In an open, prospective, randomised, controlled trial, we compared CPR performance of three groups of medical students in a two-rescuer scenario. Group "sCPR" was taught standard BLS without continuous feedback, serving as control. Group "mfCPR" was taught BLS with mechanical audio-visual feedback (HeartStart MRx with Q-CPR-Technology™). Group "hfCPR" was taught standard BLS with human feedback. Afterwards, 326 medical students performed two-rescuer BLS on a manikin for 8 min. CPR quality parameters, such as "effective compression ratio" (ECR: compressions with correct hand position, depth and complete decompression multiplied by flow-time fraction), and other compression, ventilation and time-related parameters were assessed for all groups. RESULTS ECR was comparable between the hfCPR and the mfCPR group (0.33 vs. 0.35, p = 0.435). The hfCPR group needed less time until starting chest compressions (2 vs. 8 s, p < 0.001) and showed fewer incorrect decompressions (26 vs. 33 %, p = 0.044). On the other hand, absolute hands-off time was higher in the hfCPR group (67 vs. 60 s, p = 0.021). CONCLUSIONS The quality of CPR with human feedback or by using a mechanical audio-visual feedback device was similar. Further studies should investigate whether extended human feedback training could further increase CPR quality at comparable costs for training.

NARRATIVAS AUDIOVISUAIS NO PIAUI: Cultura e Sociedade na Linguagem do Cinema Universidade Metodista de São Paulo Programa de Pós-Graduação em Comunicação Social São Bernardo

O trabalho propõe uma investigação sobre a produção audiovisual piauiense, buscando entender como são negociadas as questões culturais e as especificidades das narrativas apresentadas no desenvolvimento histórico da cultura visual piauiense. Analisa-se, assim, as questões discursivas, imagéticas e tecnológicas que abordam aquela realidade cultural influenciada pelas vanguardas cinematográficas brasileiras, no caso o cinema novo e o cinema marginal. A pesquisa tem como corpus as produções superoitistas feitas a partir de 1972 até meados 1985, quando se encerra um segundo ciclo cinematográfico. Este estudo, de natureza qualitativa, emprega pesquisa bibliográfica e documental, com apoio em entrevistas e análise de documentos da época para o estudo das narrativas apresentadas pelas produções audiovisuais. Leva-se em consideração a influência das questões social, política, tecnológica e econômica do Piauí na construção desses filmes. Conclui-se que as práticas culturais e os recursos tecnológicos constituem uma cultura visual que representa as angústias e críticas locais e traduzem a tipificação do sujeito no Piauí.

Towards an Asynchronous Cinema: how can the asynchronous use of sound in artists' moving image underpin the creation of dialectic tension between the audio, the visual and the audience?

This PhD by publication examines selected practice-based audio-visual works made by the author over a ten-year period, placing them in a critical context. Central to the publications, and the focus of the thesis, is an exploration of the role of sound in the creation of dialectic tension between the audio, the visual and the audience. By first analysing a number of texts (films/videos and key writings) the thesis locates the principal issues and debates around the use of audio in artists’ moving image practice. From this it is argued that asynchronism, first advocated in 1929 by Pudovkin as a response to the advent of synchronised sound, can be used to articulate audio-visual relationships. Central to asynchronism’s application in this paper is a recognition of the propensity for sound and image to adhere, and in visual music for there to be a literal equation of audio with the visual, often married with a quest for the synaesthetic. These elements can either be used in an illusionist fashion, or employed as part of an anti-illusionist strategy for realising dialectic. Using this as a theoretical basis, the paper examines how the publications implement asynchronism, including digital mapping to facilitate innovative reciprocal sound and image combinations, and the asynchronous use of ‘found sound’ from a range of online sources to reframe the moving image. The synthesis of publications and practice demonstrates that asynchronism can both underpin the creation of dialectic, and be an integral component in an audio-visual anti-illusionist methodology.

DIDÁCTICA DE LA GEOGRAFÍA Y DIDÁCTICA SUPERIOR

El artículo reflexiona sobre el aporte de la didáctica en el proceso de enseñanza – aprendizaje, enfatizando en su carácter teórico – práctica.  Diferencia entre una didáctica general y una didáctica específica.La Didáctica Crítica propone al profesor en Geografía como un maestro en el  saber y actuar, en el cumplimiento de sus deberes, responsabilidades y habilidades.  La enseñanza de la Geografía precisadel apoyo de numerosos instrumentos de alta tecnología, que constituyen valiosos aportes didácticos, la  incorporación de  las Tecnologías de  Información y Comunicación  (TICʼs)    es una necesidad. La formación a nivel universitario debe ofrecer las destrezas  y habilidades en la creación y composiciónde medios audiovisuales para la enseñanza  de la Geografía.  La Geografía contribuye eficazmente al desarrollo de capacidades, actitudes y competencias genéricas esenciales sobre las que ha de basarse la formación ciudadana y la educación de valores.Palabras claves: Geografía, didáctica de  la Geografía, saber geográfico, metodología de enseñanza, método didáctico.AbstractThe article reflects on the contribution from didactics in the process of teaching-learning, emphasizing in its theoretical character – practice. It differentiates between general didactics and specific didactics. Critical didactics suggests that the Geography professor, as an expert, knows and acts according to his or her duties, responsibilities and abilities.The teaching of the Geography draws upon the support of high-tech tools, which constitute valuable didactic contributions, so that incorporation of the Technology Information and Communication (TICʼs) is a necessity to be the mode of communication. Formation at the university level ought to offer talents and abilities in the creation and composition of audio-visual media in the teaching of geography. Geography effectively contributes to development of capabilities, attitudes and competencies generic essential for citizen formation and the education of values.Key Words: Geography,  didactics  of Geography,  geographical  knowledge,  teaching methodology, didactic method

An exploration of varieties of visual attention: ERP findings

A set of five tasks was designed to examine dynamic aspects of visual attention: selective attention to color, selective attention to pattern, dividing and switching attention between color and pattern, and selective attention to pattern with changing target. These varieties of visual attention were examined using the same set of stimuli under different instruction sets; thus differences between tasks cannot be attributed to differences in the perceptual features of the stimuli. ERP data are presented for each of these tasks. A within-task analysis of different stimulus types varying in similarity to the attended target feature revealed that an early frontal selection positivity (FSP) was evident in selective attention tasks, regardless of whether color was the attended feature. The scalp distribution of a later posterior selection negativity (SN) was affected by whether the attended feature was color or pattern. The SN was largely unaffected by dividing attention across color and pattern. A large widespread positivity was evident in most conditions, consisting of at least three subcomponents which were differentially affected by the attention conditions. These findings are discussed in relation to prior research and the time course of visual attention processes in the brain. (C) 1999 Elsevier Science B.V. All rights reserved.

A utilização da plataforma Educast@fccn, como ferramenta de gravação de aulas no ambiente de e-learning português

Dissertação apresentada à Escola Superior de Comunicação Social como parte dos requisitos para obtenção de grau de mestre em Audiovisual e Multimédia.