El análisis de la música cinematográfica como modelo para la propia creación musical en el entorno audiovisual

Programa de doctorado: Formación del Profesorado

Sistemas de difusión, protocolos y plataformas digitales para documentación audiovisual : herramientas y recursos Web 2.0

Ponencia presentada en el Encuentro de patrimonio audiovisual de Gran Canaria (13 de julio al 31 de agosto de 2012. Las Palmas de Gran Canaria). Resumen: La producción de filmaciones y grabaciones sonoras ya no solo se realiza con medios digitales, sino que se transmite de igual modo. Al igual que otros entornos utilizados para preservar la memoria colectiva, el ámbito audiovisual es ya impensable sin los medios que, cada día más, se encuentran al alcance no sólo de los técnicos, sino de la población en general. Su calidad y acceso a los nuevos canales de distribución a través de Internet, particularmente mediante los entornos Web 2.0 permiten que la generación de contenidos y su difusión estén casi al alcance de cualquiera. Sin embargo, los conocimientos necesarios para favorecer su consulta sin barreras, las herramientas más adecuadas, los protocolos que hay que cumplir, el enfoque de esta producción como patrimonio y las políticas de preservación a largo plazo, sigue siendo en buena parte desconocidos.

Interacción basada en visión por computador mediante reacTIVision para la generación, control e interpretación de mensajes MIDI y su utilización en la creación audiovisual

[ES] Uno de los últimos avances en tecnología que más interés está despertando es la tecnología multitouch. Esta técnica ofrece una nueva forma de interacción hombre-máquina donde las aplicaciones gráficas son manejadas usando eventos táctiles. Esto abre la puerta a una comunicación más intuitiva y más directa entre el usuario y la máquina. Para lograr esto, son necesarios dos elementos: una superficie multitouch y una interfaz adaptada para el uso con eventos táctiles y objetos físicos que tendrán unos marcadores asignados. Con este proyecto buscamos desarrollar un dispositivo multitouch y un software para la creación de música digital mediante eventos táctiles y objetos. Para ello, es necesario investigar en distintos campos como son el del procesamiento de imágenes, hardware y desarrollo de aplicaciones e interfaces gráficas.

Desarrollo de material didáctico audiovisual para prácticas de laboratorio en el Proyecto PROMETEO-ULPGC: resistencia de materiales

[ES]En el contexto del proyecto de innovación educativa PROMETEO de la ULPGC, desde el área de Mecánica de los Medios Continuos y Teoría de Estructuras se han desarrollado materiales didácticos audiovisuales con la finalidad de facilitar el seguimiento y realización de varias prácticas de laboratorio en la materia de Resistencia de Materiales correspondientes a las titulaciones de ingeniería de la Escuela de Ingenierías Industriales y Civiles de la ULPGC, que se exponen en este trabajo.

Creazione e sviluppo di corpora multimediali. Nuove metodologie di ricerca nella traduzione audiovisiva

The construction and use of multimedia corpora has been advocated for a while in the literature as one of the expected future application fields of Corpus Linguistics. This research project represents a pioneering experience aimed at applying a data-driven methodology to the study of the field of AVT, similarly to what has been done in the last few decades in the macro-field of Translation Studies. This research was based on the experience of Forlixt 1, the Forlì Corpus of Screen Translation, developed at the University of Bologna’s Department of Interdisciplinary Studies in Translation, Languages and Culture. As a matter of fact, in order to quantify strategies of linguistic transfer of an AV product, we need to take into consideration not only the linguistic aspect of such a product but all the meaning-making resources deployed in the filmic text. Provided that one major benefit of Forlixt 1 is the combination of audiovisual and textual data, this corpus allows the user to access primary data for scientific investigation, and thus no longer rely on pre-processed material such as traditional annotated transcriptions. Based on this rationale, the first chapter of the thesis sets out to illustrate the state of the art of research in the disciplinary fields involved. The primary objective was to underline the main repercussions on multimedia texts resulting from the interaction of a double support, audio and video, and, accordingly, on procedures, means, and methods adopted in their translation. By drawing on previous research in semiotics and film studies, the relevant codes at work in visual and acoustic channels were outlined. Subsequently, we concentrated on the analysis of the verbal component and on the peculiar characteristics of filmic orality as opposed to spontaneous dialogic production. In the second part, an overview of the main AVT modalities was presented (dubbing, voice-over, interlinguistic and intra-linguistic subtitling, audio-description, etc.) in order to define the different technologies, processes and professional qualifications that this umbrella term presently includes. The second chapter focuses diachronically on various theories’ contribution to the application of Corpus Linguistics’ methods and tools to the field of Translation Studies (i.e. Descriptive Translation Studies, Polysystem Theory). In particular, we discussed how the use of corpora can favourably help reduce the gap existing between qualitative and quantitative approaches. Subsequently, we reviewed the tools traditionally employed by Corpus Linguistics in regard to the construction of traditional “written language” corpora, to assess whether and how they can be adapted to meet the needs of multimedia corpora. In particular, we reviewed existing speech and spoken corpora, as well as multimedia corpora specifically designed to investigate Translation. The third chapter reviews Forlixt 1's main developing steps, from a technical (IT design principles, data query functions) and methodological point of view, by laying down extensive scientific foundations for the annotation methods adopted, which presently encompass categories of pragmatic, sociolinguistic, linguacultural and semiotic nature. Finally, we described the main query tools (free search, guided search, advanced search and combined search) and the main intended uses of the database in a pedagogical perspective. The fourth chapter lists specific compilation criteria retained, as well as statistics of the two sub-corpora, by presenting data broken down by language pair (French-Italian and German-Italian) and genre (cinema’s comedies, television’s soapoperas and crime series). Next, we concentrated on the discussion of the results obtained from the analysis of summary tables reporting the frequency of categories applied to the French-Italian sub-corpus. The detailed observation of the distribution of categories identified in the original and dubbed corpus allowed us to empirically confirm some of the theories put forward in the literature and notably concerning the nature of the filmic text, the dubbing process and Italian dubbed language’s features. This was possible by looking into some of the most problematic aspects, like the rendering of socio-linguistic variation. The corpus equally allowed us to consider so far neglected aspects, such as pragmatic, prosodic, kinetic, facial, and semiotic elements, and their combination. At the end of this first exploration, some specific observations concerning possible macrotranslation trends were made for each type of sub-genre considered (cinematic and TV genre). On the grounds of this first quantitative investigation, the fifth chapter intended to further examine data, by applying ad hoc models of analysis. Given the virtually infinite number of combinations of categories adopted, and of the latter with searchable textual units, three possible qualitative and quantitative methods were designed, each of which was to concentrate on a particular translation dimension of the filmic text. The first one was the cultural dimension, which specifically focused on the rendering of selected cultural references and on the investigation of recurrent translation choices and strategies justified on the basis of the occurrence of specific clusters of categories. The second analysis was conducted on the linguistic dimension by exploring the occurrence of phrasal verbs in the Italian dubbed corpus and by ascertaining the influence on the adoption of related translation strategies of possible semiotic traits, such as gestures and facial expressions. Finally, the main aim of the third study was to verify whether, under which circumstances, and through which modality, graphic and iconic elements were translated into Italian from an original corpus of both German and French films. After having reviewed the main translation techniques at work, an exhaustive account of possible causes for their non-translation was equally provided. By way of conclusion, the discussion of results obtained from the distribution of annotation categories on the French-Italian corpus, as well as the application of specific models of analysis allowed us to underline possible advantages and drawbacks related to the adoption of a corpus-based approach to AVT studies. Even though possible updating and improvement were proposed in order to help solve some of the problems identified, it is argued that the added value of Forlixt 1 lies ultimately in having created a valuable instrument, allowing to carry out empirically-sound contrastive studies that may be usefully replicated on different language pairs and several types of multimedia texts. Furthermore, multimedia corpora can also play a crucial role in L2 and translation teaching, two disciplines in which their use still lacks systematic investigation.

Neurolinguistic analysis: aspects of language and speech deviations in Palestinian Arabic aphasics

The characteristics of aphasics’ speech in various languages have been the core of numerous studies, but Arabic in general, and Palestinian Arabic in particular, is still a virgin field in this respect. However, it is of vital importance to have a clear picture of the specific aspects of Palestinian Arabic that might be affected in the speech of aphasics in order to establish screening, diagnosis and therapy programs based on a clinical linguistic database. Hence the central questions of this study are what are the main neurolinguistic features of the Palestinian aphasics’ speech at the phonetic-acoustic level and to what extent are the results similar or not to those obtained from other languages. In general, this study is a survey of the most prominent features of Palestinian Broca’s aphasics’ speech. The main acoustic parameters of vowels and consonants are analysed such as vowel duration, formant frequency, Voice Onset Time (VOT), intensity and frication duration. The deviant patterns among the Broca’s aphasics are displayed and compared with those of normal speakers. The nature of deficit, whether phonetic or phonological, is also discussed. Moreover, the coarticulatory characteristics and some prosodic patterns of Broca’s aphasics are addressed. Samples were collected from six Broca’s aphasics from the same local region. The acoustic analysis conducted on a range of consonant and vowel parameters displayed differences between the speech patterns of Broca’s aphasics and normal speakers. For example, impairments in voicing contrast between the voiced and voiceless stops were found in Broca’s aphasics. This feature does not exist for the fricatives produced by the Palestinian Broca’s aphasics and hence deviates from data obtained for aphasics’ speech from other languages. The Palestinian Broca’s aphasics displayed particular problems with the emphatic sounds. They exhibited deviant coarticulation patterns, another feature that is inconsistent with data obtained from studies from other languages. However, several other findings are in accordance with those reported from various other languages such as impairments in the VOT. The results are in accordance with the suggestions that speech production deficits in Broca’s aphasics are not related to phoneme selection but rather to articulatory implementation and some speech output impairments are related to timing and planning deficits.

The effect of arts speech therapy on cerebral oxygenation and low frequency hemodynamic oscillations measured using functional near-infrared spectroscopy

Introduction: As a previous study revealed, arts speech therapy (AST) affects cardiorespiratory interaction [1]. The aim of the present study was to investigate whether AST also has effects on brain oxygenation and hemodynamics measured non-invasively using near-infrared spectroscopy (NIRS). Material and methods: NIRS measurements were performed on 17 subjects (8 men and 9 women, mean age: 35.6 ± 12.7 y) during AST. Each measurement lasted 35 min, comprising 8 min pre-baseline, 10 min recitation and 20 min post-baseline. For each subject, measurements were performed for three different AST recitation tasks (recitation of alliterative, hexameter and prose verse). Relative concentration changes of oxyhemoglobin (Δ[O2Hb]) and deoxyhemoglobin (Δ[HHb]) as well as the tissue oxygenation index (TOI) were measured using a Hamamatsu NIRO300 NIRS device and a sensor placed on the subjects forehead. Movement artifacts were removed using a novel method [2]. Statistical analysis (Wilcoxon test) was applied to the data to investigate (i) if the recitation causes changes in the median values and/or in the Mayer wave power spectral density (MW-PSD, range: 0.07–0.13 Hz) of Δ[O2Hb], Δ[HHb] or TOI, and (ii) if these changes vary between the 3 recitation forms. Results: For all three recitation styles a significant (p < 0.05) decrease in Δ[O2Hb] and TOI was found, indicating a decrease in blood flow. These decreases did not vary significantly between the three styles. MW-PSD increased significantly for Δ[O2Hb] when reciting the hexameter and prose verse, and for Δ[HHb] and TOI when reciting alliterations and hexameter, representing an increase in Mayer waves. The MW-PSD increase for Δ[O2Hb] was significantly larger for the hexameter verse compared to alliterative and prose verse Conclusion: The study showed that AST affects brain hemodynamics (oxygenation, blood flow and Mayer waves). Recitation caused a significant decrease in cerebral blood flow for all recitation styles as well as an increase in Mayer waves, particularly for the hexameter, which may indicate a sympathetic activation. References 1. D. Cysarz, D. von Bonin, H. Lackner, P. Heusser, M. Moser, H. Bettermann. Am J Physiol Heart Circ Physiol, 287 (2) (2004), pp. H579–H587 2. F. Scholkmann, S. Spichtig, T. Muehlemann, M. Wolf. Physiol Meas, 31 (5) (2010), pp. 649–662

The near babylonian speech confusion in early detection of psychosis

It is barely 15 years since, in 1996, the issue theme of Schizophrenia Bulletin (Vol 22, 2) “Early Detection, and Intervention in Schizophrenia” signified the commencement of this field of research. Since that time the field of early detection research has developed rapidly and it may be translated into clinical practice by the introduction of an Attenuated Psychosis Syndrome in Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, (DSM-5) (www.dsm5.org/ProposedRevisions/Pages/proposedrevision.aspx?rid=412#). Attenuated psychotic symptoms (APS) had first been suggested as a clinical predictor of first-episode psychosis by the Personal Assessment and Crisis Evaluation (PACE) Clinic group as part of the ultrahigh risk (UHR) criteria.1 The term ultrahigh risk became broadly accepted for this set of criteria for imminent risk of developing psychosis in the late 1990s. The use of the term “prodrome” for a state characterized by at-risk (AR) criteria was criticized as a retrospective concept inevitably followed by the full-blown disorder.1 Although alternative terms have been suggested, prodrome is still used in prospective studies (eg, prodromally symptomatic, potentially or putatively prodromal, prodrome-like state/symptoms). Some alternative suggestions such as prepsychotic state/symptoms, subthreshold psychotic symptoms, early psychosis, subsyndromal psychosis, hypopsychosis, or subpsychosis were short-lived. Other terms still in use include UHR, at-risk mental state (ARMS), AR, high risk, clinical high risk (CHR), or early and late AR state. Further, the term psychotic-like experiences (PLEs) has recently (re-)entered early detection research. …

Effects and limitations of an AED with audiovisual feedback for cardiopulmonary resuscitation: a randomized manikin study

Correctly performed basic life support (BLS) and early defibrillation are the most effective measures to treat sudden cardiac arrest. Audiovisual feedback improves BLS. Automated external defibrillators (AED) with feedback technology may play an important role in improving CPR quality. The aim of this simulation study was to investigate if an AED with audiovisual feedback improves CPR parameters during standard BLS performed by trained laypersons.

Speech Enhancement With Inventory Style Speech Resynthesis

We present a new method for the enhancement of speech. The method is designed for scenarios in which targeted speaker enrollment as well as system training within the typical noise environment are feasible. The proposed procedure is fundamentally different from most conventional and state-of-the-art denoising approaches. Instead of filtering a distorted signal we are resynthesizing a new “clean” signal based on its likely characteristics. These characteristics are estimated from the distorted signal. A successful implementation of the proposed method is presented. Experiments were performed in a scenario with roughly one hour of clean speech training data. Our results show that the proposed method compares very favorably to other state-of-the-art systems in both objective and subjective speech quality assessments. Potential applications for the proposed method include jet cockpit communication systems and offline methods for the restoration of audio recordings.

Assessment of speech perception and communication skills in adolescents with cochlear implants for pre- and peri-lingual deafness

This study aimed to assess speech perception and communication skills in adolescents between ages 8 and 18 that received cochlear implants for pre- and peri-lingual deafness.

Variation of vector quantisation and speech waveform coding

A new idea for waveform coding using vector quantisation (VQ) is introduced. This idea makes it possible to deal with codevectors much larger than before for a fixed bit per sample rate. Also a solution to the matching problem (inherent in the present context) in the &-norm describing a measure of neamess is presented. The overall computational complexity of this solution is O(n3 log, n). Sample results are presented to demonstrate the advantage of using this technique in the context of coding of speech waveforms.