Entre o sonoro e o visual: relações e processos nas artes

Pós-graduação em Artes - IA

Towards an Asynchronous Cinema: how can the asynchronous use of sound in artists' moving image underpin the creation of dialectic tension between the audio, the visual and the audience?

This PhD by publication examines selected practice-based audio-visual works made by the author over a ten-year period, placing them in a critical context. Central to the publications, and the focus of the thesis, is an exploration of the role of sound in the creation of dialectic tension between the audio, the visual and the audience. By first analysing a number of texts (films/videos and key writings) the thesis locates the principal issues and debates around the use of audio in artists’ moving image practice. From this it is argued that asynchronism, first advocated in 1929 by Pudovkin as a response to the advent of synchronised sound, can be used to articulate audio-visual relationships. Central to asynchronism’s application in this paper is a recognition of the propensity for sound and image to adhere, and in visual music for there to be a literal equation of audio with the visual, often married with a quest for the synaesthetic. These elements can either be used in an illusionist fashion, or employed as part of an anti-illusionist strategy for realising dialectic. Using this as a theoretical basis, the paper examines how the publications implement asynchronism, including digital mapping to facilitate innovative reciprocal sound and image combinations, and the asynchronous use of ‘found sound’ from a range of online sources to reframe the moving image. The synthesis of publications and practice demonstrates that asynchronism can both underpin the creation of dialectic, and be an integral component in an audio-visual anti-illusionist methodology.

Tempêtes. Composition audiovisuelle.

La version intégrale de cette thèse est disponible uniquement pour consultation individuelle à la Bibliothèque de musique de l’Université de Montréal (www.bib.umontreal.ca/MU ).

De la musique au-delà des frontières du son

La musique aujourd’hui est régulièrement accompagnée d’environnements visuels. Depuis les propositions en provenance du vidéoclip jusqu’aux œuvres installatives en passant par l’art web ou le cinéma, l’audiovisuel occupe une place considérable dans notre monde médiatisé et constitue un foyer important du développement des pratiques musicales. L’alliage entre son et image est souvent rattachée à l’histoire du cinéma mais les prémisses entourant l’audiovisuel remontent en réalité à l’Antiquité. Les correspondances entre sons et couleurs ont pris racine en premier chez les Pythagoriciens et cet intérêt se poursuit encore aujourd’hui. L’avènement de différentes technologies est venu reformuler au fil des siècles cette recherche qui retourne du décloisonnement artistique. L’arrivée de l’électricité permet au XIXe siècle le développement d’une lutherie expérimentale avec entre autres l’orgue à couleur d’Alexander Rimington. Ces instruments audiovisuels donnent naissance plus tard au Lumia, un art de la couleur et du mouvement se voulant proche de la musique et qui ne donne pourtant rien à entendre. Parallèlement à ces nouvelles propositions artistiques, il se développe dès les tout début du XXe siècle au sein des avant-gardes cinématographiques un corpus d’œuvres qui sera ensuite appelé musique visuelle. Les possibilités offertes par le support filmique vient offrir de nouvelles possibilités quant à l’organisation de la couleur et du mouvement. La pratique de cet art hybride est ensuite reformulée par les artistes associés à l’art vidéo avant de connaitre une vaste phase de démocratisation avec l’avènement des ordinateurs domestiques depuis les années 1990. Je retrace le parcours historique de ces pratiques audiovisuelles qui s’inscrivent résolument sur le terrain du musical. Un parcours appuyé essentiellement sur des œuvres et des ouvrages théoriques tout en étant parsemé de réflexions personnelles. Je traite des enjeux théoriques associés à ces propositions artistiques en les différenciant d’un autre format audiovisuel majeur soit le cinéma. Cet exposé permet de préparer le terrain afin de présenter et contextualiser mon travail de création. Je traite de deux œuvres, Trombe (2011) et Lungta (2012), des propositions qui héritent à la fois des musiques visuelles, de l’art interactif et de l’art cinétique.

Music Notes: Exploring Music Listening Data as a Visual Representation of Self

Thesis (Master's)--University of Washington, 2016-06

Seeing music performance: Visual influences on perception and experience

Drawing from ethnographic, empirical, and historical/cultural perspectives, we examine the extent to which visual aspects of music contribute to the communication that takes place between performers and their listeners. First, we introduce a framework for understanding how media and genres shape aural and visual experiences of music. Second, we present case studies of two performances, and describe the relation between visual and aural aspects of performance. Third, we report empirical evidence that visual aspects of performance reliably influence perceptions of musical structure (pitch related features) and affective interpretations of music. Finally, we trace new and old media trajectories of aural and visual dimensions of music, and highlight how our conceptions, perceptions and appreciation of music are intertwined with technological innovation and media deployment strategies.

Natação para crianças portadoras de deficiencia visual : uma proposta de ensino

Universidade Estadual de Campinas. Faculdade de Educação Física

Time processing in visual cortices: How the visual brain encodes and keeps track of time

Time is embedded in any sensory experience: the movements of a dance, the rhythm of a piece of music, the words of a speaker are all examples of temporally structured sensory events. In humans, if and how visual cortices perform temporal processing remains unclear. Here we show that both primary visual cortex (V1) and extrastriate area V5/MT are causally involved in encoding and keeping time in memory and that this involvement is independent from low-level visual processing. Most importantly we demonstrate that V1 and V5/MT are functionally linked and temporally synchronized during time encoding whereas they are functionally independent and operate serially (V1 followed by V5/MT) while maintaining temporal information in working memory. These data challenge the traditional view of V1 and V5/MT as visuo-spatial features detectors and highlight the functional contribution and the temporal dynamics of these brain regions in the processing of time in millisecond range. The present project resulted in the paper entitled: 'How the visual brain encodes and keeps track of time' by Paolo Salvioni, Lysiann Kalmbach, Micah Murray and Domenica Bueti that is now submitted for publication to the Journal of Neuroscience.

How the visual brain encodes and keeps track of time.

Time is embedded in any sensory experience: the movements of a dance, the rhythm of a piece of music, the words of a speaker are all examples of temporally structured sensory events. In humans, if and how visual cortices perform temporal processing remains unclear. Here we show that both primary visual cortex (V1) and extrastriate area V5/MT are causally involved in encoding and keeping time in memory and that this involvement is independent from low-level visual processing. Most importantly we demonstrate that V1 and V5/MT come into play simultaneously and seem to be functionally linked during interval encoding, whereas they operate serially (V1 followed by V5/MT) and seem to be independent while maintaining temporal information in working memory. These data help to refine our knowledge of the functional properties of human visual cortex, highlighting the contribution and the temporal dynamics of V1 and V5/MT in the processing of the temporal aspects of visual information.

The Arts (Music, Arts&Crafts and Drama) as a procedure for learning pre-school content: Grupo Edebé Project

There is nothing as amazing and fascinating as children learning process. Between 0 and 6 years old, a child brain develops in a waythat will never be repeated. At this age, children are eager to discover and they have great potential of active and affective life.Because of this, their learning capacity in this period is incalculable. (Jordan-Decarbo y Nelson, 2002; Wild, 1999).Pre-school Education is a unique and special stage, with self identity, which aims are:attending children as a whole,motivate them to learn,give them an affective and stable environment in which they can grow up and get to be balanced and confident people and inwhich they can relate to others, learn, enjoy and be happy.Arts, Music, Visual Arts and Drama (Gardner, 1994) can provide a framework of special, even unique, personal expression.With the aim of introducing qualitative improvements in the education of children and to ensure their emotional wellbeing, and havingnoticed that teachers had important needs and concerns as regards to diversity in their student groups, we developed a programbased on the detection of needs and concerns explained by professionals in education.This program of Grupo edebé, object of our research, is a multicultural, interdisciplinary and globalizing project the aims of which are:developing children's talent and personality,keeping their imagination and creativity and using these as a learning resource,promoting reasoning, favouring expression and communication,providing children with the tools to manage their emotions,and especially, introducing Arts as a procedure to increase learning.We wanted to start the research by studying the impact (Brice, 2003) that this last point had on the learning of five-year-old childrenschooled in multicultural environments.Therefore, the main goal of the research was the assessment of the implementation of a child education programme attending todiversity in a population of five-year-old children, specifically in the practice of procedures based on the use of Arts (music, arts andcrafts and theatre) as a vehicle or procedure for learning contents in Pre-school stage.Because children emotional welfare was a subject of our concern, and bearing in mind that the affective aspects are of vitalimportance for learning and child development (Parke and Gauvain, 2009), Grupo Edebé has also evaluated the starting, evolving andfinal impact in five-year-old children given that they finish Pre-school education at that age.

Skill Development in Music Reading. The Eye-Movement Approach

This dissertation examined skill development in music reading by focusing on the visual processing of music notation in different music-reading tasks. Each of the three experiments of this dissertation addressed one of the three types of music reading: (i) sight-reading, i.e. reading and performing completely unknown music, (ii) rehearsed reading, during which the performer is already familiar with the music being played, and (iii) silent reading with no performance requirements. The use of the eye-tracking methodology allowed the recording of the readers’ eye movements from the time of music reading with extreme precision. Due to the lack of coherence in the smallish amount of prior studies on eye movements in music reading, the dissertation also had a heavy methodological emphasis. The present dissertation thus aimed to promote two major issues: (1) it investigated the eye-movement indicators of skill and skill development in sight-reading, rehearsed reading and silent reading, and (2) developed and tested suitable methods that can be used by future studies on the topic. Experiment I focused on the eye-movement behaviour of adults during their first steps of learning to read music notation. The longitudinal experiment spanned a nine-month long music-training period, during which 49 participants (university students taking part in a compulsory music course) sight-read and performed a series of simple melodies in three measurement sessions. Participants with no musical background were entitled as “novices”, whereas “amateurs” had had musical training prior to the experiment. The main issue of interest was the changes in the novices’ eye movements and performances across the measurements while the amateurs offered a point of reference for the assessment of the novices’ development. The experiment showed that the novices tended to sight-read in a more stepwise fashion than the amateurs, the latter group manifesting more back-and-forth eye movements. The novices’ skill development was reflected by the faster identification of note symbols involved in larger melodic intervals. Across the measurements, the novices also began to show sensitivity to the melodies’ metrical structure, which the amateurs demonstrated from the very beginning. The stimulus melodies consisted of quarter notes, making the effects of meter and larger melodic intervals distinguishable from effects caused by, say, different rhythmic patterns. Experiment II explored the eye movements of 40 experienced musicians (music education students and music performance students) during temporally controlled rehearsed reading. This cross-sectional experiment focused on the eye-movement effects of one-bar-long melodic alterations placed within a familiar melody. The synchronizing of the performance and eye-movement recordings enabled the investigation of the eye-hand span, i.e., the temporal gap between a performed note and the point of gaze. The eye-hand span was typically found to remain around one second. Music performance students demonstrated increased professing efficiency by their shorter average fixation durations as well as in the two examined eye-hand span measures: these participants used larger eye-hand spans more frequently and inspected more of the musical score during the performance of one metrical beat than students of music education. Although all participants produced performances almost indistinguishable in terms of their auditory characteristics, the altered bars indeed affected the reading of the score: the general effects of expertise in terms of the two eye- hand span measures, demonstrated by the music performance students, disappeared in the face of the melodic alterations. Experiment III was a longitudinal experiment designed to examine the differences between adult novice and amateur musicians’ silent reading of music notation, as well as the changes the 49 participants manifested during a nine-month long music course. From a methodological perspective, an opening to research on eye movements in music reading was the inclusion of a verbal protocol in the research design: after viewing the musical image, the readers were asked to describe what they had seen. A two-way categorization for verbal descriptions was developed in order to assess the quality of extracted musical information. More extensive musical background was related to shorter average fixation duration, more linear scanning of the musical image, and more sophisticated verbal descriptions of the music in question. No apparent effects of skill development were observed for the novice music readers alone, but all participants improved their verbal descriptions towards the last measurement. Apart from the background-related differences between groups of participants, combining verbal and eye-movement data in a cluster analysis identified three styles of silent reading. The finding demonstrated individual differences in how the freely defined silent-reading task was approached. This dissertation is among the first presentations of a series of experiments systematically addressing the visual processing of music notation in various types of music-reading tasks and focusing especially on the eye-movement indicators of developing music-reading skill. Overall, the experiments demonstrate that the music-reading processes are affected not only by “top-down” factors, such as musical background, but also by the “bottom-up” effects of specific features of music notation, such as pitch heights, metrical division, rhythmic patterns and unexpected melodic events. From a methodological perspective, the experiments emphasize the importance of systematic stimulus design, temporal control during performance tasks, and the development of complementary methods, for easing the interpretation of the eye-movement data. To conclude, this dissertation suggests that advances in comprehending the cognitive aspects of music reading, the nature of expertise in this musical task, and the development of educational tools can be attained through the systematic application of the eye-tracking methodology also in this specific domain.

Metal music as critical dystopia : humans, technology and the future in 1990s science fiction metal

Metal Music as Critical Dystopia: Humans, Technology and the Future in 1990s Science Fiction Metal seeks to demonstrate that the dystopian elements in metal music are not merely or necessarily a sonic celebration of disaster. Rather, metal music's fascination with dystopian imagery is often critical in intent, borrowing themes and imagery from other literary and cinematic traditions in an effort to express a form of social commentary. The artists and musical works examined in this thesis maintain strong ties with the science fiction genre, in particular, and tum to science fiction conventions in order to examine the long-term implications of humanity's complex relationship with advanced technology. Situating metal's engagements with science fiction in relation to a broader practice of blending science fiction and popular music and to the technophobic tradition in writing and film, this thesis analyzes the works of two science fiction metal bands, VOlvod and Fear Factory, and provides close readings of four futuristic albums from the mid to late 1990s that address humanity's relationship with advanced technology in musical and visual imagery as well as lyrics. These recorded texts, described here as cyber metal for their preoccupation with technology in subject matter and in sound, represent prime examples of the critical dystopia in metal music. While these albums identify contemporary problems as the root bf devastation yet to come, their musical narratives leave room for the possibility of hope , allowing for the chance that dystopia is not our inevitable future.

Baby Santa's : music box.

Se muestra a bebés y preescolares el mundo de la Navidad, con sus tradiciones y celebraciones propias. Las imágenes se acompañan con la música de clásicos como Tchaikovsky o Mozart.

Meteorological phenomena in Western classical orchestral music

Depictions of the weather are common throughout the arts. Unlike in the visual arts, however, there has been little study of meteorological inspiration in music. This article catalogues and analyzes the frequencies with which weather is depicted in a sample of classical orchestral music. The depictions vary from explicit mimicry using traditional and specialized orchestral instruments, through to subtle suggestions. It is found that composers are generally influenced by their own environment in the type of weather they choose to represent. As befits the national stereotype, British composers seem disproportionately keen to depict the UK's variable weather patterns and stormy coastline