Entre o sonoro e o visual: relações e processos nas artes

Pós-graduação em Artes - IA

Towards an Asynchronous Cinema: how can the asynchronous use of sound in artists' moving image underpin the creation of dialectic tension between the audio, the visual and the audience?

This PhD by publication examines selected practice-based audio-visual works made by the author over a ten-year period, placing them in a critical context. Central to the publications, and the focus of the thesis, is an exploration of the role of sound in the creation of dialectic tension between the audio, the visual and the audience. By first analysing a number of texts (films/videos and key writings) the thesis locates the principal issues and debates around the use of audio in artists’ moving image practice. From this it is argued that asynchronism, first advocated in 1929 by Pudovkin as a response to the advent of synchronised sound, can be used to articulate audio-visual relationships. Central to asynchronism’s application in this paper is a recognition of the propensity for sound and image to adhere, and in visual music for there to be a literal equation of audio with the visual, often married with a quest for the synaesthetic. These elements can either be used in an illusionist fashion, or employed as part of an anti-illusionist strategy for realising dialectic. Using this as a theoretical basis, the paper examines how the publications implement asynchronism, including digital mapping to facilitate innovative reciprocal sound and image combinations, and the asynchronous use of ‘found sound’ from a range of online sources to reframe the moving image. The synthesis of publications and practice demonstrates that asynchronism can both underpin the creation of dialectic, and be an integral component in an audio-visual anti-illusionist methodology.

Cutaneous Grooves: Composing for the Sense of Touch

What if the traditional relationship between touch and music was essentially turned upside down, making the tactile sensation the aesthetic end? This paper presents a novel coupling of haptics technology and music, introducing the notion of tactile composition or aesthetic composition for the sense of touch. A system that facilitates the composition and perception of intricate, musically structured spatio-temporal patterns of vibration on the surface of the body is described. Relevant work from disciplines including sensory substitution, electronic musical instrument design, simulation design, entertainment technology, and visual music is considered. The psychophysical parameter space for our sense of touch is summarized and the building blocks of a compositional language for touch are explored. A series of concerts held for the skin and ears is described, as well as some of the lessons learned along the way. In conclusion, some potential evolutionary branches of tactile composition are posited.

Tempêtes. Composition audiovisuelle.

La version intégrale de cette thèse est disponible uniquement pour consultation individuelle à la Bibliothèque de musique de l’Université de Montréal (www.bib.umontreal.ca/MU ).

De la musique au-delà des frontières du son

La musique aujourd’hui est régulièrement accompagnée d’environnements visuels. Depuis les propositions en provenance du vidéoclip jusqu’aux œuvres installatives en passant par l’art web ou le cinéma, l’audiovisuel occupe une place considérable dans notre monde médiatisé et constitue un foyer important du développement des pratiques musicales. L’alliage entre son et image est souvent rattachée à l’histoire du cinéma mais les prémisses entourant l’audiovisuel remontent en réalité à l’Antiquité. Les correspondances entre sons et couleurs ont pris racine en premier chez les Pythagoriciens et cet intérêt se poursuit encore aujourd’hui. L’avènement de différentes technologies est venu reformuler au fil des siècles cette recherche qui retourne du décloisonnement artistique. L’arrivée de l’électricité permet au XIXe siècle le développement d’une lutherie expérimentale avec entre autres l’orgue à couleur d’Alexander Rimington. Ces instruments audiovisuels donnent naissance plus tard au Lumia, un art de la couleur et du mouvement se voulant proche de la musique et qui ne donne pourtant rien à entendre. Parallèlement à ces nouvelles propositions artistiques, il se développe dès les tout début du XXe siècle au sein des avant-gardes cinématographiques un corpus d’œuvres qui sera ensuite appelé musique visuelle. Les possibilités offertes par le support filmique vient offrir de nouvelles possibilités quant à l’organisation de la couleur et du mouvement. La pratique de cet art hybride est ensuite reformulée par les artistes associés à l’art vidéo avant de connaitre une vaste phase de démocratisation avec l’avènement des ordinateurs domestiques depuis les années 1990. Je retrace le parcours historique de ces pratiques audiovisuelles qui s’inscrivent résolument sur le terrain du musical. Un parcours appuyé essentiellement sur des œuvres et des ouvrages théoriques tout en étant parsemé de réflexions personnelles. Je traite des enjeux théoriques associés à ces propositions artistiques en les différenciant d’un autre format audiovisuel majeur soit le cinéma. Cet exposé permet de préparer le terrain afin de présenter et contextualiser mon travail de création. Je traite de deux œuvres, Trombe (2011) et Lungta (2012), des propositions qui héritent à la fois des musiques visuelles, de l’art interactif et de l’art cinétique.

Seeing music performance : visual influences on perception and experience

Drawing from ethnographic, empirical, and historical / cultural perspectives, we examine the extent to which visual aspects of music contribute to the communication that takes place between performers and their listeners. First, we introduce a framework for understanding how media and genres shape aural and visual experiences of music. Second, we present case studies of two performances, and describe the relation between visual and aural aspects of performance. Third, we report empirical evidence that visual aspects of performance reliably influence perceptions of musical structure (pitch related features) and affective interpretations of music. Finally, we trace new and old media trajectories of aural and visual dimensions of music, and highlight how our conceptions, perceptions and appreciation of music are intertwined with technological innovation and media deployment strategies.

Music selection using a touch screen interface : effect of auditory and visual feedback on driving and usability

This study aimed to examine the effects on driving, usability and subjective workload of performing music selection tasks using a touch screen interface. Additionally, to explore whether the provision of visual and/or auditory feedback offers any performance and usability benefits. Thirty participants performed music selection tasks with a touch screen interface while driving. The interface provided four forms of feedback: no feedback, auditory feedback, visual feedback, and a combination of auditory and visual feedback. Performance on the music selection tasks significantly increased subjective workload and degraded performance on a range of driving measures including lane keeping variation and number of lane excursions. The provision of any form of feedback on the touch screen interface did not significantly affect driving performance, usability or subjective workload, but was preferred by users over no feedback. Overall, the results suggest that touch screens may not be a suitable input device for navigating scrollable lists.

Music, Magic, and the Mythic: The Dynamics of Visual and Aural Discourse in Souleymane Cissé’s Yeelen

The 1980s saw a wave of African films that aimed to represent, on both local and international screens, a sophisticated pre-colonial Africa, thus debunking notions of the continent as primitive. Toward this aim the films inscribed the conventions of oral performance within their visual styles, denying spectator identification with the protagonists and emphasising the presence of the narrator. However, some critics argued that these films exoticised Africa, while their use of oral performance’s distancing effect echoed the ‘scientific’ distance structured by the ethnographic film, in which African societies were represented as ‘the other’. Souleymane Cissé’s Yeelen exemplifies this tension, transposing into cinematic form oral storytelling techniques in the depiction of a power struggle within the covert cult of the komo, a Bambara initiation society unfamiliar to most non-Bambara viewers. This paper demonstrates how the film negotiates this tension via music, which interpellates the international spectator by eliciting a greater identification with the protagonists than that determined at a visual level, while encoding a verisimilitude to rituals that may otherwise be read as the superstitious practices of ‘the other’. In this way, music and image in Yeelen operate as parallel, though often overlapping, discourses, bridging the gap between the film’s culturally specific narrative and formal components, and its international spectators.

Music Notes: Exploring Music Listening Data as a Visual Representation of Self

Thesis (Master's)--University of Washington, 2016-06

Seeing music performance: Visual influences on perception and experience

Drawing from ethnographic, empirical, and historical/cultural perspectives, we examine the extent to which visual aspects of music contribute to the communication that takes place between performers and their listeners. First, we introduce a framework for understanding how media and genres shape aural and visual experiences of music. Second, we present case studies of two performances, and describe the relation between visual and aural aspects of performance. Third, we report empirical evidence that visual aspects of performance reliably influence perceptions of musical structure (pitch related features) and affective interpretations of music. Finally, we trace new and old media trajectories of aural and visual dimensions of music, and highlight how our conceptions, perceptions and appreciation of music are intertwined with technological innovation and media deployment strategies.

Restrung New Chamber Festival : Bridging the Gap Between High Art Music and Popular Sensibilities

The Restrung New Chamber Festival was a practice-led research project which explored the intricacies of musical relationships. Specifically, it investigated the relationships between new music ensembles and pop-oriented bands inspired by the new music genre. The festival, held at the Brisbane Powerhouse (28 February-2 March 2009) comprised 17 diverse groups including the Brodsky Quartet, Topology, Wood, Fourplay and CODA. Restrung used a new and distinctive model which presented new music and syncretic musical genres within an immersive environment. Restrung brought together approaches used in both contemporary classical and popular music festivals, using musical, visual and spatial aspects to engage audiences. Interactivity was encouraged through video and sound installations, workshops and forums. This paper will investigate some of the issues surrounding the conception and design of the Restrung model, within the context of an overview of European new music trends. It includes a discussion of curating such an event in a musically sensitive and effective way, and approaches to identifying new and receptive audiences. As a guide to programming Restrung, I formulated a working definition of new music, further developed by interviews with specialists in Australia and Europe, and this will be outlined below.

Performing with grid music systems

Grid music systems provide discrete geometric methods for simplified music-making by providing spatialised input to construct patterned music on a 2D matrix layout. While they are conceptually simple, grid systems may be layered to enable complex and satisfying musical results. Grid music systems have been applied to a range of systems from small portable devices up to larger systems. In this paper we will discuss the use of grid music systems in general and present an overview of the HarmonyGrid system we have developed as a new interactive performance system. We discuss a range of issues related to the design and use of larger-scale grid- based interactive performance systems such as the HarmonyGrid.

Music, Mime & Metamorphosis: Interdisciplinary intersections, interactions and influences between music, mime and corporate communication

This doctoral thesis comprises three distinct yet related projects which investigate interdisciplinary practice across: music collaboration; mime performance; and corporate communication. Both the processes and underpinning research of these projects explore, expose and exploit areas where disparate and apparently conflicting fields of professional practice successfully and effectively; intersect, interact, and inform each other - rather than conflict - thereby enhancing each, both individually and collectively. Informed by three decades of professional practice across: music; stage performance; television; corporate communication; design; and tertiary education, the three projects have produced innovative, creative, and commercial viable outcomes, manifest in a variety of media including: music; written text; digital, audio/visual; and internet. In exploring new practice and creating new knowledge, these project outcomes clearly demonstrate the value and effectiveness of reconciling disparate fields of practice through the application of inter-disciplinary creativity and innovation to professional practice.

Navigating technological contexts and experience design in music education

For some time we have jokingly referred to our network jamming research with jam2jam as ‘Switched on Orff’ (Brown, Sorensen and Dillon 2002; Dillon 2003; Dillon 2006; Dillon 2006; Brown and Dillon 2007). The connection with electronic music and Wendy Carlos’ classic work ‘Switched on Bach’ was obvious; we were using electronic music in schools and with children. The deeper connection with Orff however was about recognising that electronic music and instruments could have cultural values and knowledge embedded in their design and practice in same way as what has come to be known as the Orff method (Orff and Keetman 1958-66). However whilst the Orff method focuses upon Western art music perceptual framework electronic instruments have the potential to have more fluid musical environments and even to move to interdisciplinary study by including visual media. Whilst the Orff method focused on making sense of Western art music through experience electronic environments potentially can make sense of the world of multi media that pervades our lives.