"Rebirth: Combined art performance"--Shirin Madg & Sweet Sound Ensemble

Vanessa Mafe-Keane was invited to participate as choreographer in Iranian singer Shirin Madg 's project, Rebirth: Combined art performance. This project integrated singing, music, visual-art, film, dance and is based on the dissident poetry of female Iranian poet, Forough Farrokhzad. The choreographic dance movement focused on simple, lyrical, flowing classical dance forms that also incorporated everyday gestures and actions performed by two Queensland dancers, Caitlin MacKenzie and Abby Johnson. The choreographic intention was not to attempt to re-create Iranian dance practices instead, to draw inspiration and reference specific movement qualities. This was achieved through the subtle inclusion of spinning movements and focusing attention on the dancers’ arms and upper torso. This fusion became an underlying theme reflected throughout the choreographic component. Additionally, this project presented an opportunity to draw on past experiences and problem-solve ways to construct choreographic work where the dancers and the musical assemble group could be staged side by side. This experience highlighted differing approaches to rehearsal protocols within disciplines, the practicalities of staging different artists, understanding musical cues and the diversity of audience engagement. Performances: BEMAC Multicultural Centre, Brisbane 06 February 2015 and Helensvale Cultural Centre, Gold Coast 07 February 2015

Semi-automated annotation of environmental acoustic recordings

Environmental sensors collect massive amounts of audio data. This thesis investigates computational methods to support human analysts in identifying faunal vocalisations from that audio. A series of experiments was conducted to trial the effectiveness of novel user interfaces. This research examines the rapid scanning of spectrograms, decision support tools for users, and cleaning methods for folksonomies. Together, these investigations demonstrate that providing computational support to human analysts increases their efficiency and accuracy; this allows bioacoustics projects to efficiently utilise their valuable human analysts.

A cluster-voting approach for speaker diarization and linking of Australian broadcast news recordings

We present a clustering-only approach to the problem of speaker diarization to eliminate the need for the commonly employed and computationally expensive Viterbi segmentation and realignment stage. We use multiple linear segmentations of a recording and carry out complete-linkage clustering within each segmentation scenario to obtain a set of clustering decisions for each case. We then collect all clustering decisions, across all cases, to compute a pairwise vote between the segments and conduct complete-linkage clustering to cluster them at a resolution equal to the minimum segment length used in the linear segmentations. We use our proposed cluster-voting approach to carry out speaker diarization and linking across the SAIVT-BNEWS corpus of Australian broadcast news data. We compare our technique to an equivalent baseline system with Viterbi realignment and show that our approach can outperform the baseline technique with respect to the diarization error rate (DER) and attribution error rate (AER).

Generalised features for bird vocalisation retrieval in acoustic recordings

Bioacoustic monitoring has become a significant research topic for species diversity conservation. Due to the development of sensing techniques, acoustic sensors are widely deployed in the field to record animal sounds over a large spatial and temporal scale. With large volumes of collected audio data, it is essential to develop semi-automatic or automatic techniques to analyse the data. This can help ecologists make decisions on how to protect and promote the species diversity. This paper presents generic features to characterize a range of bird species for vocalisation retrieval. In the implementation, audio recordings are first converted to spectrograms using short-time Fourier transform, then a ridge detection method is applied to the spectrogram for detecting points of interest. Based on the detected points, a new region representation are explored for describing various bird vocalisations and a local descriptor including temporal entropy, frequency bin entropy and histogram of counts of four ridge directions is calculated for each sub-region. To speed up the retrieval process, indexing is carried out and the retrieved results are ranked according to similarity scores. The experiment results show that our proposed feature set can achieve 0.71 in term of retrieval success rate which outperforms spectral ridge features alone (0.55) and Mel frequency cepstral coefficients (0.36).

Clustering acoustic events in environmental recordings for species richness surveys

Environmental acoustic recordings can be used to perform avian species richness surveys, whereby a trained ornithologist can observe the species present by listening to the recording. This could be made more efficient by using computational methods for iteratively selecting the richest parts of a long recording for the human observer to listen to, a process known as “smart sampling”. This allows scaling up to much larger ecological datasets. In this paper we explore computational approaches based on information and diversity of selected samples. We propose to use an event detection algorithm to estimate the amount of information present in each sample. We further propose to cluster the detected events for a better estimate of this amount of information. Additionally, we present a time dispersal approach to estimating diversity between iteratively selected samples. Combinations of approaches were evaluated on seven 24-hour recordings that have been manually labeled by bird watchers. The results show that on average all the methods we have explored would allow annotators to observe more new species in fewer minutes compared to a baseline of random sampling at dawn.

Birdcall retrieval from environmental acoustic recordings using image processing

Acoustic recordings of the environment provide an effective means to monitor bird species diversity. To facilitate exploration of acoustic recordings, we describe a content-based birdcall retrieval algorithm. A query birdcall is a region of spectrogram bounded by frequency and time. Retrieval depends on a similarity measure derived from the orientation and distribution of spectral ridges. The spectral ridge detection method caters for a broad range of birdcall structures. In this paper, we extend previous work by incorporating a spectrogram scaling step in order to improve the detection of spectral ridges. Compared to an existing approach based on MFCC features, our feature representation achieves better retrieval performance for multiple bird species in noisy recordings.

Detection of anuran calling activity in long field recordings for bio-acoustic monitoring

This paper presents a system to analyze long field recordings with low signal-to-noise ratio (SNR) for bio-acoustic monitoring. A method based on spectral peak track, Shannon entropy, harmonic structure and oscillation structure is proposed to automatically detect anuran (frog) calling activity. Gaussian mixture model (GMM) is introduced for modelling those features. Four anuran species widespread in Queensland, Australia, are selected to evaluate the proposed system. A visualization method based on extracted indices is employed for detection of anuran calling activity which achieves high accuracy.

Compact features for birdcall retrieval from environmental acoustic recordings

Bioacoustic data can be used for monitoring animal species diversity. The deployment of acoustic sensors enables acoustic monitoring at large temporal and spatial scales. We describe a content-based birdcall retrieval algorithm for the exploration of large data bases of acoustic recordings. In the algorithm, an event-based searching scheme and compact features are developed. In detail, ridge events are detected from audio files using event detection on spectral ridges. Then event alignment is used to search through audio files to locate candidate instances. A similarity measure is then applied to dimension-reduced spectral ridge feature vectors. The event-based searching method processes a smaller list of instances for faster retrieval. The experimental results demonstrate that our features achieve better success rate than existing methods and the feature dimension is greatly reduced.

De Profundis (Theatre Soundtrack)

This creative work is an original soundtrack for the multimedia performance adaptation of Oscar Wilde’s De Profundis, led by David Fenton and Brian Lucas and produced by Metro Arts. Intermediality offers unique challenges to the composer creating towards live performance. Given the text-based nature of the piece, and the prevalence of screen content, music had a distinct role to play in supporting the intermedial performance environment. Drawing from Oscar Wilde’s own writings in the initial stages [“...richer cadences…more curious effects” “…the cry of Marysas” and the “deferred resolution of Chopin”] , the deliberately risky compositional process experimented with improvised location recordings and found sounds, random and fragmented assemblages of vintage recordings, rough methods and obsolete recording technology, and the sonic kinship of the hissing sibilances of the sea, theatrical applause and the crackle of antique recording devices (which had just been invented in Wilde’s time) worked into wefts of sound. As the soundtrack emerged, is was clearly resistant to ‘concepts’ imposed from the outside, and as the field of possibilities expanded and engaged in dialogue with the other elements of the performance (live and projected) certain pieces were selected by the director and curated into the emerging work. Thus leitmotifs emerged, rather than being imposed from the outset, with a particular through line holding: if it was too obviously like ‘music’, (which is usually used in theatre as emotional lubrication and narrative signpost) it didn’t work, and if it sounded like avant-garde sound-art, it was too grating and detracted from the primacy of the text. As a composer I worked this sweet spot inbetween these two poles as well as serving David Fenton’s curation: he determined which compositions to incorporate, reiterate and omit as part of the process of writing text, action and image and the compositional process responded with organic elaborations and variations on these selections. Musical resolution was mostly deferred until the closing stages of the performance. The soundtrack was present for the duration of the show, and Artshub reviewed the musical component thus: “...the score by David Megarrity is a refined, understated ambient scaffolding.” It premiered at the Visy Theatre, Brisbane Powerhouse, on 22 April 2015.

Normal-mode sound propagation in an ocean with sinusoidal surface waves

The normal-mode solution to the problem of acoustic wave propagation in an isovelocity ocean with a wavy surface is considered. The surface wave amplitude is assumed to be small compared to the acoustic wavelength, and the method of multiple scales is employed to study the interaction between normal-mode acoustic waves and the surface waves. A nonresonant interaction causes small fluctuations of the amplitude and phase of the acoustic wave at a rate dependent on the frequency of the surface wave. Backscatter occurs if the wavenumber of the surface wave is larger than that of the acoustic wave. The interaction becomes resonant if appropriate phase-matching conditions are satisfied. In this case, two acoustic normal modes get coupled, resulting in a large-scale periodic exchange of energy from one mode to another.

Cortical processing of speech and non-speech sounds in adults and newborns

Comprehension of a complex acoustic signal - speech - is vital for human communication, with numerous brain processes required to convert the acoustics into an intelligible message. In four studies in the present thesis, cortical correlates for different stages of speech processing in a mature linguistic system of adults were investigated. In two further studies, developmental aspects of cortical specialisation and its plasticity in adults were examined. In the present studies, electroencephalographic (EEG) and magnetoencephalographic (MEG) recordings of the mismatch negativity (MMN) response elicited by changes in repetitive unattended auditory events and the phonological mismatch negativity (PMN) response elicited by unexpected speech sounds in attended speech inputs served as the main indicators of cortical processes. Changes in speech sounds elicited the MMNm, the magnetic equivalent of the electric MMN, that differed in generator loci and strength from those elicited by comparable changes in non-speech sounds, suggesting intra- and interhemispheric specialisation in the processing of speech and non-speech sounds at an early automatic processing level. This neuronal specialisation for the mother tongue was also reflected in the more efficient formation of stimulus representations in auditory sensory memory for typical native-language speech sounds compared with those formed for unfamiliar, non-prototype speech sounds and simple tones. Further, adding a speech or non-speech sound context to syllable changes was found to modulate the MMNm strength differently in the left and right hemispheres. Following the acoustic-phonetic processing of speech input, phonological effort related to the selection of possible lexical (word) candidates was linked with distinct left-hemisphere neuronal populations. In summary, the results suggest functional specialisation in the neuronal substrates underlying different levels of speech processing. Subsequently, plasticity of the brain's mature linguistic system was investigated in adults, in whom representations for an aurally-mediated communication system, Morse code, were found to develop within the same hemisphere where representations for the native-language speech sounds were already located. Finally, recording and localization of the MMNm response to changes in speech sounds was successfully accomplished in newborn infants, encouraging future MEG investigations on, for example, the state of neuronal specialisation at birth.

Sensory auditory processing and intuitive sound detection : an investigation of musical experts and nonexperts

The auditory system can detect occasional changes (deviants) in acoustic regularities without the need for subjects to focus their attention on the sound material. Deviant detection is reflected in the elicitation of the mismatch negativity component (MMN) of the event-related potentials. In the studies presented in this thesis, the MMN is used to investigate the auditory abilities for detecting similarities and regularities in sound streams. To investigate the limits of these processes, professional musicians have been tested in some of the studies. The results show that auditory grouping is already more advanced in musicians than in nonmusicians and that the auditory system of musicians can, unlike that of nonmusicians, detect a numerical regularity of always four tones in a series. These results suggest that sensory auditory processing in musicians is not only a fine tuning of universal abilities, but is also qualitatively more advanced than in nonmusicians. In addition, the relationship between the auditory change-detection function and perception is examined. It is shown that, contrary to the generally accepted view, MMN elicitation does not necessarily correlate with perception. The outcome of the auditory change-detection function can be implicit and the implicit knowledge of the sound structure can, after training, be utilized for behaviorally correct intuitive sound detection. These results illustrate the automatic character of the sensory change detection function.

Processing of Stimulus Repetition and Change in the Somatosensory System: Recordings of Electrical and Magnetic Brain Responses

In the present work, effects of stimulus repetition and change in a continuous stimulus stream on the processing of somatosensory information in the human brain were studied. Human scalp-recorded somatosensory event-related potentials (ERPs) and magnetoencephalographic (MEG) responses rapidly diminished with stimulus repetition when mechanical or electric stimuli were applied to fingers. On the contrary, when the ERPs and multi-unit a ctivity (MUA) were directly recorded from the primary (SI) and secondary (SII) somatosensory cortices in a monkey, there was no marked decrement in the somatosensory responses as a function of stimulus repetition. These results suggest that this rate effect is not due to the response diminution in the SI and SII cortices. Obviously the responses to the first stimulus after a long "silent" period are nhanced due to unspecific initial orientation, originating in more broadly distributed and/or deeper neural structures, perhaps in the prefrontal cortices. With fast repetition rates not only the late unspecific but also some early specific somatosensory ERPs were diminished in amplitude. The fast decrease of the ERPs as a function of stimulus repetition is mainly due to the disappearance of the orientation effect and with faster repetition rates additively due to stimulus specific refractoriness. A sudden infrequent change in the continuous stimulus stream also enhanced somatosensory MEG responses to electric stimuli applied to different fingers. These responses were quite similar to those elicited by the deviant stimuli alone when the frequent standard stimuli were omitted. This enhancement was obviously due to the release from refractoriness because the neural structures generating the responses to the infrequent deviants had more time to recover from the refractoriness than the respective structures for the standards. Infrequent deviant mechanical stimuli among frequent standard stimuli also enhanced somatosensory ERPs and, in addition, they elicited a new negative wave which did not occur in the deviants-alone condition. This extra negativity could be recorded to deviations in the stimulation site and in the frequency of the vibratory stimuli. This response is probably a somatosensory analogue of the auditory mismatch negativity (MMN) which has been suggested to reflect a neural mismatch process between the sensory input and the sensory memory trace.