Gestural communication of music structure during solo classical piano performance

The production and perception of music is a multimodal activity involving auditory, visual and conceptual processing, integrating these with prior knowledge and environmental experience. Musicians utilise expressive physical nuances to highlight salient features of the score. The question arises within the literature as to whether performers’ non-technical, non-sound-producing movements may be communicatively meaningful and convey important structural information to audience members and co-performers. In the light of previous performance research (Vines et al., 2006, Wanderley, 2002, Davidson, 1993), and considering findings within co-speech gestural research and auditory and audio-visual neuroscience, this thesis examines the nature of those movements not directly necessary for the production of sound, and their particular influence on audience perception. Within the current research 3D performance analysis is conducted using the Vicon 12- camera system and Nexus data-processing software. Performance gestures are identified as repeated patterns of motion relating to music structure, which not only express phrasing and structural hierarchy but are consistently and accurately interpreted as such by a perceiving audience. Gestural characteristics are analysed across performers and performance style using two Chopin preludes selected for their diverse yet comparable structures (Opus 28:7 and 6). Effects on perceptual judgements of presentation modes (visual-only, auditory-only, audiovisual, full- and point-light) and viewing conditions are explored. This thesis argues that while performance style is highly idiosyncratic, piano performers reliably generate structural gestures through repeated patterns of upper-body movement. The shapes and locations of phrasing motions are identified particular to the sample of performers investigated. Findings demonstrate that despite the personalised nature of the gestures, performers use increased velocity of movements to emphasise musical structure and that observers accurately and consistently locate phrasing junctures where these patterns and variation in motion magnitude, shape and velocity occur. By viewing performance motions in polar (spherical) rather than cartesian coordinate space it is possible to get mathematically closer to the movement generated by each of the nine performers, revealing distinct patterns of motion relating to phrasing structures, regardless of intended performance style. These patterns are highly individualised both to each performer and performed piece. Instantaneous velocity analysis indicates a right-directed bias of performance motion variation at salient structural features within individual performances. Perceptual analyses demonstrate that audience members are able to accurately and effectively detect phrasing structure from performance motion alone. This ability persists even for degraded point-light performances, where all extraneous environmental information has been removed. The relative contributions of audio, visual and audiovisual judgements demonstrate that the visual component of a performance does positively impact on the over- all accuracy of phrasing judgements, indicating that receivers are most effective in their recognition of structural segmentations when they can both see and hear a performance. Observers appear to make use of a rapid online judgement heuristics, adjusting response processes quickly to adapt and perform accurately across multiple modes of presentation and performance style. In line with existent theories within the literature, it is proposed that this processing ability may be related to cognitive and perceptual interpretation of syntax within gestural communication during social interaction and speech. Findings of this research may have future impact on performance pedagogy, computational analysis and performance research, as well as potentially influencing future investigations of the cognitive aspects of musical and gestural understanding.

Investigation of multisensory processing and structural brain differences in Autism Spectrum Disorder

This thesis is an investigation of structural brain abnormalities, as well as multisensory and unisensory processing deficits in autistic traits and Autism Spectrum Disorder (ASD). To achieve this, structural and functional magnetic resonance imaging (fMRI) and psychophysical techniques were employed. ASD is a neurodevelopmental condition which is characterised by the social communication and interaction deficits, as well as repetitive patterns of behaviour, interests and activities. These traits are thought to be present in a typical population. The Autism Spectrum Quotient questionnaire (AQ) was developed to assess the prevalence of autistic traits in the general population. Von dem Hagen et al. (2011) revealed a link between AQ with white matter (WM) and grey matter (GM) volume (using voxel-based-morphometry). However, their findings revealed no difference in GM in areas associated with social cognition. Cortical thickness (CT) measurements are known to be a more direct measure of cortical morphology than GM volume. Therefore, Chapter 2 investigated the relationship between AQ scores and CT in the same sample of participants. This study showed that AQ scores correlated with CT in the left temporo-occipital junction, left posterior cingulate, right precentral gyrus and bilateral precentral sulcus, in a typical population. These areas were previously associated with structural and functional differences in ASD. Thus the findings suggest, to some extent, autistic traits are reflected in brain structure - in the general population. The ability to integrate auditory and visual information is crucial to everyday life, and results are mixed regarding how ASD influences audiovisual integration. To investigate this question, Chapter 3 examined the Temporal Integration Window (TIW), which indicates how precisely sight and sound need to be temporally aligned so that a unitary audiovisual event can be perceived. 26 adult males with ASD and 26 age and IQ-matched typically developed males were presented with flash-beep (BF), point-light drummer, and face-voice (FV) displays with varying degrees of asynchrony and asked to make Synchrony Judgements (SJ) and Temporal Order Judgements (TOJ). Analysis of the data included fitting Gaussian functions as well as using an Independent Channels Model (ICM) to fit the data (Garcia-Perez & Alcala-Quintana, 2012). Gaussian curve fitting for SJs showed that the ASD group had a wider TIW, but for TOJ no group effect was found. The ICM supported these results and model parameters indicated that the wider TIW for SJs in the ASD group was not due to sensory processing at the unisensory level, but rather due to decreased temporal resolution at a decisional level of combining sensory information. Furthermore, when performing TOJ, the ICM revealed a smaller Point of Subjective Simultaneity (PSS; closer to physical synchrony) in the ASD group than in the TD group. Finding that audiovisual temporal processing is different in ASD encouraged us to investigate the neural correlates of multisensory as well as unisensory processing using functional magnetic resonance imaging fMRI. Therefore, Chapter 4 investigated audiovisual, auditory and visual processing in ASD of simple BF displays and complex, social FV displays. During a block design experiment, we measured the BOLD signal when 13 adults with ASD and 13 typically developed (TD) age-sex- and IQ- matched adults were presented with audiovisual, audio and visual information of BF and FV displays. Our analyses revealed that processing of audiovisual as well as unisensory auditory and visual stimulus conditions in both the BF and FV displays was associated with reduced activation in ASD. Audiovisual, auditory and visual conditions of FV stimuli revealed reduced activation in ASD in regions of the frontal cortex, while BF stimuli revealed reduced activation the lingual gyri. The inferior parietal gyrus revealed an interaction between stimulus sensory condition of BF stimuli and group. Conjunction analyses revealed smaller regions of the superior temporal cortex (STC) in ASD to be audiovisual sensitive. Against our predictions, the STC did not reveal any activation differences, per se, between the two groups. However, a superior frontal area was shown to be sensitive to audiovisual face-voice stimuli in the TD group, but not in the ASD group. Overall this study indicated differences in brain activity for audiovisual, auditory and visual processing of social and non-social stimuli in individuals with ASD compared to TD individuals. These results contrast previous behavioural findings, suggesting different audiovisual integration, yet intact auditory and visual processing in ASD. Our behavioural findings revealed audiovisual temporal processing deficits in ASD during SJ tasks, therefore we investigated the neural correlates of SJ in ASD and TD controls. Similar to Chapter 4, we used fMRI in Chapter 5 to investigate audiovisual temporal processing in ASD in the same participants as recruited in Chapter 4. BOLD signals were measured while the ASD and TD participants were asked to make SJ on audiovisual displays of different levels of asynchrony: the participants’ PSS, audio leading visual information (audio first), visual leading audio information (visual first). Whereas no effect of group was found with BF displays, increased putamen activation was observed in ASD participants compared to TD participants when making SJs on FV displays. Investigating SJ on audiovisual displays in the bilateral superior temporal gyrus (STG), an area involved in audiovisual integration (see Chapter 4), we found no group differences or interaction between group and levels of audiovisual asynchrony. The investigation of different levels of asynchrony revealed a complex pattern of results indicating a network of areas more involved in processing PSS than audio first and visual first, as well as areas responding differently to audio first compared to video first. These activation differences between audio first and video first in different brain areas are constant with the view that audio leading and visual leading stimuli are processed differently.

The structural and mechanical integrity of historic wood

Little is known about historic wood as it ages naturally. Instead, most studies focus on biological decay, as it is often assumed that wood remains otherwise stable with age. This PhD project was organised by Historic Scotland and the University of Glasgow to investigate the natural chemical and physical aging of wood. The natural aging of wood was a concern for Historic Scotland as traditional timber replacement is the standard form of repair used in wooden cultural heritage; replacing rotten timber with new timber of the same species. The project was set up to look at what differences could exist both chemically and physically between old and new wood, which could put unforeseen stress on the joint between them. Through Historic Scotland it was possible to work with genuine historic wood from two species, Oak and Scots pine, both from the 1500’s, rather than relying on artificial aging. Artificial aging of wood is still a debated topic, with consideration given to whether it is truly mimicking the aging process or just damaging the wood cells. The chemical stability of wood was investigated using Fourier-transform infrared (FTIR) microscopy, as well as wet chemistry methods including a test for soluble sugars from the possible breakdown of the wood polymers. The physical properties assessed included using a tensile testing machine to uncover possible differences in mechanical properties. An environmental chamber was used to test the reaction to moisture of wood of different ages, as moisture is the most damaging aspect of the environment to wooden cultural objects. The project uncovered several differences, both physical and chemical, between the modern and historic wood which could affect the success of traditional ‘like for like’ repairs. Both oak and pine lost acetyl groups, over historic time, from their hemicellulose polymers. This chemical reaction releases acetic acid, which had no effect on the historic oak but was associated with reduced stiffness in historic pine, probably due to degradation of the hemicellulose polymers by acid hydrolysis. The stiffness of historic oak and pine was also reduced by decay. Visible pest decay led to loss of wood density but there was evidence that fungal decay, extending beyond what was visible, degraded the S2 layer of the pine cell walls, reducing the stiffness of the wood by depleting the cellulose microfibrils most aligned with the grain. Fungal decay of polysaccharides in pine wood left behind sugars that attracted increased levels of moisture. The degradation of essential polymers in the wood structure due to age had different impacts on the two species of wood, and raised questions concerning both the mechanism of aging of wood and the ways in which traditional repairs are implemented, especially in Scots pine. These repairs need to be done with more care and precision, especially in choosing new timber to match the old. Within this project a quantitative method of measuring the microfibril angle (MFA) of wood using polarised Fourier transform infrared (FTIR) microscopy has been developed, allowing the MFA of both new and historic pine to be measured. This provides some of the information needed for a more specific match when selecting replacement timbers for historic buildings.

A contribution towards the performance and structural understanding of copper and nickel salts as ammonia stores

The ongoing depletion of fossil fuels and the severe consequences of the greenhouse effect make the development of alternative energy systems crucially important. While hydrogen is, in principle, a promising alternative, releasing nothing but energy and pure water. Hydrogen storage is complicated and no completely viable technique has been proposed so far. This work is concerned with the study of one potential alternative to pure hydrogen: ammonia, and more specifically its storage in solids. Ammonia, NH3, can be regarded as a chemical hydrogen carrier with the advantages of strongly reduced flammability and explosiveness as compared to hydrogen. Furthermore, ammine metal salts presented here as promising ammonia stores easily store up to 50 wt.-% ammonia, giving them a volumetric energy density comparable to natural gas. The model system NiX2–NH3 ( X = Cl, Br, I) is studied thoroughly with respect to ammine salt formation, thermal decomposition, air stability and structural effects. The system CuX2–NH3 ( X = Cl, Br) has an adverse thermal decomposition behaviour, making it impractical for use as an ammonia store. This system is, however, most interesting from a structural point of view and some work concerning the study of the structural behaviour of this system is presented. Finally, close chemical relatives to the metal ammine halides, the metal ammine nitrates are studied. They exhibit interesting anion arrangements, which is an impressive showcase for the combination of diffraction and spectroscopic information. The characterisation techniques in this thesis range from powder diffraction over single crystal diffraction, spectroscopy, computational modelling, thermal analyses to gravimetric uptake experiments. Further highlights are the structure solutions and refinements from powder data of (NH4)2[NiCl4(H2O)(NH3)] and Ni(NH3)2(NO3)2, the combination of crystallographic and chemical information for the elucidation of the (NH4)2[NiCl4(H2O)(NH3)] formation reaction and the growth of single crystals under ammonia flow, a technique allowing the first documented successful growth and single crystal diffraction measurement for [Cu(NH3)6]Cl2.