Towards an emotion-driven interactive music system: ridging the gaps between affect, physiology and sound generation

Report for the scientific sojourn at the Stanford University from January until June 2007. Music is well known for affecting human emotional states, yet the relationship between specific musical parameters and emotional responses is still not clear. With the advent of new human-computer interaction (HCI) technologies, it is now possible to derive emotion-related information from physiological data and use it as an input to interactive music systems. Providing such implicit musical HCI will be highly relevant for a number of applications including music therapy, diagnosis, nteractive gaming, and physiologically-based musical instruments. A key question in such physiology-based compositions is how sound synthesis parameters can be mapped to emotional states of valence and arousal. We used both verbal and heart rate responses to evaluate the affective power of five musical parameters. Our results show that a significant correlation exists between heart rate and the subjective evaluation of well-defined musical parameters. Brightness and loudness showed to be arousing parameters on subjective scale while harmonicity and even partial attenuation factor resulted in heart rate changes typically associated to valence. This demonstrates that a rational approach to designing emotion-driven music systems for our public installations and music therapy applications is possible.

Sound Texture Modelling

In the PhD thesis “Sound Texture Modeling” we deal with statistical modelling or textural sounds like water, wind, rain, etc. For synthesis and classification. Our initial model is based on a wavelet tree signal decomposition and the modeling of the resulting sequence by means of a parametric probabilistic model, that can be situated within the family of models trainable via expectation maximization (hidden Markov tree model ). Our model is able to capture key characteristics of the source textures (water, rain, fire, applause, crowd chatter ), and faithfully reproduces some of the sound classes. In terms of a more general taxonomy of natural events proposed by Graver, we worked on models for natural event classification and segmentation. While the event labels comprise physical interactions between materials that do not have textural propierties in their enterity, those segmentation models can help in identifying textural portions of an audio recording useful for analysis and resynthesis. Following our work on concatenative synthesis of musical instruments, we have developed a pattern-based synthesis system, that allows to sonically explore a database of units by means of their representation in a perceptual feature space. Concatenative syntyhesis with “molecules” built from sparse atomic representations also allows capture low-level correlations in perceptual audio features, while facilitating the manipulation of textural sounds based on their physical and perceptual properties. We have approached the problem of sound texture modelling for synthesis from different directions, namely a low-level signal-theoretic point of view through a wavelet transform, and a more high-level point of view driven by perceptual audio features in the concatenative synthesis setting. The developed framework provides unified approach to the high-quality resynthesis of natural texture sounds. Our research is embedded within the Metaverse 1 European project (2008-2011), where our models are contributting as low level building blocks within a semi-automated soundscape generation system.

Methodology for studying the numerical speed of sound in finite differences schemes

The space and time discretization inherent to all FDTD schemesintroduce non-physical dispersion errors, i.e. deviations ofthe speed of sound from the theoretical value predicted bythe governing Euler differential equations. A generalmethodologyfor computing this dispersion error via straightforwardnumerical simulations of the FDTD schemes is presented.The method is shown to provide remarkable accuraciesof the order of 1/1000 in a wide variety of twodimensionalfinite difference schemes.

The emergence of biofuels and the co-movement between crude oil and agricultural prices

Biofuels are becoming an alternative to non-renewable energy sources but we know little about the economic mechanisms influencing their prices. This paper studies the interrelationships between the spot prices of oil and those of agricultural commodities used as biofuel feedstocks. Using daily data since 1988, we identify a co-movement after 2005 that does not appear for other food-related commodities and is not due to general economic variables. We also find traces of the co-movement in the prices of a large biofuel stock. The results amount to the first systematic piece of empirical evidence linking spot oil and agricultural markets via the emergence of biofuels.

On the drivers of commodity co-movement: Evidence from biofuels

We use the recent introduction of biofuels to study the effect of industry factors on the relationshipsbetween wholesale commodity prices. Correlations between agricultural products and oilare strongest in the 2005-09 period, coinciding with the boom of biofuels, and remain substantialuntil 2011. We disentangle three possible drivers for the linkage: substitution, energy costs, andfinancialization. The timing and magnitude of the biofuels-to-oil relationships are different to thoseof other commodities, and far higher than can be justified by costs and financialization. Substitutionand costs drive the monthly correlations of long-term futures, and each of the three contributeequally to the daily co-movement of the short-term ones. The findings survive many robustnesschecks and appear in the stock market.

Comparative study of algal communities in acid and alkaline waters from Tinto, Odiel and Piedras river basins (SW Spain)

The distribution patterns of benthic algal assemblages in the Tinto, Odiel and Piedras rivers were analyzed during the winter of 2005 in 18 sampling stations. The main objective was to assess and compare the algal communities and parameters affecting them both in the zones affected by acid mine drainage (AMD) and in naturally alkaline waters. A total of 108 benthic diatom taxa and 31 non-diatom taxa were identi ed. Results showed large differences between algal communities in the two environments: Pinnularia acoricola, P. subcapitata and Eunotia exigua were the most frequent diatom taxa in regions affected by acid mine drainage, along with algae like Klebsormidium and Euglena mutabilis were the most relevant non-diatom taxa. In alkaline waters the dominant diatom taxa were Planothidium frequentissimum, Gomphonema angustum, Fragilaria capucina, and some species of Navicula (N. viridula, N. veneta or N. radiosa), accompanied by Oscillatoria and Anabaena as well as by streptophytes of the group of zygnemataceae and desmidiaceae

Statistical modeling of bowing control applied to violin sound synthesis

Excitation-continuous music instrument control patterns are often not explicitly represented in current sound synthesis techniques when applied to automatic performance. Both physical model-based and sample-based synthesis paradigmswould benefit from a flexible and accurate instrument control model, enabling the improvement of naturalness and realism. Wepresent a framework for modeling bowing control parameters inviolin performance. Nearly non-intrusive sensing techniques allow for accurate acquisition of relevant timbre-related bowing control parameter signals.We model the temporal contour of bow velocity, bow pressing force, and bow-bridge distance as sequences of short Bézier cubic curve segments. Considering different articulations, dynamics, and performance contexts, a number of note classes are defined. Contours of bowing parameters in a performance database are analyzed at note-level by following a predefined grammar that dictates characteristics of curve segment sequences for each of the classes in consideration. As a result, contour analysis of bowing parameters of each note yields an optimal representation vector that is sufficient for reconstructing original contours with significant fidelity. From the resulting representation vectors, we construct a statistical model based on Gaussian mixtures suitable for both the analysis and synthesis of bowing parameter contours. By using the estimated models, synthetic contours can be generated through a bow planning algorithm able to reproduce possible constraints caused by the finite length of the bow. Rendered contours are successfully used in two preliminary synthesis frameworks: digital waveguide-based bowed stringphysical modeling and sample-based spectral-domain synthesis.

Gesture sampling for instrumental sound synthesis: violin bowing as a case study

This paper presents a framework in which samples of bowing gesture parameters are retrieved and concatenated from a database of violin performances by attending to an annotated input score. Resulting bowing parameter signals are then used to synthesize sound by means of both a digital waveguide violin physical model, and an spectral-domainadditive synthesizer.

A comparative study of the intercellular connections between parasite/host in two Gelidiocolax/Gelidium algal parasitic systems

Comparative ultrastructural study of the intercellular connections between parasite and host cells in two algal parasitic systems, Gelidiocolax christianae Feldmann and Feldmann/Ge/iV/ium spathulatum (Kutz.) Bornet and Gelidiocolax deformans Seoane Camba/Gelidium sesquipedale (Clem.) Thur, shows quantitative and structural differences. The number of free conjunctor cells (before fusión with the adjacent host cells) differs between the two parasitic systems and is inversely related to the number of complex pit connections. The fibrillar cell wall structure of the conjunctor cells and the lamellar structure of the complex pit plugs in the two systems are also different A hypothesis concerning the different activity of the conjuntor cell wall in the two parasitic systems, related with the different structural appearance, is proposed.

Sexual dimorphism of sonic apparatus and extreme intersexual variation of sounds in Ophidion rochei (Ophidiidae): first evidence of a tight relationship between morphology and sound characteristics in Ophidiidae

Many Ophidiidae are active in dark environments and display complex sonic apparatus morphologies. However, sound recordings are scarce and little is known about acoustic communication in this family. This paper focuses on Ophidion rochei which is known to display an important sexual dimorphism in swimbladder and anterior skeleton. The aims of this study were to compare the sound producing morphology, and the resulting sounds in juveniles, females and males of O. rochei.Results: Males, females, and juveniles possessed different morphotypes. Females and juveniles contrasted with males because they possessed dramatic differences in morphology of their sonic muscles, swimbladder, supraoccipital crest, and first vertebrae and associated ribs. Further, they lacked the ‘rocker bone’ typically found in males. Sounds from each morphotype were highly divergent. Males generally produced non harmonic, multiple-pulsed sounds that lasted for several seconds (3.5 ± 1.3 s) with a pulse period of ca. 100 ms. Juvenile and female sounds were recorded for the first time in ophidiids. Female sounds were harmonic, had shorter pulse period (±3.7 ms), and never exceeded a few dozen milliseconds (18 ± 11 ms). Moreover, unlike male sounds, female sounds did not have alternating long and short pulse periods. Juvenile sounds were weaker but appear to be similar to female sounds.Conclusions: Although it is not possible to distinguish externally male from female in O. rochei, they show a sonic apparatus and sounds that are dramatically different. This difference is likely due to their nocturnal habits that may have favored the evolution of internal secondary sexual characters that help to distinguish males from females and that could facilitate mate choice by females. Moreover, the comparison of different morphotypes in this study shows that these morphological differences result from a peramorphosis that takes place during the development of the gonads

Animal sound : què se sent quan s’és una cabra?

Aquest projecte treballa sobre la possibilitat d'aplicar tècniques d'enregistrament binaural en animals, en concret, en una cabra domèstica. Mitjançant uns micròfons col·locats dins les seves orelles i una petita càmera de vídeo muntada sobre el seu cap, s'obté un material audiovisual que permet fer-se una idea aproximada de com és la seva percepció del món. En base a un estudi de cognició comparada, es pretén trobar maneres de transformar els enregistraments obtinguts per tal d'adaptar el marc psicoacústic humà al de l'animal. L'objectiu és que una persona pugui sentir com ho fa un animal, encara que sigui d'una manera aproximada. Els materials obtinguts al llarg dels enregistraments són el punt de partida per a la construcció de paisatges sonors i peces audiovisuals diverses. Així doncs, el treball s'inicia amb el disseny i construcció d'un micròfon binaural, continua amb enregistraments de camp en animals i acaba amb l'edició, el processat i la composició dels paisatges sonors finals.