Towards an emotion-driven interactive music system: ridging the gaps between affect, physiology and sound generation

Report for the scientific sojourn at the Stanford University from January until June 2007. Music is well known for affecting human emotional states, yet the relationship between specific musical parameters and emotional responses is still not clear. With the advent of new human-computer interaction (HCI) technologies, it is now possible to derive emotion-related information from physiological data and use it as an input to interactive music systems. Providing such implicit musical HCI will be highly relevant for a number of applications including music therapy, diagnosis, nteractive gaming, and physiologically-based musical instruments. A key question in such physiology-based compositions is how sound synthesis parameters can be mapped to emotional states of valence and arousal. We used both verbal and heart rate responses to evaluate the affective power of five musical parameters. Our results show that a significant correlation exists between heart rate and the subjective evaluation of well-defined musical parameters. Brightness and loudness showed to be arousing parameters on subjective scale while harmonicity and even partial attenuation factor resulted in heart rate changes typically associated to valence. This demonstrates that a rational approach to designing emotion-driven music systems for our public installations and music therapy applications is possible.

Banc electrònic d'objectes i recursos d'aprenentatge dels estudis de comunicació audiovisual

OdACAV és un banc electrònic d’objectes d’aprenentatge (OdA) que te com a finalitat principal servir als docents de les assignatures troncals dels Estudis de Comunicació Audiovisual (CAV) de la UB (obert també a altres universitats catalanes) implicats o que es vulguin implicar en la innovació docent i pretén facilitar i potenciar la documentació per a la investigació i la recerca a l’entorn del paper innovador dels OdA digitals; així com la patrimonialització dels mateixos. Què és i que no és un OdA de CAV? És tot allò que serveix en un procés d'ensinistrament, d'aprenentatge, de formació - que en els cas que ens ocupa, es tradueix en una col·lecció d'imatges fixes, en un hipertext, en una hipermèdia, un vídeo, etc.-, i la missió dels quals és suscitar l’interès i l'aprofitament en el transvasament dels continguts de les assignatures dels estudis actuals (de la Llicenciatura) i futurs (del Grau) de Comunicació Audiovisual. El projecte sorgeix com a necessitat orgànica de la mateixa naturalesa de l'ensenyament de Comunicació Audiovisual, on en el pla dels continguts els exemples, els referents, els models no són encara prou desenvolupats i costa molt disposar d'OdAs suficientment competents. Les fonts documentals del cinema, de la televisió i dels mitjans en general són la base sobre la que s’han bastit els OdA de la base de dades del web http://www.lmi.ub.es/repositori/ amb l’esperança que resultin adequats a la innovació en els Estudis de Comunicació Audiovisual. És, doncs, un repositori de condició cooperativa, dinàmic i flexible; amb esperit blog/wiki, els objectius del qual són: la creació d'un sistema de dipositació dels OdA; l'establiment d'un sistema de recuperació dels mateixos; la implantació de fluxos d'entrada i sortida; la consolidació d'un observatori d'investigació i de recerca sobre la innovació dels OdA en els entorns i els sistemes educatius actuals; l’articulació de possibles accions patrimonials al voltant de la creació i preservació d’aquests.

Sound Texture Modelling

In the PhD thesis “Sound Texture Modeling” we deal with statistical modelling or textural sounds like water, wind, rain, etc. For synthesis and classification. Our initial model is based on a wavelet tree signal decomposition and the modeling of the resulting sequence by means of a parametric probabilistic model, that can be situated within the family of models trainable via expectation maximization (hidden Markov tree model ). Our model is able to capture key characteristics of the source textures (water, rain, fire, applause, crowd chatter ), and faithfully reproduces some of the sound classes. In terms of a more general taxonomy of natural events proposed by Graver, we worked on models for natural event classification and segmentation. While the event labels comprise physical interactions between materials that do not have textural propierties in their enterity, those segmentation models can help in identifying textural portions of an audio recording useful for analysis and resynthesis. Following our work on concatenative synthesis of musical instruments, we have developed a pattern-based synthesis system, that allows to sonically explore a database of units by means of their representation in a perceptual feature space. Concatenative syntyhesis with “molecules” built from sparse atomic representations also allows capture low-level correlations in perceptual audio features, while facilitating the manipulation of textural sounds based on their physical and perceptual properties. We have approached the problem of sound texture modelling for synthesis from different directions, namely a low-level signal-theoretic point of view through a wavelet transform, and a more high-level point of view driven by perceptual audio features in the concatenative synthesis setting. The developed framework provides unified approach to the high-quality resynthesis of natural texture sounds. Our research is embedded within the Metaverse 1 European project (2008-2011), where our models are contributting as low level building blocks within a semi-automated soundscape generation system.

When the mummy is digital : preservation and dissemination

Most educational institutions include nowadays a digital repository as part of their development and positioning strategy. The main goals of a digital repository are preservation and dissemination, which are some how contradictory, especially if the repository follows an open approach, as it is designed, built and managed from an institutional perspective, although it is intended to be used by teachers and learners. This fact may lead to a low level of usage, as final users are not able to integrate the learning object repository into their learning process. In this paper we will discuss how to promote open educational resources by connecting open repositories with open social networks, bridging the gap between resources and final users (teachers and learners).

WePreserve 2009: DPE/Planets/CASPAR/nestor Joint Training Event: The Preservation challenge: basic concepts and practical applications (Barcelona, 23-27 març 2009)

Reseña del congreso WePreserve 2009 que tuvo lugar los pasados 23 al 27 de marzo en Barcelona, organizadopor la Facultad de Biblioteconomía y Documentación de la Universitat de Barcelona, con la colaboración del Institut d'Estudis Catalans, la Biblioteca de Catalunya y el Consorci de Biblioteques de Barcelona. El seminario 2009 WePreserve se celebra anualmente desde el año 2007 y participan todas las figuras de referencia europeas en materia de investigación en sistemas y metodologías que garantizan la preservación digital de los documentos: Digital Preservation Europe (DPE), Preservation and Long-term Access Through Networked Services (Planets), Cultural Artistic and Scientific Knowledge for Preservation, Access and Retrieval (CASPAR), y Network of expertise in Digital long-term preservation (nestor).

Methodology for studying the numerical speed of sound in finite differences schemes

The space and time discretization inherent to all FDTD schemesintroduce non-physical dispersion errors, i.e. deviations ofthe speed of sound from the theoretical value predicted bythe governing Euler differential equations. A generalmethodologyfor computing this dispersion error via straightforwardnumerical simulations of the FDTD schemes is presented.The method is shown to provide remarkable accuraciesof the order of 1/1000 in a wide variety of twodimensionalfinite difference schemes.

Statistical modeling of bowing control applied to violin sound synthesis

Excitation-continuous music instrument control patterns are often not explicitly represented in current sound synthesis techniques when applied to automatic performance. Both physical model-based and sample-based synthesis paradigmswould benefit from a flexible and accurate instrument control model, enabling the improvement of naturalness and realism. Wepresent a framework for modeling bowing control parameters inviolin performance. Nearly non-intrusive sensing techniques allow for accurate acquisition of relevant timbre-related bowing control parameter signals.We model the temporal contour of bow velocity, bow pressing force, and bow-bridge distance as sequences of short Bézier cubic curve segments. Considering different articulations, dynamics, and performance contexts, a number of note classes are defined. Contours of bowing parameters in a performance database are analyzed at note-level by following a predefined grammar that dictates characteristics of curve segment sequences for each of the classes in consideration. As a result, contour analysis of bowing parameters of each note yields an optimal representation vector that is sufficient for reconstructing original contours with significant fidelity. From the resulting representation vectors, we construct a statistical model based on Gaussian mixtures suitable for both the analysis and synthesis of bowing parameter contours. By using the estimated models, synthetic contours can be generated through a bow planning algorithm able to reproduce possible constraints caused by the finite length of the bow. Rendered contours are successfully used in two preliminary synthesis frameworks: digital waveguide-based bowed stringphysical modeling and sample-based spectral-domain synthesis.

Gesture sampling for instrumental sound synthesis: violin bowing as a case study

This paper presents a framework in which samples of bowing gesture parameters are retrieved and concatenated from a database of violin performances by attending to an annotated input score. Resulting bowing parameter signals are then used to synthesize sound by means of both a digital waveguide violin physical model, and an spectral-domainadditive synthesizer.

On the preservation of combinatorial types for maps on trees

We study the preservation of the periodic orbits of an A-monotone tree map f:T→T in the class of all tree maps g:S→S having a cycle with the same pattern as A. We prove that there is a period-preserving injective map from the set of (almost all) periodic orbits of ƒ into the set of periodic orbits of each map in the class. Moreover, the relative positions of the corresponding orbits in the trees T and S (which need not be homeomorphic) are essentially preserved

Forage preservation (grazing vs. hay) fed to ewes affects the fatty acid profile of milk and CPT1B gene expression in the sheep mammary gland

Background: Alterations in lipid metabolism occur when animals are exposed to different feeding systems. In the last few decades, the characterisation of genes involved in fat metabolism and technological advances have enabled the study of the effect of diet on the milk fatty acid (FA) profile in the mammary gland and aided in the elucidation of the mechanisms of the response to diet. The aim of this study was to evaluate the effect of different forage diets (grazing vs. hay) near the time of ewe parturition on the relationship between the fatty acid profile and gene expression in the mammary gland of the Churra Tensina sheep breed. Results: In this study, the forage type affected the C18:2 cis-9 trans-11 (CLA) and long-chain saturated fatty acid (LCFA) content, with higher percentages during grazing than during hay feeding. This may suggest that these FAs act as regulatory factors for the transcriptional control of the carnitine palmitoyltransferase 1B (CPT1B) gene, which was more highly expressed in the grazing group (GRE). The most highly expressed gene in the mammary gland at the fifth week of lactation is CAAT/ enhancer- binding protein beta (CEBPB), possibly due to its role in milk fat synthesis in the mammary gland. More stable housekeeping genes in the ovine mammary gland that would be appropriate for use in gene expression studies were ribosomal protein L19 (RPL19) and glyceraldehyde- 3- phosphate dehydrogenase (GAPDH). Conclusions: Small changes in diet, such as the forage preservation (grazing vs. hay), can affect the milk fatty acid profile and the expression of the CPT1B gene, which is associated with the oxidation of fatty acids. When compared to hay fed indoors, grazing fresh low mountain pastures stimulates the milk content of CLA and LCFA via mammary uptake. In this sense, LCFA in milk may be acting as a regulatory factor for transcriptional control of the CPT1B gene, which was more highly expressed in the grazing group.

Shout and Act: an Algorithm for Digital Objects Preservation Inspired from Rescue Robots

We adapt the Shout and Act algorithm to Digital Objects Preservation where agents explore file systems looking for digital objects to be preserved (victims). When they find something they “shout” so that agent mates can hear it. The louder the shout, the urgent or most important the finding is. Louder shouts can also refer to closeness. We perform several experiments to show that this system works very scalably, showing that heterogeneous teams of agents outperform homogeneous ones over a wide range of tasks complexity. The target at-risk documents are MS Office documents (including an RTF file) with Excel content or in Excel format. Thus, an interesting conclusion from the experiments is that fewer heterogeneous (varying skills) agents can equal the performance of many homogeneous (combined super-skilled) agents, implying significant performance increases with lower overall cost growth. Our results impact the design of Digital Objects Preservation teams: a properly designed combination of heterogeneous teams is cheaper and more scalable when confronted with uncertain maps of digital objects that need to be preserved. A cost pyramid is proposed for engineers to use for modeling the most effective agent combinations

Sexual dimorphism of sonic apparatus and extreme intersexual variation of sounds in Ophidion rochei (Ophidiidae): first evidence of a tight relationship between morphology and sound characteristics in Ophidiidae

Many Ophidiidae are active in dark environments and display complex sonic apparatus morphologies. However, sound recordings are scarce and little is known about acoustic communication in this family. This paper focuses on Ophidion rochei which is known to display an important sexual dimorphism in swimbladder and anterior skeleton. The aims of this study were to compare the sound producing morphology, and the resulting sounds in juveniles, females and males of O. rochei.Results: Males, females, and juveniles possessed different morphotypes. Females and juveniles contrasted with males because they possessed dramatic differences in morphology of their sonic muscles, swimbladder, supraoccipital crest, and first vertebrae and associated ribs. Further, they lacked the ‘rocker bone’ typically found in males. Sounds from each morphotype were highly divergent. Males generally produced non harmonic, multiple-pulsed sounds that lasted for several seconds (3.5 ± 1.3 s) with a pulse period of ca. 100 ms. Juvenile and female sounds were recorded for the first time in ophidiids. Female sounds were harmonic, had shorter pulse period (±3.7 ms), and never exceeded a few dozen milliseconds (18 ± 11 ms). Moreover, unlike male sounds, female sounds did not have alternating long and short pulse periods. Juvenile sounds were weaker but appear to be similar to female sounds.Conclusions: Although it is not possible to distinguish externally male from female in O. rochei, they show a sonic apparatus and sounds that are dramatically different. This difference is likely due to their nocturnal habits that may have favored the evolution of internal secondary sexual characters that help to distinguish males from females and that could facilitate mate choice by females. Moreover, the comparison of different morphotypes in this study shows that these morphological differences result from a peramorphosis that takes place during the development of the gonads

Animal sound : què se sent quan s’és una cabra?

Aquest projecte treballa sobre la possibilitat d'aplicar tècniques d'enregistrament binaural en animals, en concret, en una cabra domèstica. Mitjançant uns micròfons col·locats dins les seves orelles i una petita càmera de vídeo muntada sobre el seu cap, s'obté un material audiovisual que permet fer-se una idea aproximada de com és la seva percepció del món. En base a un estudi de cognició comparada, es pretén trobar maneres de transformar els enregistraments obtinguts per tal d'adaptar el marc psicoacústic humà al de l'animal. L'objectiu és que una persona pugui sentir com ho fa un animal, encara que sigui d'una manera aproximada. Els materials obtinguts al llarg dels enregistraments són el punt de partida per a la construcció de paisatges sonors i peces audiovisuals diverses. Així doncs, el treball s'inicia amb el disseny i construcció d'un micròfon binaural, continua amb enregistraments de camp en animals i acaba amb l'edició, el processat i la composició dels paisatges sonors finals.