Towards an emotion-driven interactive music system: ridging the gaps between affect, physiology and sound generation

Report for the scientific sojourn at the Stanford University from January until June 2007. Music is well known for affecting human emotional states, yet the relationship between specific musical parameters and emotional responses is still not clear. With the advent of new human-computer interaction (HCI) technologies, it is now possible to derive emotion-related information from physiological data and use it as an input to interactive music systems. Providing such implicit musical HCI will be highly relevant for a number of applications including music therapy, diagnosis, nteractive gaming, and physiologically-based musical instruments. A key question in such physiology-based compositions is how sound synthesis parameters can be mapped to emotional states of valence and arousal. We used both verbal and heart rate responses to evaluate the affective power of five musical parameters. Our results show that a significant correlation exists between heart rate and the subjective evaluation of well-defined musical parameters. Brightness and loudness showed to be arousing parameters on subjective scale while harmonicity and even partial attenuation factor resulted in heart rate changes typically associated to valence. This demonstrates that a rational approach to designing emotion-driven music systems for our public installations and music therapy applications is possible.

The spatio-temporal brain dynamics of processing and integrating sound localization cues in humans.

Interaural intensity and time differences (IID and ITD) are two binaural auditory cues for localizing sounds in space. This study investigated the spatio-temporal brain mechanisms for processing and integrating IID and ITD cues in humans. Auditory-evoked potentials were recorded, while subjects passively listened to noise bursts lateralized with IID, ITD or both cues simultaneously, as well as a more frequent centrally presented noise. In a separate psychophysical experiment, subjects actively discriminated lateralized from centrally presented stimuli. IID and ITD cues elicited different electric field topographies starting at approximately 75 ms post-stimulus onset, indicative of the engagement of distinct cortical networks. By contrast, no performance differences were observed between IID and ITD cues during the psychophysical experiment. Subjects did, however, respond significantly faster and more accurately when both cues were presented simultaneously. This performance facilitation exceeded predictions from probability summation, suggestive of interactions in neural processing of IID and ITD cues. Supra-additive neural response interactions as well as topographic modulations were indeed observed approximately 200 ms post-stimulus for the comparison of responses to the simultaneous presentation of both cues with the mean of those to separate IID and ITD cues. Source estimations revealed differential processing of IID and ITD cues initially within superior temporal cortices and also at later stages within temporo-parietal and inferior frontal cortices. Differences were principally in terms of hemispheric lateralization. The collective psychophysical and electrophysiological results support the hypothesis that IID and ITD cues are processed by distinct, but interacting, cortical networks that can in turn facilitate auditory localization.

Sound objects in time, space and action

SOUND OBJECTS IN TIME, SPACE AND ACTIONThe term "sound object" describes an auditory experience that is associated with an acoustic event produced by a sound source. At cortical level, sound objects are represented by temporo-spatial activity patterns within distributed neural networks. This investigation concerns temporal, spatial and action aspects as assessed in normal subjects using electrical imaging or measurement of motor activity induced by transcranial magnetic stimulation (TMS).Hearing the same sound again has been shown to facilitate behavioral responses (repetition priming) and to modulate neural activity (repetition suppression). In natural settings the same source is often heard again and again, with variations in spectro-temporal and spatial characteristics. I have investigated how such repeats influence response times in a living vs. non-living categorization task and the associated spatio-temporal patterns of brain activity in humans. Dynamic analysis of distributed source estimations revealed differential sound object representations within the auditory cortex as a function of the temporal history of exposure to these objects. Often heard sounds are coded by a modulation in a bilateral network. Recently heard sounds, independently of the number of previous exposures, are coded by a modulation of a left-sided network.With sound objects which carry spatial information, I have investigated how spatial aspects of the repeats influence neural representations. Dynamics analyses of distributed source estimations revealed an ultra rapid discrimination of sound objects which are characterized by spatial cues. This discrimination involved two temporo-spatially distinct cortical representations, one associated with position-independent and the other with position-linked representations within the auditory ventral/what stream.Action-related sounds were shown to increase the excitability of motoneurons within the primary motor cortex, possibly via an input from the mirror neuron system. The role of motor representations remains unclear. I have investigated repetition priming-induced plasticity of the motor representations of action sounds with the measurement of motor activity induced by TMS pulses applied on the hand motor cortex. TMS delivered to the hand area within the primary motor cortex yielded larger magnetic evoked potentials (MEPs) while the subject was listening to sounds associated with manual than non- manual actions. Repetition suppression was observed at motoneuron level, since during a repeated exposure to the same manual action sound the MEPs were smaller. I discuss these results in terms of specialized neural network involved in sound processing, which is characterized by repetition-induced plasticity.Thus, neural networks which underlie sound object representations are characterized by modulations which keep track of the temporal and spatial history of the sound and, in case of action related sounds, also of the way in which the sound is produced.LES OBJETS SONORES AU TRAVERS DU TEMPS, DE L'ESPACE ET DES ACTIONSLe terme "objet sonore" décrit une expérience auditive associée avec un événement acoustique produit par une source sonore. Au niveau cortical, les objets sonores sont représentés par des patterns d'activités dans des réseaux neuronaux distribués. Ce travail traite les aspects temporels, spatiaux et liés aux actions, évalués à l'aide de l'imagerie électrique ou par des mesures de l'activité motrice induite par stimulation magnétique trans-crânienne (SMT) chez des sujets sains. Entendre le même son de façon répétitive facilite la réponse comportementale (amorçage de répétition) et module l'activité neuronale (suppression liée à la répétition). Dans un cadre naturel, la même source est souvent entendue plusieurs fois, avec des variations spectro-temporelles et de ses caractéristiques spatiales. J'ai étudié la façon dont ces répétitions influencent le temps de réponse lors d'une tâche de catégorisation vivant vs. non-vivant, et les patterns d'activité cérébrale qui lui sont associés. Des analyses dynamiques d'estimations de sources ont révélé des représentations différenciées des objets sonores au niveau du cortex auditif en fonction de l'historique d'exposition à ces objets. Les sons souvent entendus sont codés par des modulations d'un réseau bilatéral. Les sons récemment entendus sont codé par des modulations d'un réseau du côté gauche, indépendamment du nombre d'expositions. Avec des objets sonores véhiculant de l'information spatiale, j'ai étudié la façon dont les aspects spatiaux des sons répétés influencent les représentations neuronales. Des analyses dynamiques d'estimations de sources ont révélé une discrimination ultra rapide des objets sonores caractérisés par des indices spatiaux. Cette discrimination implique deux représentations corticales temporellement et spatialement distinctes, l'une associée à des représentations indépendantes de la position et l'autre à des représentations liées à la position. Ces représentations sont localisées dans la voie auditive ventrale du "quoi".Des sons d'actions augmentent l'excitabilité des motoneurones dans le cortex moteur primaire, possiblement par une afférence du system des neurones miroir. Le rôle des représentations motrices des sons d'actions reste peu clair. J'ai étudié la plasticité des représentations motrices induites par l'amorçage de répétition à l'aide de mesures de potentiels moteurs évoqués (PMEs) induits par des pulsations de SMT sur le cortex moteur de la main. La SMT appliquée sur le cortex moteur primaire de la main produit de plus grands PMEs alors que les sujets écoutent des sons associée à des actions manuelles en comparaison avec des sons d'actions non manuelles. Une suppression liée à la répétition a été observée au niveau des motoneurones, étant donné que lors de l'exposition répétée au son de la même action manuelle les PMEs étaient plus petits. Ces résultats sont discuté en termes de réseaux neuronaux spécialisés impliqués dans le traitement des sons et caractérisés par de la plasticité induite par la répétition. Ainsi, les réseaux neuronaux qui sous-tendent les représentations des objets sonores sont caractérisés par des modulations qui gardent une trace de l'histoire temporelle et spatiale du son ainsi que de la manière dont le son a été produit, en cas de sons d'actions.

Sound Texture Modelling

In the PhD thesis “Sound Texture Modeling” we deal with statistical modelling or textural sounds like water, wind, rain, etc. For synthesis and classification. Our initial model is based on a wavelet tree signal decomposition and the modeling of the resulting sequence by means of a parametric probabilistic model, that can be situated within the family of models trainable via expectation maximization (hidden Markov tree model ). Our model is able to capture key characteristics of the source textures (water, rain, fire, applause, crowd chatter ), and faithfully reproduces some of the sound classes. In terms of a more general taxonomy of natural events proposed by Graver, we worked on models for natural event classification and segmentation. While the event labels comprise physical interactions between materials that do not have textural propierties in their enterity, those segmentation models can help in identifying textural portions of an audio recording useful for analysis and resynthesis. Following our work on concatenative synthesis of musical instruments, we have developed a pattern-based synthesis system, that allows to sonically explore a database of units by means of their representation in a perceptual feature space. Concatenative syntyhesis with “molecules” built from sparse atomic representations also allows capture low-level correlations in perceptual audio features, while facilitating the manipulation of textural sounds based on their physical and perceptual properties. We have approached the problem of sound texture modelling for synthesis from different directions, namely a low-level signal-theoretic point of view through a wavelet transform, and a more high-level point of view driven by perceptual audio features in the concatenative synthesis setting. The developed framework provides unified approach to the high-quality resynthesis of natural texture sounds. Our research is embedded within the Metaverse 1 European project (2008-2011), where our models are contributting as low level building blocks within a semi-automated soundscape generation system.

Evaluation de la qualité de vie après prélèvement de la veine fémorale superficielle lors de reconstructions infra-inguinales [Quality of life after superficial femoral vein harvest for infra-inguinal reconstructions]

BACKGROUND: The superficial femoral vein (SFV) is a well-established alternative conduit for infra-inguinal reconstructivenous hypertension after SFV harvest may however result in significant morbidity. This study reports the efficiency of SFV as conduit for infra-inguinal reconstructions and characterize the anatomic and physiologic changes in harvest limbs and their relationship to the development of venous complications. METHODS: From May 1999 through November 2003, 23 SFV were harvested from 21 patients undergoing infra-inguinal reconstructions. Bypasses were controlled by regular duplex-ultrasound. The venous morbidity was assessed by measurements of leg circumferences, strain-gauge plethysmography and quality of life, investigated by the VEINES-QOL scale. RESULTS: At a mean follow-up of 10.4 months (range 1-56), primary, secondary patency and limb salvage rates of infra-inguinal bypasses using SFV are 71.4%, 76.2% and 85.7% respectively. No patient had major venous claudication. Oedema was significantly present in nine patients. Strain-gauge plethysmography showed outflow obstruction in all patients. The VEINES-QOL assessment showed no limitation in social and domestic activity, moderate complain about leg heaviness despite presence of oedema. CONCLUSION: The SFV harvest is a reliable conduit for infra-inguinal reconstructions and results in moderate venous morbidity in terms of functional consequences and quality of life.

Methodology for studying the numerical speed of sound in finite differences schemes

The space and time discretization inherent to all FDTD schemesintroduce non-physical dispersion errors, i.e. deviations ofthe speed of sound from the theoretical value predicted bythe governing Euler differential equations. A generalmethodologyfor computing this dispersion error via straightforwardnumerical simulations of the FDTD schemes is presented.The method is shown to provide remarkable accuraciesof the order of 1/1000 in a wide variety of twodimensionalfinite difference schemes.

A Review of the Revitalize Iowa's Sound Economy Program, Administered by the Department of Transportation, July 1, 1986 through June 30, 2004

Other Audit Reports - Review

Importância da criação de infra-estruturas e da formação de recursos humanos no desenvolvimento

Este Trabalho procurou ser uma analise da importância das infrastuturas e da formação de recursios humanos no desevolvimento dos pequenos estados insulares em desenvolvimento. Sendo paises com especialidades muito proprias que lhes conferem tambem niveis diferentes de vilnerabilidade, o estudo empirico procurou mostrar que no caso de Cabo Verde e de São Tomé e Principe o investimento na criação de infra-estruturas, quando associado à prestação de serviços de qualidade, pode ser directamente gerador de valor, induzir outros investimentos ou criar condições sociais para o desenvolvimneto.... 6 D = ) + + ! + + * 7 ! ) " *+ + " , * ! , " > - * , + = , ! 4 + ! , + , # + 2 + + - " ) , + + ! , + + + , + + + + ’ 6 , E + ) , " 6 ! + + + ) ! 6 ! + ) 2 ! *+ ! + + + 6 = ! 7 ! F ( + + = , B , " D ! 4 ) + + 9 " 4 " 6 , " > - * * , ! ) " ) 6 D = ) + + ! + + * 7 ! ) " *+ + " , * ! , " > - * , + = , ! 4 + ! , + , # + 2 + + - " ) , + + ! , + + + , + + + + ’ 6 , E + ) , " 6 ! + + + ) ! 6 ! + ) 2 ! *+ ! + + + 6 = ! 7 ! F ( + + = , B , " D ! 4 ) + + 9 " 4 " 6 , " > - * * , ! ) + + v

Infra-estrutura de dados espaciais como ferramenta de integração, disseminação e análise de dados para o sistema estatístico nacional de Cabo Verde

O maior benefício de uma Infra-estrutura de Dados Espaciais (IDE) é essencialmente prover as condições para a coordenação, integração, troca e partilha de dados geográficos entre diferentes actores de vários níveis da comunidade de dados espaciais. Assumem uma grande importância no contexto da gestão de informação espacial para a tomada de decisões, como forma de garantir um desenvolvimento economicamente e ambientalmente equilibrado. Ao longo dos últimos anos têm sido apontados vários casos de sucesso de implementação de IDE em diferentes países, e, a diferentes níveis políticoadministrativos. Porém, nem sempre o desenvolvimento e a implementação de IDE é realizada de uma forma integrada quer para promover a interacção entre diferentes níveis de implementação, quer para assegurar o envolvimento das comunidades, de uma forma geral, de todos aqueles que se beneficiam com seu desenvolvimento. Há ainda um longo caminho a percorrer particularmente na África onde poucas são as iniciativas apoiadas por estruturas do governo central, e onde é muito popular se encontrar outras entidades responsáveis pelas iniciativas. Com essa dissertação faz-se uma análise da importância e pertinência da criação de uma Infra-estrutura de Dados Espaciais que permita a disseminação e análise de dados produzidos pelo Órgãos Produtores das Estatísticas Oficiais do Sistema Estatístico Nacional de Cabo Verde, de forma a facilitar o acesso à toda informação estatística produzida.

Sistema de Informação Geográfica na Internet para apoio à gestão de Infra-estruturas, baseado em ferramentas Open Source

A gestão de infra-estruturas é um processo de coordenação, avaliação sistemática e manutenção efectiva das infra-estruturas relacionadas com os serviços básicos. A gestão eficaz de infra-estruturas exige que a disposição dos equipamentos no meio urbano seja conhecida, bem como sua relação com o uso do solo. Sendo a gestão de infra-estruturas uma questão de natureza espacial, é natural pensar na utilização de um Sistema de Informação Geográfica como ferramenta com potencial de grande utilidade, pois permite a georreferenciação de dados espaciais e sua interligação com atributos alfanuméricos, para além da realização de análises complexas e a possibilidade de simular diversos cenários de modo a propiciar uma tomada de decisão eficaz. O elevado custo das licenças e a especificidade de utilização a nível técnico, tornam-se obstáculos na criação e manutenção de dados geográficos. Contudo os recentes desenvolvimentos em tecnologias da Internet têm contribuído para o acesso, publicação, exploração e distribuição da Informação Geográfica. A utilização de SIG distribuídos na Internet (WebGIS), nomeadamente na vertente livre, pode ser uma solução adequada visto que coloca funcionalidades de SIG ao alcance de utilizadores, através de um simples browser, sem necessidade de investimentos em relação a software ou mesmo em formação técnica especializada. Assim, com este projecto pretende-se a criação de um sistema de Informação geográfico na Internet, utilizando softwares livres, que disponibiliza toda a informação geográfica e alfanumérica das infra-estruturas construídas e sob a responsabilidade do Ministério das Infra-estruturas e Economia Marítima, permitindo a visualização e a realização de pesquisas e operações de análise.

Infra-estrutura Tecnológica de Suporte –Censo 2010

Com a graduação de Cabo Verde para país de rendimento médio, admite-se que o Sistema Estatístico Nacional (SEN) deverá estar à altura dos novos desafios impostos pela necessidade de responder com mais rigor e qualidade aos decisores políticos, investidores, investigadores e ao público em geral. O projecto desenvolvido durante o estágio, que ora se apresenta, aborda a infra-estrutura tecnológica de suporte – Censo 2010, cujos principais objectivos são a montagem e a instalação do Servidor que suporta o armazenamento de dados do Recenseamento Geral da população recolhida no terreno, cujas metodologias utilizadas para materialização do trabalho foram a pesquisa em Internet, o estágio propriamente dito e a concretização do Servidor com sistema operativo Windows Server 2003. O estágio teve como resultado a montagem do Servidor, como Controlador de Domínio, desempenhando o papel de Active Directory, a criação de um interface de transmissão de dados on line. O Servidor armazena dados que são transmitidos via Portal do INE, mediante um acesso restrito, permitido somente aos utilizadores credenciados. Toda a infra-estrutura foi pensada para garantir o sucesso na transmissão de dados on line, uma vez que, ao contrário nos censos anteriores, em que a recolha era feita de forma tradicional, isto é utilizando questionários em formato de papel, nestes censos, o INE opta por utilizar questionários em formatos digitais e introduziu como uma grande inovação tecnológica o uso de Computadores de mão (PDA) para a recolha de dados no terreno. Para compilação de informação no terreno teve-se como suporte um PDA baseado no geoprocessamento dos dados com utilização do Global Position System (GPS), sendo uma de suas vantagens permitir a captura das respectivas coordenadas do ponto de recolha. O último Recenseamento Geral realizou-se no ano 2000. Com essa operação, nesta fase, pretende-se actualizar os dados existentes anteriormente, e obter novos dados dos edifícios construídos, o que permitirá fazer a actualização da cartografia censitária, dados esses que servirão de base à realização do Recenseamento Geral da População Habitacional (RGPH) 2010.

Where sound position influences sound object representations: a 7-T fMRI study.

Evidence from human and non-human primate studies supports a dual-pathway model of audition, with partially segregated cortical networks for sound recognition and sound localisation, referred to as the What and Where processing streams. In normal subjects, these two networks overlap partially on the supra-temporal plane, suggesting that some early-stage auditory areas are involved in processing of either auditory feature alone or of both. Using high-resolution 7-T fMRI we have investigated the influence of positional information on sound object representations by comparing activation patterns to environmental sounds lateralised to the right or left ear. While unilaterally presented sounds induced bilateral activation, small clusters in specific non-primary auditory areas were significantly more activated by contra-laterally presented stimuli. Comparison of these data with histologically identified non-primary auditory areas suggests that the coding of sound objects within early-stage auditory areas lateral and posterior to primary auditory cortex AI is modulated by the position of the sound, while that within anterior areas is not.

Affective state and voice : cross-cultural assessment of speaking behavior and voice sound characteristics : a normative multicenter study of 577 + 36 healthy subjects

BACKGROUND: Human speech is greatly influenced by the speakers' affective state, such as sadness, happiness, grief, guilt, fear, anger, aggression, faintheartedness, shame, sexual arousal, love, amongst others. Attentive listeners discover a lot about the affective state of their dialog partners with no great effort, and without having to talk about it explicitly during a conversation or on the phone. On the other hand, speech dysfunctions, such as slow, delayed or monotonous speech, are prominent features of affective disorders. METHODS: This project was comprised of four studies with healthy volunteers from Bristol (English: n = 117), Lausanne (French: n = 128), Zurich (German: n = 208), and Valencia (Spanish: n = 124). All samples were stratified according to gender, age, and education. The specific study design with different types of spoken text along with repeated assessments at 14-day intervals allowed us to estimate the 'natural' variation of speech parameters over time, and to analyze the sensitivity of speech parameters with respect to form and content of spoken text. Additionally, our project included a longitudinal self-assessment study with university students from Zurich (n = 18) and unemployed adults from Valencia (n = 18) in order to test the feasibility of the speech analysis method in home environments. RESULTS: The normative data showed that speaking behavior and voice sound characteristics can be quantified in a reproducible and language-independent way. The high resolution of the method was verified by a computerized assignment of speech parameter patterns to languages at a success rate of 90%, while the correct assignment to texts was 70%. In the longitudinal self-assessment study we calculated individual 'baselines' for each test person along with deviations thereof. The significance of such deviations was assessed through the normative reference data. CONCLUSIONS: Our data provided gender-, age-, and language-specific thresholds that allow one to reliably distinguish between 'natural fluctuations' and 'significant changes'. The longitudinal self-assessment study with repeated assessments at 1-day intervals over 14 days demonstrated the feasibility and efficiency of the speech analysis method in home environments, thus clearing the way to a broader range of applications in psychiatry. © 2014 S. Karger AG, Basel.