Game Audio in Audio Games : Towards a Theory on the Roles and Functions of Sound in Audio Games

For the past few decades, researchers have increased our understanding of how sound functions within various audio–visual media formats. With a different focus in mind, this study aims to identify the roles and functions of sound in relation to the game form Audio Games, in order to explore the potential of sound when acting as an autonomous narrative form. Because this is still a relatively unexplored research field, the main purpose of this study is to help establish a theoretical ground and stimulate further research within the field of audio games. By adopting an interdisciplinary approach to the topic, this research relies on theoretical studies, examinations of audio games and contact with the audio game community. In order to reveal the roles of sound, the gathered data is analyzed according to both a contextual and a functional perspective. The research shows that a distinction between the terms ‘function’ and ‘role’ is important when analyzing sound in digital games. The analysis therefore results in the identification of two analytical levels that help define the functions and roles of an entity within a social context, named the Functional and the Interfunctional levels. In addition to successfully identifying three main roles of sound within audio games—each describing the relationship between sound and the entities game system, player and virtual environment—many other issues are also addressed. Consequently, and in accordance with its purpose, this study provides a broad foundation for further research of sound in both audio games and video games.

L’écoute de la peur : une étude du son dans les jeux vidéo d’horreur

S’intéressant aux différents rôles du son dans les jeux vidéo d’horreur, ce mémoire vise à exposer le travail cognitif effectué par le joueur lors de son activité de jeu. De la structuration des sons jusqu’à la production de sens à partir de leurs fonctions, cette recherche mesure l’implication du phénomène sonore dans la mise en scène de la peur vidéoludique. Dans cette optique, elle présente, critique et développe une pluralité de concepts portant sur la jouabilité, les postures d’écoute, la diégèse, les générateurs sonores, les fonctions sonores systémiques et immersives ainsi que sur la création de la peur à l’aide de différentes stratégies sonores.

The functions of Japanese Sound Symbolic Words in Different Types of Texts and Their Translation

Though sound symbolic words (onomatopoeia and mimetic words, or giongo and gitaigo in Japanese) exist in other languages, it would not be so easy to compare them to those in Japanese. This is because unlike in Japanese, in many other languages (here we see English and Spanish) sound symbolic words do not have distinctive forms that separate them immediately from the rest of categories of words. In Japanese, a sound symbolic word has a radical (that is based on the elaborated Japanese sound symbolic system), and often a suffix that shows subtle nuance. Together they give the word a distinctive form that differentiates it from other categories of words, though its grammatical functions could vary, especially in the case of mimetic words (gitaigo). Without such an obvious feature, in other languages, it would not be always easy to separate sound symbolic words from the rest. These expressions are extremely common and used in almost all types of text in Japanese, but their elaborated sound symbolic system and possibly their various grammatical functions are making giongo and gitaigo one of the most difficult challenges for the foreign students and translators. Studying the translation of these expressions into other languages might give some indication related to the comparison of Japanese sound symbolic words and those in other languages. Though sound symbolic words are present in many types of texts in Japanese, their functions in traditional forms of text (letters only) and manga (Japanese comics)are different and they should be treated separately. For example, in traditional types of text such as novels, the vast majority of the sound symbolic words used are mimetic words (gitaigo) and most of them are used as adverbs, whereas in manga, the majority of the sound symbolic words used (excluding those appear within the speech bubbles) are onomatopoeias (giongo) and often used on their own (i.e. not as a part of a sentence). Naturally, the techniques used to translate these expressions in the above two types of documents differ greatly. The presentation will focus on i) grammatical functions of Japanese sound symbolic words in traditional types of texts (novels/poems) and in manga works, and ii) whether their features and functions are maintained (i.e. whether they are translated as sound symbolic words) when translated into other languages (English and Spanish). The latter point should be related to a comparison of sound symbolic words in Japanese and other languages, which will be also discussed.

Structure-based modeling of head -related transfer functions towards interactive customization of binaural sound systems

One of the most popular techniques for creating spatialized virtual sounds is based on the use of Head-Related Transfer Functions (HRTFs). HRTFs are signal processing models that represent the modifications undergone by the acoustic signal as it travels from a sound source to each of the listener's eardrums. These modifications are due to the interaction of the acoustic waves with the listener's torso, shoulders, head and pinnae, or outer ears. As such, HRTFs are somewhat different for each listener. For a listener to perceive synthesized 3-D sound cues correctly, the synthesized cues must be similar to the listener's own HRTFs. ^ One can measure individual HRTFs using specialized recording systems, however, these systems are prohibitively expensive and restrict the portability of the 3-D sound system. HRTF-based systems also face several computational challenges. This dissertation presents an alternative method for the synthesis of binaural spatialized sounds. The sound entering the pinna undergoes several reflective, diffractive and resonant phenomena, which determine the HRTF. Using signal processing tools, such as Prony's signal modeling method, an appropriate set of time delays and a resonant frequency were used to approximate the measured Head-Related Impulse Responses (HRIRs). Statistical analysis was used to find out empirical equations describing how the reflections and resonances are determined by the shape and size of the pinna features obtained from 3D images of 15 experimental subjects modeled in the project. These equations were used to yield “Model HRTFs” that can create elevation effects. ^ Listening tests conducted on 10 subjects show that these model HRTFs are 5% more effective than generic HRTFs when it comes to localizing sounds in the frontal plane. The number of reversals (perception of sound source above the horizontal plane when actually it is below the plane and vice versa) was also reduced by 5.7%, showing the perceptual effectiveness of this approach. The model is simple, yet versatile because it relies on easy to measure parameters to create an individualized HRTF. This low-order parameterized model also reduces the computational and storage demands, while maintaining a sufficient number of perceptually relevant spectral cues. ^

The role of the right parietal cortex in sound localization: a chronometric single pulse transcranial magnetic stimulation study.

Auditory spatial functions, including the ability to discriminate between the positions of nearby sound sources, are subserved by a large temporo-parieto-frontal network. With the aim of determining whether and when the parietal contribution is critical for auditory spatial discrimination, we applied single pulse transcranial magnetic stimulation on the right parietal cortex 20, 80, 90 and 150 ms post-stimulus onset while participants completed a two-alternative forced choice auditory spatial discrimination task in the left or right hemispace. Our results reveal that transient TMS disruption of right parietal activity impairs spatial discrimination when applied at 20 ms post-stimulus onset for sounds presented in the left (controlateral) hemispace and at 80 ms for sounds presented in the right hemispace. We interpret our finding in terms of a critical role for controlateral temporo-parietal cortices over initial stages of the building-up of auditory spatial representation and for a right hemispheric specialization in integrating the whole auditory space over subsequent, higher-order processing stages.

Efficient calculation of two-dimensional periodic and waveguide acoustic Green’s functions

New representations and efficient calculation methods are derived for the problem of propagation from an infinite regularly spaced array of coherent line sources above a homogeneous impedance plane, and for the Green's function for sound propagation in the canyon formed by two infinitely high, parallel rigid or sound soft walls and an impedance ground surface. The infinite sum of source contributions is replaced by a finite sum and the remainder is expressed as a Laplace-type integral. A pole subtraction technique is used to remove poles in the integrand which lie near the path of integration, obtaining a smooth integrand, more suitable for numerical integration, and a specific numerical integration method is proposed. Numerical experiments show highly accurate results across the frequency spectrum for a range of ground surface types. It is expected that the methods proposed will prove useful in boundary element modeling of noise propagation in canyon streets and in ducts, and for problems of scattering by periodic surfaces.

How do laryngeal and respiratory functions contribute to differentiate actors/actresses and untrained voices?

Purpose. The present study aimed to compare actors/actresses's voices and vocally trained subjects through aerodynamic and electroglottographic (EGG) analyses. We hypothesized that glottal and breathing functions would reflect technical and physiological differences between vocally trained and untrained subjects.Methods. Forty participants with normal voices participated in this study (20 professional theater actors and 20 untrained participants). In each group, 10 male and 10 female subjects were assessed. All participants underwent aerodynamic and EGG assessment of voice. From the Phonatory Aerodynamic System, three protocols were used: comfortable sustained phonation with EGG, voice efficiency with EGG, and running speech. Contact quotient was calculated from EGG. All phonatory tasks were produced at three different loudness levels. Mean sound pressure level and fundamental frequency were also assessed. Univariate, multivariate, and correlation statistical analyses were performed.Results. Main differences between vocally trained and untrained participants were found in the following variables: mean sound pressure level, phonatory airflow, subglottic pressure, inspiratory airflow duration, inspiratory airflow, and inspiratory volume. These variables were greater for trained participants. Mean pitch was found to be lower for trained voices.Conclusions. The glottal source seemed to have a weak contribution when differentiating the training status in speaking voice. More prominent changes between vocally trained and untrained participants are demonstrated in respiratory-related variables. These findings may be related to better management of breathing function (better breath support).

Hydrodynamic correlation functions of a driven granular fluid in steady state

We study a homogeneously driven granular fluid of hard spheres at intermediate volume fractions and focus on time-delayed correlation functions in the stationary state. Inelastic collisions are modeled by incomplete normal restitution, allowing for efficient simulations with an event-driven algorithm. The incoherent scattering function Fincoh(q,t ) is seen to follow time-density superposition with a relaxation time that increases significantly as the volume fraction increases. The statistics of particle displacements is approximately Gaussian. For the coherent scattering function S(q,ω), we compare our results to the predictions of generalized fluctuating hydrodynamics, which takes into account that temperature fluctuations decay either diffusively or with a finite relaxation rate, depending on wave number and inelasticity. For sufficiently small wave number q we observe sound waves in the coherent scattering function S(q,ω) and the longitudinal current correlation function Cl(q,ω). We determine the speed of sound and the transport coefficients and compare them to the results of kinetic theory.

3-dimensional binaural sound rendering using acoustic measurements stored in the SOFA format

Several groups all over the world are researching in several ways to render 3D sounds. One way to achieve this is to use Head Related Transfer Functions (HRTFs). These measurements contain the Frequency Response of the human head and torso for each angle. Some years ago, was only possible to measure these Frequency Responses only in the horizontal plane. Nowadays, several improvements have made possible to measure and use 3D data for this purpose. The problem was that the groups didn't have a standard format file to store the data. That was a problem when a third part wanted to use some different HRTFs for 3D audio rendering. Every of them have different ways to store the data. The Spatially Oriented Format for Acoustics or SOFA was created to provide a solution to this problem. It is a format definition to unify all the previous different ways of storing any kind of acoustics data. At the moment of this project they have defined some basis for the format and some recommendations to store HRTFs. It is actually under development, so several changes could come. The SOFA[1] file format uses a numeric container called netCDF[2], specifically the Enhaced data model described in netCDF 4 that is based on HDF5[3]. The SoundScape Renderer (SSR) is a tool for real-time spatial audio reproduction providing a variety of rendering algorithms. The SSR was developed at the Quality and Usability Lab at TU Berlin and is now further developed at the Institut für Nachrichtentechnik at Universität Rostock [4]. This project is intended to be an introduction to the use of SOFA files, providing a C++ API to manipulate them and adapt the binaural renderer of the SSR for working with the SOFA format. RESUMEN. El SSR (SoundScape Renderer) es un programa que está siendo desarrollado actualmente por la Universität Rostock, y previamente por la Technische Universität Berlin. El SSR es una herramienta diseñada para la reproducción y renderización de audio 2D en tiempo real. Para ello utiliza diversos algoritmos, algunos orientados a sistemas formados por arrays de altavoces en diferentes configuraciones y otros algoritmos diseñados para cascos. El principal objetivo de este proyecto es dotar al SSR de la capacidad de renderizar sonidos binaurales en 3D. Este proyecto está centrado en el binaural renderer del SSR. Este algoritmo se basa en el uso de HRTFs (Head Related Transfer Function). Las HRTFs representan la función de transferencia del sistema formado por la cabeza y el torso del oyente. Esta función es medida desde diferentes ángulos. Con estos datos el binaural renderer puede generar audio en tiempo real simulando la posición de diferentes fuentes. Para poder incluir una base de datos con HRTFs en 3D se ha hecho uso del nuevo formato SOFA (Spatially Oriented Format for Acoustics). Este nuevo formato se encuentra en una fase bastante temprana de su desarrollo. Está pensado para servir como formato estándar para almacenar HRTFs y cualquier otro tipo de medidas acústicas, ya que actualmente cada laboratorio cuenta con su propio formato de almacenamiento y esto hace bastante difícil usar varias bases de datos diferentes en un mismo proyecto. El formato SOFA hace uso del contenedor numérico netCDF, que a su vez esta basado en un contenedor más básico llamado HRTF-5. Para poder incluir el formato SOFA en el binaural renderer del SSR se ha desarrollado una API en C++ para poder crear y leer archivos SOFA con el fin de utilizar los datos contenidos en ellos dentro del SSR.

Discovering the Parameters of Sound Creation: Physics Informed String Pedagogy

Thesis (D.M.A.)--University of Washington, 2016-06

Fast algorithms for automatic mapping with space-limited covariance functions

In this paper we discuss a fast Bayesian extension to kriging algorithms which has been used successfully for fast, automatic mapping in emergency conditions in the Spatial Interpolation Comparison 2004 (SIC2004) exercise. The application of kriging to automatic mapping raises several issues such as robustness, scalability, speed and parameter estimation. Various ad-hoc solutions have been proposed and used extensively but they lack a sound theoretical basis. In this paper we show how observations can be projected onto a representative subset of the data, without losing significant information. This allows the complexity of the algorithm to grow as O(n m 2), where n is the total number of observations and m is the size of the subset of the observations retained for prediction. The main contribution of this paper is to further extend this projective method through the application of space-limited covariance functions, which can be used as an alternative to the commonly used covariance models. In many real world applications the correlation between observations essentially vanishes beyond a certain separation distance. Thus it makes sense to use a covariance model that encompasses this belief since this leads to sparse covariance matrices for which optimised sparse matrix techniques can be used. In the presence of extreme values we show that space-limited covariance functions offer an additional benefit, they maintain the smoothness locally but at the same time lead to a more robust, and compact, global model. We show the performance of this technique coupled with the sparse extension to the kriging algorithm on synthetic data and outline a number of computational benefits such an approach brings. To test the relevance to automatic mapping we apply the method to the data used in a recent comparison of interpolation techniques (SIC2004) to map the levels of background ambient gamma radiation. © Springer-Verlag 2007.

Reduced-parameter model of head-related transfer functions for synthesized spatial audio

Digital systems can generate left and right audio channels that create the effect of virtual sound source placement (spatialization) by processing an audio signal through pairs of Head-Related Transfer Functions (HRTFs) or, equivalently, Head-Related Impulse Responses (HRIRs). The spatialization effect is better when individually-measured HRTFs or HRIRs are used than when generic ones (e.g., from a mannequin) are used. However, the measurement process is not available to the majority of users. There is ongoing interest to find mechanisms to customize HRTFs or HRIRs to a specific user, in order to achieve an improved spatialization effect for that subject. Unfortunately, the current models used for HRTFs and HRIRs contain over a hundred parameters and none of those parameters can be easily related to the characteristics of the subject. This dissertation proposes an alternative model for the representation of HRTFs, which contains at most 30 parameters, all of which have a defined functional significance. It also presents methods to obtain the value of parameters in the model to make it approximately equivalent to an individually-measured HRTF. This conversion is achieved by the systematic deconstruction of HRIR sequences through an augmented version of the Hankel Total Least Squares (HTLS) decomposition approach. An average 95% match (fit) was observed between the original HRIRs and those re-constructed from the Damped and Delayed Sinusoids (DDSs) found by the decomposition process, for ipsilateral source locations. The dissertation also introduces and evaluates an HRIR customization procedure, based on a multilinear model implemented through a 3-mode tensor, for mapping of anatomical data from the subjects to the HRIR sequences at different sound source locations. This model uses the Higher-Order Singular Value Decomposition (HOSVD) method to represent the HRIRs and is capable of generating customized HRIRs from easily attainable anatomical measurements of a new intended user of the system. Listening tests were performed to compare the spatialization performance of customized, generic and individually-measured HRIRs when they are used for synthesized spatial audio. Statistical analysis of the results confirms that the type of HRIRs used for spatialization is a significant factor in the spatialization success, with the customized HRIRs yielding better results than generic HRIRs.

In-car sound linear simulation

In recent years, vehicle acoustics have gained significant importance in new car development: increasingly advanced infotainment systems for spatial audio and sound enhancement algorithms have become the norm in modern vehicles. In the past, car manufacturers had to build numerous prototypes to study the sound behaviour inside the car cabin or the effect of new algorithms under development. Nowadays, advanced simulation techniques can reduce development costs and time. In this work, after selecting the reference test vehicle, a modern luxury sedan equipped with a high-end sound system, two independent tools were developed: a simulation tool created in the Comsol Multiphysics environment and an auralization tool developed in the Cycling ‘74 MAX environment. The simulation tool can calculate the impulse response and acoustic spectrum at a specific position inside the cockpit. Its input data are the vehicle’s geometry, acoustic absorption parameters of materials, the acoustic characteristics and position of loudspeakers, and the type and position of virtual microphones (or microphone arrays). The simulation tool can also provide binaural impulse responses thanks to Head Related Transfer Functions (HRTFs) and an innovative algorithm able to compute the HRTF at any distance and angle from the head. Impulse responses from simulations or acoustic measurements inside the car cabin are processed and fed into the auralization tool, enabling real-time interaction by applying filters, changing the channels gain or displaying the acoustic spectrum. Since the acoustic simulation of a vehicle involves multiple topics, the focus of this work has not only been the development of two tools but also the study and application of new techniques for acoustic characterization of the materials that compose the cockpit and the loudspeaker simulation. Specifically, three different methods have been applied for material characterization through the use of a pressure-velocity probe, a Laser Doppler Vibrometer (LDV), and a microphone array.

Mechanisms Predisposing Penile Fracture And Long-term Outcomes On Erectile And Voiding Functions.

Purpose. To determine the mechanisms predisposing penile fracture as well as the rate of long-term penile deformity and erectile and voiding functions. Methods. All fractures were repaired on an emergency basis via subcoronal incision and absorbable suture with simultaneous repair of eventual urethral lesion. Patients' status before fracture and voiding and erectile functions at long term were assessed by periodic follow-up and phone call. Detailed history included cause, symptoms, and single-question self-report of erectile and voiding functions. Results. Among the 44 suspicious cases, 42 (95.4%) were confirmed, mean age was 34.5 years (range: 18-60), mean follow-up 59.3 months (range 9-155). Half presented the classical triad of audible crack, detumescence, and pain. Heterosexual intercourse was the most common cause (28 patients, 66.7%), followed by penile manipulation (6 patients, 14.3%), and homosexual intercourse (4 patients, 9.5%). Woman on top was the most common heterosexual position (n = 14, 50%), followed by doggy style (n = 8, 28.6%). Four patients (9.5%) maintained the cause unclear. Six (14.3%) patients had urethral injury and two (4.8%) had erectile dysfunction, treated by penile prosthesis and PDE-5i. No patient showed urethral fistula, voiding deterioration, penile nodule/curve or pain. Conclusions. Woman on top was the potentially riskiest sexual position (50%). Immediate surgical treatment warrants long-term very low morbidity.

Two-component System Vicrk Regulates Functions Associated With Establishment Of Streptococcus Sanguinis In Biofilms.

Streptococcus sanguinis is a commensal pioneer colonizer of teeth and an opportunistic pathogen of infectious endocarditis. The establishment of S. sanguinis in host sites likely requires dynamic fitting of the cell wall in response to local stimuli. In this study, we investigated the two-component system (TCS) VicRK in S. sanguinis (VicRKSs), which regulates genes of cell wall biogenesis, biofilm formation, and virulence in opportunistic pathogens. A vicK knockout mutant obtained from strain SK36 (SKvic) showed slight reductions in aerobic growth and resistance to oxidative stress but an impaired ability to form biofilms, a phenotype restored in the complemented mutant. The biofilm-defective phenotype was associated with reduced amounts of extracellular DNA during aerobic growth, with reduced production of H2O2, a metabolic product associated with DNA release, and with inhibitory capacity of S. sanguinis competitor species. No changes in autolysis or cell surface hydrophobicity were detected in SKvic. Reverse transcription-quantitative PCR (RT-qPCR), electrophoretic mobility shift assays (EMSA), and promoter sequence analyses revealed that VicR directly regulates genes encoding murein hydrolases (SSA_0094, cwdP, and gbpB) and spxB, which encodes pyruvate oxidase for H2O2 production. Genes previously associated with spxB expression (spxR, ccpA, ackA, and tpK) were not transcriptionally affected in SKvic. RT-qPCR analyses of S. sanguinis biofilm cells further showed upregulation of VicRK targets (spxB, gbpB, and SSA_0094) and other genes for biofilm formation (gtfP and comE) compared to expression in planktonic cells. This study provides evidence that VicRKSs regulates functions crucial for S. sanguinis establishment in biofilms and identifies novel VicRK targets potentially involved in hydrolytic activities of the cell wall required for these functions.