Impact Study of Nonverbal Facial Cues on Spontaneous Chatting with Virtual Humans

Non-verbal communication (NVC) is considered to represent more than 90 percent of everyday communication. In virtual world, this important aspect of interaction between virtual humans (VH) is strongly neglected. This paper presents a user-test study to demonstrate the impact of automatically generated graphics-based NVC expression on the dialog quality: first, we wanted to compare impassive and emotion facial expression simulation for impact on the chatting. Second, we wanted to see whether people like chatting within a 3D graphical environment. Our model only proposes facial expressions and head movements induced from spontaneous chatting between VHs. Only subtle facial expressions are being used as nonverbal cues - i.e. related to the emotional model. Motion capture animations related to hand gestures, such as cleaning glasses, were randomly used to make the virtual human lively. After briefly introducing the technical architecture of the 3D-chatting system, we focus on two aspects of chatting through VHs. First, what is the influence of facial expressions that are induced from text dialog? For this purpose, we exploited an emotion engine extracting an emotional content from a text and depicting it into a virtual character developed previously [GAS11]. Second, as our goal was not addressing automatic generation of text, we compared the impact of nonverbal cues in conversation with a chatbot or with a human operator with a wizard of oz approach. Among main results, the within group study -involving 40 subjects- suggests that subtle facial expressions impact significantly not only on the quality of experience but also on dialog understanding.

Interactive Ray Tracing for Virtual TV Studio Applications

In the last years, the well known ray tracing algorithm gained new popularity with the introduction of interactive ray tracing methods. The high modularity and the ability to produce highly realistic images make ray tracing an attractive alternative to raster graphics hardware. Interactive ray tracing also proved its potential in the field of Mixed Reality rendering and provides novel methods for seamless integration of real and virtual content. Actor insertion methods, a subdomain of Mixed Reality and closely related to virtual television studio techniques, can use ray tracing for achieving high output quality in conjunction with appropriate visual cues like shadows and reflections at interactive frame rates. In this paper, we show how interactive ray tracing techniques can provide new ways of implementing virtual studio applications.

Precise Near-to-Head Acoustics with Binaural Synthesis

For enhanced immersion into a virtual scene more than just the visual sense should be addressed by a Virtual Reality system. Additional auditory stimulation appears to have much potential, as it realizes a multisensory system. This is especially useful when the user does not have to wear any additional hardware, e.g., headphones. Creating a virtual sound scene with spatially distributed sources requires a technique for adding spatial cues to audio signals and an appropriate reproduction. In this paper we present a real-time audio rendering system that combines dynamic crosstalk cancellation and multi-track binaural synthesis for virtual acoustical imaging. This provides the possibility of simulating spatially distributed sources and, in addition to that, near-to-head sources for a freely moving listener in room-mounted virtual environments without using any headphones. A special focus will be put on near-to-head acoustics, and requirements in respect of the head-related transfer function databases are discussed.

Italia: La chiusura della Casa dell’Emigrante come spunto riflessione sui rientri degli emigrati italiani

In the literature on migration, as well as in social policies regarding this phenomenon, the situation of returning emigrants receives scant attention. This essay establishes an intricate connection between attitudes and policies that prevail in a country regarding emigration and those concerning immigration. The case of Italy provides a prime example for this as it once was a classical country of emigration, only to turn, in recent decades, into a country that appears highly attractive (and relatively accessible) to immigrants. The essay traces the pervasive ambiguity that characterizes this country’s attitudes towards emigration from the beginning of mass emigration shortly after the unification of Italy in 1868 to the emigration policies of the fascist regime of Mussolini and the post-World War II waves of emigration right through to the corresponding ambiguity concerning the status of immigrants in contemporary society, including the indifferent treatment of returning Italian emigrants who constitute a considerable numerical phenomenon. These reflections take their origin from the impending closure of a reception centre in Lazio, the Casa dell’Emigrante near Sant’Elia Fiumerapido, Province of Frosinone, ostensibly for financial reasons. This centre had been the only one of its kind in the whole of Italy dealing officially with the needs of repatriated Italians. It had assisted returning emigrants both with practical matters, such as negotiating the labyrinth of Italian bureaucracy , and with psychological implications of a return, which are often considerable given the time lag of experiences with current social realities and the frequently unrealistic expectations associated with the return. Questions of identity become highly acute in those circumstances. The threatened closure of the centre illustrates the unwillingness of the state to face up to the factual prevalence of migratory experiences in the country as a whole and as a core element of national history, experiences of migration in both directions. The statistics speak for themselves: of the 4.660.427 persons who left Italy between 1880 and 1950, 2.322.451 have returned, almost exactly 50%. To those have to be added 3.628.430 returnees of the 5.109.860 emigrants who left Italy between the end of World War II and 1976 for Europe alone. Attitudes towards people leaving changed ostensibly over time. In the first two decades after Unification parliament on the one hand wanted to show some concern over the fate of its citizens, not wanting to abandon those newly created citizens entirely to their own destiny, while on the other portraying their decisions to emigrate as expressions of individual liberty and responsibility and not necessitated by want and poverty. Emigrants had to prove, paradoxically that they had the requisite means to emigrate when in fact poverty was largely driving them to emigrate. To admit that publicly would have amounted to admission of economic and political failure made evident through emigration. In contrast to that Mussolini’s emigration policies not only enforced large population movements within the territory of Italy to balance unemployment between regions and particularly between North and South, but also declared it citizen’s duty to be ready to move also to the colonies, thereby ‘turning emigration as a sign of social crisis into a sign of national strength and the success of the country’s political agenda’ (Gaspari 2001, p. 34). The duplicity continued even after World War II when secret deals were done with the USA to allow a continuous flow of Italian immigrants and EU membership obviously further facilitated the departure of unemployed, impoverished Italians. With the growing prosperity of Italy the reversal of the direction of migration became more obvious. On the basis of empirical research conducted by one of the author on returning emigrants four types of motives for returning can be distinguished: 1. Return as a result of failure – particularly the emigrants who left during the 1950-1970 period usually had no linguistic preparation, and in any case the gap between the spoken and the written language is enormous with the latter often being insurmountable. This gives rise to nostalgic sentiments which motivates a return into an environment where language is familiar 2. Return as a means of preserving an identity – the life of emigrants often takes place within ghetto-like conditions where familiarity is being reproduced but under restricted conditions and hence not entirely authentic. The necessity for saving money permits only a partial entry into the host society and at the same time any accumulating savings add to the desire to return home where life can be lived fully again – or so it seems. 3. Return of investment – the impossibility to become fully part of another society often motivates migrants to accumulate not so much material wealth but new experiences and competences which they then aim to reinvest in their home country. 4. Return to retire – for many emigrants returning home becomes acute once they leave a productive occupation and feelings of estrangement build up, in conjunction with the efforts of having invested in building a house back home. All those motives are associated with a variety of difficulties on the actual return home because, above all, time in relation to the country of origin has been suspended for the emigrant and the encounter with the reality of that country reveals constant discrepancies and requires constant readjustment. This is where the need for assistance to returning emigrants arises. The fact that such an important centre of assistance has been closed is further confirmation of the still prevailing politics of ambiguity which nominally demand integration from nationals and non-nationals alike but deny the means of achieving this. Citizenship is not a natural result of nationality but requires the means for active participation in society. Furthermore, the experiences of returning immigrants provide important cues for the double ambivalence in which immigrants to Italy live between the demands made on them to integrate, the simultaneous threats of repatriation and the alienation from the immigrants’ home country which grows inexorably during the absence. The state can only regain its credibility by putting an end to this ambiguity and provide to returning emigrants, and immigrants alike, the means of reconstructing strong communal identities.

Spatial audition in a static virtual environment : the role of auditory-visual interaction

The integration of the auditory modality in virtual reality environments is known to promote the sensations of immersion and presence. However it is also known from psychophysics studies that auditory-visual interaction obey to complex rules and that multisensory conflicts may disrupt the adhesion of the participant to the presented virtual scene. It is thus important to measure the accuracy of the auditory spatial cues reproduced by the auditory display and their consistency with the spatial visual cues. This study evaluates auditory localization performances under various unimodal and auditory-visual bimodal conditions in a virtual reality (VR) setup using a stereoscopic display and binaural reproduction over headphones in static conditions. The auditory localization performances observed in the present study are in line with those reported in real conditions, suggesting that VR gives rise to consistent auditory and visual spatial cues. These results validate the use of VR for future psychophysics experiments with auditory and visual stimuli. They also emphasize the importance of a spatially accurate auditory and visual rendering for VR setups.

Investigations into Velocity and Distance Perception Based on Different Types of Moving Sound Sources with Respect to Auditory Virtual Environments

The characteristics of moving sound sources have strong implications on the listener's distance perception and the estimation of velocity. Modifications of the typical sound emissions as they are currently occurring due to the tendency towards electromobility have an impact on the pedestrian's safety in road traffic. Thus, investigations of the relevant cues for velocity and distance perception of moving sound sources are not only of interest for the psychoacoustic community, but also for several applications, like e.g. virtual reality, noise pollution and safety aspects of road traffic. This article describes a series of psychoacoustic experiments in this field. Dichotic and diotic stimuli of a set of real-life recordings taken from a passing passenger car and a motorcycle were presented to test subjects who in turn were asked to determine the velocity of the object and its minimal distance from the listener. The results of these psychoacoustic experiments show that the estimated velocity is strongly linked to the object's distance. Furthermore, it could be shown that binaural cues contribute significantly to the perception of velocity. In a further experiment, it was shown that - independently of the type of the vehicle - the main parameter for distance determination is the maximum sound pressure level at the listener's position. The article suggests a system architecture for the adequate consideration of moving sound sources in virtual auditory environments. Virtual environments can thus be used to investigate the influence of new vehicle powertrain concepts and the related sound emissions of these vehicles on the pedestrians' ability to estimate the distance and velocity of moving objects.