Impact Study of Nonverbal Facial Cues on Spontaneous Chatting with Virtual Humans

Non-verbal communication (NVC) is considered to represent more than 90 percent of everyday communication. In virtual world, this important aspect of interaction between virtual humans (VH) is strongly neglected. This paper presents a user-test study to demonstrate the impact of automatically generated graphics-based NVC expression on the dialog quality: first, we wanted to compare impassive and emotion facial expression simulation for impact on the chatting. Second, we wanted to see whether people like chatting within a 3D graphical environment. Our model only proposes facial expressions and head movements induced from spontaneous chatting between VHs. Only subtle facial expressions are being used as nonverbal cues - i.e. related to the emotional model. Motion capture animations related to hand gestures, such as cleaning glasses, were randomly used to make the virtual human lively. After briefly introducing the technical architecture of the 3D-chatting system, we focus on two aspects of chatting through VHs. First, what is the influence of facial expressions that are induced from text dialog? For this purpose, we exploited an emotion engine extracting an emotional content from a text and depicting it into a virtual character developed previously [GAS11]. Second, as our goal was not addressing automatic generation of text, we compared the impact of nonverbal cues in conversation with a chatbot or with a human operator with a wizard of oz approach. Among main results, the within group study -involving 40 subjects- suggests that subtle facial expressions impact significantly not only on the quality of experience but also on dialog understanding.

Approximate decoding approaches for network coded correlated data

This paper considers a framework where data from correlated sources are transmitted with the help of network coding in ad hoc network topologies. The correlated data are encoded independently at sensors and network coding is employed in the intermediate nodes in order to improve the data delivery performance. In such settings, we focus on the problem of reconstructing the sources at decoder when perfect decoding is not possible due to losses or bandwidth variations. We show that the source data similarity can be used at decoder to permit decoding based on a novel and simple approximate decoding scheme. We analyze the influence of the network coding parameters and in particular the size of finite coding fields on the decoding performance. We further determine the optimal field size that maximizes the expected decoding performance as a trade-off between information loss incurred by limiting the resolution of the source data and the error probability in the reconstructed data. Moreover, we show that the performance of the approximate decoding improves when the accuracy of the source model increases even with simple approximate decoding techniques. We provide illustrative examples showing how the proposed algorithm can be deployed in sensor networks and distributed imaging applications.

Involvement of a region of 23S ribosomal RNA of {\it E. coli\/} in decoding of termination codons

Involvement of E. coli 23S ribosomal RNA (rRNA) in decoding of termination codons was first indicated by the characterization of a 23S rRNA mutant that causes UGA-specific nonsense suppression. The work described here was begun to test the hypothesis that more 23S rRNA suppressors of specific nonsense mutations can be isolated and that they would occur non-randomly in the rRNA genes and be clustered in specific, functionally significant regions of rRNA.^ Approximately 2 kilobases of the gene for 23S rRNA were subjected to PCR random mutagenesis and the amplified products screened for suppression of nonsense mutations in trpA. All of the suppressor mutations obtained were located in a thirty-nucleotide part of the GTPase center, a conserved rRNA sequence and structure, and they and others made in that region by site-directed mutagenesis were shown to be UGA-specific in their suppression of termination codon mutations. These results proved the initial hypothesis and demonstrated that a group of nucleotides in this region are involved in decoding of the UGA termination codon. Further, it was shown that limitation of cellular availability or synthesis of L11, a ribosomal protein that binds to the GTPase center rRNA, resulted in suppression of termination codon mutations, suggesting the direct involvement of L11 in termination in vivo.^ Finally, in vivo analysis of certain site-specific mutations made in the GTPase center RNA demonstrated that (a) the G$\cdot$A base pair closing the hexanucleotide hairpin loop was not essential for normal termination, (b) the "U-turn" structure in the 1093 to 1098 hexaloop is critical for normal termination, (c) nucleotides A1095 and A1067, necessary for the binding to ribosomes of thiostrepton, an antibiotic that inhibits polypeptide release factor binding to ribosomes in vitro, are also necessary for normal peptide chain termination in vivo, and (d) involvement of this region of rRNA in termination is determined by some unique subset structure that includes particular nucleotides rather than merely by a general structural feature of the GTPase center.^ This work advances the understanding of peptide chain termination by demonstrating that the GTPase region of 23S rRNA participates in recognition of termination codons, through an associated ribosomal protein and specific conserved nucleotides and structural motifs in its RNA. ^

Functions and intramolecular interactions of ribosomal RNA in decoding of termination codons

Two regions in the 3$\prime$ domain of 16S rRNA (the RNA of the small ribosomal subunit) have been implicated in decoding of termination codons. Using segment-directed PCR random mutagenesis, I isolated 33 translational suppressor mutations in the 3$\prime$ domain of 16S rRNA. Characterization of the mutations by both genetic and biochemical methods indicated that some of the mutations are defective in UGA-specific peptide chain termination and that others may be defective in peptide chain termination at all termination codons. The studies of the mutations at an internal loop in the non-conserved region of helix 44 also indicated that this structure, in a non-conserved region of 16S rRNA, is involved in both peptide chain termination and assembly of 16S rRNA.^ With a suppressible trpA UAG nonsense mutation, a spontaneously arising translational suppressor mutation was isolated in the rrnB operon cloned into a pBR322-derived plasmid. The mutation caused suppression of UAG at two codon positions in trpA but did not suppress UAA or UGA mutations at the same trpA positions. The specificity of the rRNA suppressor mutation suggests that it may cause a defect in UAG-specific peptide chain termination. The mutation is a single nucleotide deletion (G2484$\Delta$) in helix 89 of 23S rRNA (the large RNA of the large ribosomal subunit). The result indicates a functional interaction between two regions of 23S rRNA. Furthermore, it provides suggestive in vivo evidence for the involvement of the peptidyl-transferase center of 23S rRNA in peptide chain termination. The $\Delta$2484 and A1093/$\Delta$2484 (double) mutations were also observed to alter the decoding specificity of the suppressor tRNA lysT(U70), which has a mutation in its acceptor stem. That result suggests that there is an interaction between the stem-loop region of helix 89 of 23S rRNA and the acceptor stem of tRNA during decoding and that the interaction is important for the decoding specificity of tRNA.^ Using gene manipulation procedures, I have constructed a new expression vector to express and purify the cellular protein factors required for a recently developed, realistic in vitro termination assay. The gene for each protein was cloned into the newly constructed vector in such a way that expression yielded a protein with an N-terminal affinity tag, for specific, rapid purification. The amino terminus was engineered so that, after purification, the unwanted N-terminal tag can be completely removed from the protein by thrombin cleavage, yielding a natural amino acid sequence for each protein. I have cloned the genes for EF-G and all three release factors into this new expression vector and the genes for all the other protein factors into a pCAL-n expression vector. These constructs will allow our laboratory group to quickly and inexpensively purify all the protein factors needed for the new in vitro termination assay. (Abstract shortened by UMI.) ^

Decoding Delay Minimization in Inter-Session Network Coding

Intra-session network coding has been shown to offer significant gains in terms of achievable throughput and delay in settings where one source multicasts data to several clients. In this paper, we consider a more general scenario where multiple sources transmit data to sets of clients over a wireline overlay network. We propose a novel framework for efficient rate allocation in networks where intermediate network nodes have the opportunity to combine packets from different sources using randomized network coding. We formulate the problem as the minimization of the average decoding delay in the client population and solve it with a gradient-based stochastic algorithm. Our optimized inter-session network coding solution is evaluated in different network topologies and is compared with basic intra-session network coding solutions. Our results show the benefits of proper coding decisions and effective rate allocation for lowering the decoding delay when the network is used by concurrent multicast sessions.

Nonverbal social communication and gesture control in schizophrenia

Schizophrenia patients are severely impaired in nonverbal communication, including social perception and gesture production. However, the impact of nonverbal social perception on gestural behavior remains unknown, as is the contribution of negative symptoms, working memory, and abnormal motor behavior. Thus, the study tested whether poor nonverbal social perception was related to impaired gesture performance, gestural knowledge, or motor abnormalities. Forty-six patients with schizophrenia (80%), schizophreniform (15%), or schizoaffective disorder (5%) and 44 healthy controls matched for age, gender, and education were included. Participants completed 4 tasks on nonverbal communication including nonverbal social perception, gesture performance, gesture recognition, and tool use. In addition, they underwent comprehensive clinical and motor assessments. Patients presented impaired nonverbal communication in all tasks compared with controls. Furthermore, in contrast to controls, performance in patients was highly correlated between tasks, not explained by supramodal cognitive deficits such as working memory. Schizophrenia patients with impaired gesture performance also demonstrated poor nonverbal social perception, gestural knowledge, and tool use. Importantly, motor/frontal abnormalities negatively mediated the strong association between nonverbal social perception and gesture performance. The factors negative symptoms and antipsychotic dosage were unrelated to the nonverbal tasks. The study confirmed a generalized nonverbal communication deficit in schizophrenia. Specifically, the findings suggested that nonverbal social perception in schizophrenia has a relevant impact on gestural impairment beyond the negative influence of motor/frontal abnormalities.

Nonverbal synchrony in social interactions of patients with schizophrenia indicates socio-communicative deficits

Background Disordered interpersonal communication can be a serious problem in schizophrenia. Recent advances in computer-based measures allow reliable and objective quantification of nonverbal behavior. Research using these novel measures has shown that objective amounts of body and head movement in patients with schizophrenia during social interactions are closely related to the symptom profiles of these patients. In addition to and above mere amounts of movement, the degree of synchrony, or imitation, between patients and normal interactants may be indicative of core deficits underlying various problems in domains related to interpersonal communication, such as symptoms, social competence, and social functioning. Methods Nonverbal synchrony was assessed objectively using Motion Energy Analysis (MEA) in 378 brief, videotaped role-play scenes involving 27 stabilized outpatients diagnosed with paranoid-type schizophrenia. Results Low nonverbal synchrony was indicative of symptoms, low social competence, impaired social functioning, and low self-evaluation of competence. These relationships remained largely significant when correcting for the amounts of patients‘ movement. When patients showed reduced imitation of their interactants’ movements, negative symptoms were likely to be prominent. Conversely, positive symptoms were more prominent in patients when their interaction partners’ imitation of their movements was reduced. Conclusions Nonverbal synchrony can be an objective and sensitive indicator of the severity of patients’ problems. Furthermore, quantitative analysis of nonverbal synchrony may provide novel insights into specific relationships between symptoms, cognition, and core communicative problems in schizophrenia.