Learning Data-Driven Models of Non-Verbal Behaviors for Building Rapport Using an Intelligent Virtual Agent

There is a growing societal need to address the increasing prevalence of behavioral health issues, such as obesity, alcohol or drug use, and general lack of treatment adherence for a variety of health problems. The statistics, worldwide and in the USA, are daunting. Excessive alcohol use is the third leading preventable cause of death in the United States (with 79,000 deaths annually), and is responsible for a wide range of health and social problems. On the positive side though, these behavioral health issues (and associated possible diseases) can often be prevented with relatively simple lifestyle changes, such as losing weight with a diet and/or physical exercise, or learning how to reduce alcohol consumption. Medicine has therefore started to move toward finding ways of preventively promoting wellness, rather than solely treating already established illness. Evidence-based patient-centered Brief Motivational Interviewing (BMI) interven- tions have been found particularly effective in helping people find intrinsic motivation to change problem behaviors after short counseling sessions, and to maintain healthy lifestyles over the long-term. Lack of locally available personnel well-trained in BMI, however, often limits access to successful interventions for people in need. To fill this accessibility gap, Computer-Based Interventions (CBIs) have started to emerge. Success of the CBIs, however, critically relies on insuring engagement and retention of CBI users so that they remain motivated to use these systems and come back to use them over the long term as necessary. Because of their text-only interfaces, current CBIs can therefore only express limited empathy and rapport, which are the most important factors of health interventions. Fortunately, in the last decade, computer science research has progressed in the design of simulated human characters with anthropomorphic communicative abilities. Virtual characters interact using humans’ innate communication modalities, such as facial expressions, body language, speech, and natural language understanding. By advancing research in Artificial Intelligence (AI), we can improve the ability of artificial agents to help us solve CBI problems. To facilitate successful communication and social interaction between artificial agents and human partners, it is essential that aspects of human social behavior, especially empathy and rapport, be considered when designing human-computer interfaces. Hence, the goal of the present dissertation is to provide a computational model of rapport to enhance an artificial agent’s social behavior, and to provide an experimental tool for the psychological theories shaping the model. Parts of this thesis were already published in [LYL+12, AYL12, AL13, ALYR13, LAYR13, YALR13, ALY14].

Learning data-driven models of non-verbal behaviors for building rapport using an intelligent virtual agent

There is a growing societal need to address the increasing prevalence of behavioral health issues, such as obesity, alcohol or drug use, and general lack of treatment adherence for a variety of health problems. The statistics, worldwide and in the USA, are daunting. Excessive alcohol use is the third leading preventable cause of death in the United States (with 79,000 deaths annually), and is responsible for a wide range of health and social problems. On the positive side though, these behavioral health issues (and associated possible diseases) can often be prevented with relatively simple lifestyle changes, such as losing weight with a diet and/or physical exercise, or learning how to reduce alcohol consumption. Medicine has therefore started to move toward finding ways of preventively promoting wellness, rather than solely treating already established illness.^ Evidence-based patient-centered Brief Motivational Interviewing (BMI) interventions have been found particularly effective in helping people find intrinsic motivation to change problem behaviors after short counseling sessions, and to maintain healthy lifestyles over the long-term. Lack of locally available personnel well-trained in BMI, however, often limits access to successful interventions for people in need. To fill this accessibility gap, Computer-Based Interventions (CBIs) have started to emerge. Success of the CBIs, however, critically relies on insuring engagement and retention of CBI users so that they remain motivated to use these systems and come back to use them over the long term as necessary.^ Because of their text-only interfaces, current CBIs can therefore only express limited empathy and rapport, which are the most important factors of health interventions. Fortunately, in the last decade, computer science research has progressed in the design of simulated human characters with anthropomorphic communicative abilities. Virtual characters interact using humans’ innate communication modalities, such as facial expressions, body language, speech, and natural language understanding. By advancing research in Artificial Intelligence (AI), we can improve the ability of artificial agents to help us solve CBI problems.^ To facilitate successful communication and social interaction between artificial agents and human partners, it is essential that aspects of human social behavior, especially empathy and rapport, be considered when designing human-computer interfaces. Hence, the goal of the present dissertation is to provide a computational model of rapport to enhance an artificial agent’s social behavior, and to provide an experimental tool for the psychological theories shaping the model. Parts of this thesis were already published in [LYL+12, AYL12, AL13, ALYR13, LAYR13, YALR13, ALY14].^

Detecting deception: Identifying differences in liars' and truth tellers' verbal strategies

Deception research has traditionally focused on three methods of identifying liars and truth tellers: observing non-verbal or behavioral cues, analyzing verbal cues, and monitoring changes in physiological arousal during polygraph tests. Research shows that observers are often incapable of discriminating between liars and truth tellers with better than chance accuracy when they use these methods. One possible explanation for observers' poor performance is that they are not properly applying existing lie detection methods. An alternative explanation is that the cues on which these methods — and observers' judgments — are based do not reliably discriminate between liars and truth tellers. It may be possible to identify more reliable cues, and potentially improve observers' ability to discriminate, by developing a better understanding of how liars and truth tellers try to tell a convincing story. ^ This research examined (a) the verbal strategies used by truthful and deceptive individuals during interviews concerning an assigned activity, and (b) observers' ability to discriminate between them based on their verbal strategies. In Experiment I, pre-interview instructions manipulated participants' expectations regarding verifiability; each participant was led to believe that the interviewer could check some types of details, but not others, before deciding whether the participant was being truthful or deceptive. Interviews were then transcribed and scored for quantity and type of information provided. In Experiment II, observers listened to a random sample of the Experiment I interviews and rendered veracity judgments; half of the observers were instructed to judge the interviews according to the verbal strategies used by liars and truth tellers and the other half were uninstructed. ^ Results of Experiment I indicate that liars and truth tellers use different verbal strategies, characterized by a differential amount of detail. Overall, truthful participants provided more information than deceptive participants. This effect was moderated by participants' expectations regarding verifiability such that truthful participants provided more information only with regard to verifiable details. Results of Experiment II indicate that observers instructed about liars' and truth tellers' verbal strategies identify them with greater accuracy than uninstructed observers. ^

Interpersonal Techniques Boost your "Hospitality Batting Average"

The ability to listen and comprehend the intrinsic meaning behind the words people are saying is an important skill for those in the hospitality industry. The author provides some prescriptions for "winning friends and influencing people.”

Cognitive performance and the thymus among HIV-infected subjects receiving HAART

Objective: To evaluate the impact of alcohol use, which is widespread in human immunodeficiency virus (HIV)+ individuals, on highly active antiretroviral therapy (HAART)-associated immune and cognitive improvements and the relationship between those two responses. Methods: In a case-control longitudinal study, thymic volume, cognition, and immune responses were evaluated at baseline and after 6 months therapy in HIV+ and HIV- controls. Cognitive performance was evaluated using the HIV Dementia Score (HDS) and the California Verbal Learning Test (CVLT). Results: Prior to HAART, thymic volume varied considerably from 2.7 to 29.3 cm3 (11 ± 7.2 cm3). Thymic volume at baseline showed a significantly inverse correlation with the patient’s number of years of drinking (r2 = 0.207; p < 0.01), as well as HDS and the CVLT scores in both HIV-infected (r2 = 0.37, p = 0.03) and noninfected (r2 = 0.8, p = 0.01). HIV-infected individuals with a small thymic volume scored in the demented range, as compared with those with a larger thymus (7 ± 2.7 vs. 12 ± 2.3, p = 0.005). After HAART, light/moderate drinkers exhibited thymus size twice that of heavy drinkers (14.8 ± 10.4 vs. 6.9 ± 3.3 cm3). Conclusions: HAART-associated increases of thymus volume appear to be negatively affected by alcohol consumption and significantly related to their cognitive status. This result could have important clinical implications.

The differential effects of *American and British trial proceedings on juror decision-making

Seven basic elements differentiate British from American trial procedures: confining attorneys to their tables; dealing with objections outside the presence of the jury; resolving disagreements between attorneys prior to objections being made; presenting the defense opening statement at the close of the prosecution case; the judge directly questions witnesses and has a wider latitude in controlling the evidence; and the judge gives a summation of all the evidence presented to the jury (Fulero & Turner, 1997). The present experiment examined the influence of these different courtroom procedures, judges' non-verbal behavior, and evidence strength on juror decision-making. Using models of persuasion to understand how the varying elements may effect juror decision-making, it was predicted that trials following American courtroom procedures would be more distracting for jurors and as such, they would be more likely to rely on the peripheral cue of the judge's expectations for trial outcome as expressed in his nonverbal behavior. In trials following British procedures jurors should be less distracted and better able to scrutinize the strength of the evidence that in turn should minimize the influence of the judge's nonverbal behavior. Two hundred forty-five participants viewed a mock civil trial in which courtroom procedure, judge's nonverbal behavior, and evidence strength were varied. Analyses suggest that courtroom procedure and evidence strength influenced the direction of participants' verdicts, but that judge's nonverbal behavior did not have a direct impact on verdict preference. Judge's nonverbal behavior appeared to influence other measures related to verdict. Participants were more confident in their verdicts when they agreed with judge's nonverbal behavior and when they viewed British courtroom procedures. Participants were more likely to return estimates of the defendant's liability that reflected judge's nonverbal behavior and a congruency with evidence strength. Participants also recalled more facts in the British conditions than in the American conditions. These findings are interpreted as indicating the importance of the impact of trial procedures and of nonverbal influence. ^

Flexible pavement layer moduli determination : an adaptive artificial neural network approach

The estimation of pavement layer moduli through the use of an artificial neural network is a new concept which provides a less strenuous strategy for backcalculation procedures. Artificial Neural Networks are biologically inspired models of the human nervous system. They are specifically designed to carry out a mapping characteristic. This study demonstrates how an artificial neural network uses non-destructive pavement test data in determining flexible pavement layer moduli. The input parameters include plate loadings, corresponding sensor deflections, temperature of pavement surface, pavement layer thicknesses and independently deduced pavement layer moduli.

Drawing as a technique to facilitate childrens memory

This study examined a technique to assist children to recall more information about witnessed events. Thirty-eight fourth-grade children from a public grade school in Miami Florida participated in the experiment. The participants watched a Red Cross demonstration and were interviewed one week later about details of the demonstration. All of the children were interviewed using a police style interview. In addition, half of the children were instructed to draw during the interview. The current study supported previous findings that the instruction to draw increased the amount of information recalled. The effect of drawing was greatest for high-visual events. In addition, the instruction to draw prompted an increase in non-verbal information, which had an unusually high accuracy rate.

Emotional intelligence and leadership in organizations: A meta-analytic test of process mechanisms

The present study—employing psychometric meta-analysis of 92 independent studies with sample sizes ranging from 26 to 322 leaders—examined the relationship between EI and leadership effectiveness. Overall, the results supported a linkage between leader EI and effectiveness that was moderate in nature (ρ = .25). In addition, the positive manifold of the effect sizes presented in this study, ranging from .10 to .44, indicate that emotional intelligence has meaningful relations with myriad leadership outcomes including effectiveness, transformational leadership, LMX, follower job satisfaction, and others. Furthermore, this paper examined potential process mechanisms that may account for the EI-leadership effectiveness relationship and showed that both transformational leadership and LMX partially mediate this relationship. However, while the predictive validities of EI were moderate in nature, path analysis and hierarchical regression suggests that EI contributes less than or equal to 1% of explained variance in leadership effectiveness once personality and intelligence are accounted for. ^

An Analysis of the Proposition Density, Sentence and Clause Types, and Non-Finite Verbal Usage in Two College Textbooks

This study investigated the proposition density, sentence and clause type usage and non-finite verbal usage in two college textbooks. The teaching implications are presented.

The relationships between elementary school principals' emotional intelligence, leadership style and school culture

The role of the principal in school settings and the principal's perceived effect on student achievement have frequently been considered vital factors in school reform. The relationships between emotional intelligence, leadership style and school culture have been widely studied. The literature reveals agreement among scholars regarding the principal's vital role in developing and fostering a positive school culture. The purpose of this study was to explore the relationships between elementary school principals' emotional intelligence, leadership style and school culture. ^ The researcher implemented a non-experimental ex post facto research design to investigate four specific research hypotheses. Utilizing the Qualtrics Survey Software, 57 elementary school principals within a large urban school district in southeast Florida completed the Emotional Quotient Inventory (EQ-i), and 850 of their faculty members completed the Multifactor Leadership Questionnaire (MLQ Form 5X). Faculty responses to the school district's School Climate Survey retrieved from the district's web site were used as the measure of school culture. ^ Linear regression analyses revealed significant positive associations between emotional intelligence and the following leadership measures: Idealized Influence-Attributes (β = .23, p = < .05), Idealized Influence-Behaviors (β = .34, p = < .01), Inspirational Motivation (β = .39, p = < .01) and Contingent Reward (β = .33, p = < .01). Hierarchical regression analyses revealed positive associations between school culture and both transformational and transactional leadership measures, and negative associations between school culture and passive-avoidant leadership measures. Significant positive associations were found between school culture and the principals' emotional intelligence over and above leadership style. Hierarchical linear regressions to test the statistical hypothesis developed to account for alternative explanations revealed significant associations between leadership style and school culture over and above school grade. ^ These results suggest that emotional intelligence merits consideration in the development of leadership theory. Practical implications include suggestions that principals employ both transformational and transactional leadership strategies, and focus on developing their level of emotional intelligence. The associations between emotional intelligence, transformational leadership, Contingent Reward and school culture found in this study validate the role of the principal as the leader of school reform.^

Emotional Intelligence and Leadership in Organization: A Meta-analytic Test of Process Mechanisms

The present study – employing psychometric meta-analysis of 92 independent studies with sample sizes ranging from 26 to 322 leaders – examined the relationship between EI and leadership effectiveness. Overall, the results supported a linkage between leader EI and effectiveness that was moderate in nature (ρ = .25). In addition, the positive manifold of the effect sizes presented in this study, ranging from .10 to .44, indicate that emotional intelligence has meaningful relations with myriad leadership outcomes including effectiveness, transformational leadership, LMX, follower job satisfaction, and others. Furthermore, this paper examined potential process mechanisms that may account for the EI-leadership effectiveness relationship and showed that both transformational leadership and LMX partially mediate this relationship. However, while the predictive validities of EI were moderate in nature, path analysis and hierarchical regression suggests that EI contributes less than or equal to 1% of explained variance in leadership effectiveness once personality and intelligence are accounted for.

Evoking non-repertory verbal behavior across operant classes : the effects of motor echoic sign language training within the context of a motivating operation

The individual effects that echoic, mand, and sign language training procedures have on the acquisition of verbal behavior have been widely demonstrated, but more efficient strategies are still needed. This study combined all three treatment strategies into one treatment intervention in order to investigate the joint effects they may have on verbal behavior. Six participants took part in the study. Intervention totaled 1 hour/day for 5 days/week until mastery criterion for motor echoic behavior was achieved. Although motor echoic behavior were solely targeted for acquisition, significant increases in spontaneous motor mands were noted in all treatment participants. Additionally, 4 treatment participants also demonstrated significant gains in vocal echoics and spontaneous vocal mands. No significant increases were noted for the control participant. Results suggest that the aforementioned procedure may provide more efficient results as a first-step to teaching a functional repertoire of verbal behavior to developmentally delayed children.

The Relationships between Elementary School Principals' Emotional Intelligence, Leadership Style and School Culture

The role of the principal in school settings and the principal’s perceived effect on student achievement have frequently been considered vital factors in school reform. The relationships between emotional intelligence, leadership style and school culture have been widely studied. The literature reveals agreement among scholars regarding the principal’s vital role in developing and fostering a positive school culture. The purpose of this study was to explore the relationships between elementary school principals’ emotional intelligence, leadership style and school culture. The researcher implemented a non-experimental ex post facto research design to investigate four specific research hypotheses. Utilizing the Qualtrics Survey Software, 57 elementary school principals within a large urban school district in southeast Florida completed the Emotional Quotient Inventory (EQ-i), and 850 of their faculty members completed the Multifactor Leadership Questionnaire (MLQ Form 5X). Faculty responses to the school district’s School Climate Survey retrieved from the district’s web site were used as the measure of school culture. Linear regression analyses revealed significant positive associations between emotional intelligence and the following leadership measures: Idealized Influence-Attributes (β = .23, p = < .05), Idealized Influence-Behaviors (β = .34, p = < .01), Inspirational Motivation (β = .39, p = < .01) and Contingent Reward (β = .33, p = < .01). Hierarchical regression analyses revealed positive associations between school culture and both transformational and transactional leadership measures, and negative associations between school culture and passive-avoidant leadership measures. Significant positive associations were found between school culture and the principals’ emotional intelligence over and above leadership style. Hierarchical linear regressions to test the statistical hypothesis developed to account for alternative explanations revealed significant associations between leadership style and school culture over and above school grade. These results suggest that emotional intelligence merits consideration in the development of leadership theory. Practical implications include suggestions that principals employ both transformational and transactional leadership strategies, and focus on developing their level of emotional intelligence. The associations between emotional intelligence, transformational leadership, Contingent Reward and school culture found in this study validate the role of the principal as the leader of school reform.