Learning Data-Driven Models of Non-Verbal Behaviors for Building Rapport Using an Intelligent Virtual Agent

There is a growing societal need to address the increasing prevalence of behavioral health issues, such as obesity, alcohol or drug use, and general lack of treatment adherence for a variety of health problems. The statistics, worldwide and in the USA, are daunting. Excessive alcohol use is the third leading preventable cause of death in the United States (with 79,000 deaths annually), and is responsible for a wide range of health and social problems. On the positive side though, these behavioral health issues (and associated possible diseases) can often be prevented with relatively simple lifestyle changes, such as losing weight with a diet and/or physical exercise, or learning how to reduce alcohol consumption. Medicine has therefore started to move toward finding ways of preventively promoting wellness, rather than solely treating already established illness. Evidence-based patient-centered Brief Motivational Interviewing (BMI) interven- tions have been found particularly effective in helping people find intrinsic motivation to change problem behaviors after short counseling sessions, and to maintain healthy lifestyles over the long-term. Lack of locally available personnel well-trained in BMI, however, often limits access to successful interventions for people in need. To fill this accessibility gap, Computer-Based Interventions (CBIs) have started to emerge. Success of the CBIs, however, critically relies on insuring engagement and retention of CBI users so that they remain motivated to use these systems and come back to use them over the long term as necessary. Because of their text-only interfaces, current CBIs can therefore only express limited empathy and rapport, which are the most important factors of health interventions. Fortunately, in the last decade, computer science research has progressed in the design of simulated human characters with anthropomorphic communicative abilities. Virtual characters interact using humans’ innate communication modalities, such as facial expressions, body language, speech, and natural language understanding. By advancing research in Artificial Intelligence (AI), we can improve the ability of artificial agents to help us solve CBI problems. To facilitate successful communication and social interaction between artificial agents and human partners, it is essential that aspects of human social behavior, especially empathy and rapport, be considered when designing human-computer interfaces. Hence, the goal of the present dissertation is to provide a computational model of rapport to enhance an artificial agent’s social behavior, and to provide an experimental tool for the psychological theories shaping the model. Parts of this thesis were already published in [LYL+12, AYL12, AL13, ALYR13, LAYR13, YALR13, ALY14].

Learning data-driven models of non-verbal behaviors for building rapport using an intelligent virtual agent

There is a growing societal need to address the increasing prevalence of behavioral health issues, such as obesity, alcohol or drug use, and general lack of treatment adherence for a variety of health problems. The statistics, worldwide and in the USA, are daunting. Excessive alcohol use is the third leading preventable cause of death in the United States (with 79,000 deaths annually), and is responsible for a wide range of health and social problems. On the positive side though, these behavioral health issues (and associated possible diseases) can often be prevented with relatively simple lifestyle changes, such as losing weight with a diet and/or physical exercise, or learning how to reduce alcohol consumption. Medicine has therefore started to move toward finding ways of preventively promoting wellness, rather than solely treating already established illness.^ Evidence-based patient-centered Brief Motivational Interviewing (BMI) interventions have been found particularly effective in helping people find intrinsic motivation to change problem behaviors after short counseling sessions, and to maintain healthy lifestyles over the long-term. Lack of locally available personnel well-trained in BMI, however, often limits access to successful interventions for people in need. To fill this accessibility gap, Computer-Based Interventions (CBIs) have started to emerge. Success of the CBIs, however, critically relies on insuring engagement and retention of CBI users so that they remain motivated to use these systems and come back to use them over the long term as necessary.^ Because of their text-only interfaces, current CBIs can therefore only express limited empathy and rapport, which are the most important factors of health interventions. Fortunately, in the last decade, computer science research has progressed in the design of simulated human characters with anthropomorphic communicative abilities. Virtual characters interact using humans’ innate communication modalities, such as facial expressions, body language, speech, and natural language understanding. By advancing research in Artificial Intelligence (AI), we can improve the ability of artificial agents to help us solve CBI problems.^ To facilitate successful communication and social interaction between artificial agents and human partners, it is essential that aspects of human social behavior, especially empathy and rapport, be considered when designing human-computer interfaces. Hence, the goal of the present dissertation is to provide a computational model of rapport to enhance an artificial agent’s social behavior, and to provide an experimental tool for the psychological theories shaping the model. Parts of this thesis were already published in [LYL+12, AYL12, AL13, ALYR13, LAYR13, YALR13, ALY14].^

Immigrant high school students' in-depth understandings of the value of heritage language and bilingualism

The purpose of this research was to explore perceptions among 9 th through 12th grade students from Brazil, Haiti and Jamaica, with respect to their heritage languages: Portuguese, Haitian Creole, and Jamaican Patois. An additional purpose was to understand in greater detail possible variations of perception with respect to heritage language maintenance (or loss) in relation to one’s gender, first language, and place of birth. The research implemented semi-structured interviews with male and female adolescents with these heritage language backgrounds. Participants’ responses were recorded and transcribed. The transcriptions were analyzed via a categorizing of themes emerging from the data. Data were analyzed using inductive analysis. Three categories emerged from the inductive analysis of the data: (a) heritage language, (b) bilingualism, and (c) English as a second language. The analysis reveals that as participants learn English, they continue to value their heritage language and feel positively toward bilingualism, but differ in their preference regarding use of native language and English in a variety of contexts. There seems to be a mismatch between a positive attitude and an interest in learning their heritage language. Families and teachers, as agents, may not be helping students fully understand the advantages of bilingualism. Students seem to have a lack of understanding of bilingualism’s cognitive and bi-literacy benefits. Instead, employment seems to be perceived as the number one reason for becoming bilingual. Also, the students have a desire to add culture to the heritage language curriculum. The study was conducted at one of the most diverse and largest high schools in Palm Beach, in Palm Beach County, Florida. The results of this study imply that given the positive attitude toward heritage language and bilingualism, students need to be guided in exploring their understanding of heritage language and bilingualism. Implications for teaching and learning, as well as recommendations for further research, are included.

Examining argumentative coherence in essays by undergraduate students of English as a foreign language in Mainland China and their English speaking peers in the United States

I conducted this study to provide insights toward deepening understanding of association between culture and writing by building, assessing, and refining a conceptual model of second language writing. To do this, I examined culture and coherence as well as the relationship between them through a mixed methods research design. Coherence has been an important and complex concept in ESL/EFL writing. I intended to study the concept of coherence in the research context of contrastive rhetoric, comparing the coherence quality in argumentative essays written by undergraduates in Mainland China and their U.S. peers. In order to analyze the complex concept of coherence, I synthesized five linguistic theories of coherence: Halliday and Hasan's cohesion theory, Carroll's theory of coherence, Enkvist's theory of coherence, Topical Structure Analysis, and Toulmin's Model. Based upon the synthesis, 16 variables were generated. Across these 16 variables, Hotelling t-test statistical analysis was conducted to predict differences in argumentative coherence between essays written by two groups of participants. In order to complement the statistical analysis, I conducted 30 interviews of the writers in the studies. Participants' responses were analyzed with open and axial coding. By analyzing the empirical data, I refined the conceptual model by adding more categories and establishing associations among them. The study found that U.S. students made use of more pronominal reference. Chinese students adopted more lexical devices of reiteration and extended paralleling progression. The interview data implied that the difference may be associated with the difference in linguistic features and rhetorical conventions in Chinese and English. As far as Toulmin's Model is concerned, Chinese students scored higher on data than their U.S. peers. According to the interview data, this may be due to the fact that Toulmin's Model, modified as three elements of arguments, have been widely and long taught in Chinese writing instruction while U.S. interview participants said that they were not taught to write essays according to Toulmin's Model. Implications were generated from the process of textual data analysis and the formulation of structural model defining coherence. These implications were aimed at informing writing instruction, assessment, peer-review, and self-revision.

English Language Learners: A Correlational Study of the Relationship Between A Proficiency Level Assessment and End of Course Test Scores at one Georgia High School

Understanding the language of one’s cultural environment is important for effective communication and function. As such, students entering U.S. schools from foreign countries are given access to English to Speakers of Other Languages (ESOL) programs and they are referred to as English Language Learner (ELL) students. This dissertation examined the correlation of ELL ACCESS Composite Performance Level (CPL) score to the End of Course tests (EOCTs) and the Georgia High School Graduation Tests (GHSGTs) in the four content courses (language arts, mathematics, science, and social studies). A premise of this study was that English language proficiency is critical in meeting or exceeding state and county assessment standards. A quantitative descriptive research design was conducted using Cross-sectional archival data from a secondary source. There were 148 participants from school years 2011-2012 to 2013- 2014 from Grades 9-12. A Pearson product moment correlation was run to assess the relationship between the ACCESS CPL (independent variable) and the EOCT scores and the GHSGT scores (dependent variables). The findings showed that there was a positive correlation between ACCESS CPL scores and the EOCT scores where language arts showed a strong positive correlation and mathematics showed a positive weak correlation. Also, there was a positive correlation between ACCESS CPL scores and GHSGT scores where language arts showed a weak positive correlation. The results of this study indicated that that there is a relationship between the stated variables, ACCESS CPL, EOCT and GHSGT. Also, the results of this study showed that there were positive correlations at varying degrees for each grade levels. While the null hypothesis for Research Question 1 and Research Question 2 were rejected, there was a slight relationship between the variables.

Curriculum development in a heritage language community-based school: A Qualitative inquiry regarding a Brazilian-Portuguese program in South Florida

This research aimed to describe, understand, and discuss the curriculum development process of a Brazilian-Portuguese heritage language community-based school in South Florida. This study was guided by the following research questions: (a) What roles does this HL community-based school aim to play for its students? This investigation was also related to the subsidiary question: (b) How does this HL community-based school organize its curriculum development process? In order to explore these research questions, I observed and interviewed teachers and coordinators based on a qualitative research approach. I analyzed the interviews’ transcripts, and the program’s website with a central focus of describing and understanding their curriculum development process. Hopefully, the findings will help Brazilian and other HL community schools toward discussing and elaborating their own curriculum development, as well as to look for specific teacher training courses.

Toward an Understanding of Faculty Perceptions about Factors that Influence Student Success in Online Education

This paper explores factors that faculty perceive as significant in influencing student success in online education. Technological support, degree of comfort with technology and a student’s personal characteristics were deemed by faculty as critical to success. Understanding these key factors could help to improve student success in online education.

Fiber Pathways for Language in the Developing Brain: A Diffusion Tensor Imaging (DTI) Study

The present study characterized two fiber pathways important for language, the superior longitudinal fasciculus/arcuate fasciculus (SLF/AF) and the frontal aslant tract (FAT), and related these tracts to speech, language, and literacy skill in children five to eight years old. We used Diffusion Tensor Imaging (DTI) to characterize the fiber pathways and administered several language assessments. The FAT was identified for the first time in children. Results showed no age-related change in integrity of the FAT, but did show age-related change in the left (but not right) SLF/AF. Moreover, only the integrity of the right FAT was related to phonology but not audiovisual speech perception, articulation, language, or literacy. Both the left and right SLF/AF related to language measures, specifically receptive and expressive language, and language content. These findings are important for understanding the neurobiology of language in the developing brain, and can be incorporated within contemporary dorsal-ventral-motor models for language.