Learning Data-Driven Models of Non-Verbal Behaviors for Building Rapport Using an Intelligent Virtual Agent

There is a growing societal need to address the increasing prevalence of behavioral health issues, such as obesity, alcohol or drug use, and general lack of treatment adherence for a variety of health problems. The statistics, worldwide and in the USA, are daunting. Excessive alcohol use is the third leading preventable cause of death in the United States (with 79,000 deaths annually), and is responsible for a wide range of health and social problems. On the positive side though, these behavioral health issues (and associated possible diseases) can often be prevented with relatively simple lifestyle changes, such as losing weight with a diet and/or physical exercise, or learning how to reduce alcohol consumption. Medicine has therefore started to move toward finding ways of preventively promoting wellness, rather than solely treating already established illness. Evidence-based patient-centered Brief Motivational Interviewing (BMI) interven- tions have been found particularly effective in helping people find intrinsic motivation to change problem behaviors after short counseling sessions, and to maintain healthy lifestyles over the long-term. Lack of locally available personnel well-trained in BMI, however, often limits access to successful interventions for people in need. To fill this accessibility gap, Computer-Based Interventions (CBIs) have started to emerge. Success of the CBIs, however, critically relies on insuring engagement and retention of CBI users so that they remain motivated to use these systems and come back to use them over the long term as necessary. Because of their text-only interfaces, current CBIs can therefore only express limited empathy and rapport, which are the most important factors of health interventions. Fortunately, in the last decade, computer science research has progressed in the design of simulated human characters with anthropomorphic communicative abilities. Virtual characters interact using humans’ innate communication modalities, such as facial expressions, body language, speech, and natural language understanding. By advancing research in Artificial Intelligence (AI), we can improve the ability of artificial agents to help us solve CBI problems. To facilitate successful communication and social interaction between artificial agents and human partners, it is essential that aspects of human social behavior, especially empathy and rapport, be considered when designing human-computer interfaces. Hence, the goal of the present dissertation is to provide a computational model of rapport to enhance an artificial agent’s social behavior, and to provide an experimental tool for the psychological theories shaping the model. Parts of this thesis were already published in [LYL+12, AYL12, AL13, ALYR13, LAYR13, YALR13, ALY14].

Learning data-driven models of non-verbal behaviors for building rapport using an intelligent virtual agent

There is a growing societal need to address the increasing prevalence of behavioral health issues, such as obesity, alcohol or drug use, and general lack of treatment adherence for a variety of health problems. The statistics, worldwide and in the USA, are daunting. Excessive alcohol use is the third leading preventable cause of death in the United States (with 79,000 deaths annually), and is responsible for a wide range of health and social problems. On the positive side though, these behavioral health issues (and associated possible diseases) can often be prevented with relatively simple lifestyle changes, such as losing weight with a diet and/or physical exercise, or learning how to reduce alcohol consumption. Medicine has therefore started to move toward finding ways of preventively promoting wellness, rather than solely treating already established illness.^ Evidence-based patient-centered Brief Motivational Interviewing (BMI) interventions have been found particularly effective in helping people find intrinsic motivation to change problem behaviors after short counseling sessions, and to maintain healthy lifestyles over the long-term. Lack of locally available personnel well-trained in BMI, however, often limits access to successful interventions for people in need. To fill this accessibility gap, Computer-Based Interventions (CBIs) have started to emerge. Success of the CBIs, however, critically relies on insuring engagement and retention of CBI users so that they remain motivated to use these systems and come back to use them over the long term as necessary.^ Because of their text-only interfaces, current CBIs can therefore only express limited empathy and rapport, which are the most important factors of health interventions. Fortunately, in the last decade, computer science research has progressed in the design of simulated human characters with anthropomorphic communicative abilities. Virtual characters interact using humans’ innate communication modalities, such as facial expressions, body language, speech, and natural language understanding. By advancing research in Artificial Intelligence (AI), we can improve the ability of artificial agents to help us solve CBI problems.^ To facilitate successful communication and social interaction between artificial agents and human partners, it is essential that aspects of human social behavior, especially empathy and rapport, be considered when designing human-computer interfaces. Hence, the goal of the present dissertation is to provide a computational model of rapport to enhance an artificial agent’s social behavior, and to provide an experimental tool for the psychological theories shaping the model. Parts of this thesis were already published in [LYL+12, AYL12, AL13, ALYR13, LAYR13, YALR13, ALY14].^

Years 2 to 6 students' ability to generalise : models, representations and theory for teaching and learning

Over the last three years, in our Early Algebra Thinking Project, we have been studying Years 3 to 5 students’ ability to generalise in a variety of situations, namely, compensation principles in computation, the balance principle in equivalence and equations, change and inverse change rules with function machines, and pattern rules with growing patterns. In these studies, we have attempted to involve a variety of models and representations and to build students’ abilities to switch between them (in line with the theories of Dreyfus, 1991, and Duval, 1999). The results have shown the negative effect of closure on generalisation in symbolic representations, the predominance of single variance generalisation over covariant generalisation in tabular representations, and the reduced ability to readily identify commonalities and relationships in enactive and iconic representations. This chapter uses the results to explore the interrelation between generalisation and verbal and visual comprehension of context. The studies evidence the importance of understanding and communicating aspects of representational forms which allowed commonalities to be seen across or between representations. Finally the chapter explores the implications of the studies for a theory that describes a growth in integration of models and representations that leads to generalisation.

A tale of six fish: achieving social presence through discussion forums in an offline learning environment

It is widely held, through the Socratic tradition, that discussion is at the heart of learning. Moderated discussion forums have been shown to replicate the debate, argument and verbal defence of viewpoints that we have come to expect in face-to-face learning environments and that we generally accept to underpin learning. While much has been written about discussion forums in educational settings, particularly in how to moderate and promote effective interaction with students at a distance, this paper takes a different approach. It looks at how forums may be used to support face-to-face learning in the contemporary context of the massification of on-campus classes. Further to this, it will argue for discussion forums as an indicator of social presence in the learning environment. It will cautiously conclude that, through purposeful design, this form of asynchronous communication has a valuable role to play in creating a positive and supportive environment for students entering university. Discussion forums are tools with a versatility yet to be fully exploited.

Verbal instruction abolishes fear conditioned to racial out-group faces

Previous research has shown resistance to extinction of fear conditioned to racial out-group faces, suggesting that these stimuli may be subject to prepared fear learning. The current study replicated and extended previous research by using a different racial out-group, and testing the prediction that prepared fear learning is unaffected by verbal instructions. Four groups of Caucasian participants were trained with male in-group (Caucasian) or out-group (Chinese) faces as conditional stimuli; one paired with an electro-tactile shock (CS+) and one presented alone (CS). Before extinction, half the participants were instructed that no more shocks would be presented. Fear conditioning, indexed by larger electrodermal responses to, and blink startle modulation during the CS+, occurred during acquisition in all groups. Resistance to extinction of fear learning was found only in the racial out-group, no instruction condition. Fear conditioned to a racial out-group face was reduced following verbal instructions, contrary to predictions for the nature of prepared fear learning.

Talk about a YouTube video in preschool : the mutual production of shared understanding for learning with digital technology

Much of what is written about digital technologies in preschool contexts focuses on young children’s acquisition of skills rather than their meaning-making during use of technologies. In this paper, we consider how the viewing of a YouTube video was used by a teacher and children to produce shared understandings about it. Conversation analysis of talk and interaction during the viewing of the video establishes some of the ways that individual accounts of events were produced for others and then endorsed as shared understandings. The analysis establishes how adults and children made use of verbal and embodied actions during interactions to produce shared understandings of the YouTube video, the events it recorded and written commentary about those events

A prospective cohort study examining the preferred learning styles of acute care registered nurses

Objectives This paper reports on the preferred learning styles of Registered Nurses practicing in acute care environments and relationships between gender, age, post-graduate experience and the identified preferred learning styles. Methods A prospective cohort study design was used. Participants completed a demographic questionnaire and the Felder-Silverman Index of Learning Styles (ILS) questionnaire to determine preferred learning styles. Results Most of the Registered Nurse participants were balanced across the Active-Reflective (n = 77, 54%), and Sequential-Global (n = 96, 68%) scales. Across the other scales, sensing (n = 97, 68%) and visual (n = 76, 53%) were the most common preferred learning style. There were only a small proportion who had a preferred learning style of reflective (n = 21, 15%), intuitive (n = 5, 4%), verbal (n = 11, 8%) or global learning (n = 15, 11%). Results indicated that gender, age and years since undergraduate education were not related to the identified preferred learning styles. Conclusions The identification of Registered Nurses’ learning style provides information that nurse educators and others can use to make informed choices about modification, development and strengthening of professional hospital-based educational programs. The use of the Index of Learning Styles questionnaire and its ability to identify ‘balanced’ learning style preferences may potentially yield additional preferred learning style information for other health-related disciplines.

Enhancing learning in large undergraduate classes using custom made audio-visual cases

This is an exploratory study into the effective use of embedding custom made audiovisual case studies (AVCS) in enhancing the student’s learning experience. This paper describes a project that used AVCS for a large divergent cohort of undergraduate students, enrolled in an International Business course. The study makes a number of key contributions to advancing learning and teaching within the discipline. AVCS provide first hand reporting of the case material, where the students have the ability to improve their understanding from both verbal and nonverbal cues. The paper demonstrates how AVCS can be embedded in a student-centred teaching approach to capture the students’ interest and to enhance a deep approach to learning by providing real-world authentic experience.

Framing student evaluations of university learning and teaching : discursive strategies and textual outcomes

Evaluation in higher education is an evolving social practice, that is, it involves what people, institutions and broader systems do and say, how they do and say it, what they value, the effects of these practices and values, and how meanings are ascribed. The textual products (verbal, written, visual, gestural) that inform and are produced by, for and through evaluative practices are important as they promulgate particular kinds of meanings and values in specific contexts. This paper reports on an exploratory study that sought to investigate, using discourse analysis, the types of evaluative practices that were ascribed value, and the student responses that ensued, in different evaluative instruments. Findings indicate that when a reflective approach is taken to evaluation, students’ responses are more considered, they interrogate their own engagement in the learning context and they are more likely to demonstrate reconstructive thought. These findings have implications for reframing evaluation as reflective learning.

Auditory - Visual Matching in Learning Disabilities : Intervention Studies from Finland and Sweden

The present thesis discusses relevant issues in education: 1) learning disabilities including the role of comorbidity in LDs, and 2) the use of research-based interventions. This thesis consists of a series of four studies (three articles), which deepens the knowledge of the field of special education. Intervention studies (N=242) aimed to examine whether training using a nonverbal auditory-visual matching computer program had a remedial effect in different learning disabilities, such as developmental dyslexia, Attention Deficit Disorder (ADD) and Specific Language Impairment (SLI). These studies were conducted in both Finland and Sweden. The intervention’s non-verbal character made an international perspective possible. The results of the intervention studies confirmed, that the auditory-visual matching computer program, called Audilex had positive intervention effects. In Study I of children with developmental dyslexia there were also improvements in reading skills, specifically in reading nonsense words and reading speed. These improvements in tasks, which are thought to rely on phonological processing, suggest that such reading difficulties in dyslexia may stem in part from more basic perceptual difficulties, including those required to manage the visual and auditory components of the decoding task. In Study II the intervention had a positive effect on children with dyslexia; older students with dyslexia and surprisingly, students with ADD also benefited from this intervention. In conclusion, the role of comorbidity was apparent. An intervention effect was evident also in students’ school behavior. Study III showed that children with SLI experience difficulties very similar to those of children with dyslexia in auditory-visual matching. Children with language-based learning disabilities, such as dyslexia and SLI benefited from the auditory-visual matching intervention. Also comorbidity was evident among these children; in addition to formal diagnoses, comorbidity was explored with an assessment inventory, which was developed for this thesis. Interestingly, an overview of the data of this thesis shows positive intervention effects in all studies despite learning disability, language, gender or age. These findings have been described by a concept inter-modal transpose. Self-evidently these issues need further studies. In learning disabilities the aim in the future will also be to identify individuals at risk rather than by deficit; this aim can be achieved by using research-based interventions, intensified support in general education and inclusive special education. Keywords: learning disabilities, developmental dyslexia, attention deficit disorder, specific language impairment, language-based learning disabilities, comorbidity, auditory-visual matching, research-based interventions, inter-modal transpose

Learning style and digital activity: An ecological study

In order to understand student engagement in higher education through the use of digital technologies, it is necessary to appreciate the broader use of differing technologies. Forty-eight first-year university students completed an online survey that queried patterns of digital activity across home, school and community contexts and that included rating scale items that measured learning style (i.e., active-reflective, sensing-intuitive, visual-verbal, sequential-global). Results suggest that students vary widely in digital activities and that such variation is related to differences in learning style. For example, active learners were more likely than reflective learners to engage in digital activities in the community and users of some specific application, as opposed to non-users, were more likely to be verbal than visual learners. Implications for instructional applications of digital technology in higher education are presented.

Teaching, learning and assessment in MSc. Databases units at University of Greenwich

In this paper we look at ways of delivering and assessing learning on database units offered on higher degree programmes (MSc) in the School of Computing and Mathematical Sciences at the University of Greenwich. Of critical importance is the teaching methods employed for verbal disposition, practical laboratory exercises and a careful evaluation of assessment methods and assessment tools in view of the fact that databases involve not only database design but also use of practical tools, such as database management systems (DBMSs) software, human designers, database administrators (DBA) and end users. Our goal is to clearly identify potential key success factors in delivering and assessing learning in both practical and theoretical aspects of database course units.

Episodic memory and learning in patients with chronic schizophrenia

This study explored the pattern of memory functioning in 58 patients with chronic schizophrenia and compared their performance with 53 normal controls. Multiple domains of memory were assessed, including verbal and nonverbal memory span, verbal and non-verbal paired associate learning, verbal and visual long-term memory, spatial and non-spatial conditional associative learning, recognition memory and memory for temporal order. Consistent with previous studies, substantial deficits in long-term memory were observed, with relative preservation of memory span. Memory for temporal order and recognition memory was intact, although significant deficits were observed on the conditional associative learning tasks. There was no evidence of lateralized memory impairment. In these respects, the pattern of memory impairment in schizophrenia is more similar in nature to that found in patients with memory dysfunction following mesiotemporal lobe lesions, rather than that associated with focal frontal lobe damage. (C) 1999 Elsevier Science B.V. All rights reserved.