Assessment of Speech Dialog Systems using Multi-Modal Cognitive Load Analysis and Driving Performance Metrics

In this paper, cognitive load analysis via acoustic- and CAN-Bus-based driver performance metrics is employed to assess two different commercial speech dialog systems (SDS) during in-vehicle use. Several metrics are proposed to measure increases in stress, distraction and cognitive load and we compare these measures with statistical analysis of the speech recognition component of each SDS. It is found that care must be taken when designing an SDS as it may increase cognitive load which can be observed through increased speech response delay (SRD), changes in speech production due to negative emotion towards the SDS, and decreased driving performance on lateral control tasks. From this study, guidelines are presented for designing systems which are to be used in vehicular environments.

A Likelihood-Maximizing Framework for Enhanced In-Car Speech Recognition Based on Speech Dialog System Interaction

Speech recognition in car environments has been identified as a valuable means for reducing driver distraction when operating noncritical in-car systems. Under such conditions, however, speech recognition accuracy degrades significantly, and techniques such as speech enhancement are required to improve these accuracies. Likelihood-maximizing (LIMA) frameworks optimize speech enhancement algorithms based on recognized state sequences rather than traditional signal-level criteria such as maximizing signal-to-noise ratio. LIMA frameworks typically require calibration utterances to generate optimized enhancement parameters that are used for all subsequent utterances. Under such a scheme, suboptimal recognition performance occurs in noise conditions that are significantly different from that present during the calibration session – a serious problem in rapidly changing noise environments out on the open road. In this chapter, we propose a dialog-based design that allows regular optimization iterations in order to track the ever-changing noise conditions. Experiments using Mel-filterbank noise subtraction (MFNS) are performed to determine the optimization requirements for vehicular environments and show that minimal optimization is required to improve speech recognition, avoid over-optimization, and ultimately assist with semireal-time operation. It is also shown that the proposed design is able to provide improved recognition performance over frameworks incorporating a calibration session only.

Dialog

This work, commissioned by Campbelltown Arts Centre, was created as part of an online residency in Western Sydney. The residency took the form of an online survey of students in Western Sydney schools that queried participants on their favourite moments, characters and dialogue from film and television. Using this information as a starting point, the work used appropriated footage to weave together a 6-way cross-screen conversation. Spouting occasionally recognizable phrases that then devolve into meaningless cliché, the narrative content of this fragmented back-and-forth hovers somewhere in between familiarity and non-sense.

Meaning in movement : a gestural design game

This paper describes a design game that we called 'Meaning in Movement'. The purpose was to explore notions of professional dental practice with dental practioners in terms of gestures, actions and movements. The game represents a first step towards involving gestures, actions and movements in a design dialog with practioners for the purpose of designing future interactive systems which are more appropriate to the type of skilful actions and richly structured environments of dentists and dental assistants.

Likelihood-maximising frameworks for enhanced in-car speech recognition

Speech recognition in car environments has been identified as a valuable means for reducing driver distraction when operating non-critical in-car systems. Likelihood-maximising (LIMA) frameworks optimise speech enhancement algorithms based on recognised state sequences rather than traditional signal-level criteria such as maximising signal-to-noise ratio. Previously presented LIMA frameworks require calibration utterances to generate optimised enhancement parameters which are used for all subsequent utterances. Sub-optimal recognition performance occurs in noise conditions which are significantly different from that present during the calibration session - a serious problem in rapidly changing noise environments. We propose a dialog-based design which allows regular optimisation iterations in order to track the changing noise conditions. Experiments using Mel-filterbank spectral subtraction are performed to determine the optimisation requirements for vehicular environments and show that minimal optimisation assists real-time operation with improved speech recognition accuracy. It is also shown that the proposed design is able to provide improved recognition performance over frameworks incorporating a calibration session.

Impact of cognitive load and frustration on drivers’ speech [Abstract]

Secondary tasks such as cell phone calls or interaction with automated speech dialog systems (SDSs) increase the driver’s cognitive load as well as the probability of driving errors. This study analyzes speech production variations due to cognitive load and emotional state of drivers in real driving conditions. Speech samples were acquired from 24 female and 17 male subjects (approximately 8.5 h of data) while talking to a co-driver and communicating with two automated call centers, with emotional states (neutral, negative) and the number of necessary SDS query repetitions also labeled. A consistent shift in a number of speech production parameters (pitch, first format center frequency, spectral center of gravity, spectral energy spread, and duration of voiced segments) was observed when comparing SDS interaction against co-driver interaction; further increases were observed when considering negative emotion segments and the number of requested SDS query repetitions. A mel frequency cepstral coefficient based Gaussian mixture classifier trained on 10 male and 10 female sessions provided 91% accuracy in the open test set task of distinguishing co-driver interactions from SDS interactions, suggesting—together with the acoustic analysis—that it is possible to monitor the level of driver distraction directly from their speech.

Analysis and detection of cognitive load and frustration in drivers' speech

Non-driving related cognitive load and variations of emotional state may impact a driver’s capability to control a vehicle and introduces driving errors. Availability of reliable cognitive load and emotion detection in drivers would benefit the design of active safety systems and other intelligent in-vehicle interfaces. In this study, speech produced by 68 subjects while driving in urban areas is analyzed. A particular focus is on speech production differences in two secondary cognitive tasks, interactions with a co-driver and calls to automated spoken dialog systems (SDS), and two emotional states during the SDS interactions - neutral/negative. A number of speech parameters are found to vary across the cognitive/emotion classes. Suitability of selected cepstral- and production-based features for automatic cognitive task/emotion classification is investigated. A fusion of GMM/SVM classifiers yields an accuracy of 94.3% in cognitive task and 81.3% in emotion classification.

Sound thinking : tips and tools for understanding popular music

Sound Thinking provides techniques and approaches to critically listen, think, talk and write about music you hear or make. It provides tips on making music and it encourages regular and deep thinking about music activities, which helps build a musical dialog that leads to deeper understanding.

From 'what' to 'how' : capacity building in health promotion for HIV/AIDS prevention in the Solomon Islands

This paper describes a capacity building process undertaken within the HIV/AIDS prevention project of the Adventist Development and Relief Agency (ADRA) in the Solomon Islands. ADRA HIV/AIDS has recently reoriented its project structure, moving beyond its awareness raising approach to incorporate health promotion frameworks, theories, strategies and assumptions. These have been used to inform project practice in project planning, delivery and evaluation. This paper shares what has worked and not worked in the capacity building process, including a project evaluation of the initial HIV/AIDS awareness raising project and the application of a number of capacity building strategies, including utilising a volunteer Australian Youth Ambassador for Development (AYAD) funded by the Australian Agency for International Development (AusAID). Existing and new projects are outlined. The underlying theme is that any capacity building exercise must include structural support (e.g. management, national frameworks) to ensure the incorporation of new initiatives and approaches. With time this enables ownership by counterparts and external partnerships to develop. The presence of an AYAD volunteer has been an effective strategy to achieve this. Reflections from the evaluators, the AYAD volunteer and the HIV/AIDS team are included.

Evolving directions in health promotion workforce development

PROJECT CONTEXT: Leaders in the fields of public health and health promotion increasingly advocate a socio-ecological approach to meet contemporary and emerging population health challenges. It is essential that health promotion workforce development initiatives mirror the evolving direction of the field to facilitate translation of theory into practice. To date, there has been limited effort to map the socio-ecological approach into tertiary education curricula. PROJECT DESCRIPTION: This project was undertaken as part of the development process for an undergraduate health promotion degree in Queensland, Australia. A review of the health promotion workforce development literature was undertaken. Group processes, key informant interviews and a Delphi technique were used to engage health promotion academics and practitioners, including an International Health Promotion Expert Advisory Panel, and an Industry Advisory Group in defining the components of the program. FINDINGS: The consultative processes facilitated the development of an undergraduate health promotion degree program underpinned by the socio-ecological approach with strong emphases upon the processes or 'how you do it' of health promotion together with evidence-based decision making and practice. CONCLUSIONS: As the basis and practice of health promotion progresses toward a socio-ecological approach, workforce training needs to keep pace with these developments to ensure an appropriately skilled health promotion workforce to meet emerging population health challenges. The reported project and the degree program that has been developed is an example of one step towards achieving this important and necessary shift in health promotion workforce development in Australia.

Building shared situational awareness in surgery through distributed dialogue

Background Failure to convey time-critical information to team members during surgery diminishes members’ perception of the dynamic information relevant to their task, and compromises shared situational awareness. This research reports the dialog around clinical decisions made by team members in the time-pressured and high-risk context of surgery, and the impact of these communications on shared situational awareness. Methods Fieldwork methods were used to capture the dynamic integration of individual and situational elements in surgery that provided the backdrop for clinical decisions. Nineteen semi structured interviews were performed with 24 participants from anaesthesia, surgery, and nursing in the operating rooms of a large metropolitan hospital in Queensland, Australia. Thematic analysis was used. Results: The domain “coordinating decisions in surgery” was generated from textual data. Within this domain, three themes illustrated the dialog of clinical decisions, i.e., synchronizing and strategizing actions, sharing local knowledge, and planning contingency decisions based on priority. Conclusion Strategies used to convey decisions that enhanced shared situational awareness included the use of “self-talk”, closed-loop communications, and “overhearing” conversations that occurred at the operating table. Behaviours’ that compromised a team’s shared situational awareness included tunnelling and fixating on one aspect of the situation.

Talking green: From Oz to Oz

This paper is an extension of Talking Green in Red States: Stories from the Great Plains (Gibson et al., 2014) that examined the communication strategies of planners involved in sustainability planning initiatives in the U.S. Great Plains. We expand these narratives beyond the Great Plains to Queensland, Australia by interviewing planners about their experiences when communicating and deliberating about issues of sustainability. Using a semi-formal structure, interviews are conducted as casual “coffee talks” (Maynard-Moody & Musheno, 2003). Together, these collected stories help us to understand how planners are “talking green” in conservative political atmospheres, across vastly differing geographies using an international comparative framework described by Reimer et al. (2014). The paper presents comparisons between the collected narratives, concerning similarities and differences in regard to the nuances of sustainability planning dialog. We relate the lessons learned about communication strategies of planners working in sustainability in Kansas and Queensland to the broader discourse of planning and politics, communicative planning and planning as storytelling, as they relate to sustainability planning in challenging situations. From this paper, the audience will better understand how: 1) to discuss environmental and sustainable planning in communities that have varying levels of distrust and suspicion towards these concepts; 2) to develop strategies to work around these planning communication issues, and; 3) international context affects these communication challenges.

Transforming higher education and student engagement through collaborative review to inform educational design

This paper reports on staff perceptions arising from a review process designed to assist staff in making informed decisions regarding educational design, approaches to engage students in learning, and the technology to support engagement in the classroom and across multiple locations and delivery modes. The aim of the review process was to transform the level of student engagement in the business faculty of an Australian university. The process took a collaborative approach through consultation with academic staff involved in the design and delivery of the units under review, and included targeted professional development as necessary. An institutional framework that characterises engagement indicator contexts and their attributes facilitated dialog during the review process. This paper reports on a mixed method study that included a survey of participants, and purposeful interviews to evaluate the effectiveness of the process. Although the study identified factors that hindered implementation and operationalization of review recommendations in some instances, study participants were generally of the view that recommendations would enhance student engagement. It is demonstrated that the bottom-up approach described in this paper is consistent with theoretical frameworks for transformational change in teaching and learning and the adoption of innovations.