The evaluation of qualitative methods selection in the field of design and emotion

The aim of this paper is to aid researchers in selecting appropriate qualitative methods in order to develop and improve future studies in the field of emotional design. These include observations, think-aloud protocols, questionnaires, diaries and interviews. Based on the authors’ experiences, it is proposed that the methods under review can be successfully used for collecting data on emotional responses to evaluate user product relationships. This paper reviews the methods; discusses the suitability, advantages and challenges in relation to design and emotion studies. Furthermore, the paper outlines the potential impact of technology on the application of these methods, discusses the implications of these methods for emotion research and concludes with recommendations for future work in this area.

Teaching design and emotion in the foundation year : an industrial design approach

This paper showcases two design tools; the ‘storyboard’ and ‘a day in the life’ demonstrated to design students in their foundational year (first year) of study. By employing these tools during the design process the aim was to provoke students to consider and design for emotional experiences for potential users. The assessment asked students to design an MP3 player using these tools. This is demonstrated through a student project that successfully used the tools and method introduced. The teaching theory, project context, student outcome as well as challenges faced by students using this approach are discussed. The paper concludes with implications for teaching emotion theory at an undergraduate level and potential future directions.

The design and development of music eScape: A new iPhone mood management music App

The physical, emotional, educational and social developmental challenges of adolescence can be associated with high levels of emotional vulnerability. Thus, the development of effective emotion-regulation strategies is crucial during this time period. Young people commonly use music to identify, express and regulate their emotions. Modern mobile technology provides an engaging, easily accessible means of assisting young people through music. A systematic contextual review identified 20 iPhone applications addressing emotions through music and two independent raters, using the Mobile App Rating Scale (MARS), evaluated the quality of the apps. Their characteristics, key features and overall quality will be presented. Three participatory design workshops (N=13, 6 males, 7 females; age 15-25) were conducted to explore young people’s use of music to enhance wellbeing. Young people were also asked to trial existing mood and music apps and to conceptualise their ultimate mood targeting music application. A thematic analysis of the participatory design workshops content identified the following music affect-regulation strategies: relationship building, modifying cognitions, modifying emotions, and immersing in emotions. The application of the key learnings from the mobile app review and participatory design workshops and the design and development of the music eScape app were presented and implications for future research was discussed.

Wearable neuroimaging and emotion : investigating emotional responses to architectural environments with functional near-infrared spectroscopy (fNIRS)

Supported by contemporary theories of architectural aesthetics and neuro-aesthetics this paper presents a case for the use of portable fNIRS imaging in the assessment of emotional responses to spatial environments experienced by both blind and sighted. The aim of the paper is to outline the implications of fNIRS for spatial research and practice within the field of architecture, thereby suggesting a potential taxonomy of particular formations of space and affect. Empirical neurological study of affect and spatial experience from an architectural design perspective remains in many instances unchartered. Clinical research using the portable non-invasive neuro-imaging device, functional near infrared spectroscopy (fNIRS) is proving convincing in its ability to detect emotional responses to visual, spatio-auditory and task based stimuli, providing a firm basis to potentially track cortical activity in the appraisal of architectural environments. Additionally, recent neurological studies have sought to explore the manifold sensory abilities of the visually impaired to better understand spatial perception in general. Key studies reveal that early blind participants perform as well as sighted due to higher auditory and somato-sensory spatial acuity. For instance, face vision enables the visually impaired to detect environments through skin pressure, enabling at times an instantaneous impression of the layout of an unfamiliar environment. Studies also report pleasant and unpleasant emotional responses such as ‘weightedness’ or ‘claustrophobia’ within certain interior environments, revealing a deeper perceptual sensitivity then would be expected. We conclude with justification that comparative fNIRS studies between the sighted and blind concerning spatial experience have the potential to provide greater understanding of emotional responses to architectural environments.

The role of design in the future of digital channels: Conceptual insights and future research directions

Purpose In the past channel literature has looked to other disciplines in developing and refining their theories, models and methods in order to evolve the field. This paper traces such history and highlights the substantial changes caused by the digital age. In light of this, the inclusion of design theory into future channel management is presented to overcome existing concerns. Design/methodology/approach A comprehensive review of literature on the history of channels, the emotional experience (people), limitations of digital innovation (technology) and the role of design (business) has been conducted to create a new approach, built upon the theory of the techno-economic innovation model. Findings The findings of this study propose design-led channel management as a new research area, providing novel research questions and future research directions. The inclusion of design and emotion theories indicates that the future of digital channel design requires a deeper understanding of customers and needs to go beyond technological advances. Theoretical implications The findings provide an opportunity to explore dynamic theories and methodologies within the field of design that will broaden the horizons and challenge existing notions in channel literature. Originality/value This paper is the first paper that introduces the theory of Emotionate, as the next evolution of channel literature. The value of Emotionate lies in providing a new design-led process of integrating emotion to provide advice to practitioners as well as identifies research areas for academia, thereby extending the reach and richness of this emerging research field.

User-centric need-driven affect modeling for spoken conversational agents: design and evaluation

It is easy to get frustrated at spoken conversational agents (SCAs), perhaps because they seem to be callous. By and large, the quality of human-computer interaction is affected due to the inability of the SCAs to recognise and adapt to user emotional state. Now with the mass appeal of artificially-mediated communication, there has been an increasing need for SCAs to be socially and emotionally intelligent, that is, to infer and adapt to their human interlocutors’ emotions on the fly, in order to ascertain an affective, empathetic and naturalistic interaction. An enhanced quality of interaction would reduce users’ frustrations and consequently increase their satisfactions. These reasons have motivated the development of SCAs towards including socio-emotional elements, turning them into affective and socially-sensitive interfaces. One barrier to the creation of such interfaces has been the lack of methods for modelling emotions in a task-independent environment. Most emotion models for spoken dialog systems are task-dependent and thus cannot be used “as-is” in different applications. This Thesis focuses on improving this, in which it concerns computational modeling of emotion, personality and their interrelationship for task-independent autonomous SCAs. The generation of emotion is driven by needs, inspired by human’s motivational systems. The work in this Thesis is organised in three stages, each one with its own contribution. The first stage involved defining, integrating and quantifying the psychological-based motivational and emotional models sourced from. Later these were transformed into a computational model by implementing them into software entities. The computational model was then incorporated and put to test with an existing SCA host, a HiFi-control agent. The second stage concerned automatic prediction of affect, which has been the main challenge towards the greater aim of infusing social intelligence into the HiFi agent. In recent years, studies on affect detection from voice have moved on to using realistic, non-acted data, which is subtler. However, it is more challenging to perceive subtler emotions and this is demonstrated in tasks such as labelling and machine prediction. In this stage, we attempted to address part of this challenge by considering the roles of user satisfaction ratings and conversational/dialog features as the respective target and predictors in discriminating contentment and frustration, two types of emotions that are known to be prevalent within spoken human-computer interaction. The final stage concerned the evaluation of the emotional model through the HiFi agent. A series of user studies with 70 subjects were conducted in a real-time environment, each in a different phase and with its own conditions. All the studies involved the comparisons between the baseline non-modified and the modified agent. The findings have gone some way towards enhancing our understanding of the utility of emotion in spoken dialog systems in several ways; first, an SCA should not express its emotions blindly, albeit positive. Rather, it should adapt its emotions to user states. Second, low performance in an SCA may be compensated by the exploitation of emotion. Third, the expression of emotion through the exploitation of prosody could better improve users’ perceptions of an SCA compared to exploiting emotions through just lexical contents. Taken together, these findings not only support the success of the emotional model, but also provide substantial evidences with respect to the benefits of adding emotion in an SCA, especially in mitigating users’ frustrations and ultimately improving their satisfactions. Resumen Es relativamente fácil experimentar cierta frustración al interaccionar con agentes conversacionales (Spoken Conversational Agents, SCA), a menudo porque parecen ser un poco insensibles. En general, la calidad de la interacción persona-agente se ve en cierto modo afectada por la incapacidad de los SCAs para identificar y adaptarse al estado emocional de sus usuarios. Actualmente, y debido al creciente atractivo e interés de dichos agentes, surge la necesidad de hacer de los SCAs unos seres cada vez más sociales y emocionalmente inteligentes, es decir, con capacidad para inferir y adaptarse a las emociones de sus interlocutores humanos sobre la marcha, de modo que la interacción resulte más afectiva, empática y, en definitiva, natural. Una interacción mejorada en este sentido permitiría reducir la posible frustración de los usuarios y, en consecuencia, mejorar el nivel de satisfacción alcanzado por los mismos. Estos argumentos justifican y motivan el desarrollo de nuevos SCAs con capacidades socio-emocionales, dotados de interfaces afectivas y socialmente sensibles. Una de las barreras para la creación de tales interfaces ha sido la falta de métodos de modelado de emociones en entornos independientes de tarea. La mayoría de los modelos emocionales empleados por los sistemas de diálogo hablado actuales son dependientes de tarea y, por tanto, no pueden utilizarse "tal cual" en diferentes dominios o aplicaciones. Esta tesis se centra precisamente en la mejora de este aspecto, la definición de modelos computacionales de las emociones, la personalidad y su interrelación para SCAs autónomos e independientes de tarea. Inspirada en los sistemas motivacionales humanos en el ámbito de la psicología, la tesis propone un modelo de generación/producción de la emoción basado en necesidades. El trabajo realizado en la presente tesis está organizado en tres etapas diferenciadas, cada una con su propia contribución. La primera etapa incluyó la definición, integración y cuantificación de los modelos motivacionales de partida y de los modelos emocionales derivados a partir de éstos. Posteriormente, dichos modelos emocionales fueron plasmados en un modelo computacional mediante su implementación software. Este modelo computacional fue incorporado y probado en un SCA anfitrión ya existente, un agente con capacidad para controlar un equipo HiFi, de alta fidelidad. La segunda etapa se orientó hacia el reconocimiento automático de la emoción, aspecto que ha constituido el principal desafío en relación al objetivo mayor de infundir inteligencia social en el agente HiFi. En los últimos años, los estudios sobre reconocimiento de emociones a partir de la voz han pasado de emplear datos actuados a usar datos reales en los que la presencia u observación de emociones se produce de una manera mucho más sutil. El reconocimiento de emociones bajo estas condiciones resulta mucho más complicado y esta dificultad se pone de manifiesto en tareas tales como el etiquetado y el aprendizaje automático. En esta etapa, se abordó el problema del reconocimiento de las emociones del usuario a partir de características o métricas derivadas del propio diálogo usuario-agente. Gracias a dichas métricas, empleadas como predictores o indicadores del grado o nivel de satisfacción alcanzado por el usuario, fue posible discriminar entre satisfacción y frustración, las dos emociones prevalentes durante la interacción usuario-agente. La etapa final corresponde fundamentalmente a la evaluación del modelo emocional por medio del agente Hifi. Con ese propósito se llevó a cabo una serie de estudios con usuarios reales, 70 sujetos, interaccionando con diferentes versiones del agente Hifi en tiempo real, cada uno en una fase diferente y con sus propias características o capacidades emocionales. En particular, todos los estudios realizados han profundizado en la comparación entre una versión de referencia del agente no dotada de ningún comportamiento o característica emocional, y una versión del agente modificada convenientemente con el modelo emocional propuesto. Los resultados obtenidos nos han permitido comprender y valorar mejor la utilidad de las emociones en los sistemas de diálogo hablado. Dicha utilidad depende de varios aspectos. En primer lugar, un SCA no debe expresar sus emociones a ciegas o arbitrariamente, incluso aunque éstas sean positivas. Más bien, debe adaptar sus emociones a los diferentes estados de los usuarios. En segundo lugar, un funcionamiento relativamente pobre por parte de un SCA podría compensarse, en cierto modo, dotando al SCA de comportamiento y capacidades emocionales. En tercer lugar, aprovechar la prosodia como vehículo para expresar las emociones, de manera complementaria al empleo de mensajes con un contenido emocional específico tanto desde el punto de vista léxico como semántico, ayuda a mejorar la percepción por parte de los usuarios de un SCA. Tomados en conjunto, los resultados alcanzados no sólo confirman el éxito del modelo emocional, sino xv que constituyen además una evidencia decisiva con respecto a los beneficios de incorporar emociones en un SCA, especialmente en cuanto a reducir el nivel de frustración de los usuarios y, en última instancia, mejorar su satisfacción.

I feel you: the design and evaluation of a domotic affect-sensitive spoken conversational agent

We describe the work on infusion of emotion into a limited-task autonomous spoken conversational agent situated in the domestic environment, using a need-inspired task-independent emotion model (NEMO). In order to demonstrate the generation of affect through the use of the model, we describe the work of integrating it with a natural-language mixed-initiative HiFi-control spoken conversational agent (SCA). NEMO and the host system communicate externally, removing the need for the Dialog Manager to be modified, as is done in most existing dialog systems, in order to be adaptive. The first part of the paper concerns the integration between NEMO and the host agent. The second part summarizes the work on automatic affect prediction, namely, frustration and contentment, from dialog features, a non-conventional source, in the attempt of moving towards a more user-centric approach. The final part reports the evaluation results obtained from a user study, in which both versions of the agent (non-adaptive and emotionally-adaptive) were compared. The results provide substantial evidences with respect to the benefits of adding emotion in a spoken conversational agent, especially in mitigating users' frustrations and, ultimately, improving their satisfaction.

The Narrative Value of Dance Communication: "habitus", musicality and emotion

The richness of dance comes from the need to work with an individual body. Still, the body of the dancer belongs to plural context, crossed by artistic and social traditions, which locate the artists in a given field. We claim that role conflict is an essential component of the structure of collective artistic creativity. We address the production of discourse in a British dance company, with data that spawns from the ethnography ‘Dance and Cognition’, directed by David Kirsh at the University of California, together with WayneMcGregor-Random Dance. Our Critical Discourse Analysis is based on multiple interviews to the dancers and choreographer. Our findings show how creativity in dance seems to be empirically observable, and thus embodied and distributed shaped by the dance habitus of the particular social context.

Guidelines on building design and operation to minimize air quality impacts from major roads in transit oriented development

Poor air quality has a huge detrimental effect, both economic and on the quality of life, in Australia. Transit oriented design (TOD), which aims to minimise urban sprawl and lower dependency on vehicles, leads to an increasing number of buildings close to transport corridors. This project aims at providing guidelines that are appropriate to include within City Plan to inform future planning along road corridors, and provide recommendations on when mitigation measures should be utilised.