Contributions to the contextualization of human-machine spoken interaction systems

The design and development of spoken interaction systems has been a thoroughly studied research scope for the last decades. The aim is to obtain systems with the ability to interact with human agents with a high degree of naturalness and efficiency, allowing them to carry out the actions they desire using speech, as it is the most natural means of communication between humans. To achieve that degree of naturalness, it is not enough to endow systems with the ability to accurately understand the user’s utterances and to properly react to them, even considering the information provided by the user in his or her previous interactions. The system has also to be aware of the evolution of the conditions under which the interaction takes place, in order to act the most coherent way as possible at each moment. Consequently, one of the most important features of the system is that it has to be context-aware. This context awareness of the system can be reflected in the modification of the behaviour of the system taking into account the current situation of the interaction. For instance, the system should decide which action it has to carry out, or the way to perform it, depending on the user that requests it, on the way that the user addresses the system, on the characteristics of the environment in which the interaction takes place, and so on. In other words, the system has to adapt its behaviour to these evolving elements of the interaction. Moreover that adaptation has to be carried out, if possible, in such a way that the user: i) does not perceive that the system has to make any additional effort, or to devote interaction time to perform tasks other than carrying out the requested actions, and ii) does not have to provide the system with any additional information to carry out the adaptation, which could imply a lesser efficiency of the interaction, since users should devote several interactions only to allow the system to become adapted. In the state-of-the-art spoken dialogue systems, researchers have proposed several disparate strategies to adapt the elements of the system to different conditions of the interaction (such as the acoustic characteristics of a specific user’s speech, the actions previously requested, and so on). Nevertheless, to our knowledge there is not any consensus on the procedures to carry out these adaptation. The approaches are to an extent unrelated from one another, in the sense that each one considers different pieces of information, and the treatment of that information is different taking into account the adaptation carried out. In this regard, the main contributions of this Thesis are the following ones: Definition of a contextualization framework. We propose a unified approach that can cover any strategy to adapt the behaviour of a dialogue system to the conditions of the interaction (i.e. the context). In our theoretical definition of the contextualization framework we consider the system’s context as all the sources of variability present at any time of the interaction, either those ones related to the environment in which the interaction takes place, or to the human agent that addresses the system at each moment. Our proposal relies on three aspects that any contextualization approach should fulfill: plasticity (i.e. the system has to be able to modify its behaviour in the most proactive way taking into account the conditions under which the interaction takes place), adaptivity (i.e. the system has also to be able to consider the most appropriate sources of information at each moment, both environmental and user- and dialogue-dependent, to effectively adapt to the conditions aforementioned), and transparency (i.e. the system has to carry out the contextualizaton-related tasks in such a way that the user neither perceives them nor has to do any effort in providing the system with any information that it needs to perform that contextualization). Additionally, we could include a generality aspect to our proposed framework: the main features of the framework should be easy to adopt in any dialogue system, regardless of the solution proposed to manage the dialogue. Once we define the theoretical basis of our contextualization framework, we propose two cases of study on its application in a spoken dialogue system. We focus on two aspects of the interaction: the contextualization of the speech recognition models, and the incorporation of user-specific information into the dialogue flow. One of the modules of a dialogue system that is more prone to be contextualized is the speech recognition system. This module makes use of several models to emit a recognition hypothesis from the user’s speech signal. Generally speaking, a recognition system considers two types of models: an acoustic one (that models each of the phonemes that the recognition system has to consider) and a linguistic one (that models the sequences of words that make sense for the system). In this work we contextualize the language model of the recognition system in such a way that it takes into account the information provided by the user in both his or her current utterance and in the previous ones. These utterances convey information useful to help the system in the recognition of the next utterance. The contextualization approach that we propose consists of a dynamic adaptation of the language model that is used by the recognition system. We carry out this adaptation by means of a linear interpolation between several models. Instead of training the best interpolation weights, we make them dependent on the conditions of the dialogue. In our approach, the system itself will obtain these weights as a function of the reliability of the different elements of information available, such as the semantic concepts extracted from the user’s utterance, the actions that he or she wants to carry out, the information provided in the previous interactions, and so on. One of the aspects more frequently addressed in Human-Computer Interaction research is the inclusion of user specific characteristics into the information structures managed by the system. The idea is to take into account the features that make each user different from the others in order to offer to each particular user different services (or the same service, but in a different way). We could consider this approach as a user-dependent contextualization of the system. In our work we propose the definition of a user model that contains all the information of each user that could be potentially useful to the system at a given moment of the interaction. In particular we will analyze the actions that each user carries out throughout his or her interaction. The objective is to determine which of these actions become the preferences of that user. We represent the specific information of each user as a feature vector. Each of the characteristics that the system will take into account has a confidence score associated. With these elements, we propose a probabilistic definition of a user preference, as the action whose likelihood of being addressed by the user is greater than the one for the rest of actions. To include the user dependent information into the dialogue flow, we modify the information structures on which the dialogue manager relies to retrieve information that could be needed to solve the actions addressed by the user. Usage preferences become another source of contextual information that will be considered by the system towards a more efficient interaction (since the new information source will help to decrease the need of the system to ask users for additional information, thus reducing the number of turns needed to carry out a specific action). To test the benefits of the contextualization framework that we propose, we carry out an evaluation of the two strategies aforementioned. We gather several performance metrics, both objective and subjective, that allow us to compare the improvements of a contextualized system against the baseline one. We will also gather the user’s opinions as regards their perceptions on the behaviour of the system, and its degree of adaptation to the specific features of each interaction. Resumen El diseño y el desarrollo de sistemas de interacción hablada ha sido objeto de profundo estudio durante las pasadas décadas. El propósito es la consecución de sistemas con la capacidad de interactuar con agentes humanos con un alto grado de eficiencia y naturalidad. De esta manera, los usuarios pueden desempeñar las tareas que deseen empleando la voz, que es el medio de comunicación más natural para los humanos. A fin de alcanzar el grado de naturalidad deseado, no basta con dotar a los sistemas de la abilidad de comprender las intervenciones de los usuarios y reaccionar a ellas de manera apropiada (teniendo en consideración, incluso, la información proporcionada en previas interacciones). Adicionalmente, el sistema ha de ser consciente de las condiciones bajo las cuales transcurre la interacción, así como de la evolución de las mismas, de tal manera que pueda actuar de la manera más coherente en cada instante de la interacción. En consecuencia, una de las características primordiales del sistema es que debe ser sensible al contexto. Esta capacidad del sistema de conocer y emplear el contexto de la interacción puede verse reflejada en la modificación de su comportamiento debida a las características actuales de la interacción. Por ejemplo, el sistema debería decidir cuál es la acción más apropiada, o la mejor manera de llevarla a término, dependiendo del usuario que la solicita, del modo en el que lo hace, etcétera. En otras palabras, el sistema ha de adaptar su comportamiento a tales elementos mutables (o dinámicos) de la interacción. Dos características adicionales son requeridas a dicha adaptación: i) el usuario no ha de percibir que el sistema dedica recursos (temporales o computacionales) a realizar tareas distintas a las que aquél le solicita, y ii) el usuario no ha de dedicar esfuerzo alguno a proporcionar al sistema información adicional para llevar a cabo la interacción. Esto último implicaría una menor eficiencia de la interacción, puesto que los usuarios deberían dedicar parte de la misma a proporcionar información al sistema para su adaptación, sin ningún beneficio inmediato. En los sistemas de diálogo hablado propuestos en la literatura, se han propuesto diferentes estrategias para llevar a cabo la adaptación de los elementos del sistema a las diferentes condiciones de la interacción (tales como las características acústicas del habla de un usuario particular, o a las acciones a las que se ha referido con anterioridad). Sin embargo, no existe una estrategia fija para proceder a dicha adaptación, sino que las mismas no suelen guardar una relación entre sí. En este sentido, cada una de ellas tiene en cuenta distintas fuentes de información, la cual es tratada de manera diferente en función de las características de la adaptación buscada. Teniendo en cuenta lo anterior, las contribuciones principales de esta Tesis son las siguientes: Definición de un marco de contextualización. Proponemos un criterio unificador que pueda cubrir cualquier estrategia de adaptación del comportamiento de un sistema de diálogo a las condiciones de la interacción (esto es, el contexto de la misma). En nuestra definición teórica del marco de contextualización consideramos el contexto del sistema como todas aquellas fuentes de variabilidad presentes en cualquier instante de la interacción, ya estén relacionadas con el entorno en el que tiene lugar la interacción, ya dependan del agente humano que se dirige al sistema en cada momento. Nuestra propuesta se basa en tres aspectos que cualquier estrategia de contextualización debería cumplir: plasticidad (es decir, el sistema ha de ser capaz de modificar su comportamiento de la manera más proactiva posible, teniendo en cuenta las condiciones en las que tiene lugar la interacción), adaptabilidad (esto es, el sistema ha de ser capaz de considerar la información oportuna en cada instante, ya dependa del entorno o del usuario, de tal manera que adecúe su comportamiento de manera eficaz a las condiciones mencionadas), y transparencia (que implica que el sistema ha de desarrollar las tareas relacionadas con la contextualización de tal manera que el usuario no perciba la manera en que dichas tareas se llevan a cabo, ni tampoco deba proporcionar al sistema con información adicional alguna). De manera adicional, incluiremos en el marco propuesto el aspecto de la generalidad: las características del marco de contextualización han de ser portables a cualquier sistema de diálogo, con independencia de la solución propuesta en los mismos para gestionar el diálogo. Una vez hemos definido las características de alto nivel de nuestro marco de contextualización, proponemos dos estrategias de aplicación del mismo a un sistema de diálogo hablado. Nos centraremos en dos aspectos de la interacción a adaptar: los modelos empleados en el reconocimiento de habla, y la incorporación de información específica de cada usuario en el flujo de diálogo. Uno de los módulos de un sistema de diálogo más susceptible de ser contextualizado es el sistema de reconocimiento de habla. Este módulo hace uso de varios modelos para generar una hipótesis de reconocimiento a partir de la señal de habla. En general, un sistema de reconocimiento emplea dos tipos de modelos: uno acústico (que modela cada uno de los fonemas considerados por el reconocedor) y uno lingüístico (que modela las secuencias de palabras que tienen sentido desde el punto de vista de la interacción). En este trabajo contextualizamos el modelo lingüístico del reconocedor de habla, de tal manera que tenga en cuenta la información proporcionada por el usuario, tanto en su intervención actual como en las previas. Estas intervenciones contienen información (semántica y/o discursiva) que puede contribuir a un mejor reconocimiento de las subsiguientes intervenciones del usuario. La estrategia de contextualización propuesta consiste en una adaptación dinámica del modelo de lenguaje empleado en el reconocedor de habla. Dicha adaptación se lleva a cabo mediante una interpolación lineal entre diferentes modelos. En lugar de entrenar los mejores pesos de interpolación, proponemos hacer los mismos dependientes de las condiciones actuales de cada diálogo. El propio sistema obtendrá estos pesos como función de la disponibilidad y relevancia de las diferentes fuentes de información disponibles, tales como los conceptos semánticos extraídos a partir de la intervención del usuario, o las acciones que el mismo desea ejecutar. Uno de los aspectos más comúnmente analizados en la investigación de la Interacción Persona-Máquina es la inclusión de las características específicas de cada usuario en las estructuras de información empleadas por el sistema. El objetivo es tener en cuenta los aspectos que diferencian a cada usuario, de tal manera que el sistema pueda ofrecer a cada uno de ellos el servicio más apropiado (o un mismo servicio, pero de la manera más adecuada a cada usuario). Podemos considerar esta estrategia como una contextualización dependiente del usuario. En este trabajo proponemos la definición de un modelo de usuario que contenga toda la información relativa a cada usuario, que pueda ser potencialmente utilizada por el sistema en un momento determinado de la interacción. En particular, analizaremos aquellas acciones que cada usuario decide ejecutar a lo largo de sus diálogos con el sistema. Nuestro objetivo es determinar cuáles de dichas acciones se convierten en las preferencias de cada usuario. La información de cada usuario quedará representada mediante un vector de características, cada una de las cuales tendrá asociado un valor de confianza. Con ambos elementos proponemos una definición probabilística de una preferencia de uso, como aquella acción cuya verosimilitud es mayor que la del resto de acciones solicitadas por el usuario. A fin de incluir la información dependiente de usuario en el flujo de diálogo, llevamos a cabo una modificación de las estructuras de información en las que se apoya el gestor de diálogo para recuperar información necesaria para resolver ciertos diálogos. En dicha modificación las preferencias de cada usuario pasarán a ser una fuente adicional de información contextual, que será tenida en cuenta por el sistema en aras de una interacción más eficiente (puesto que la nueva fuente de información contribuirá a reducir la necesidad del sistema de solicitar al usuario información adicional, dando lugar en consecuencia a una reducción del número de intervenciones necesarias para llevar a cabo una acción determinada). Para determinar los beneficios de las aplicaciones del marco de contextualización propuesto, llevamos a cabo una evaluación de un sistema de diálogo que incluye las estrategias mencionadas. Hemos recogido diversas métricas, tanto objetivas como subjetivas, que nos permiten determinar las mejoras aportadas por un sistema contextualizado en comparación con el sistema sin contextualizar. De igual manera, hemos recogido las opiniones de los participantes en la evaluación acerca de su percepción del comportamiento del sistema, y de su capacidad de adaptación a las condiciones concretas de cada interacción.

DAEDALUS at PAN 2014: Guessing tweet author's gender and age

This paper describes our participation at PAN 2014 author profiling task. Our idea was to define, develop and evaluate a simple machine learning classifier able to guess the gender and the age of a given user based on his/her texts, which could become part of the solution portfolio of the company. We were interested in finding not the best possible classifier that achieves the highest accuracy, but to find the optimum balance between performance and throughput using the most simple strategy and less dependent of external systems. Results show that our software using Naive Bayes Multinomial with a term vector model representation of the text is ranked quite well among the rest of participants in terms of accuracy.

Towards automatic tweet generation: A comparative study from the text summarization perspective in the journalism genre

In recent years, Twitter has become one of the most important microblogging services of the Web 2.0. Among the possible uses it allows, it can be employed for communicating and broadcasting information in real time. The goal of this research is to analyze the task of automatic tweet generation from a text summarization perspective in the context of the journalism genre. To achieve this, different state-of-the-art summarizers are selected and employed for producing multi-lingual tweets in two languages (English and Spanish). A wide experimental framework is proposed, comprising the creation of a new corpus, the generation of the automatic tweets, and their assessment through a quantitative and a qualitative evaluation, where informativeness, indicativeness and interest are key criteria that should be ensured in the proposed context. From the results obtained, it was observed that although the original tweets were considered as model tweets with respect to their informativeness, they were not among the most interesting ones from a human viewpoint. Therefore, relying only on these tweets may not be the ideal way to communicate news through Twitter, especially if a more personalized and catchy way of reporting news wants to be performed. In contrast, we showed that recent text summarization techniques may be more appropriate, reflecting a balance between indicativeness and interest, even if their content was different from the tweets delivered by the news providers.

Deconstruction-analysis-explanation:contextualization in professional digital discourse

It is already a truism that emerging communication technologies have changed the landscape of communication in every aspect of our lives, but this is specifically true for how we communicate at work. Advances in communication technologies have enabled a wide range of digital communication modes to be utilized for both internal and external business communication; including audio and visual communication and voice-over protocols, as well as text-based channels, such as email, forums, instant messaging and social media. In spite of the wide range of available audio-visual channels, and despite the ever-increasing popularity of email, real-time text-based communication technologies (instant messaging or IM) are also on the rise (see Mak, 2014; Pazos et al., 2013; Radicati & Levenstein, 2013; and Markman in this volume). The prominence of IM is evident in the rise of this mode of communication, not only as a tool for internal business communication, but as a front-stage channel, particularly for customer service encounters or professional-client conversations (Makarem et al., 2009; Pearce et al., 2013; L. Zhang et al., 2011).

Documenting the Contextualization and Implementation of mhGAP-HIG in Post-earthquake Nepal

Background: The burden of mental health is increased in humanitarian settings, and needs to be addressed in emergency situations. The World Health Organization has recently released the mental health Global Action Programme Humanitarian Intervention Guide (mhGAP-HIG) in order to scale up mental health service delivery in humanitarian settings through task-shifting. This study aims to evaluate, contextualize and identify possible barriers and challenges to mhGAP-HIG manual content, training and implementation in post-earthquake Nepal.

Methods: This qualitative study was conducted in Kathmandu, Nepal. Key informant interviews were conducted with fourteen psychiatrists involved in a mhGAP-HIG Training of Trainers and Supervisors (ToTS) in order to assess the mhGAP-HIG, ToTS training, and the potential challenges and barriers to mhGAP-HIG implementation. Themes identified by informants were supplemented by process notes taken by the researcher during observed training sessions and meetings.

Results: Key themes emerging from key informant interviews include the need to take three factors into account in manual contextualization: culture, health systems and the humanitarian setting. This includes translation of the manual into the local language, adding or expanding upon conditions prevalent in Nepal, and more consideration to improving feasibility of manual use by non-specialists.

Conclusion: The mhGAP-HIG must be tailored to specific humanitarian settings for effective implementation. This study shows the importance of conducting a manual contextualization workshop prior to training in order to maximize the feasibility and success in training health care workers in mhGAP.

Two studies on Twitter networks and tweet content in relation to Amyotrophic Lateral Sclerosis (ALS): conversation, information, and ‘Diary of a Daily Life’

To date, there is no research examining how adults with Amyotrophic Lateral Sclerosis (ALS) or Motor Neurone Disease (MND) and severe communication disability use Twitter, nor the use of Twitter in relation to ALS/MND beyond its use for fundraising and raising awareness. In this paper we (a) outline a rationale for the use of Twitter as a method of communication and information exchange for adults with ALS/MND, (b) detail multiple qualitative and quantitative methods used to analyse Twitter networks and tweet content in the our studies, and (c) present the results of two studies designed to provide insights on the use of Twitter by an adult with ALS/MND and by #ALS and #MND hashtag communities in Twitter. We will also discuss findings across the studies, implications for health service providers in Twitter, and directions for future Twitter research in relation to ALS/MND.

Student resources for the use of supervisors

The materials presented here are intended to: a) accompany the document Supervisor Resource and b) provide technology supervisors with materials that may be readily shared with students. These resources are not designed to be distributed to students without contextualization, they are intended for use in workshops or in discussions between supervisors and students. As authors, we anticipate that supervisors or workshop facilitators are most likely to extract individual resources of interest for particular occasions. The materials have been developed from conversations with supervisors from the technology disciplines.

A phenomenological study of contemplative experiences : implications for interior design

This research reports on a project concerned with the relationship between the person and the environment in the context of achieving a contemplative or existential state – a state which can be experienced either consciously or subconsciously. The need for such a study originated with the desire to contribute to the design of multicultural spaces which could be used for a range of activities within the public and the personal arena, activities including contemplation, meditation and prayer. The concept of ‘sacred’ is explored in the literature review and in primary interviews with the participants of this study. Given that the word ‘sacred’ is highly value-laden and potentially alienating for some people, it was decided to use the more accessible term ‘contemplative’. The outcomes of the study inform the practice of interior design and architecture which tends currently to neglect the potential for all spaces to be existentially meaningful. Informed by phenomenological methodology, data were collected from a diverse group of people, using photo-elicitation and interviews. The technique of photo-elicitation proved to be highly effective in helping people reveal their everyday lived experience of contemplative spaces. Reflective analysis (Van Manen 2000) was used to explore the data collected. The initial stage of analysis produced three categories of data: varying conceptions of contemplation, aspects of the person involved in the contemplation, and aspects of environment involved in contemplation. From this, it was found that achieving a state of contemplation involves both the person and the environment in a dialectic process of unfolding. The unfolding has various physical, psycho-social, and existential dimensions or qualities which operate sequentially and simultaneously. Two concepts emerged as being central to unfolding: ‘Cleansing’ and ‘Nothingness’. Unfolding is found to comprise the Core; Distinction; Manifestation; Cleansing; Creation; and Sharing. This has a parallel with Mircea Eliade’s (1959) definition of sacred as something that manifests itself as different from the profane. The power of design, re-contextualization through utility and purpose, and the existential engagements between the person and environment are used as a basis for establishing the potential contribution of the study to interior design. In this way, the study makes a contribution to our understanding of how space and its elements inspire, support and sustain person environment interaction – particularly at the existential level – as well as to our understanding of the multi-dimensional and holistic nature of this interaction. In addition, it points to the need for a phenomenological re-conceptualisation of the design/client relationship. In summary, the contributions of this research are: the exploration of contemplative experience as sacred experience; an understanding of the design of space as creating engagement between person and environment; a rationale for the introduction of a phenomenological approach to the relationship between designer and clients; and raising awareness of the spiritual in a holistic approach to design.

Contextualizing the tensions and weaknesses of information privacy and data breach notification laws

Data breach notification laws have detailed numerous failures relating to the protection of personal information that have blighted both corporate and governmental institutions. There are obvious parallels between data breach notification and information privacy law as they both involve the protection of personal information. However, a closer examination of both laws reveals conceptual differences that give rise to vertical tensions between each law and shared horizontal weaknesses within both laws. Tensions emanate from conflicting approaches to the implementation of information privacy law that results in different regimes and the implementation of different types of protections. Shared weaknesses arise from an overt focus on specified types of personal information which results in ‘one size fits all’ legal remedies. The author contends that a greater contextual approach which promotes the importance of social context is required and highlights the effect that contextualization could have on both laws.

Microblogging as a literacy practice for educational communities

Microblogging is an emergent adolescent and adult literacy practice that has become popularized through platforms such as Twitter, Plurk and Jaiku, in the rise of Web 2.0 – “the social web”. Yet the potentials of microblogging for literacy learning in educational contexts is currently underexplored in the research and literature. This article draws on new research with 150 adolescent and adult participants in school and university contexts, which was made possible through cross-disciplinary collaboration between specialists English and Information and Communication Technologies (ICT) educators. Strategies are provided for teachers to establish their own microblogging networks, with suggested activities to enhance the literacy learning of adolescents in educational contexts.

Standing on the shoulders of ants : stigmergy in the Web

Stigmergy is a biological term used when discussing insect or swarm behaviour, and describes a model supporting environmental communication separately from artefacts or agents. This phenomenon is demonstrated in the behavior of ants and their food gathering process when following pheromone trails, or similarly termites and their termite mound building process. What is interesting with this mechanism is that highly organized societies are achieved with a lack of any apparent management structure. Stigmergic behavior is implicit in the Web where the volume of users provides a self-organizing and self-contextualization of content in sites which facilitate collaboration. However, the majority of content is generated by a minority of the Web participants. A significant contribution from this research would be to create a model of Web stigmergy, identifying virtual pheromones and their importance in the collaborative process. This paper explores how exploiting stigmergy has the potential of providing a valuable mechanism for identifying and analyzing online user behavior recording actionable knowledge otherwise lost in the existing web interaction dynamics. Ultimately this might assist our building better collaborative Web sites.

New methodologies for researching news discussion on Twitter

Twitter has become a major instrument for the rapid dissemination and subsequent debate of news stories. It has been instrumental both in drawing attention to events as they unfolded (such as the emergency landing of a plane in New York’s Hudson River in 2009) and in facilitating a sustained discussion of major stories over timeframes measured in weeks and months (including the continuing saga around Wikileaks and Julian Assange), sometimes still keeping stories alive even if mainstream media attention has moved on elsewhere. More comprehensive methodologies for research into news discussion on Twitter – beyond anecdotal or case study approaches – are only now beginning to emerge. This paper presents a large-scale quantitative approach to studying public communication in the Australian Twittersphere, developed as part of a three-year ARC Discovery project that also examines blogs and other social media spaces. The paper will both outline the innovative research tools developed for this work, and present outcomes from an application of these methodologies to recent and present news themes. Our methodology enables us to identify major themes in Twitter’s discussion of these events, trace their development and decline over time, and map the dynamics of the discussion networks formed ad hoc around specific themes (in part with the help of Twitter #hashtags: brief identifiers which mark a tweet as taking part in an established discussion). It is also able to identify links to major news stories and other online resources, and to track their dissemination across the wider Twittersphere.