User-centric need-driven affect modeling for spoken conversational agents: design and evaluation

It is easy to get frustrated at spoken conversational agents (SCAs), perhaps because they seem to be callous. By and large, the quality of human-computer interaction is affected due to the inability of the SCAs to recognise and adapt to user emotional state. Now with the mass appeal of artificially-mediated communication, there has been an increasing need for SCAs to be socially and emotionally intelligent, that is, to infer and adapt to their human interlocutors’ emotions on the fly, in order to ascertain an affective, empathetic and naturalistic interaction. An enhanced quality of interaction would reduce users’ frustrations and consequently increase their satisfactions. These reasons have motivated the development of SCAs towards including socio-emotional elements, turning them into affective and socially-sensitive interfaces. One barrier to the creation of such interfaces has been the lack of methods for modelling emotions in a task-independent environment. Most emotion models for spoken dialog systems are task-dependent and thus cannot be used “as-is” in different applications. This Thesis focuses on improving this, in which it concerns computational modeling of emotion, personality and their interrelationship for task-independent autonomous SCAs. The generation of emotion is driven by needs, inspired by human’s motivational systems. The work in this Thesis is organised in three stages, each one with its own contribution. The first stage involved defining, integrating and quantifying the psychological-based motivational and emotional models sourced from. Later these were transformed into a computational model by implementing them into software entities. The computational model was then incorporated and put to test with an existing SCA host, a HiFi-control agent. The second stage concerned automatic prediction of affect, which has been the main challenge towards the greater aim of infusing social intelligence into the HiFi agent. In recent years, studies on affect detection from voice have moved on to using realistic, non-acted data, which is subtler. However, it is more challenging to perceive subtler emotions and this is demonstrated in tasks such as labelling and machine prediction. In this stage, we attempted to address part of this challenge by considering the roles of user satisfaction ratings and conversational/dialog features as the respective target and predictors in discriminating contentment and frustration, two types of emotions that are known to be prevalent within spoken human-computer interaction. The final stage concerned the evaluation of the emotional model through the HiFi agent. A series of user studies with 70 subjects were conducted in a real-time environment, each in a different phase and with its own conditions. All the studies involved the comparisons between the baseline non-modified and the modified agent. The findings have gone some way towards enhancing our understanding of the utility of emotion in spoken dialog systems in several ways; first, an SCA should not express its emotions blindly, albeit positive. Rather, it should adapt its emotions to user states. Second, low performance in an SCA may be compensated by the exploitation of emotion. Third, the expression of emotion through the exploitation of prosody could better improve users’ perceptions of an SCA compared to exploiting emotions through just lexical contents. Taken together, these findings not only support the success of the emotional model, but also provide substantial evidences with respect to the benefits of adding emotion in an SCA, especially in mitigating users’ frustrations and ultimately improving their satisfactions. Resumen Es relativamente fácil experimentar cierta frustración al interaccionar con agentes conversacionales (Spoken Conversational Agents, SCA), a menudo porque parecen ser un poco insensibles. En general, la calidad de la interacción persona-agente se ve en cierto modo afectada por la incapacidad de los SCAs para identificar y adaptarse al estado emocional de sus usuarios. Actualmente, y debido al creciente atractivo e interés de dichos agentes, surge la necesidad de hacer de los SCAs unos seres cada vez más sociales y emocionalmente inteligentes, es decir, con capacidad para inferir y adaptarse a las emociones de sus interlocutores humanos sobre la marcha, de modo que la interacción resulte más afectiva, empática y, en definitiva, natural. Una interacción mejorada en este sentido permitiría reducir la posible frustración de los usuarios y, en consecuencia, mejorar el nivel de satisfacción alcanzado por los mismos. Estos argumentos justifican y motivan el desarrollo de nuevos SCAs con capacidades socio-emocionales, dotados de interfaces afectivas y socialmente sensibles. Una de las barreras para la creación de tales interfaces ha sido la falta de métodos de modelado de emociones en entornos independientes de tarea. La mayoría de los modelos emocionales empleados por los sistemas de diálogo hablado actuales son dependientes de tarea y, por tanto, no pueden utilizarse "tal cual" en diferentes dominios o aplicaciones. Esta tesis se centra precisamente en la mejora de este aspecto, la definición de modelos computacionales de las emociones, la personalidad y su interrelación para SCAs autónomos e independientes de tarea. Inspirada en los sistemas motivacionales humanos en el ámbito de la psicología, la tesis propone un modelo de generación/producción de la emoción basado en necesidades. El trabajo realizado en la presente tesis está organizado en tres etapas diferenciadas, cada una con su propia contribución. La primera etapa incluyó la definición, integración y cuantificación de los modelos motivacionales de partida y de los modelos emocionales derivados a partir de éstos. Posteriormente, dichos modelos emocionales fueron plasmados en un modelo computacional mediante su implementación software. Este modelo computacional fue incorporado y probado en un SCA anfitrión ya existente, un agente con capacidad para controlar un equipo HiFi, de alta fidelidad. La segunda etapa se orientó hacia el reconocimiento automático de la emoción, aspecto que ha constituido el principal desafío en relación al objetivo mayor de infundir inteligencia social en el agente HiFi. En los últimos años, los estudios sobre reconocimiento de emociones a partir de la voz han pasado de emplear datos actuados a usar datos reales en los que la presencia u observación de emociones se produce de una manera mucho más sutil. El reconocimiento de emociones bajo estas condiciones resulta mucho más complicado y esta dificultad se pone de manifiesto en tareas tales como el etiquetado y el aprendizaje automático. En esta etapa, se abordó el problema del reconocimiento de las emociones del usuario a partir de características o métricas derivadas del propio diálogo usuario-agente. Gracias a dichas métricas, empleadas como predictores o indicadores del grado o nivel de satisfacción alcanzado por el usuario, fue posible discriminar entre satisfacción y frustración, las dos emociones prevalentes durante la interacción usuario-agente. La etapa final corresponde fundamentalmente a la evaluación del modelo emocional por medio del agente Hifi. Con ese propósito se llevó a cabo una serie de estudios con usuarios reales, 70 sujetos, interaccionando con diferentes versiones del agente Hifi en tiempo real, cada uno en una fase diferente y con sus propias características o capacidades emocionales. En particular, todos los estudios realizados han profundizado en la comparación entre una versión de referencia del agente no dotada de ningún comportamiento o característica emocional, y una versión del agente modificada convenientemente con el modelo emocional propuesto. Los resultados obtenidos nos han permitido comprender y valorar mejor la utilidad de las emociones en los sistemas de diálogo hablado. Dicha utilidad depende de varios aspectos. En primer lugar, un SCA no debe expresar sus emociones a ciegas o arbitrariamente, incluso aunque éstas sean positivas. Más bien, debe adaptar sus emociones a los diferentes estados de los usuarios. En segundo lugar, un funcionamiento relativamente pobre por parte de un SCA podría compensarse, en cierto modo, dotando al SCA de comportamiento y capacidades emocionales. En tercer lugar, aprovechar la prosodia como vehículo para expresar las emociones, de manera complementaria al empleo de mensajes con un contenido emocional específico tanto desde el punto de vista léxico como semántico, ayuda a mejorar la percepción por parte de los usuarios de un SCA. Tomados en conjunto, los resultados alcanzados no sólo confirman el éxito del modelo emocional, sino xv que constituyen además una evidencia decisiva con respecto a los beneficios de incorporar emociones en un SCA, especialmente en cuanto a reducir el nivel de frustración de los usuarios y, en última instancia, mejorar su satisfacción.

Virtual-bound, filamentary and layered states in a box-shaped quantum dot of square potential form the exact numerical solution of the effective mass Schrodinger equation

The effective mass Schrodinger equation of a QD of parallelepipedic shape with a square potential well is solved by diagonalizing the exact Hamiltonian matrix developed in a basis of separation-of-variables wavefunctions. The expected below bandgap bound states are found not to differ very much from the former approximate calculations. In addition, the presence of bound states within the conduction band is confirmed. Furthermore, filamentary states bounded in two dimensions and extended in one dimension and layered states with only one dimension bounded, all within the conduction band which are similar to those originated in quantum wires and quantum wells coexist with the ordinary continuum spectrum of plane waves. All these subtleties are absent in spherically shaped quantum dots, often used for modeling.

Distribution of the coal flow in the mill-duct system of the As Pontes Power Plant using CFD modeling

The efficiency of a Power Plant is affected by the distribution of the pulverized coal within the furnace. The coal, which is pulverized in the mills, is transported and distributed by the primary gas through the mill-ducts to the interior of the furnace. This is done with a double function: dry and enter the coal by different levels for optimizing the combustion in the sense that a complete combustion occurs with homogeneous heat fluxes to the walls. The mill-duct systems of a real Power Plant are very complex and they are not yet well understood. In particular, experimental data concerning the mass flows of coal to the different levels are very difficult to measure. CFD modeling can help to determine them. An Eulerian/Lagrangian approach is used due to the low solid–gas volume ratio.

Discrete element analysis for the assessment of the accuracy of load cell-based dynamic weighing systems in grape harvesters under different ground conditions

Dynamic weighing systems based on load cells are commonly used to estimate crop yields in the field. There is lack of data, however, regarding the accuracy of such weighing systems mounted on harvesting machinery, especially on that used to collect high value crops such as fruits and vegetables. Certainly, dynamic weighing systems mounted on the bins of grape harvesters are affected by the displacement of the load inside the bin when moving over terrain of changing topography. In this work, the load that would be registered in a grape harvester bin by a dynamic weighing system based on the use of a load cell was inferred by using the discrete element method (DEM). DEM is a numerical technique capable of accurately describing the behaviour of granular materials under dynamic situations and it has been proven to provide successful predictions in many different scenarios. In this work, different DEM models of a grape harvester bin were developed contemplating different influencing factors. Results obtained from these models were used to infer the output given by the load cell of a real bin. The mass detected by the load cell when the bin was inclined depended strongly on the distribution of the load within the bin, but was underestimated in all scenarios. The distribution of the load was found to be dependent on the inclination of the bin caused by the topography of the terrain, but also by the history of inclination (inclination rate, presence of static periods, etc.) since the effect of the inertia of the particles (i.e., representing the grapes) was not negligible. Some recommendations are given to try to improve the accuracy of crop load measurement in the field.

Modeling the impact of global tuberculosis control strategies

An epidemiological model of tuberculosis has been developed and applied to five regions of the world. Globally, 6.7 million new cases of tuberculosis and 2.4 million deaths from tuberculosis are estimated for 1998. Based on current trends in uptake of the World Health Organization’s strategy of directly observed treatment, short-course, we expect a total of 225 million new cases and 79 million deaths from tuberculosis between 1998 and 2030. Active case-finding by using mass miniature radiography could save 23 million lives over this period. A single contact treatment for tuberculosis could avert 24 million cases and 11 million deaths; combined with active screening, it could reduce mortality by nearly 40%. A new vaccine with 50% efficacy could lower incidence by 36 million cases and mortality by 9 million deaths. Support for major extensions to global tuberculosis control strategies will occur only if the size of the problem and the potential for action are recognized more widely.

A numerical strategy for efficient modeling of the equatorial wave guide

Convection in the tropics is observed to involve a wide-ranging hierarchy of scales from a few kilometers to the planetary scales and also has a profound impact on short-term climate. The mechanisms responsible for this behavior present a major unsolved problem. A promising emerging approach to address these issues is cloud-resolving modeling. Here a family of numerical models is introduced specifically to model the feedback of small-scale deep convection on tropical planetary waves and tropical circulation in a highly efficient manner compatible with the approach through cloud-resolving modeling. Such a procedure is also useful for theoretical purposes. The basic idea in the approach is to use low-order truncation in the meriodonal direction through Gauss–Hermite quadrature projected onto a simple discrete radiation condition. In this fashion, the cloud-resolving modeling of equatorially trapped planetary waves reduces to the solution of a small number of purely zonal two-dimensional wave systems along a few judiciously chosen meriodonal layers that are coupled only by some additional source terms. The approach is analyzed in detail with full mathematical rigor for linearized equatorial primitive equations with source terms.

Mathematical modeling of drying process of unripe banana slices

Unripe banana flour (UBF) production employs bananas not submitted to maturation process, is an interesting alternative to minimize the fruit loss reduction related to inappropriate handling or fast ripening. The UBF is considered as a functional ingredient improving glycemic and plasma insulin levels in blood, have also shown efficacy on the control of satiety, insulin resistance. The aim of this work was to study the drying process of unripe banana slabs (Musa cavendishii, Nanicão) developing a transient drying model through mathematical modeling with simultaneous moisture and heat transfer. The raw material characterization was performed and afterwards the drying process was conducted at 40 ºC, 50 ºC e 60 ºC, the product temperature was recorded using thermocouples, the air velocity inside the chamber was 4 m·s-1. With the experimental data was possible to validate the diffusion model based on the Fick\'s second law and Fourier. For this purpose, the sorption isotherms were measured and fitted to the GAB model estimating the equilibrium moisture content (Xe), 1.76 [g H2O/100g d.b.] at 60 ºC and 10 % of relative humidity (RH), the thermophysical properties (k, Cp, ?) were also measured to be used in the model. Five cases were contemplated: i) Constant thermophysical properties; ii) Variable properties; iii) Mass (hm), heat transfer (h) coefficient and effective diffusivity (De) estimation 134 W·m-2·K-1, 4.91x10-5 m-2·s-1 and 3.278?10-10 m·s-2 at 60 ºC, respectively; iv) Variable De, it presented a third order polynomial behavior as function of moisture content; v) The shrinkage had an effect on the mathematical model, especially in the 3 first hours of process, the thickness experienced a contraction of about (30.34 ± 1.29) % out of the initial thickness, finding two decreasing drying rate periods (DDR I and DDR II), 3.28x10-10 m·s-2 and 1.77x10-10 m·s-2, respectively. COMSOL Multiphysics simulations were possible to perform through the heat and mass transfer coefficient estimated by the mathematical modeling.

EnvironmentalWaveletTool: Continuous and discrete wavelet analysis and filtering for environmental time series

A MATLAB-based computer code has been developed for the simultaneous wavelet analysis and filtering of several environmental time series, particularly focused on the analyses of cave monitoring data. The continuous wavelet transform, the discrete wavelet transform and the discrete wavelet packet transform have been implemented to provide a fast and precise time–period examination of the time series at different period bands. Moreover, statistic methods to examine the relation between two signals have been included. Finally, the entropy of curves and splines based methods have also been developed for segmenting and modeling the analyzed time series. All these methods together provide a user-friendly and fast program for the environmental signal analysis, with useful, practical and understandable results.

Logic-Based Outer Approximation for the Design of Discrete-Continuous Dynamic Systems with Implicit Discontinuities

We address the optimization of discrete-continuous dynamic optimization problems using a disjunctive multistage modeling framework, with implicit discontinuities, which increases the problem complexity since the number of continuous phases and discrete events is not known a-priori. After setting a fixed alternative sequence of modes, we convert the infinite-dimensional continuous mixed-logic dynamic (MLDO) problem into a finite dimensional discretized GDP problem by orthogonal collocation on finite elements. We use the Logic-based Outer Approximation algorithm to fully exploit the structure of the GDP representation of the problem. This modelling framework is illustrated with an optimization problem with implicit discontinuities (diver problem).

Quiescent thermal emission from neutron stars in low-mass X-ray binaries

Context. We monitored the quiescent thermal emission from neutron stars in low-mass X-ray binaries after active periods of intense activity in X-rays (outbursts). Aims. The theoretical modeling of the thermal relaxation of the neutron star crust may be used to establish constraints on the crust composition and transport properties, depending on the astrophysical scenarios assumed. Methods. We numerically simulated the thermal evolution of the neutron star crust and compared them with inferred surface temperatures for five sources: MXB 1659−29, KS 1731−260, XTE J1701−462, EXO 0748−676  and IGR J17480−2446. Results. We find that the evolution of MXB 1659−29, KS 1731−260 and EXO 0748−676 can be well described within a deep crustal cooling scenario. Conversely, we find that the other two sources can only be explained with models beyond crustal cooling. For the peculiar emission of XTE J1701−462 we propose alternative scenarios such as residual accretion during quiescence, additional heat sources in the outer crust, and/or thermal isolation of the inner crust due to a buried magnetic field. We also explain the very recent reported temperature of IGR J17480−2446 with an additional heat deposition in the outer crust from shallow sources.

Logical Modeling and Dynamical Analysis of Cellular Networks

The logical (or logic) formalism is increasingly used to model regulatory and signaling networks. Complementing these applications, several groups contributed various methods and tools to support the definition and analysis of logical models. After an introduction to the logical modeling framework and to several of its variants, we review here a number of recent methodological advances to ease the analysis of large and intricate networks. In particular, we survey approaches to determine model attractors and their reachability properties, to assess the dynamical impact of variations of external signals, and to consistently reduce large models. To illustrate these developments, we further consider several published logical models for two important biological processes, namely the differentiation of T helper cells and the control of mammalian cell cycle.

Modeling and validation of the thermal behavior of buildings for the development of demand response methods

The representation of the thermal behaviour of the building is achieved through a relatively simple dynamic model that takes into account the effects due to the thermal mass of the building components. The model of a intra-floor apartment has been built in the Matlab-Simulink environment and considers the heat transmission through the external envelope, wall and windows, the internal thermal masses, (i.e. furniture, internal wall and floor slabs) and the sun gain due to opaque and see-through surfaces of the external envelope. The simulations results for the entire year have been compared and the model validated, with the one obtained with the dynamic building simulation software Energyplus.

The influence of parent's body mass index on peer selection: An experimental approach using virtual reality

Relatively little is known about the influence of psychosocial factors, such as familial role modeling and social network on the development and maintenance of childhood obesity. We investigated peer se- lection using an immersive virtual reality environment. In a virtual schoolyard, children were confronted with normal weight and overweight avatars either eating or playing. Fifty-seven children aged 7–13 participated. Interpersonal distance to the avatars, child's BMI, self-perception, eating behavior and parental BMI were assessed. Parental BMI was the strongest predictor for the children's minimal distance to the avatars. Specifically, a higher mothers' BMI was associated with greater interpersonal distance and children approached closer to overweight eating avatars. A higher father's BMI was associated with a lower interpersonal distance to the avatars. These children approached normal weight playing and overweight eating avatar peers closest. The importance of parental BMI for the child's social approach/ avoidance behavior can be explained through social modeling mechanisms. Differential effects of pa- ternal and maternal BMI might be due to gender specific beauty ideals. Interventions to promote social interaction with peer groups could foster weight stabilization or weight loss in children.

Survey of fire modeling efforts with application to transportation vehicles. Final report.

Transportation Department, Secretary of Transportation, Washington, D.C.