Controlled Software Evaluation Methodology: Case Simulation and Cognitive User Analysis

Software for use with patient records is challenging to design and difficult to evaluate because of the tremendous variability of patient circumstances. A method was devised by the authors to overcome a number of difficulties. The method evaluates and compares objectively various software products for use in emergency departments and compares software to conventional methods like dictation and templated chart forms. The technique utilizes oral case simulation and video recording for analysis. The methodology and experiences of executing a study using this case simulation are discussed in this presentation.

Institutions as key drivers of collective action in WUAs [Water User Associations] of Uzbekistan

This paper presents a multifactor approach for performance assessment of Water Users Associations (WUAs) in Uzbekistan in order to identify the drivers for improved and effi cient performance of WUAs. The study was carried out in the Fergana Valley where the WUAs were created along the South Fergana Main Canal during the last 10 years. The farmers and the employees of 20 WUAs were questioned about the WUAs’ activities and the quantitative and qualitative data were obtained. This became a base for the calculation of 36 indicators divided into 6 groups: Water supply, technical conditions, economic conditions, social and cultural conditions, organizational conditions and information conditions. All the indicators assessed with a differentiated point system adjusted for subjectivity of several of them give the total maximal result for the associations of 250 point. The WUAs of the Fergana Valley showed the score between 145 and 219 points, what refl ects a highly diverse level of the WUAs performance in the region. The analysis of the indicators revealed that the key points of the WUA’s success are the organizational and institutional conditions including the participatory factors and awareness of both the farmers and employees about the work of WUA. The research showed that the low performance of the WUAs is always explained by the low technical and economic conditions along with weak organization and information dissemination conditions. It is clear that it is complicated to improve technical and economic conditions immediately because they are cost-based and cost-induced. However, it is possible to improve the organizational conditions and to strengthen the institutional basis via formal and information institutions which will gradually lead to improvement of economic and technical conditions of WUAs. Farmers should be involved into the WUA Governance and into the process of making common decisions and solving common problems together via proper institutions. Their awareness can also be improved by leading additional trainings for increasing farmers’ agronomic and irrigation knowledge, teaching them water saving technologies and acquainting them with the use of water measuring equipment so it can bring reliable water supply, transparent budgeting and adequate as well as equitable water allocation to the water users.

Extending private-collective innovation: a case study

The private-collective innovation model proposes incentives for individuals and firms to privately invest resources to create public goods innovations. Such innovations are characterized by non-rivalry and non-exclusivity in consumption. Examples include open source software, user-generated media products, drug formulas, and sport equipment designs. There is still limited empirical research on private-collective innovation. We present a case study to (1) provide empirical evidence of a case of private-collective innovation, showing specific benefits, and (2) to extend the private-collective innovation model by analyzing the hidden costs for the company involved. We examine the development of the Nokia Internet Tablet, which builds on both proprietary and open source software development, and that involves both Nokia developers and volunteers who are not employed by the company. Seven benefits for Nokia are identified, as are five hidden costs: difficulty to differentiate, guarding business secrets, reducing community entry barriers, giving up control, and organizational inertia. We examine the actions taken by the management to mitigate these costs throughout the development period.

Carbohydrate Estimation by a Mobile Phone-Based System Versus Self-Estimations of Individuals With Type 1 Diabetes Mellitus: A Comparative Study

Background: Diabetes mellitus is spreading throughout the world and diabetic individuals have been shown to often assess their food intake inaccurately; therefore, it is a matter of urgency to develop automated diet assessment tools. The recent availability of mobile phones with enhanced capabilities, together with the advances in computer vision, have permitted the development of image analysis apps for the automated assessment of meals. GoCARB is a mobile phone-based system designed to support individuals with type 1 diabetes during daily carbohydrate estimation. In a typical scenario, the user places a reference card next to the dish and acquires two images using a mobile phone. A series of computer vision modules detect the plate and automatically segment and recognize the different food items, while their 3D shape is reconstructed. Finally, the carbohydrate content is calculated by combining the volume of each food item with the nutritional information provided by the USDA Nutrient Database for Standard Reference. Objective: The main objective of this study is to assess the accuracy of the GoCARB prototype when used by individuals with type 1 diabetes and to compare it to their own performance in carbohydrate counting. In addition, the user experience and usability of the system is evaluated by questionnaires. Methods: The study was conducted at the Bern University Hospital, “Inselspital” (Bern, Switzerland) and involved 19 adult volunteers with type 1 diabetes, each participating once. Each study day, a total of six meals of broad diversity were taken from the hospital’s restaurant and presented to the participants. The food items were weighed on a standard balance and the true amount of carbohydrate was calculated from the USDA nutrient database. Participants were asked to count the carbohydrate content of each meal independently and then by using GoCARB. At the end of each session, a questionnaire was completed to assess the user’s experience with GoCARB. Results: The mean absolute error was 27.89 (SD 38.20) grams of carbohydrate for the estimation of participants, whereas the corresponding value for the GoCARB system was 12.28 (SD 9.56) grams of carbohydrate, which was a significantly better performance ( P=.001). In 75.4% (86/114) of the meals, the GoCARB automatic segmentation was successful and 85.1% (291/342) of individual food items were successfully recognized. Most participants found GoCARB easy to use. Conclusions: This study indicates that the system is able to estimate, on average, the carbohydrate content of meals with higher accuracy than individuals with type 1 diabetes can. The participants thought the app was useful and easy to use. GoCARB seems to be a well-accepted supportive mHealth tool for the assessment of served-on-a-plate meals.

Feasibility of stereotactic MRI-based image guidance for the treatment of vascular malformations: a phantom study.

PURPOSE Treatment of vascular malformations requires the placement of a needle within vessels which may be as small as 1 mm, with the current state of the art relying exclusively on two-dimensional fluoroscopy images for guidance. We hypothesize that the combination of stereotactic image guidance with existing targeting methods will result in faster and more reproducible needle placements, as well as reduced radiationexposure, when compared to standard methods based on fluoroscopy alone. METHODS The proposed navigation approach was evaluated in a phantom experiment designed to allow direct comparison with the conventional method. An anatomical phantom of the left forearm was constructed, including an independent control mechanism to indicate the attainment of the target position. Three interventionalists (one inexperienced, two of them frequently practice the conventional fluoroscopic technique) performed 45 targeting attempts utilizing the combined and 45 targeting attempts utilizing the standard approaches. RESULTS In all 45 attempts, the users were able to reach the target when utilizing the combined approach. In two cases, targeting was stopped after 15 min without reaching the target when utilizing only the C-arm. The inexperienced user was faster when utilizing the combined approach and applied significantly less radiation than when utilizing the conventional approach. Conversely, both experienced users were faster when using the conventional approach, in one case significantly so, with no significant difference in radiation dose when compared to the combined approach. CONCLUSIONS This work presents an initial evaluation of a combined navigation fluoroscopy targeting technique in a phantom study. The results suggest that, especially for inexperienced interventionalists, navigation may help to reduce the time and the radiation dose. Future work will focus on the improvement and clinical evaluation of the proposed method.

How did the crisis affect small and medium-sized enterprises? -- from a field study of the metal-working industry in Java

This paper focuses on the impact of Indonesia's economic crisis on small and medium-sized enterprises (SMEs). It shows how the performance of SMEs during the crisis varied widely even in the same industrial subsector, and found that the factors most affecting performance have been market orientation and the linkages that the SMEs have formed with the buyers of their products. Well-performing SMEs were found to have utilized putting-out linkages with wholesalers which enabled them to switch to products having better markets. On the other hand, the SMEs which had subcontracting linkages with assemblers or contracting linkages with user-factories (with the exception of SMEs having export-oriented linkages) suffered badly in the crisis because of specificity of products with little room for switching. The paper also found that exposure to debt due to borrowing for investment has been another factor affecting performance, but that enterprise size has had no linear correlation with performance.

A comparative study on unconstrained hand biometrics

Biometrics applied to mobile devices are of great interest for security applications. Daily scenarios can benefit of a combination of both the most secure systems and most simple and extended devices. This document presents a hand biometric system oriented to mobile devices, proposing a non-intrusive, contact-less acquisition process where final users should take a picture of their hand in free-space with a mobile device without removals of rings, bracelets or watches. The main contribution of this paper is threefold: firstly, a feature extraction method is proposed, providing invariant hand measurements to previous changes; second contribution consists of providing a template creation based on hand geometric distances, requiring information from only one individual, without considering data from the rest of individuals within the database; finally, a proposal for template matching is proposed, minimizing the intra-class similarity and maximizing the inter-class likeliness. The proposed method is evaluated using three publicly available contact-less, platform-free databases. In addition, the results obtained with these databases will be compared to the results provided by two competitive pattern recognition techniques, namely Support Vector Machines (SVM) and k-Nearest Neighbour, often employed within the literature. Therefore, this approach provides an appropriate solution to adapt hand biometrics to mobile devices, with an accurate results and a non-intrusive acquisition procedure which increases the overall acceptance from the final user.

A Method for Developing Ontologies from User-Generated Classication Systems

Abstract Web 2.0 applications enabled users to classify information resources using their own vocabularies. The bottom-up nature of these user-generated classification systems have turned them into interesting knowledge sources, since they provide a rich terminology generated by potentially large user communities. Previous research has shown that it is possible to elicit some emergent semantics from the aggregation of individual classifications in these systems. However the generation of ontologies from them is still an open research problem. In this thesis we address the problem of how to tap into user-generated classification systems for building domain ontologies. Our objective is to design a method to develop domain ontologies from user-generated classifications systems. To do so, we rely on ontologies in the Web of Data to formalize the semantics of the knowledge collected from the classification system. Current ontology development methodologies have recognized the importance of reusing knowledge from existing resources. Thus, our work is framed within the NeOn methodology scenario for building ontologies by reusing and reengineering non-ontological resources. The main contributions of this work are: An integrated method to develop ontologies from user-generated classification systems. With this method we extract a domain terminology from the classification system and then we formalize the semantics of this terminology by reusing ontologies in the Web of Data. Identification and adaptation of existing techniques for implementing the activities in the method so that they can fulfill the requirements of each activity. A novel study about emerging semantics in user-generated lists. Resumen La web 2.0 permitió a los usuarios clasificar recursos de información usando su propio vocabulario. Estos sistemas de clasificación generados por usuarios son recursos interesantes para la extracción de conocimiento debido principalmente a que proveen una extensa terminología generada por grandes comunidades de usuarios. Se ha demostrado en investigaciones previas que es posible obtener una semántica emergente de estos sistemas. Sin embargo la generación de ontologías a partir de ellos es todavía un problema de investigación abierto. Esta tesis trata el problema de cómo aprovechar los sistemas de clasificación generados por usuarios en la construcción de ontologías de dominio. Así el objetivo de la tesis es diseñar un método para desarrollar ontologías de dominio a partir de sistemas de clasificación generados por usuarios. El método propuesto reutiliza conceptualizaciones existentes en ontologías publicadas en la Web de Datos para formalizar la semántica del conocimiento que se extrae del sistema de clasificación. Por tanto, este trabajo está enmarcado dentro del escenario para desarrollar ontologías mediante la reutilización y reingeniería de recursos no ontológicos que se ha definido en la Metodología NeOn. Las principales contribuciones de este trabajo son: Un método integrado para desarrollar una ontología de dominio a partir de sistemas de clasificación generados por usuarios. En este método se extrae una terminología de dominio del sistema de clasificación y posteriormente se formaliza su semántica reutilizando ontologías en la Web de Datos. La identificación y adaptación de un conjunto de técnicas para implementar las actividades propuestas en el método de tal manera que puedan cumplir automáticamente los requerimientos de cada actividad. Un novedoso estudio acerca de la semántica emergente en las listas generadas por usuarios en la Web.

User benefit assessment by a dynamic LUTI model for Madrid

Assessing users’ benefit in a transport policy implementation has been studied by many researchers using theoretical or empirical measures. However, few of them measure users’ benefit in a different way from the consumer surplus. Therefore, this paper aims to assess a new measure of user benefits by weighting consumer surplus in order to include equity assessment for different transport policies simulated in a dynamic middle-term LUTI model adapted to the case study of Madrid. Three different transport policies, including road pricing, parking charge and public transport improvement have been simulated through the Metropolitan Activity Relocation Simulator, MARS, the LUTI calibrated model for Madrid). A social welfare function (WF) is defined using a cost benefit analysis function that includes mainly costs and benefits of users and operators of the transport system. Particularly, the part of welfare function concerning the users, (i.e. consumer surplus), is modified by a compensating weight (CW) which represents the inverse of household income level. Based on the modified social welfare function, the effects on the measure of users benefits are estimated and compared with the old WF ́s results as well. The result of the analysis shows that road pricing leads a negative effect on the users benefits specially on the low income users. Actually, the road pricing and parking charge implementation results like a regressive policy especially at long term. Public transport improvement scenario brings more positive effects on low income user benefits. The integrated (road pricing and increasing public services) policy scenario is the one which receive the most user benefits. The results of this research could be a key issue to understanding the relationship between transport systems policies and user benefits distribution in a metropolitan context.

Distributed primary user identification from imprecise location information

We study a cognitive radio scenario in which the network of sec- ondary users wishes to identify which primary user, if any, is trans- mitting. To achieve this, the nodes will rely on some form of location information. In our previous work we proposed two fully distributed algorithms for this task, with and without a pre-detection step, using propagation parameters as the only source of location information. In a real distributed deployment, each node must estimate its own po- sition and/or propagation parameters. Hence, in this work we study the effect of uncertainty, or error in these estimates on the proposed distributed identification algorithms. We show that the pre-detection step significantly increases robustness against uncertainty in nodes' locations.

Does high speed rail compete fairly with other transportation modes? Madrid-Barcelona case study

Transportation modes produce many external costs such as congestion, accidents, and environmental impacts (pollution, noise and so on). From the microeconomic theory it is well known that in order to maximize social welfare, transportation modes should internalize the marginal costs they produce. Allocative efficiency is achieved when all transportation modes are priced at their social marginal cost. The objective of this research is to evaluate to what extent different passenger transport modes internalize their social marginal costs. This analysis is important since it affects the competitiveness of the different transport modes for a given OD pair. The case study analyzed is the corridor Madrid-Barcelona in Spain and the different transport modes have been considered (cars, buses, high-speed train and air). The research calculates the marginal social cost per user for each transportation mode, and it compares it with the average fare—allowing for the effect of discriminatory taxes—currently paid by the users. The external costs are calculated according to the guidelines established by the European Union. The gap between the marginal social cost and the price paid by users will provide the extra cost per passenger that each transport mode should have to pay for internalizing the external cost it produces. The research shows that external costs already produced by road and air transport modes are much higher than those produced by rail. However, the results show that road transport already internalizes every external costs it produces because users pay high fuel taxes. In other words, although rail transportation produces lower external costs, road transportation pays more than it should on the basis of the social marginal costs. The results of this work might be of help for Europ ean policy actions to be undertaken in the future.

User Experience in Human-Technology Interaction. Communication, context and evaluation methodology

En esta Tesis se presentan dos líneas de investigación relacionadas y que contribuyen a las áreas de Interacción Hombre-Tecnología (o Máquina; siglas en inglés: HTI o HMI), lingüística computacional y evaluación de la experiencia del usuario. Las dos líneas en cuestión son el diseño y la evaluación centrada en el usuario de sistemas de Interacción Hombre-Máquina avanzados. En la primera parte de la Tesis (Capítulos 2 a 4) se abordan cuestiones fundamentales del diseño de sistemas HMI avanzados. El Capítulo 2 presenta una panorámica del estado del arte de la investigación en el ámbito de los sistemas conversacionales multimodales, con la que se enmarca el trabajo de investigación presentado en el resto de la Tesis. Los Capítulos 3 y 4 se centran en dos grandes aspectos del diseño de sistemas HMI: un gestor del diálogo generalizado para tratar la Interacción Hombre-Máquina multimodal y sensible al contexto, y el uso de agentes animados personificados (ECAs) para mejorar la robustez del diálogo, respectivamente. El Capítulo 3, sobre gestión del diálogo, aborda el tratamiento de la heterogeneidad de la información proveniente de las modalidades comunicativas y de los sensores externos. En este capítulo se propone, en un nivel de abstracción alto, una arquitectura para la gestión del diálogo con influjos heterogéneos de información, apoyándose en el uso de State Chart XML. En el Capítulo 4 se presenta una contribución a la representación interna de intenciones comunicativas, y su traducción a secuencias de gestos a ejecutar por parte de un ECA, diseñados específicamente para mejorar la robustez en situaciones de diálogo críticas que pueden surgir, por ejemplo, cuando se producen errores de entendimiento en la comunicación entre el usuario humano y la máquina. Se propone, en estas páginas, una extensión del Functional Mark-up Language definido en el marco conceptual SAIBA. Esta extensión permite representar actos comunicativos que realizan intenciones del emisor (la máquina) que no se pretende sean captadas conscientemente por el receptor (el usuario humano), pero con las que se pretende influirle a éste e influir el curso del diálogo. Esto se consigue mediante un objeto llamado Base de Intenciones Comunicativas (en inglés, Communication Intention Base, o CIB). La representación en el CIB de intenciones “no claradas” además de las explícitas permite la construcción de actos comunicativos que realizan simultáneamente varias intenciones comunicativas. En el Capítulo 4 también se describe un sistema experimental para el control remoto (simulado) de un asistente domótico, con autenticación de locutor para dar acceso, y con un ECA en el interfaz de cada una de estas tareas. Se incluye una descripción de las secuencias de comportamiento verbal y no verbal de los ECAs, que fueron diseñados específicamente para determinadas situaciones con objeto de mejorar la robustez del diálogo. Los Capítulos 5 a 7 conforman la parte de la Tesis dedicada a la evaluación. El Capítulo 5 repasa antecedentes relevantes en la literatura de tecnologías de la información en general, y de sistemas de interacción hablada en particular. Los principales antecedentes en el ámbito de la evaluación de la interacción sobre los cuales se ha desarrollado el trabajo presentado en esta Tesis son el Technology Acceptance Model (TAM), la herramienta Subjective Assessment of Speech System Interfaces (SASSI), y la Recomendación P.851 de la ITU-T. En el Capítulo 6 se describen un marco y una metodología de evaluación aplicados a la experiencia del usuario con sistemas HMI multimodales. Se desarrolló con este propósito un novedoso marco de evaluación subjetiva de la calidad de la experiencia del usuario y su relación con la aceptación por parte del mismo de la tecnología HMI (el nombre dado en inglés a este marco es Subjective Quality Evaluation Framework). En este marco se articula una estructura de clases de factores subjetivos relacionados con la satisfacción y aceptación por parte del usuario de la tecnología HMI propuesta. Esta estructura, tal y como se propone en la presente tesis, tiene dos dimensiones ortogonales. Primero se identifican tres grandes clases de parámetros relacionados con la aceptación por parte del usuario: “agradabilidad ” (likeability: aquellos que tienen que ver con la experiencia de uso, sin entrar en valoraciones de utilidad), rechazo (los cuales sólo pueden tener una valencia negativa) y percepción de utilidad. En segundo lugar, este conjunto clases se reproduce para distintos “niveles, o focos, percepción del usuario”. Éstos incluyen, como mínimo, un nivel de valoración global del sistema, niveles correspondientes a las tareas a realizar y objetivos a alcanzar, y un nivel de interfaz (en los casos propuestos en esta tesis, el interfaz es un sistema de diálogo con o sin un ECA). En el Capítulo 7 se presenta una evaluación empírica del sistema descrito en el Capítulo 4. El estudio se apoya en los mencionados antecedentes en la literatura, ampliados con parámetros para el estudio específico de los agentes animados (los ECAs), la auto-evaluación de las emociones de los usuarios, así como determinados factores de rechazo (concretamente, la preocupación por la privacidad y la seguridad). También se evalúa el marco de evaluación subjetiva de la calidad propuesto en el capítulo anterior. Los análisis de factores efectuados revelan una estructura de parámetros muy cercana conceptualmente a la división de clases en utilidad-agradabilidad-rechazo propuesta en dicho marco, resultado que da cierta validez empírica al marco. Análisis basados en regresiones lineales revelan estructuras de dependencias e interrelación entre los parámetros subjetivos y objetivos considerados. El efecto central de mediación, descrito en el Technology Acceptance Model, de la utilidad percibida sobre la relación de dependencia entre la intención de uso y la facilidad de uso percibida, se confirma en el estudio presentado en la presente Tesis. Además, se ha encontrado que esta estructura de relaciones se fortalece, en el estudio concreto presentado en estas páginas, si las variables consideradas se generalizan para cubrir más ampliamente las categorías de agradabilidad y utilidad contempladas en el marco de evaluación subjetiva de calidad. Se ha observado, asimismo, que los factores de rechazo aparecen como un componente propio en los análisis de factores, y además se distinguen por su comportamiento: moderan la relación entre la intención de uso (que es el principal indicador de la aceptación del usuario) y su predictor más fuerte, la utilidad percibida. Se presentan también resultados de menor importancia referentes a los efectos de los ECAs sobre los interfaces de los sistemas de diálogo y sobre los parámetros de percepción y las valoraciones de los usuarios que juegan un papel en conformar su aceptación de la tecnología. A pesar de que se observa un rendimiento de la interacción dialogada ligeramente mejor con ECAs, las opiniones subjetivas son muy similares entre los dos grupos experimentales (uno interactuando con un sistema de diálogo con ECA, y el otro sin ECA). Entre las pequeñas diferencias encontradas entre los dos grupos destacan las siguientes: en el grupo experimental sin ECA (es decir, con interfaz sólo de voz) se observó un efecto más directo de los problemas de diálogo (por ejemplo, errores de reconocimiento) sobre la percepción de robustez, mientras que el grupo con ECA tuvo una respuesta emocional más positiva cuando se producían problemas. Los ECAs parecen generar inicialmente expectativas más elevadas en cuanto a las capacidades del sistema, y los usuarios de este grupo se declaran más seguros de sí mismos en su interacción. Por último, se observan algunos indicios de efectos sociales de los ECAs: la “amigabilidad ” percibida los ECAs estaba correlada con un incremento la preocupación por la seguridad. Asimismo, los usuarios del sistema con ECAs tendían más a culparse a sí mismos, en lugar de culpar al sistema, de los problemas de diálogo que pudieran surgir, mientras que se observó una ligera tendencia opuesta en el caso de los usuarios del sistema con interacción sólo de voz. ABSTRACT This Thesis presents two related lines of research work contributing to the general fields of Human-Technology (or Machine) Interaction (HTI, or HMI), computational linguistics, and user experience evaluation. These two lines are the design and user-focused evaluation of advanced Human-Machine (or Technology) Interaction systems. The first part of the Thesis (Chapters 2 to 4) is centred on advanced HMI system design. Chapter 2 provides a background overview of the state of research in multimodal conversational systems. This sets the stage for the research work presented in the rest of the Thesis. Chapers 3 and 4 focus on two major aspects of HMI design in detail: a generalised dialogue manager for context-aware multimodal HMI, and embodied conversational agents (ECAs, or animated agents) to improve dialogue robustness, respectively. Chapter 3, on dialogue management, deals with how to handle information heterogeneity, both from the communication modalities or from external sensors. A highly abstracted architectural contribution based on State Chart XML is proposed. Chapter 4 presents a contribution for the internal representation of communication intentions and their translation into gestural sequences for an ECA, especially designed to improve robustness in critical dialogue situations such as when miscommunication occurs. We propose an extension of the functionality of Functional Mark-up Language, as envisaged in much of the work in the SAIBA framework. Our extension allows the representation of communication acts that carry intentions that are not for the interlocutor to know of, but which are made to influence him or her as well as the flow of the dialogue itself. This is achieved through a design element we have called the Communication Intention Base. Such r pr s ntation of “non- clar ” int ntions allows th construction of communication acts that carry several communication intentions simultaneously. Also in Chapter 4, an experimental system is described which allows (simulated) remote control to a home automation assistant, with biometric (speaker) authentication to grant access, featuring embodied conversation agents for each of the tasks. The discussion includes a description of the behavioural sequences for the ECAs, which were designed for specific dialogue situations with particular attention given to the objective of improving dialogue robustness. Chapters 5 to 7 form the evaluation part of the Thesis. Chapter 5 reviews evaluation approaches in the literature for information technologies, as well as in particular for speech-based interaction systems, that are useful precedents to the contributions of the present Thesis. The main evaluation precedents on which the work in this Thesis has built are the Technology Acceptance Model (TAM), the Subjective Assessment of Speech System Interfaces (SASSI) tool, and ITU-T Recommendation P.851. Chapter 6 presents the author’s work in establishing an valuation framework and methodology applied to the users’ experience with multimodal HMI systems. A novel user-acceptance Subjective Quality Evaluation Framework was developed by the author specifically for this purpose. A class structure arises from two orthogonal sets of dimensions. First we identify three broad classes of parameters related with user acceptance: likeability factors (those that have to do with the experience of using the system), rejection factors (which can only have a negative valence) and perception of usefulness. Secondly, the class structure is further broken down into several “user perception levels”; at the very least: an overall system-assessment level, task and goal-related levels, and an interface level (e.g., a dialogue system with or without an ECA). An empirical evaluation of the system described in Chapter 4 is presented in Chapter 7. The study was based on the abovementioned precedents in the literature, expanded with categories covering the inclusion of an ECA, the users’ s lf-assessed emotions, and particular rejection factors (privacy and security concerns). The Subjective Quality Evaluation Framework proposed in the previous chapter was also scrutinised. Factor analyses revealed an item structure very much related conceptually to the usefulness-likeability-rejection class division introduced above, thus giving it some empirical weight. Regression-based analysis revealed structures of dependencies, paths of interrelations, between the subjective and objective parameters considered. The central mediation effect, in the Technology Acceptance Model, of perceived usefulness on the dependency relationship of intention-to-use with perceived ease of use was confirmed in this study. Furthermore, the pattern of relationships was stronger for variables covering more broadly the likeability and usefulness categories in the Subjective Quality Evaluation Framework. Rejection factors were found to have a distinct presence as components in factor analyses, as well as distinct behaviour: they were found to moderate the relationship between intention-to-use (the main measure of user acceptance) and its strongest predictor, perceived usefulness. Insights of secondary importance are also given regarding the effect of ECAs on the interface of spoken dialogue systems and the dimensions of user perception and judgement attitude that may have a role in determining user acceptance of the technology. Despite observing slightly better performance values in the case of the system with the ECA, subjective opinions regarding both systems were, overall, very similar. Minor differences between two experimental groups (one interacting with an ECA, the other only through speech) include a more direct effect of dialogue problems (e.g., non-understandings) on perceived dialogue robustness for the voice-only interface test group, and a more positive emotional response for the ECA test group. Our findings further suggest that the ECA generates higher initial expectations, and users seem slightly more confident in their interaction with the ECA than do those without it. Finally, mild evidence of social effects of ECAs was also found: the perceived friendliness of the ECA increased security concerns, and ECA users may tend to blame themselves rather than the system when dialogue problems are encountered, while the opposite may be true for voice-only users.

Case study on mobile applications UX: effect of the usage of a crosss-platform development framework

Cross-platform development frameworks for mobile applications promise important advantages in cost cuttings and easy maintenance, posing as a very good option for organizations interested in the design of mobile applications for several platforms. Given that platform conventions are especially important for the User eXperience (UX) of mobile applications, the usage of framework where the same code defines the behavior of the app in different platforms could have negative impact in the UX. The objetive of this study is comparing the cross-platform and the native approach for being able to determine if the selected development approach has any impact on the users in terms of UX. To be able to set a base line under this subject, study on cross-platform frameworks was performed to select the most appropriate one from a UX point of view. In order to achieve the objectives of this work, two development teams have developed two versions of the same application; one using framework that generates Android and iOS versions automatically, and another team developing native versions of the same application. The alternative versions for each platform have been evaluated with 37 users with a combination of a laboratory usability test and a longitudinal study. The results show that differences are minimal in the Android version, but in iOS, even if a reasonable good UX can be obtained with the usage of this framework by an UX-conscious design team, a higher level of UX can be obtained directly developing in native code.

Enhancing Evacuation Plans with a Situation Awareness System Based on End-User Knowledge Provision.

Recent disasters have shown that having clearly defined preventive procedures and decisions is a critical component that minimizes evacuation hazards and ensures a rapid and successful evolution of evacuation plans. In this context, we present our Situation-Aware System for enhancing Evacuation Plans (SASEP) system, which allows creating end-user business rules that technically support the specific events, conditions and actions related to evacuation plans. An experimental validation was carried out where 32 people faced a simulated emergency situation, 16 of them using SASEP and the other 16 using a legacy system based on static signs. From the results obtained, we compare both techniques and discuss in which situations SASEP offers a better evacuation route option, confirming that it is highly valuable when there is a threat in the evacuation route. In addition, a study about user satisfaction using both systems is presented showing in which cases the systems are assessed as satisfactory, relevant and not frustrating.

Confidence: dependencies and their critical role in fostering user acceptance in pervasive applications

Pervasive computing offers new scenarios where users are surrounded by invisible and proactive technology making smart spaces. Although the utility and power of solutions developed using this computer paradigm are proved, there are unresolved problems that hinder their acceptance and inclusion in our private life. Users have problems understanding the operations of a pervasive computing solution, and therefore they should trust that the solution works properly and according to their expectations. Nevertheless, the concept of trust is already framed in a specific use within the ecosystem of applications that can populate a smart space. To take this concept of trust to the whole space, we propose to study and define the concept of confidence. In contrast to the concept of trust, confidence has deeper psychological implications.