Can prospective usability evaluation predict data errors?

Increasing amounts of clinical research data are collected by manual data entry into electronic source systems and directly from research subjects. For this manual entered source data, common methods of data cleaning such as post-entry identification and resolution of discrepancies and double data entry are not feasible. However data accuracy rates achieved without these mechanisms may be higher than desired for a particular research use. We evaluated a heuristic usability method for utility as a tool to independently and prospectively identify data collection form questions associated with data errors. The method evaluated had a promising sensitivity of 64% and a specificity of 67%. The method was used as described in the literature for usability with no further adaptations or specialization for predicting data errors. We conclude that usability evaluation methodology should be further investigated for use in data quality assurance.

Study on heuristic usability evaluation for mobile applications

Usability guidelines are a useful tool for the developers to improve interaction with systems. It includes knowledge of different disciplines related to usability and provides solutions and best practices to achieve the objectives of usability. Heuristic evaluation is one of the methods most widely used to evaluate and user interfaces. The objective of this study is to enrich the process of heuristic evaluation with the design guidelines focusing it on the evaluation of applications for mobile devices. As well as generate a homogeneous classification of guidelines content, in order to help that from design and development process, be included solutions and good practices provided by the guidelines. In order to achieve the objectives of this work, it is provides a method for generating heuristics for mobile applications, with which four applications were evaluated, and a web tool has also been developed that allows access to the content of the guidelines using the homogeneous classification of guidelines content. The results showed the ease and utility of performing heuristic evaluations using a set of heuristics focused on mobile applications.

Evaluation of multi-sensory feedback on the usability of a virtual assembly environment

Virtual assembly environment (VAE) technology has the great potential for benefiting the manufacturing applications in industry. Usability is an important aspect of the VAE. This paper presents the usability evaluation of a developed multi-sensory VAE. The evaluation is conducted by using its three attributes: (a) efficiency of use; (b) user satisfaction; and (c) reliability. These are addressed by using task completion times (TCTs), questionnaires, and human performance error rates (HPERs), respectively. A peg-in-a-hole and a Sener electronic box assembly task have been used to perform the experiments, using sixteen participants. The outcomes showed that the introduction of 3D auditory and/or visual feedback could improve the usability. They also indicated that the integrated feedback (visual plus auditory) offered better usability than either feedback used in isolation. Most participants preferred the integrated feedback to either feedback (visual or auditory) or no feedback. The participants' comments demonstrated that nonrealistic or inappropriate feedback had negative effects on the usability, and easily made them feel frustrated. The possible reasons behind the outcomes are also analysed. © 2007 ACADEMY PUBLISHER.

Tag-based interaction in online and mobile banking : a preliminary study of the effect on usability

In this paper we describe tag-based interaction afforded by a tag-based interface in online and mobile banking, and present our preliminary usability evaluation findings. We conducted a pilot usability study with a group of banking users by comparing the present 'conventional' interface and tag-based interface. The results show that participants perceive the tag-based interface as more usable in both online and mobile contexts. Participants also rated the tag-based interface better despite their unfamiliarity and perceived it as more user-friendly. Additionally, the results highlight that tag-based interaction is more effective in the mobile context especially to inexperienced mobile banking users. This in turn could have a positive effect on the adoption and acceptance of mobile banking in general and also specifically in Australia. We discuss our findings in more detail in the later sections of this paper and conclude with a discussion on future work.

Application of Heuristic Evaluation Method by Universal Design Experts

This study highlights how heuristic evaluation as a usability evaluation method can feed into current building design practice to conform to universal design principles. It provides a definition of universal usability that is applicable to an architectural design context. It takes the seven universal design principles as a set of heuristics and applies an iterative sequence of heuristic evaluation in a shopping mall, aiming to achieve a cost-effective evaluation process. The evaluation was composed of three consecutive sessions. First, five evaluators from different professions were interviewed regarding the construction drawings in terms of universal design principles. Then, each evaluator was asked to perform the predefined task scenarios. In subsequent interviews, the evaluators were asked to re-analyze the construction drawings. The results showed that heuristic evaluation could successfully integrate universal usability into current building design practice in two ways: (i) it promoted an iterative evaluation process combined with multi-sessions rather than relying on one evaluator and on one evaluation session to find the maximum number of usability problems, and (ii) it highlighted the necessity of an interdisciplinary ad hoc committee regarding the heuristic abilities of each profession. A multi-session and interdisciplinary heuristic evaluation method can save both the project budget and the required time, while ensuring a reduced error rate for the universal usage of the built environments.

Evaluating the appropriateness of speech input in marine applications : a field evaluation

This paper discusses the first of three studies which collectively represent a convergence of two ongoing research agendas: (1) the empirically-based comparison of the effects of evaluation environment on mobile usability evaluation results; and (2) the effect of environment - in this case lobster fishing boats - on achievable speech-recognition accuracy. We describe, in detail, our study and outline our results to date based on preliminary analysis. Broadly speaking, the potential for effective use of speech for data collection and vessel control looks very promising - surprisingly so! We outline our ongoing analysis and further work.

Evaluating the appropriateness of speech input in marine applications:a field evaluation

This paper discusses the first of three studies which collectively represent a convergence of two ongoing research agendas: (1) the empirically-based comparison of the effects of evaluation environment on mobile usability evaluation results; and (2) the effect of environment - in this case lobster fishing boats - on achievable speech-recognition accuracy. We describe, in detail, our study and outline our results to date based on preliminary analysis. Broadly speaking, the potential for effective use of speech for data collection and vessel control looks very promising - surprisingly so! We outline our ongoing analysis and further work.

Managing process model complexity via concrete syntax modifications

While Business Process Management (BPM) is an established discipline, the increased adoption of BPM technology in recent years has introduced new challenges. One challenge concerns dealing with the ever-growing complexity of business process models. Mechanisms for dealing with this complexity can be classified into two categories: 1) those that are solely concerned with the visual representation of the model and 2) those that change its inner structure. While significant attention is paid to the latter category in the BPM literature, this paper focuses on the former category. It presents a collection of patterns that generalize and conceptualize various existing mechanisms to change the visual representation of a process model. Next, it provides a detailed analysis of the degree of support for these patterns in a number of state-of-the-art languages and tools. This paper concludes with the results of a usability evaluation of the patterns conducted with BPM practitioners.

Visual support for work assignment in process-aware information systems : framework formalisation and implementation

Process-aware information systems, ranging from generic workflow systems to dedicated enterprise information systems, use work-lists to offer so-called work items to users. In real scenarios, users can be confronted with a very large number of work items that stem from multiple cases of different processes. In this jungle of work items, users may find it hard to choose the right item to work on next. The system cannot autonomously decide which is the right work item, since the decision is also dependent on conditions that are somehow outside the system. For instance, what is “best” for an organisation should be mediated with what is “best” for its employees. Current work-list handlers show work items as a simple sorted list and therefore do not provide much decision support for choosing the right work item. Since the work-list handler is the dominant interface between the system and its users, it is worthwhile to provide an intuitive graphical interface that uses contextual information about work items and users to provide suggestions about prioritisation of work items. This paper uses the so-called map metaphor to visualise work items and resources (e.g., users) in a sophisticated manner. Moreover, based on distance notions, the work-list handler can suggest the next work item by considering different perspectives. For example, urgent work items of a type that suits the user may be highlighted. The underlying map and distance notions may be of a geographical nature (e.g., a map of a city or office building), but may also be based on process designs, organisational structures, social networks, due dates, calendars, etc. The framework proposed in this paper is generic and can be applied to any process-aware information system. Moreover, in order to show its practical feasibility, the paper discusses a full-fledged implementation developed in the context of the open-source workflow environment YAWL, together with two real examples stemming from two very different scenarios. The results of an initial usability evaluation of the implementation are also presented, which provide a first indication of the validity of the approach.

Using Participatory Design (PD) as youth engagement strategy to inform design and development of evidence-based wellbeing e-tools

The paper presents a participatory design research framework as a primary method for structuring youth engagement, participation and contribution to the design, development and usability evaluation of three evidencebased e-tools for wellbeing, which include smart phone mobile apps as well as e-health websites. The three projects are part of a series of six e-tools part of Safe and Supportive program under Young and Well CRC. The participatory design method, developed by Zelenko (2012) for application in design of online health promoting technologies, was further piloted in partnership with Inspire USA for specific application within the CRC, deploying a combination of creative design workshops and speculative design activities in developing e-tool prototypes with young people. This paper presents the resulting participatory research framework as it was implemented across the e-tool projects to facilitate active youth participation in co-designing the e-tools and ensuring the final designs are relevant to young people and deliver health messages in engaging ways. The principles of Participatory Design (PD) that inform the new framework include a high degree of participant agency in creative decisionmaking and a commitment to the process of co-designing, with young people working alongside designers and developers. The paper will showcase how the PD framework was applied across three projects to increase young people’s contribution to final design outcome.

Efficient web browsing with a single-switch

This paper presents a new approach to web browsing in situ- ations where the user can only provide the device with a sin- gle input command device (switch). Switches have been de- veloped for example for people with locked-in syndrome and are used in combination with scanning to navigate virtual keyboards and desktop interfaces. Our proposed approach leverages the hierarchical structure of webpages to operate a multi-level scan of actionable elements of webpages (links or form elements). As there are a few methods already exist- ing to facilitate browsing under these conditions, we present a theoretical usability evaluation of our approach in com- parison to the existing ones, which takes into account the average time taken to reach any part of a web page (such as a link or a form) but also the number of clicks necessary to reach the goal. We argue that these factors contribute together to usability. In addition, we propose that our ap- proach presents additional usability benefits.

Avaliação de usabilidade em jogo eletrônico: estudo de caso

Esta pesquisa tem como objetivo documentar o processo de redução de riscos e incertezas de um jogo eletrônico em desenvolvimento por meio da aplicação de métodos de avaliação de Usabilidade. Foi realizado um estudo de caso da utilização de métodos e técnicas de avaliação de Usabilidade durante a produção do jogo eletrônico Dungeonland, conduzido entre 2010 a 2013 ao longo de diversas iterações do produto, da pré-produção ao lançamento. Foram utilizados os métodos de observação direta baseada em problemas, avaliação cooperativa, questionário e entrevista semi-estruturada. Os dados coletados demonstram a evolução do design do jogo, as diferentes metodologias empregadas em cada estágio de desenvolvimento, e o impacto da avaliação no projeto. Apesar de problemas e limitações no emprego dos testes de Usabilidade no produto em questão, o impacto da avaliação foi visto como muito grande e muito positivo pelos desenvolvedores - através de dados qualitativos como protocolos verbais e de gameplay de usuários, e de dados quantitativos sobre suas experiências com o produto que possam ser comparados estatisticamente, os desenvolvedores de jogos têm à sua disposição poderosas ferramentas para estabelecer processos de Design claros, centrados no usuário, e que ofereçam um ambiente onde problemas são rapidamente identificados e soluções são validadas com usuários reais.

一种新的笔式交互范式的可用性评估

笔式用户界面是Post-WIMP界面的一种重要形态，具有自然、和谐的交互特性。尽管笔式用户界面广泛应用于无处不在计算环境中，但很少专门对它的可用性进行评估。针对笔式用户界面的特征，提出了一种新的笔式用户界面范式，基于此范式设计并实现了一个面向教学的笔式交互系统(PenOffice)，利用此系统对范式从易学性、效率、出错率和用户满意度这几个方面进行了可用性评估，从中分析了笔式用户界面以及这种新的界面范式的优点和不足。

Comunicação humano-robô através de linguagem falada

Nos últimos anos, as tecnologias que dão suporte à robótica avançaram expressivamente. É possível encontrar robôs de serviço nos mais variados campos. O próximo passo é o desenvolvimento de robôs inteligentes, com capacidade de comunicação em linguagem falada e de realizar trabalhos úteis em interação/cooperação com humanos. Torna-se necessário, então, encontrar um modo de interagir eficientemente com esses robôs, e com agentes inteligentes de maneira geral, que permita a transmissão de conhecimento em ambos os sentidos. Partiremos da hipótese de que é possível desenvolver um sistema de diálogo baseado em linguagem natural falada que resolva esse problema. Assim, o objetivo principal deste trabalho é a definição, implementação e avaliação de um sistema de diálogo utilizável na interação baseada em linguagem natural falada entre humanos e agentes inteligentes. Ao longo deste texto, mostraremos os principais aspectos da comunicação por linguagem falada, tanto entre os humanos, como também entre humanos e máquinas. Apresentaremos as principais categorias de sistemas de diálogo, com exemplos de alguns sistemas implementados, assim como ferramentas para desenvolvimento e algumas técnicas de avaliação. A seguir, entre outros aspectos, desenvolveremos os seguintes: a evolução levada a efeito na arquitetura computacional do Carl, robô utilizado neste trabalho; o módulo de aquisição e gestão de conhecimento, desenvolvido para dar suporte à interação; e o novo gestor de diálogo, baseado na abordagem de “Estado da Informação”, também concebido e implementado no âmbito desta tese. Por fim, uma avaliação experimental envolvendo a realização de diversas tarefas de interação com vários participantes voluntários demonstrou ser possível interagir com o robô e realizar as tarefas solicitadas. Este trabalho experimental incluiu avaliação parcial de funcionalidades, avaliação global do sistema de diálogo e avaliação de usabilidade.