Handbook of research on user interface design and evaluation for mobile technology

In recent years, mobile technology has been one of the major growth areas in computing. Designing the user interface for mobile applications, however, is a very complex undertaking which is made even more challenging by the rapid technological developments in mobile hardware. Mobile human-computer interaction, unlike desktop-based interaction, must be cognizant of a variety of complex contextual factors affecting both users and technology. The Handbook of Research on User Interface Design and Evaluation provides students, researchers, educators, and practitioners with a compendium of research on the key issues surrounding the design and evaluation of mobile user interfaces, such as the physical environment and social context in which a mobile device is being used and the impact of multitasking behavior typically exhibited by mobile-device users. Compiling the expertise of over 150 leading experts from 26 countries, this exemplary reference tool will make an indispensable addition to every library collection.

Quality of experience evaluation of voice communication: an affect-based approach

Voice communication systems such as Voice-over IP (VoIP), Public Switched Telephone Networks, and Mobile Telephone Networks, are an integral means of human tele-interaction. These systems pose distinctive challenges due to their unique characteristics such as low volume, burstiness and stringent delay/loss requirements across heterogeneous underlying network technologies. Effective quality evaluation methodologies are important for system development and refinement, particularly by adopting user feedback based measurement. Presently, most of the evaluation models are system-centric (Quality of Service or QoS-based), which questioned us to explore a user-centric (Quality of Experience or QoE-based) approach as a step towards the human-centric paradigm of system design. We research an affect-based QoE evaluation framework which attempts to capture users' perception while they are engaged in voice communication. Our modular approach consists of feature extraction from multiple information sources including various affective cues and different classification procedures such as Support Vector Machines (SVM) and k-Nearest Neighbor (kNN). The experimental study is illustrated in depth with detailed analysis of results. The evidences collected provide the potential feasibility of our approach for QoE evaluation and suggest the consideration of human affective attributes in modeling user experience.

Utilising presence in places to support mobile interaction

Physical places are given contextual meaning by the objects and people that make up the space. Presence in physical places can be utilised to support mobile interaction by making access to media and notifications on a smartphone easier and more visible to other people. Smartphone interfaces can be extended into the physical world in a meaningful way by anchoring digital content to artefacts, and interactions situated around physical artefacts can provide contextual meaning to private manipulations with a mobile device. Additionally, places themselves are designed to support a set of tasks, and the logical structure of places can be used to organise content on the smartphone. Menus that adapt the functionality of a smartphone can support the user by presenting the tools most likely to be needed just-in-time, so that information needs can be satisfied quickly and with little cognitive effort. Furthermore, places are often shared with people whom the user knows, and the smartphone can facilitate social situations by providing access to content that stimulates conversation. However, the smartphone can disrupt a collaborative environment, by alerting the user with unimportant notifications, or sucking the user in to the digital world with attractive content that is only shown on a private screen. Sharing smartphone content on a situated display creates an inclusive and unobtrusive user experience, and can increase focus on a primary task by allowing content to be read at a glance. Mobile interaction situated around artefacts of personal places is investigated as a way to support users to access content from their smartphone while managing their physical presence. A menu that adapts to personal places is evaluated to reduce the time and effort of app navigation, and coordinating smartphone content on a situated display is found to support social engagement and the negotiation of notifications. Improving the sensing of smartphone users in places is a challenge that is out-with the scope of this thesis. Instead, interaction designers and developers should be provided with low-cost positioning tools that utilise presence in places, and enable quantitative and qualitative data to be collected in user evaluations. Two lightweight positioning tools are developed with the low-cost sensors that are currently available: The Microsoft Kinect depth sensor allows movements of a smartphone user to be tracked in a limited area of a place, and Bluetooth beacons enable the larger context of a place to be detected. Positioning experiments with each sensor are performed to highlight the capabilities and limitations of current sensing techniques for designing interactions with a smartphone. Both tools enable prototypes to be built with a rapid prototyping approach, and mobile interactions can be tested with more advanced sensing techniques as they become available. Sensing technologies are becoming pervasive, and it will soon be possible to perform reliable place detection in-the-wild. Novel interactions that utilise presence in places can support smartphone users by making access to useful functionality easy and more visible to the people who matter most in everyday life.

Cyber foraging for green computing, improving performance and prolonging battery life of mobile devices

The increasing dependency of everyday life on mobile devices also increases the number and complexity of computing tasks to be supported by these devices. However, the inherent requirement of mobility restricts them from being resources rich both in terms of energy (battery capacity) and other computing resources such as processing capacity, memory and other resources. This thesis looks into cyber foraging technique of offloading computing tasks. Various experiments on android mobile devices are carried out to evaluate offloading benefits in terms of sustainability advantage, prolonging battery life and augmenting the performance of mobile devices. This thesis considers two scenarios of cyber foraging namely opportunistic offloading and competitive offloading. These results show that the offloading scenarios are important for both green computing and resource augmentation of mobile devices. A significant advantage in battery life gain and performance enhancement is obtained. Moreover, cyber foraging is proved to be efficient in minimizing energy consumption per computing tasks. The work is based on scavenger cyber foraging system. In addition, the work can be used as a basis for studying cyber foraging and other similar approaches such as mobile cloud/edge computing for internet of things devices and improving the user experiences of applications by minimizing latencies through the use of potential nearby surrogates.

Engineering searchable encryption of mobile cloud networks : when QoE meets QoP

Mobile cloud computing can effectively address the resource limitations of mobile devices, and is therefore essential to enable extensive resource consuming mobile computing and communication applications. Of all the mobile cloud computing applications, data outsourcing, such as iCloud, is fundamental, which outsources a mobile user's data to external cloud servers and accordingly provides a scalable and always on approach for public data access. With the security and privacy issues related to outsourced data becoming a rising concern, encryption on outsourced data is often necessary. Although encryption increases the quality of protection (QoP) of data outsourcing, it significantly reduces data usability and thus harms the mobile user's quality of experience (QoE). How to strike a balance between QoP and QoE is therefore an important yet challenging task. In this article we focus on the fundamental problem of QoP and QoE provisioning in searchable encryption of data outsourcing. We develop a fine-grained data search scheme and discuss its implementation on encrypted mobile cloud data, which is an effective balance between QoE and QoP in mobile cloud data outsourcing.

User interaction monitoring and analysis framework

An overview is given of a user interaction monitoring and analysis framework called BaranC. Monitoring and analysing human-digital interaction is an essential part of developing a user model as the basis for investigating user experience. The primary human-digital interaction, such as on a laptop or smartphone, is best understood and modelled in the wider context of the user and their environment. The BaranC framework provides monitoring and analysis capabilities that not only records all user interaction with a digital device (e.g. smartphone), but also collects all available context data (such as from sensors in the digital device itself, a fitness band or a smart appliances). The data collected by BaranC is recorded as a User Digital Imprint (UDI) which is, in effect, the user model and provides the basis for data analysis. BaranC provides functionality that is useful for user experience studies, user interface design evaluation, and providing user assistance services. An important concern for personal data is privacy, and the framework gives the user full control over the monitoring, storing and sharing of their data.

User-designer collaboration during the early stage of the product development process

As an understanding of users' tacit knowledge and latent needs embedded in user experience has played a critical role in product development, users’ direct involvement in design has become a necessary part of the design process. Various ways of accessing users' tacit knowledge and latent needs have been explored in the field of user-centred design, participatory design, and design for experiencing. User-designer collaboration has been used unconsciously by traditional designers to facilitate the transfer of users' tacit knowledge and to elicit new knowledge. However, what makes user-designer collaboration an effective strategy has rarely been reported on or explored. Therefore, interaction patterns between the users and the designers in three industry-supported user involvement cases were studied. In order to develop a coding system, collaboration was defined as a set of coordinated and joint problem solving activities, measured by the elicitation of new knowledge from collaboration. The analysis of interaction patterns in the user involvement cases revealed that allowing users to challenge or modify their contextual experiences facilitates the transfer of knowledge and new knowledge generation. It was concluded that users can be more effectively integrated into the product development process by employing collaboration strategies to intensify the depth of user involvement.

Cipher cities

Cipher Cities was a practice-led research project developed in 3 stages between 2005 and 2007 resulting in the creation of a unique online community, ‘Cipher Cities’, that provides simple authoring tools and processes for individuals and groups to create their own mobile events and event journals, build community profile and participate in other online community activities. Cipher Cities was created to revitalise peoples relationship to everyday places by giving them the opportunity and motivation to create and share complex digital stories in simple and engaging ways. To do so we developed new design processes and methods for both the research team and the end user to appropriate web and mobile technologies. To do so we collaborated with ethnographers, designers and ICT researchers and developers. In teams we ran a series of workshops in a wide variety of cities in Australia to refine an engagement process and to test a series of iteratively developed prototypes to refine the systems that supported community motivation and collaboration. The result of the research is 2 fold: 1. a sophisticated prototype for researchers and designers to further experiment with community engagement methodologies using existing and emerging communications technologies. 2. A ‘human dimensions matrix’. This matrix assists in the identification and modification of place based interventions in the social, technical, spatial, cultural, pedagogical conditions of any given community. This matrix has now become an essential part of a number of subsequent projects and assists design collaborators to successfully conceptualise, generate and evaluate interactive experiences. the research team employed practice-led action research methodologies that involved a collaborative effort across the fields of interaction design and social science, in particular ethnography, in order to: 1. seek, contest, refine a design methodology that would maximise the successful application of a dynamic system to create new kinds of interactions between people, places and artefacts’. 2. To design and deploy an application that intervenes in place-based and mobile technologies and offers people simple interfaces to create and share digital stories. Cipher Cities was awarded 3 separate CRC competitive grants (over $270,000 in total) to assist 3 stages of research covering the development of the Ethnographic Design Methodologies, the development of the tools, and the testing and refinement of both the engagement models and technologies. The resulting methodologies and tools are in the process of being commercialised by the Australasian CRC for Interaction Design.

Street computing workshop 2009

The Street Computing workshop, held in conjunction with OZCHI 2009, solicits papers discussing new research directions, early research results, works-in-progress and critical surveys of prior research work in the areas of ubiquitous computing and interaction design for urban environments. Urban spaces have unique characteristics. Typically, they are densely populated, buzzing with life twenty-four hours a day, seven days a week. These traits afford many opportunities, but they also present many challenges: traffic jams, smog and pollution, stress placed on public services, and more. Computing technology, particularly the kind that can be placed in the hands of citizens, holds much promise in combating some of these challenges. Yet, computation is not merely a tool for overcoming challenges; rather, when embedded appropriately in our everyday lives, it becomes a tool of opportunity, for shaping how our cities evolve, for enabling us to interact with our city and its people in new ways, and for uncovering useful, but hidden relationships and correlations between elements of the city. The increasing availability of an urban computing infrastructure has lead to new and exciting ways inhabitants can interact with their city. This includes interaction with a wide range of services (e.g. public transport, public services), conceptual representations of the city (e.g. local weather and traffic conditions), the availability of a variety of shared and personal displays (e.g. public, ambient, mobile) and the use of different interaction modes (e.g. tangible, gesture-based, token-based). This workshop solicits papers that address the above themes in some way. We encourage researchers to submit work that deals with challenges and possibilities that the availability of urban computing infrastructure such as sensors and middleware for sensor networks pose. This includes new and innovative ways of interacting with and within urban environments; user experience design and participatory design approaches for urban environments; social aspects of urban computing; and other related areas.

Mobile learning, social learning

Understanding the future development of interaction design as it applies to learning and training scenarios is crucial to effective development of curriculum and appropriate application of social and mobile communication technologies. As Attewell & Saville-Smith have recognised (2004), the use of mobile communication devices for improved literacy and numeracy is a desirable prospect among young people who represent the average age of undergraduate students. Further, with the growing penetration of broadband internet access, the ubiquity of wireless access in educational locations, the rise of ultra-mobile portable computers and the proliferation of social software applications in educational contexts, there are a growing number of channels for facilitation of learning. Nevertheless, there has been insufficient consideration of the interaction design issues that affect the effective facilitation of such learning. This paper contends that there is a clear need to design mobile and social learning to accommodate the benefits of these diverse channels for interaction. Additionally, there is a need to implement suitable testing processes to ensure participants in mobile and social learning are contributing effectively and maximising their learning. Through the presentation of case studies in mobile and social learning, the paper attempts to demonstrate how considered interaction design techniques can improve the effectiveness of new learning channels.

The embodied hybrid space : designing ubiquitous computing towards an amplification of situated real world experiences

The emergence of mobile and ubiquitous computing technology has created what is often referred to as the hybrid space – a virtual layer of digital information and interaction opportunities that sit on top of and augment the physical environment. Embodied media materialise digital information as observable and sometimes interactive parts of the physical environment. The aim of this work is to explore ways to enhance people’s situated real world experience, and to find out what the role and impact of embodied media in achieving this goal can be. The Edge, an initiative of the State Library of Queensland in Brisbane, Australia, and case study of this thesis, envisions to be a physical place for people to meet, explore, experience, learn and teach each other creative practices in various areas related to digital technology and arts. Guided by an Action Research approach, this work applies Lefebvre’s triad of space (1991) to investigate the Edge as a social space from a conceived, perceived and lived point of view. Based on its creators’ vision and goals on the conceived level, different embodied media are iteratively designed, implemented and evaluated towards shaping and amplifying the Edge’s visitor experience on the perceived and lived level.

User-driven saliency maps for evaluating Region-of-Interest detection

Detection of Region of Interest (ROI) in a video leads to more efficient utilization of bandwidth. This is because any ROIs in a given frame can be encoded in higher quality than the rest of that frame, with little or no degradation of quality from the perception of the viewers. Consequently, it is not necessary to uniformly encode the whole video in high quality. One approach to determine ROIs is to use saliency detectors to locate salient regions. This paper proposes a methodology for obtaining ground truth saliency maps to measure the effectiveness of ROI detection by considering the role of user experience during the labelling process of such maps. User perceptions can be captured and incorporated into the definition of salience in a particular video, taking advantage of human visual recall within a given context. Experiments with two state-of-the-art saliency detectors validate the effectiveness of this approach to validating visual saliency in video. This paper will provide the relevant datasets associated with the experiments.

Remixing social media for location sharing on public urban screens

This paper introduces Sapporo World Window (hereafter SWW), an interactive social media mash-up deployed in a newly built urban public underground space utilising ten public displays and urban dwellers’ mobile phones. SWW enables users to share their favourite locations with fellow citizens and visitors through integrating various social media contents to a coherent whole. The system aims to engage citizens in socio-cultural and technological interactions, turning the underground space into a creative and lively social space. We present first insight from an initial user study in a real world setting.

Identity management : strengthening one-time password authentication through usability

Usability in HCI (Human-Computer Interaction) is normally understood as the simplicity and clarity with which the interaction with a computer program or a web site is designed. Identity management systems need to provide adequate usability and should have a simple and intuitive interface. The system should not only be designed to satisfy service provider requirements but it has to consider user requirements, otherwise it will lead to inconvenience and poor usability for users when managing their identities. With poor usability and a poor user interface with regard to security, it is highly likely that the system will have poor security. The rapid growth in the number of online services leads to an increasing number of different digital identities each user needs to manage. As a result, many people feel overloaded with credentials, which in turn negatively impacts their ability to manage them securely. Passwords are perhaps the most common type of credential used today. To avoid the tedious task of remembering difficult passwords, users often behave less securely by using low entropy and weak passwords. Weak passwords and bad password habits represent security threats to online services. Some solutions have been developed to eliminate the need for users to create and manage passwords. A typical solution is based on generating one-time passwords, i.e. passwords for single session or transaction usage. Unfortunately, most of these solutions do not satisfy scalability and/or usability requirements, or they are simply insecure. In this thesis, the security and usability aspects of contemporary methods for authentication based on one-time passwords (OTP) are examined and analyzed. In addition, more scalable solutions that provide a good user experience while at the same time preserving strong security are proposed.

Orientation passport : using gamification to engage university students

Adding game elements to an application to motivate use and enhance the user experience is a growing trend known as gamification. This study explores the use of game achievements when applied to a mobile application designed to help new students at university. This paper describes the foundations of a design framework used to integrate game elements to Orientation Passport, a personalised orientation event application for smart phones. Orientation Passport utilises game achievements to present orientation information in an engaging way and to encourage use of the application. The system is explained in terms of the design framework, and the findings of a pilot study involving 26 new students are presented. This study contributes the foundations of a design framework for general gamified achievement design. It also suggests that added game elements can be enjoyable but can potentially encourage undesirable use by some, and aren't as enjoyable if not enforced properly by the technology. Consideration is also needed when enforcing stricter game rules as usability can be affected.