The Human in the Loop: User Participation in Self-Adaptive Software

Self-adaptive software provides a profound solution for adapting applications to changing contexts in dynamic and heterogeneous environments. Having emerged from Autonomic Computing, it incorporates fully autonomous decision making based on predefined structural and behavioural models. The most common approach for architectural runtime adaptation is the MAPE-K adaptation loop implementing an external adaptation manager without manual user control. However, it has turned out that adaptation behaviour lacks acceptance if it does not correspond to a user’s expectations – particularly for Ubiquitous Computing scenarios with user interaction. Adaptations can be irritating and distracting if they are not appropriate for a certain situation. In general, uncertainty during development and at run-time causes problems with users being outside the adaptation loop. In a literature study, we analyse publications about self-adaptive software research. The results show a discrepancy between the motivated application domains, the maturity of examples, and the quality of evaluations on the one hand and the provided solutions on the other hand. Only few publications analysed the impact of their work on the user, but many employ user-oriented examples for motivation and demonstration. To incorporate the user within the adaptation loop and to deal with uncertainty, our proposed solutions enable user participation for interactive selfadaptive software while at the same time maintaining the benefits of intelligent autonomous behaviour. We define three dimensions of user participation, namely temporal, behavioural, and structural user participation. This dissertation contributes solutions for user participation in the temporal and behavioural dimension. The temporal dimension addresses the moment of adaptation which is classically determined by the self-adaptive system. We provide mechanisms allowing users to influence or to define the moment of adaptation. With our solution, users can have full control over the moment of adaptation or the self-adaptive software considers the user’s situation more appropriately. The behavioural dimension addresses the actual adaptation logic and the resulting run-time behaviour. Application behaviour is established during development and does not necessarily match the run-time expectations. Our contributions are three distinct solutions which allow users to make changes to the application’s runtime behaviour: dynamic utility functions, fuzzy-based reasoning, and learning-based reasoning. The foundation of our work is a notification and feedback solution that improves intelligibility and controllability of self-adaptive applications by implementing a bi-directional communication between self-adaptive software and the user. The different mechanisms from the temporal and behavioural participation dimension require the notification and feedback solution to inform users on adaptation actions and to provide a mechanism to influence adaptations. Case studies show the feasibility of the developed solutions. Moreover, an extensive user study with 62 participants was conducted to evaluate the impact of notifications before and after adaptations. Although the study revealed that there is no preference for a particular notification design, participants clearly appreciated intelligibility and controllability over autonomous adaptations.

The artist and the computer : understanding the creative user

This paper is focused on understanding the creative computer user for the purposes of informing the design of future creativity support systems and related software. We present an investigation of a successful Australian artist, Jill Lewis, who paints on canvas. In particular, we highlight the interesting part that existing digital technology plays in her creative practice, and we identify and describe in detail two specific uses of this technology. We terml these uses "electronic collaging" and "media switching". We go on to attempt to relate this artist's creative process to two theoretical models of the creative process.

Combining the power of online crowdsourcing with creative writing user interfaces

The goal of this work was to provide professional and amateur writers with a new way of enhancing their productivity and mental well-being, by helping them overcoming writers block and being able to achieve a state of optimal experience while writing. Our approach is based on bringing together different components to create what we call a creative moment. A creative moment is composed by an image, a text, a mood, a location and a color. The color presented in the creative moment varied according to the mood that was associated to the creative moment. With the creative moments we hoped that our users could have a way to easily trigger their creativity and have a kick start in their work. The prototyping of a web crowdsourcing platform, named CreativeWall, and a Microsoft Word Add-In, that was used on the user study performed, is described and their implementations are discussed. The user study reveals that our approach does have a positive influence in the productivity of the participants when compared with another existing approach. The study also revealed that our approach can ease the process of achieving a state of optimal experience by enhancing one of the dimensions presented on the Flow Theory. At the end we present what we consider would be some possible future developments for the concept created during the development of this work.

Combining the power of online crowdsourcing with creative writing user interfaces

The goal of this work was to provide professional and amateur writers with a new way of enhancing their productivity and mental well-being, by helping them overcoming writers block and being able to achieve a state of optimal experience while writing. Our approach is based on bringing together different components to create what we call a creative moment. A creative moment is composed by an image, a text, a mood, a location and a color. The color presented in the creative moment varied according to the mood that was associated to the creative moment. With the creative moments we hoped that our users could have a way to easily trigger their creativity and have a kick start in their work. The prototyping of a web crowdsourcing platform, named CreativeWall, and a Microsoft Word Add-In, that was used on the user study performed, is described and their implementations are discussed. The user study reveals that our approach does have a positive influence in the productivity of the participants when compared with another existing approach. The study also revealed that our approach can ease the process of achieving a state of optimal experience by enhancing one of the dimensions presented on the Flow Theory. At the end we present what we consider would be some possible future developments for the concept created during the development of this work.

Applying Information Foraging Theory to understand user interaction with content-based image retrieval

The paper proposes an ISE (Information goal, Search strategy, Evaluation threshold) user classification model based on Information Foraging Theory for understanding user interaction with content-based image retrieval (CBIR). The proposed model is verified by a multiple linear regression analysis based on 50 users' interaction features collected from a task-based user study of interactive CBIR systems. To our best knowledge, this is the first principled user classification model in CBIR verified by a formal and systematic qualitative analysis of extensive user interaction data. Copyright 2010 ACM.

Investigation into a best practice model for providing an integrated user experience with mobile cloud applications

Mobile Cloud Computing promises to overcome the physical limitations of mobile devices by executing demanding mobile applications on cloud infrastructure. In practice, implementing this paradigm is difficult; network disconnection often occurs, bandwidth may be limited, and a large power draw is required from the battery, resulting in a poor user experience. This thesis presents a mobile cloud middleware solution, Context Aware Mobile Cloud Services (CAMCS), which provides cloudbased services to mobile devices, in a disconnected fashion. An integrated user experience is delivered by designing for anticipated network disconnection, and low data transfer requirements. CAMCS achieves this by means of the Cloud Personal Assistant (CPA); each user of CAMCS is assigned their own CPA, which can complete user-assigned tasks, received as descriptions from the mobile device, by using existing cloud services. Service execution is personalised to the user's situation with contextual data, and task execution results are stored with the CPA until the user can connect with his/her mobile device to obtain the results. Requirements for an integrated user experience are outlined, along with the design and implementation of CAMCS. The operation of CAMCS and CPAs with cloud-based services is presented, specifically in terms of service description, discovery, and task execution. The use of contextual awareness to personalise service discovery and service consumption to the user's situation is also presented. Resource management by CAMCS is also studied, and compared with existing solutions. Additional application models that can be provided by CAMCS are also presented. Evaluation is performed with CAMCS deployed on the Amazon EC2 cloud. The resource usage of the CAMCS Client, running on Android-based mobile devices, is also evaluated. A user study with volunteers using CAMCS on their own mobile devices is also presented. Results show that CAMCS meets the requirements outlined for an integrated user experience.

Structuring free-form tagging in online news

Tagging has become one of the key activities in next generation websites which allow users selecting short labels to annotate, manage, and share multimedia information such as photos, videos and bookmarks. Tagging does not require users any prior training before participating in the annotation activities as they can freely choose any terms which best represent the semantic of contents without worrying about any formal structure or ontology. However, the practice of free-form tagging can lead to several problems, such as synonymy, polysemy and ambiguity, which potentially increase the complexity of managing the tags and retrieving information. To solve these problems, this research aims to construct a lightweight indexing scheme to structure tags by identifying and disambiguating the meaning of terms and construct a knowledge base or dictionary. News has been chosen as the primary domain of application to demonstrate the benefits of using structured tags for managing the rapidly changing and dynamic nature of news information. One of the main outcomes of this work is an automatically constructed vocabulary that defines the meaning of each named entity tag, which can be extracted from a news article (including person, location and organisation), based on experts suggestions from major search engines and the knowledge from public database such as Wikipedia. To demonstrate the potential applications of the vocabulary, we have used it to provide more functionalities in an online news website, including topic-based news reading, intuitive tagging, clipping and sharing of interesting news, as well as news filtering or searching based on named entity tags. The evaluation results on the impact of disambiguating tags have shown that the vocabulary can help to significantly improve news searching performance. The preliminary results from our user study have demonstrated that users can benefit from the additional functionalities on the news websites as they are able to retrieve more relevant news, clip and share news with friends and families effectively.

Capital music : personal expression with a public display of song choice

Using information and communication technology devices in public urban places can help to create a personalised space. Looking at a mobile phone screen or listening to music on an MP3 player is a common practice avoiding direct contact with others e.g. whilst using public transport. However, such devices can also be utilised to explore how to build new meaningful connections with the urban space and the collocated people within. We present findings of work-in-progress on Capital Music, a mobile application enabling urban dwellers to listen to music songs as usual, but also allowing them to announce song titles and discover songs currently being listened to by other people in the vicinity. We study the ways that this tool can change or even enhance people’s experience of public urban spaces. Our first user study also found changes in choosing different songs. Anonymous social interactions based on users’ music selection are implemented in the first iteration of the prototype that we studied.

Remixing social media for location sharing on public urban screens

This paper introduces Sapporo World Window (hereafter SWW), an interactive social media mash-up deployed in a newly built urban public underground space utilising ten public displays and urban dwellers’ mobile phones. SWW enables users to share their favourite locations with fellow citizens and visitors through integrating various social media contents to a coherent whole. The system aims to engage citizens in socio-cultural and technological interactions, turning the underground space into a creative and lively social space. We present first insight from an initial user study in a real world setting.

Simple classification of walking activities using commodity smart phones

People interact with mobile computing devices everywhere, while sitting, walking, running or even driving. Adapting the interface to suit these contexts is important, thus this paper proposes a simple human activity classification system. Our approach uses a vector magnitude recognition technique to detect and classify when a person is stationary (or not walking), casually walking, or jogging, without any prior training. The user study has confirmed the accuracy.

Modeling player-like behaviour for game Al design

The design of artificial intelligence in computer games is an important component of a player's game play experience. As games are becoming more life-like and interactive, the need for more realistic game AI will increase. This is particularly the case with respect to AI that simulates how human players act, behave and make decisions. The purpose of this research is to establish a model of player-like behavior that may be effectively used to inform the design of artificial intelligence to more accurately mimic a player's decision making process. The research uses a qualitative analysis of player opinions and reactions while playing a first person shooter video game, with recordings of their in game actions, speech and facial characteristics. The initial studies provide player data that has been used to design a model of how a player behaves.

Impact of region-of-interest video coding on perceived quality in mobile video

Effective streaming of video can be achieved by providing more bits to the most important region in the frame at the cost of reduced bits in the less important regions. This strategy can be beneficial for delivering high quality videos in mobile devices, especially when the availability of bandwidth is usually low and limited. While the state-of-the-art video codecs such as H.264 may have been optimised for perceived quality, it is hypothesised that users will give more attention to interesting region/object when watching videos. Therefore, giving a higher quality to region of interest (ROI)while reducing quality of other areas may result in improving the overall perceived quality without necessarily increasing the bitrate. In this paper, the impact of ROI-based encoded video on perceived quality is investigated by conducting a user study for varous target bitrates. The results from the user study demonstrate that ROI-based video coding has superior perceived quality compared to normal encoded video at the same bitrate in the lower bitrate range.

Human–computer interaction : the impact of users’ cognitive styles on query reformulation behaviour during web searching

This paper discusses users’ query reformulation behaviour while searching information on the Web. Query reformulations have emerged as an important component of Web search behaviour and human-computer interaction (HCI) because a user’s success of information retrieval (IR) depends on how he or she formulates queries. There are various factors, such as cognitive styles, that influence users’ query reformulation behaviour. Understanding how users with different cognitive styles formulate their queries while performing Web searches can help HCI researchers and information systems (IS) developers to provide assistance to the users. This paper aims to examine the effects of users’ cognitive styles on their query reformation behaviour. To achieve the goal of the study, a user study was conducted in which a total of 3613 search terms and 872 search queries were submitted by 50 users who engaged in 150 scenario-based search tasks. Riding’s (1991) Cognitive Style Analysis (CSA) test was used to assess users’ cognitive style as wholist or analytic, and verbaliser or imager. The study findings show that users’ query reformulation behaviour is affected by their cognitive styles. The results reveal that analytic users tended to prefer Add queries while all other users preferred New queries. A significant difference was found among wholists and analytics in the manner they performed Remove query reformulations. Future HCI researchers and IS developers can utilize the study results to develop interactive and user-cantered search model, and to provide context-based query suggestions for users.

Relationship between the nature of the search task types and query reformulation behaviour

Success of query reformulation and relevant information retrieval depends on many factors, such as users’ prior knowledge, age, gender, and cognitive styles. One of the important factors that affect a user’s query reformulation behaviour is that of the nature of the search tasks. Limited studies have examined the impact of the search task types on query reformulation behaviour while performing Web searches. This paper examines how the nature of the search tasks affects users’ query reformulation behaviour during information searching. The paper reports empirical results from a user study in which 50 participants performed a set of three Web search tasks – exploratory, factorial and abstract. Users’ interactions with search engines were logged by using a monitoring program. 872 unique search queries were classified into five query types – New, Add, Remove, Replace and Repeat. Users submitted fewer queries for the factual task, which accounted for 26%. They completed a higher number of queries (40% of the total queries) while carrying out the exploratory task. A one-way MANOVA test indicated a significant effect of search task types on users’ query reformulation behaviour. In particular, the search task types influenced the manner in which users reformulated the New and Repeat queries.

Spotting the difference : identifying player opponent preferences in FPS games

This paper describes a study designed to understand player responses to artificially intelligent opponents in multi-player First Person Shooter games. It examines the player's ability to tell the difference between artificially intelligent opponents and other human players, and investigates the players' perceptions of these opponents. The study examines player preferences in this regard and identifies the significance of the cues and signs players use to categorise an opponent as artificial or human.