Quantifying epistemic actions in human-computer interaction

As digital systems move away from traditional desktop setups, new interaction paradigms are emerging that better integrate with users’ realworld surroundings, and better support users’ individual needs. While promising, these modern interaction paradigms also present new challenges, such as a lack of paradigm-specific tools to systematically evaluate and fully understand their use. This dissertation tackles this issue by framing empirical studies of three novel digital systems in embodied cognition – an exciting new perspective in cognitive science where the body and its interactions with the physical world take a central role in human cognition. This is achieved by first, focusing the design of all these systems on a contemporary interaction paradigm that emphasizes physical interaction on tangible interaction, a contemporary interaction paradigm; and second, by comprehensively studying user performance in these systems through a set of novel performance metrics grounded on epistemic actions, a relatively well established and studied construct in the literature on embodied cognition. The first system presented in this dissertation is an augmented Four-in-a-row board game. Three different versions of the game were developed, based on three different interaction paradigms (tangible, touch and mouse), and a repeated measures study involving 36 participants measured the occurrence of three simple epistemic actions across these three interfaces. The results highlight the relevance of epistemic actions in such a task and suggest that the different interaction paradigms afford instantiation of these actions in different ways. Additionally, the tangible version of the system supports the most rapid execution of these actions, providing novel quantitative insights into the real benefits of tangible systems. The second system presented in this dissertation is a tangible tabletop scheduling application. Two studies with single and paired users provide several insights into the impact of epistemic actions on the user experience when these are performed outside of a system’s sensing boundaries. These insights are clustered by the form, size and location of ideal interface areas for such offline epistemic actions to occur, as well as how can physical tokens be designed to better support them. Finally, and based on the results obtained to this point, the last study presented in this dissertation directly addresses the lack of empirical tools to formally evaluate tangible interaction. It presents a video-coding framework grounded on a systematic literature review of 78 papers, and evaluates its value as metric through a 60 participant study performed across three different research laboratories. The results highlight the usefulness and power of epistemic actions as a performance metric for tangible systems. In sum, through the use of such novel metrics in each of the three studies presented, this dissertation provides a better understanding of the real impact and benefits of designing and developing systems that feature tangible interaction.

Rapid prototyping of ubiquitous computing environments

Ubiquitous computing raises new usability challenges that cut across design and development. We are particularly interested in environments enhanced with sensors, public displays and personal devices. How can prototypes be used to explore the users' mobility and interaction, both explicitly and implicitly, to access services within these environments? Because of the potential cost of development and design failure, these systems must be explored using early assessment techniques and versions of the systems that could disrupt if deployed in the target environment. These techniques are required to evaluate alternative solutions before making the decision to deploy the system on location. This is crucial for a successful development, that anticipates potential user problems, and reduces the cost of redesign. This thesis reports on the development of a framework for the rapid prototyping and analysis of ubiquitous computing environments that facilitates the evaluation of design alternatives. It describes APEX, a framework that brings together an existing 3D Application Server with a modelling tool. APEX-based prototypes enable users to navigate a virtual world simulation of the envisaged ubiquitous environment. By this means users can experience many of the features of the proposed design. Prototypes and their simulations are generated in the framework to help the developer understand how the user might experience the system. These are supported through three different layers: a simulation layer (using a 3D Application Server); a modelling layer (using a modelling tool) and a physical layer (using external devices and real users). APEX allows the developer to move between these layers to evaluate different features. It supports exploration of user experience through observation of how users might behave with the system as well as enabling exhaustive analysis based on models. The models support checking of properties based on patterns. These patterns are based on ones that have been used successfully in interactive system analysis in other contexts. They help the analyst to generate and verify relevant properties. Where these properties fail then scenarios suggested by the failure provide an important aid to redesign.

Seven stories: location based story-delivery system

Location aware content-based experiences have a substantial tradition in HCI, several projects over the last two decades have explored the association of digital media to specific locations or objects. However, a large portion of the literature has little focus on the creative side of designing of the experience and on the iterative process of user evaluations. In this thesis we present two iterations in the design and evaluation of a location based story delivery system (LBSDS), inspired by local folklore and oral storytelling in Madeira. We started by testing an already existing location based story platform, PlaceWear, with short multimedia clips that recounted local traditions and folktales, to this experience we called iLand. An initial evaluation of iLand, was conducted; we shadowed users during the experience and then they responded to a questionnaire. By analyzing the evaluation results we uncovered several issues that informed the redesign of the system itself as well as part of the story content. The outcome of this re design was the 7Stories experience. In the new experience we performed the integration of visual markers in the interface and the framing of the fragmented story content through the literary technique of the narrator. This was done aiming to improving the connection of the audience to the physical context where the experience is delivered. The 7Stories experience was evaluated following a similar methodology to the iLand evaluation but the user’s experience resulted considerably different; because of the same setting for the experience in both versions and the constancy of the most of the content across the two versions we were able to assess the specific effect of the new design and discuss its strengths and shortcomings. Although we did not run a formal and strict comparative test between the two evaluations, it is evident from the collected data how the specific design changes to our LBSDS influenced the user experience.

De touch: extending service channels in an airport through touch technologies

This project aimed to create a communication and interaction channel between Madeira Airport and its passengers. We used the pre-existent touch enabled screens at the terminal since their potential was not being utilised to their full capacity. To achieve our goal, we have followed an agile strategy to create a testable prototype and take advantages of its results. The developed prototype is based on a plugin architecture turning it into a maintainable and highly customisable system. The collected usage data suggests that we have achieved the initially defined goals. There is no doubt that this new interaction channel is an improvement regarding the provided services and, supported by the usage data, there is an opportunity to explore additional developments to the channel.

WISE-MUSE Mobile: aplicação móvel para monitorização ambiental de museus

Manter o património histórico, natural e artístico por muitos anos é de vital importância pois, sem este património, as gerações não poderiam ter conhecimento nem aprender todos os grandes acontecimentos que têm passado ao longo dos anos num determinado país. Para evitar que esse património se perca foi criado o projecto WISE-MUSE, que tem como objectivo principal a conservação preventiva de obras de arte através de monitorização ambiental, usando uma Rede de Sensores Sem Fios (RSSF) quer para recolher dados quer para transmiti-los. Apesar do projecto WISE-MUSE ser relevante para todos os museus, em geral, este foi aplicado ao caso específico do Museu da Baleia, localizado na Ilha da Madeira, em Portugal. Inicialmente, este projecto preocupava-se essencialmente com a monitorização ambiental das salas de exposição e das câmaras frigoríficas, usando uma rede de sensores sem fios distribuídos pelos espaços em questão. Uma RSSF permite obter dados essenciais à preservação das obras de arte num museu, de uma forma fácil e continua. Para tal, foram usados diferentes tipos de sensores para capturar parametros ambientais, tais como temperatura, humidade, luminosidade, CO e CO2. Um dos objectivos principais do projecto WISE-MUSE é conseguir visualizar as medições recebidas pelos sensores sem fios, de uma forma rápida e fiável, para que quando seja detectado algum problema, os responsáveis pelo museu possam actuar a tempo e resolver o problema sem danos maiores. Tendo em vista este objectivo, este trabalho de mestrado consistiu no desenvolvimento de uma aplicação móvel que permita a visualização e análise dos respectivos dados, em tempo real e independentemente do local onde os utilizadores se encontrem, enviando-lhes alertas caso seja detectado algum parâmetro com valores fora do normal. Esta aplicação foi implementada sobre o sistema operativo Android, por ser um dos mais usados por todo o mundo, assim como graças à sua facilidade de utilização quer por parte dos programadores quer por parte dos utilizadores finais.

Breadcrumbs: location and context aware mobile platform for story sharing

With the current proliferation of sensor equipped mobile devices such as smartphones and tablets, location aware services are expanding beyond the mere efficiency and work related needs of users, evolving in order to incorporate fun, culture and the social life of users. Today people on the move have more and more connectivity and are expected to be able to communicate with their usual and familiar social networks. That means communications not only with their peers and colleagues, friends and family but also with unknown people that might share their interests, curiosities or happen to use the same social network. Through social networks, location aware blogging, cultural mobile applications relevant information is now available at specific geographical locations and open to feedback and conversations among friends as well as strangers. In fact, nowadays smartphone technologies aloud users to post and retrieve content while on the move, often relating to specific physical landmarks or locations, engaging and being engaged in conversations with strangers as much as their own social network. The use of such technologies and applications while on the move can often lead people to serendipitous discoveries and interactions. Throughout our thesis we are engaging on a two folded investigation: how can we foster and support serendipitous discoveries and what are the best interfaces for it? In fact, to read and write content while on the move is a cognitively intensive task. While the map serves the function of orienting the user, it also absorbs most of the user’s concentration. In order to address this kind of cognitive overload issue with Breadcrumbs we propose a 360 degrees interface that enables the user to find content around them by means of scanning the surrounding space with the mobile device. By using a loose metaphor of a periscope, harnessing the power of the smartphone sensors we designed an interactive interface capable of detecting content around the users and display it in the form of 2 dimensional bubbles which diameter depends on their distance from the users. Users will navigate the space in relation to the content that they are curious about, rather than in relation to the traditional geographical map. Through this model we envisage alleviating a certain cognitive overload generated by having to continuously confront a two dimensional map with the real three dimensional space surrounding the user, but also use the content as a navigational filter. Furthermore this alternative mean of navigating space might bring serendipitous discovery about places that user where not aware of or intending to reach. We hence conclude our thesis with the evaluation of the Breadcrumbs application and the comparison of the 360 degrees interface with a traditional 2 dimensional map displayed on the devise screen. Results from the evaluation are compiled in findings and insights for future use in designing and developing context aware mobile applications.

Combining the power of online crowdsourcing with creative writing user interfaces

The goal of this work was to provide professional and amateur writers with a new way of enhancing their productivity and mental well-being, by helping them overcoming writers block and being able to achieve a state of optimal experience while writing. Our approach is based on bringing together different components to create what we call a creative moment. A creative moment is composed by an image, a text, a mood, a location and a color. The color presented in the creative moment varied according to the mood that was associated to the creative moment. With the creative moments we hoped that our users could have a way to easily trigger their creativity and have a kick start in their work. The prototyping of a web crowdsourcing platform, named CreativeWall, and a Microsoft Word Add-In, that was used on the user study performed, is described and their implementations are discussed. The user study reveals that our approach does have a positive influence in the productivity of the participants when compared with another existing approach. The study also revealed that our approach can ease the process of achieving a state of optimal experience by enhancing one of the dimensions presented on the Flow Theory. At the end we present what we consider would be some possible future developments for the concept created during the development of this work.