Integrating eye tracking in virtual reality for stroke rehabilitation

This thesis reports on research done for the integration of eye tracking technology into virtual reality environments, with the goal of using it in rehabilitation of patients who suffered from stroke. For the last few years, eye tracking has been a focus on medical research, used as an assistive tool  to help people with disabilities interact with new technologies  and as an assessment tool  to track the eye gaze during computer interactions. However, tracking more complex gaze behaviors and relating them to motor deficits in people with disabilities is an area that has not been fully explored, therefore it became the focal point of this research. During the research, two exploratory studies were performed in which eye tracking technology was integrated in the context of a newly created virtual reality task to assess the impact of stroke. Using an eye tracking device and a custom virtual task, the system developed is able to monitor the eye gaze pattern changes over time in patients with stroke, as well as allowing their eye gaze to function as an input for the task. Based on neuroscientific hypotheses of upper limb motor control, the studies aimed at verifying the differences in gaze patterns during the observation and execution of the virtual goal-oriented task in stroke patients (N=10), and also to assess normal gaze behavior in healthy participants (N=20). Results were found consistent and supported the hypotheses formulated, showing that eye gaze could be used as a valid assessment tool on these patients. However, the findings of this first exploratory approach are limited in order to fully understand the effect of stroke on eye gaze behavior. Therefore, a novel model-driven paradigm is proposed to further understand the relation between the neuronal mechanisms underlying goal-oriented actions and eye gaze behavior.

A virtual agent to sustaining children's engagement in language learning

In this thesis we aimed to explore the potential of gamification - defined as “the use of game elements in non-game contexts” [30] - in increasing children's (aged 5 to 6) engagement with the task. This is mainly due to the fact that our world is living a technological era, and videogames are an example of this engagement by being able to maintain children’s (and adults) engagement for hours straight. For the purpose of limiting complexity, we only addressed the feedback element by introducing it with an anthropomorphic virtual agent (human-like aspect), because research shows that virtual agents (VA’s) can influence behavioural change [17], or even induce emotions on humans both through the use of feedback provided and their facial expressions, which can interpreted in the same way as of humans’ [2]. By pairing the VA with the gamification concept, we wanted to 1) create a VA that is likely to be well-received by children (appearance and behaviour), and 2) have the immediate feedback that games have, so we can give children an assessment of their actions in real-time, as opposed to waiting for feedback from someone (traditional teaching), and with this give students more chances to succeed [32, 43]. Our final system consisted on a virtual environment, where children formed words that corresponded to a given image. In order to measure the impact that the VA had on engagement, the system was developed in two versions: one version of the system was limited to provide a simple feedback environment, where the VA provided feedback, by responding with simple phrases (i.e. “correct” or “incorrect”); for the second version, the VA had a more complex approach where it tried to encourage children to complete the word – a motivational feedback - even when they weren’t succeeding. Lastly we conducted a field study with two groups of children, where one group tested the version with the simple feedback, and the other group tested the ‘motivational’ version of the system. We used a quantitative approach to analyze the collected data that measured the engagement, based on the number of tasks (words) completed and time spent with system. The results of the evaluation showed that the use of motivational feedback may carry a positive effect on engaging children.