Signal processing and information fusing algorithms for the synthesis of an alternative electromyogram/eye gaze tracking computer cursor control system

This research pursued the conceptualization and real-time verification of a system that allows a computer user to control the cursor of a computer interface without using his/her hands. The target user groups for this system are individuals who are unable to use their hands due to spinal dysfunction or other afflictions, and individuals who must use their hands for higher priority tasks while still requiring interaction with a computer. ^ The system receives two forms of input from the user: Electromyogram (EMG) signals from muscles in the face and point-of-gaze coordinates produced by an Eye Gaze Tracking (EGT) system. In order to produce reliable cursor control from the two forms of user input, the development of this EMG/EGT system addressed three key requirements: an algorithm was created to accurately translate EMG signals due to facial movements into cursor actions, a separate algorithm was created that recognized an eye gaze fixation and provided an estimate of the associated eye gaze position, and an information fusion protocol was devised to efficiently integrate the outputs of these algorithms. ^ Experiments were conducted to compare the performance of EMG/EGT cursor control to EGT-only control and mouse control. These experiments took the form of two different types of point-and-click trials. The data produced by these experiments were evaluated using statistical analysis, Fitts' Law analysis and target re-entry (TRE) analysis. ^ The experimental results revealed that though EMG/EGT control was slower than EGT-only and mouse control, it provided effective hands-free control of the cursor without a spatial accuracy limitation, and it also facilitated a reliable click operation. This combination of qualities is not possessed by either EGT-only or mouse control, making EMG/EGT cursor control a unique and practical alternative for a user's cursor control needs. ^

The role of contingency and gaze direction in the emergence of social referencing

The current study assessed the importance of infant detection of contingency and head and eye gaze direction in the emergence of social referencing. Five- to six-month-old infants' detection of affect-object relations and subsequent manual preferences for objects paired with positive expressions were assessed. In particular, the role of contingency between toys' movements and an actress's emotional expressions as well as the role of gaze direction toward the toys' location were examined. Infants were habituated to alternating films of two toys each paired with an actress's affective expression (happy and fearful) under contingent or noncontingent and gaze congruent or gaze incongruent conditions. Results indicated that gaze congruence and contingency between toys' movements and a person's affective expressions were important for infant perception of affect-object relations. Furthermore, infant perception of the relation between affective expressions and toys translated to their manual preferences for the 3-dimensional toys. Infants who received contingent affective responses to the movements of the toys spent more time touching the toy that was previously paired with the positive expression. These findings demonstrate the role of contingency and gaze direction in the emergence of social referencing in the first half year of life.^

Shifting Our Gaze to the Strengths of Alternative Education Students

This study addresses the need for educators to understand what matters for alternative education students. Three male alternative education students were interviewed about their educational memories, realities and aspirations. Results revealed family influences prevailed over inhospitable socioeconomic factors in students’ educational choices. Findings supported recommendations for alternative education teacher development.

A practical system for hands-off computer interaction based on electromyogram (EMG)/eye gaze tracking (EGT)

Effective interaction with personal computers is a basic requirement for many of the functions that are performed in our daily lives. With the rapid emergence of the Internet and the World Wide Web, computers have become one of the premier means of communication in our society. Unfortunately, these advances have not become equally accessible to physically handicapped individuals. In reality, a significant number of individuals with severe motor disabilities, due to a variety of causes such as Spinal Cord Injury (SCI), Amyothrophic Lateral Sclerosis (ALS), etc., may not be able to utilize the computer mouse as a vital input device for computer interaction. The purpose of this research was to further develop and improve an existing alternative input device for computer cursor control to be used by individuals with severe motor disabilities. This thesis describes the development and the underlying principle for a practical hands-off human-computer interface based on Electromyogram (EMG) signals and Eye Gaze Tracking (EGT) technology compatible with the Microsoft Windows operating system (OS). Results of the software developed in this thesis show a significant improvement in the performance and usability of the EMG/EGT cursor control HCI.

Types and amplitudes of touch as reinforcing stimuli in operant conditioning of infant behavior

The reinforcing effects of diverse tactile stimuli were examined in this study. The study had two purposes. First, this study expanded on the Pelaez-Nogueras, Field, Gewirtz, Cigales, Gonzalez, Sanchez and Clasky (1997) finding that stroking increases infants' gaze duration, and smiling and vocalization frequencies more than tickling/poking. Instead of presenting poking and tickling as a single stimulus combination, this study separated poking and tickling in order to measure the effects of each component separately. Further, the effects of poking, tickling/tapping and stroking intensity (i.e., tactile pressure) were compared by having both mild and intense conditions. Second, this study compared the reinforcing efficacy of mother-delivered tactile stimulation to that of infant-originated tactile exploration. Twelve infants from 2- to 5-months of age participated in this study. The experiment was conducted using a repeated measures A-B-A-C-A-D reversal design. The A phases signified baselines and reversals. The B, C, and D phases consisted of alternating treatments (either mild stroking vs. mild poking vs. mild tickling/tapping, intense stroking vs. intense poking vs. intense tickling/tapping, or mother-delivered tactile stimulation vs. infant-originated tactile exploration). Three experimental hypotheses were assessed: (1) infant leg kick rate would be greater when it produced stroking or tickling/tapping (presumptive positive reinforcers), than when it produced poking (a possible punisher), regardless of tactile pressure; (2) infant leg kick rate would be greater when it produced a more intense level of stroking or tickling/tapping and lower when it produced intense poking compared to mild poking; (3) infant leg-kick rate would be greater for mother-delivered tactile stimulation than for infant-originated tactile exploration. Visual inspection and inferential statistical methods were used to analyze the results. The data supported the first two hypotheses. Mixed support emerged for the third hypothesis. This study made several important contributions to the field of psychology. First, this was the first study to quantify the pressure of tactile stimulation, via a pressure meter developed by the researcher. Additionally, the results of this study yielded valuable information about the effects of different modalities of touch. ^

Flanerie: A post -reflexive mode of urban research inquiry

Urban ethnographic studies in social science usually proceed from within a pre-figured research framework that guides field activity and filters what is considered see-worthy and study-worthy to those phenomena which are in some way necessary for the fulfillment of specific research agendas. Reliance on formalized ethnographic research methods to explore the city, and the reflex to intellectualize that which is observed by using them, frequently leads to the neglect of much of what comprises the ethnographic richness of city-life through an overly-contemplative intellectuocentrism. ^ Flânerie, an avocational, proto-sociological ethnographic mode of urban exploration and observation originating in early-nineteenth-century Paris, is representative of an alternate approach to exploration and study of the city—one less subject to a compressed horizon of ethnographic experience. It affords insight into a broad range of phenomena taking place in interstitial urban space and time which often escape the instrumentally overly-determined gaze of professionalist research inquiry. ^ The present work addressed the fact that the concept of flânerie , though occasionally invoked in discussions of methodology in urban sociology and cultural studies, had not been subjected to a systematic attempt to relate it to the research methods of these disciplines. Though as a typological figure the flâneur is thought to have disappeared long ago, practices represented by the concept continue to be engaged in by persons inside and outside of academia who approach city life and social science with a perspective somewhat transcending the reified dichotomies of private-professional and art-and-science. This study transferred aspects of the experience of the city embodied in the flâneur to the realm of ethnographic urban studies and the building of grounded social theory, finding flânerie to be crucial to the development of a refined knowledge of urban life, and an important means by which previously overlooked social phenomena and silenced avenues of research inquiry might be exposed. ^ The conclusions reached found a “flâneuristic” approach to the city as being beneficial to research practice and the vitalization of urban ethnographic inquiry, by affording opportunities for exposure to, and immersion within, an expanded range of quotidian social phenomena that have frequently escaped the academicist gaze. ^

A multimodal and adaptive real-time human -computer interface for peoples with severe motor disability

This dissertation introduces the design of a multimodal, adaptive real-time assistive system as an alternate human computer interface that can be used by individuals with severe motor disabilities. The proposed design is based on the integration of a remote eye-gaze tracking system, voice recognition software, and a virtual keyboard. The methodology relies on a user profile that customizes eye gaze tracking using neural networks. The user profiling feature facilitates the notion of universal access to computing resources for a wide range of applications such as web browsing, email, word processing and editing. ^ The study is significant in terms of the integration of key algorithms to yield an adaptable and multimodal interface. The contributions of this dissertation stem from the following accomplishments: (a) establishment of the data transport mechanism between the eye-gaze system and the host computer yielding to a significantly low failure rate of 0.9%; (b) accurate translation of eye data into cursor movement through congregate steps which conclude with calibrated cursor coordinates using an improved conversion function; resulting in an average reduction of 70% of the disparity between the point of gaze and the actual position of the mouse cursor, compared with initial findings; (c) use of both a moving average and a trained neural network in order to minimize the jitter of the mouse cursor, which yield an average jittering reduction of 35%; (d) introduction of a new mathematical methodology to measure the degree of jittering of the mouse trajectory; (e) embedding an onscreen keyboard to facilitate text entry, and a graphical interface that is used to generate user profiles for system adaptability. ^ The adaptability nature of the interface is achieved through the establishment of user profiles, which may contain the jittering and voice characteristics of a particular user as well as a customized list of the most commonly used words ordered according to the user's preferences: in alphabetical or statistical order. This allows the system to successfully provide the capability of interacting with a computer. Every time any of the sub-system is retrained, the accuracy of the interface response improves even more. ^

Distribution of light in the human retina under natural viewing conditions

Age-related macular degeneration (AMD) is the leading cause of blindness inAmerica. The fact that AMD wreaks most of the damage in the center of the retina raises the question of whether light, integrated over long periods, is more concentrated in the macula. A method, based on eye-tracking, was developed to measure the distribution of light in the retina under natural viewing conditions. The hypothesis was that integrated over time, retinal illumination peaked in the macula. Additionally a possible relationship between age and retinal illumination was investigated. The eye tracker superimposed the subject's gaze position on a video recorded by a scene camera. Five informed subjects were employed in feasibility tests, and 58 naïve subjects participated in 5 phases. In phase 1 the subjects viewed a gray-scale image. In phase 2, they observed a sequence of photographic images. In phase 3 they viewed a video. In phase 4, they worked on a computer; in phase 5, the subjects walked around freely. The informed subjects were instructed to gaze at bright objects in the field of view and then at dark objects. Naïve subjects were allowed to gaze freely for all phases. Using the subject's gaze coordinates, and the video provided by the scene camera, the cumulative light distribution on the retina was calculated for ∼15° around the fovea. As expected for control subjects, cumulative retinal light distributions peaked and dipped in the fovea when they gazed at bright or dark objects respectively. The light distribution maps obtained from the naïve subjects presented a tendency to peak in the macula for phases 1, 2, and 3, a consistent tendency in phase 4 and a variable tendency in phase 5. The feasibility of using an eye-tracker system to measure the distribution of light in the retina was demonstrated, thus helping to understand the role played by light exposure in the etiology of AMD. Results showed that a tendency for light to peak in the macula is a characteristic of some individuals and of certain tasks. In these situations, risk of AMD could be increased. No significant difference was observed based on age.

Écfrasis y Lecturas Icono-textuales en las "Leyendas" de Gustavo Adolfo Bécquer

The relationship between literature and the visual arts is ancient and it has been studied from different conceptual frames. Scholars agree that both have a descriptive function and therefore share the common goal of portraying a fictional or nonfictional reality. Based on this correspondence between two different modes of artistic expression, the Roman poet Horace coined the well-known simile ut pictura poesis --as is painting so is in poetry-- which in turn functions as the theoretical underpinning of ekphrasis, a rhetorical device through which one medium of art tries to describe the essence and form of another medium of art, with the purpose of enhancing the original work described. Spanish post-romantic poet and writer Gustavo Adolfo Bécquer (1836-1870) mastered this rhetorical strategy by expertly weaving all of his artistic interests into his prose. The purpose of this dissertation is to analyze how Bécquer makes his readers both see and hear through his prose. My semiotic research encompasses the various forms of ekphrasis used by Bécquer in the “Leyendas”. It shows how both images and symbols produce in readers sensory experiences that enhance their role as active participants in the creation of meaning. Thus, Bécquer´s prose is like a painting which not only tells a story, but also reflects reality through the eyes of the reader’s imagination. By using these ekphrastic strategies in his collection of short stories, Bécquer makes words, paintings, and music converge and collide with iconography, visual culture, and intertextuality. These components must be read, seen, heard, and understood to be more than just complementary to the text, but rather crucial elements, equal in importance to verbal expression. This analysis shows how Bécquer’s “Leyendas” not only tackle notions such as fantasy, figuration, and imagination, but also the importance of the reader´s gaze. Bécquer integrates processes such as imaginative action, iconization and visualization, into a semantic web whereby the reader creates his own particular hermeneutic image.

Preschooler’s Comprehension of Spatial Terms

This study aims to understand individual differences in preschooler’s early comprehension of spatial language. Spatial language is defined as terms describing location, direction, shape, dimension, features, orientation, and quantity (e.g location, shape). Spatial language is considered to be one of the important factors in the development of spatial reasoning in the preschool years (Pruden, Levine, & Huttenlocher, 2011). In recent years, research has shown spatial reasoning is an important predictor of successes in STEM (Science, Technology, Engineering, and Mathematics) fields (e.g. Shea, Lubinski & Benbow, 2001; Wai, Lubinksi &Benbow, 2009). The current study focuses on when children begin to comprehend spatial terms, while previous work has mainly focused on production of spatial language. Identifying when children begin to comprehend spatial terms could lead to a better understanding of how spatial reasoning develops. We use the Intermodal Preferential Looking paradigm (IPLP) to examine three-year-old children’s ability to map spatial terms to visual representations. Fourteen spatial terms were used to test these abilities (e.g. bottom, diamond, longer). For each test trial children were presented with two different stimuli simultaneously on the left and right sides of a television screen. A female voice prompted the child to find the target spatial relation (e.g. “can you find the boy pointing to the bottom of the window”; Figure 1). A Tobii X60 eye-tracker was used to record the child’s eye gaze for each trial. For each child the proportion of looking to the target image divided by their total looking during the trial was calculated; this served as the dependent variable. Proportions above .50 indicated that the child had correctly mapped the spatial term to the target image. Preliminary data shows that the number of words comprehended in the IPLP task is correlated to parental report of the child’s comprehension of spatial terms (r[14]=.500, p<.05).

Preschooler’s Comprehension of Spatial Terms

This study aims to understand individual differences in preschooler’s early comprehension of spatial language. Spatial language is defined as terms describing location, direction, shape, dimension, features, orientation, and quantity (e.g location, shape). Spatial language is considered to be one of the important factors in the development of spatial reasoning in the preschool years (Pruden, Levine, & Huttenlocher, 2011). In recent years, research has shown spatial reasoning is an important predictor of successes in STEM (Science, Technology, Engineering, and Mathematics) fields (e.g. Shea, Lubinski & Benbow, 2001; Wai, Lubinksi &Benbow, 2009). The current study focuses on when children begin to comprehend spatial terms, while previous work has mainly focused on production of spatial language. Identifying when children begin to comprehend spatial terms could lead to a better understanding of how spatial reasoning develops. We use the Intermodal Preferential Looking paradigm (IPLP) to examine three-year-old children’s ability to map spatial terms to visual representations. Fourteen spatial terms were used to test these abilities (e.g. bottom, diamond, longer). For each test trial children were presented with two different stimuli simultaneously on the left and right sides of a television screen. A female voice prompted the child to find the target spatial relation (e.g. “can you find the boy pointing to the bottom of the window”; Figure 1). A Tobii X60 eye-tracker was used to record the child’s eye gaze for each trial. For each child the proportion of looking to the target image divided by their total looking during the trial was calculated; this served as the dependent variable. Proportions above .50 indicated that the child had correctly mapped the spatial term to the target image. Preliminary data shows that the number of words comprehended in the IPLP task is correlated to parental report of the child’s comprehension of spatial terms (r[14]=.500, p<.05).

Mother-Infant Reciprocity and the Development of Joint Attention in 12 Month Old Infants

Social contingency is the ability to connect social stimuli, such as those behaviors performed by oneself and those performed by others. Detecting social contingencies occurs by means of reciprocity through shared experiences with others. Reciprocity denotes a circumstance in which two individuals participate in a collaborative exchange, and is distinguished from an event in which two individuals engage in separate, unrelated activities. Specifically, reciprocity incorporates joint attention (JA), which occurs when two individuals simultaneously and visually attend to the same item. JA is facilitated by gazing and pointing, whereby one individual initiates the action and the second individual follows suit by, for example, gaze-following. However, little is known about the role the mother may play in the development of JA. The purpose of our study was to investigate social contingency between mothers and infants engaging in dyadic interactions. Thirty-three 12-month-old typically developing infants (M = 12.2, SD = .19; N = 19 males) were filmed for 10 minutes during free play with their mothers and toys provided by an experimenter. Reciprocity was measured by coding mother-infant interactions when a precise chain of events occurred: (1) mother initiated a bid by introducing a toy/activity or request to the infant, (2) infant accepted the bid/request by engaging in play with the given toy/activity, and (3) mother persisted by continuing to engage in play with said toy/activity. We computed a Pearson Correlation to assess the relation between the mothers’ initiations of JA and their infants’ responses to JA. We found a moderately positive correlation between the two variables (r= 0.37, p<.05). Our findings suggest that reciprocity, an important component of social relationships, during parent-infant dyads may serve as a scaffold for joint attention abilities, which have been linked to social and language development.

Distribution of Light in the Human Retina under Natural Viewing Conditions

Age-related macular degeneration (AMD) is the leading cause of blindness inAmerica. The fact that AMD wreaks most of the damage in the center of the retina raises the question of whether light, integrated over long periods, is more concentrated in the macula. A method, based on eye-tracking, was developed to measure the distribution of light in the retina under natural viewing conditions. The hypothesis was that integrated over time, retinal illumination peaked in the macula. Additionally a possible relationship between age and retinal illumination was investigated. The eye tracker superimposed the subject’s gaze position on a video recorded by a scene camera. Five informed subjects were employed in feasibility tests, and 58 naïve subjects participated in 5 phases. In phase 1 the subjects viewed a gray-scale image. In phase 2, they observed a sequence of photographic images. In phase 3 they viewed a video. In phase 4, they worked on a computer; in phase 5, the subjects walked around freely. The informed subjects were instructed to gaze at bright objects in the field of view and then at dark objects. Naïve subjects were allowed to gaze freely for all phases. Using the subject’s gaze coordinates, and the video provided by the scene camera, the cumulative light distribution on the retina was calculated for ~15° around the fovea. As expected for control subjects, cumulative retinal light distributions peaked and dipped in the fovea when they gazed at bright or dark objects respectively. The light distribution maps obtained from the naïve subjects presented a tendency to peak in the macula for phases 1, 2, and 3, a consistent tendency in phase 4 and a variable tendency in phase 5. The feasibility of using an eye-tracker system to measure the distribution of light in the retina was demonstrated, thus helping to understand the role played by light exposure in the etiology of AMD. Results showed that a tendency for light to peak in the macula is a characteristic of some individuals and of certain tasks. In these situations, risk of AMD could be increased. No significant difference was observed based on age.