A multimodal and adaptive real-time human -computer interface for peoples with severe motor disability

This dissertation introduces the design of a multimodal, adaptive real-time assistive system as an alternate human computer interface that can be used by individuals with severe motor disabilities. The proposed design is based on the integration of a remote eye-gaze tracking system, voice recognition software, and a virtual keyboard. The methodology relies on a user profile that customizes eye gaze tracking using neural networks. The user profiling feature facilitates the notion of universal access to computing resources for a wide range of applications such as web browsing, email, word processing and editing. ^ The study is significant in terms of the integration of key algorithms to yield an adaptable and multimodal interface. The contributions of this dissertation stem from the following accomplishments: (a) establishment of the data transport mechanism between the eye-gaze system and the host computer yielding to a significantly low failure rate of 0.9%; (b) accurate translation of eye data into cursor movement through congregate steps which conclude with calibrated cursor coordinates using an improved conversion function; resulting in an average reduction of 70% of the disparity between the point of gaze and the actual position of the mouse cursor, compared with initial findings; (c) use of both a moving average and a trained neural network in order to minimize the jitter of the mouse cursor, which yield an average jittering reduction of 35%; (d) introduction of a new mathematical methodology to measure the degree of jittering of the mouse trajectory; (e) embedding an onscreen keyboard to facilitate text entry, and a graphical interface that is used to generate user profiles for system adaptability. ^ The adaptability nature of the interface is achieved through the establishment of user profiles, which may contain the jittering and voice characteristics of a particular user as well as a customized list of the most commonly used words ordered according to the user's preferences: in alphabetical or statistical order. This allows the system to successfully provide the capability of interacting with a computer. Every time any of the sub-system is retrained, the accuracy of the interface response improves even more. ^

The relationship between the use of Academic Text Talk and the comprehension of scientific academic language for diverse second graders

Changing demographics impact our schools as children come from more linguistically and culturally diverse backgrounds. The various social, cultural, and economic backgrounds of the students affect their early language learning experiences which expose them to the academic language needed to succeed in school. Teachers can help students acquire academic language by introducing words that are within their Zone of Proximal Development and increasing exposure to and use of academic language. This study investigated the effects of increasing structured activities for students to orally interact with informational text on their scientific academic language development and comprehension of expository text. ^ The Academic Text Talk activities, designed to scaffold verbalization of new words and ideas, included discussion, retelling, games, and sentence walls. This study also evaluated if there were differences in scientific language proficiency and comprehension between boys and girls, and between English language learners and native English speakers. ^ A quasi-experimental design was used to determine the relationship between increasing students' oral practice with academic language and their academic language proficiency. Second graders (n = 91) from an urban public school participated in two science units over an 8 week period and were pre and post tested using the Woodcock Muñoz Language Survey-Revised and vocabulary tests from the National Energy Education Project. Analysis of covariance was performed on the pre to post scores by treatment group to determine differences in academic language proficiency for students taught using Academic Text Talk compared to students taught using a text-centered method, using the initial Florida Assessment for Instruction in Reading test as a covariate. Students taught using Academic Text Talk multimodal strategies showed significantly greater increases in their pre to posttest means on the Woodcock Muñoz Language Survey-Revised Oral Language Totals and National Energy Education Development Project Vocabulary tests than students taught using the text-centered method, ps < .05. Boys did not show significantly greater increases than girls, nor did English language learners show significantly greater increases than the native English speakers. ^ This study informs the field of reading research by evaluating the effectiveness of a multimodal combination of strategies emphasizing discourse to build academic language.^