A multimodal and adaptive real-time human -computer interface for peoples with severe motor disability

This dissertation introduces the design of a multimodal, adaptive real-time assistive system as an alternate human computer interface that can be used by individuals with severe motor disabilities. The proposed design is based on the integration of a remote eye-gaze tracking system, voice recognition software, and a virtual keyboard. The methodology relies on a user profile that customizes eye gaze tracking using neural networks. The user profiling feature facilitates the notion of universal access to computing resources for a wide range of applications such as web browsing, email, word processing and editing. ^ The study is significant in terms of the integration of key algorithms to yield an adaptable and multimodal interface. The contributions of this dissertation stem from the following accomplishments: (a) establishment of the data transport mechanism between the eye-gaze system and the host computer yielding to a significantly low failure rate of 0.9%; (b) accurate translation of eye data into cursor movement through congregate steps which conclude with calibrated cursor coordinates using an improved conversion function; resulting in an average reduction of 70% of the disparity between the point of gaze and the actual position of the mouse cursor, compared with initial findings; (c) use of both a moving average and a trained neural network in order to minimize the jitter of the mouse cursor, which yield an average jittering reduction of 35%; (d) introduction of a new mathematical methodology to measure the degree of jittering of the mouse trajectory; (e) embedding an onscreen keyboard to facilitate text entry, and a graphical interface that is used to generate user profiles for system adaptability. ^ The adaptability nature of the interface is achieved through the establishment of user profiles, which may contain the jittering and voice characteristics of a particular user as well as a customized list of the most commonly used words ordered according to the user's preferences: in alphabetical or statistical order. This allows the system to successfully provide the capability of interacting with a computer. Every time any of the sub-system is retrained, the accuracy of the interface response improves even more. ^

Design and development of a one-degree-of-freedom force-reflecting manual controller prototype for teleoperation

The present research is carried out from the viewpoint of primarily space applications where human lives may be in danger if they are to work under these conditions. This work proposes to develop a one-degree-of-freedom (1-DOF) force-reflecting manual controller (FRMC) prototype for teleoperation, and address the effects of time delays commonly found in space applications where the control is accomplished via the earth-based control stations. To test the FRMC, a mobile robot (PPRK) and a slider-bar were developed and integrated to the 1-DOF FRMC. The software developed in Visual Basic is able to telecontrol any platform that uses an SV203 controller through the internet and it allows the remote system to send feedback information which may be in the form of visual or force signals. Time delay experiments were conducted on the platform and the effects of time delay on the FRMC system operation have been studied and delineated.