A multimodal and adaptive real-time human -computer interface for peoples with severe motor disability

This dissertation introduces the design of a multimodal, adaptive real-time assistive system as an alternate human computer interface that can be used by individuals with severe motor disabilities. The proposed design is based on the integration of a remote eye-gaze tracking system, voice recognition software, and a virtual keyboard. The methodology relies on a user profile that customizes eye gaze tracking using neural networks. The user profiling feature facilitates the notion of universal access to computing resources for a wide range of applications such as web browsing, email, word processing and editing. ^ The study is significant in terms of the integration of key algorithms to yield an adaptable and multimodal interface. The contributions of this dissertation stem from the following accomplishments: (a) establishment of the data transport mechanism between the eye-gaze system and the host computer yielding to a significantly low failure rate of 0.9%; (b) accurate translation of eye data into cursor movement through congregate steps which conclude with calibrated cursor coordinates using an improved conversion function; resulting in an average reduction of 70% of the disparity between the point of gaze and the actual position of the mouse cursor, compared with initial findings; (c) use of both a moving average and a trained neural network in order to minimize the jitter of the mouse cursor, which yield an average jittering reduction of 35%; (d) introduction of a new mathematical methodology to measure the degree of jittering of the mouse trajectory; (e) embedding an onscreen keyboard to facilitate text entry, and a graphical interface that is used to generate user profiles for system adaptability. ^ The adaptability nature of the interface is achieved through the establishment of user profiles, which may contain the jittering and voice characteristics of a particular user as well as a customized list of the most commonly used words ordered according to the user's preferences: in alphabetical or statistical order. This allows the system to successfully provide the capability of interacting with a computer. Every time any of the sub-system is retrained, the accuracy of the interface response improves even more. ^

Hidden Communication Deficits in Children with Autism and Early Behavioral Interventions

Taking a behavioral systems approach to autism, early hidden communicative deficits are introduced as precursors of autistic development. This paper argues that early identification of communication (language and cognition) impairments followed by intensive behavioral interventions, as early as infancy, may have the most preventive effect on the development of autism.

An incremental multilinear system for human face learning and recognition

This dissertation establishes a novel system for human face learning and recognition based on incremental multilinear Principal Component Analysis (PCA). Most of the existing face recognition systems need training data during the learning process. The system as proposed in this dissertation utilizes an unsupervised or weakly supervised learning approach, in which the learning phase requires a minimal amount of training data. It also overcomes the inability of traditional systems to adapt to the testing phase as the decision process for the newly acquired images continues to rely on that same old training data set. Consequently when a new training set is to be used, the traditional approach will require that the entire eigensystem will have to be generated again. However, as a means to speed up this computational process, the proposed method uses the eigensystem generated from the old training set together with the new images to generate more effectively the new eigensystem in a so-called incremental learning process. In the empirical evaluation phase, there are two key factors that are essential in evaluating the performance of the proposed method: (1) recognition accuracy and (2) computational complexity. In order to establish the most suitable algorithm for this research, a comparative analysis of the best performing methods has been carried out first. The results of the comparative analysis advocated for the initial utilization of the multilinear PCA in our research. As for the consideration of the issue of computational complexity for the subspace update procedure, a novel incremental algorithm, which combines the traditional sequential Karhunen-Loeve (SKL) algorithm with the newly developed incremental modified fast PCA algorithm, was established. In order to utilize the multilinear PCA in the incremental process, a new unfolding method was developed to affix the newly added data at the end of the previous data. The results of the incremental process based on these two methods were obtained to bear out these new theoretical improvements. Some object tracking results using video images are also provided as another challenging task to prove the soundness of this incremental multilinear learning method.

The effect of the use of computer assisted instruction (CAI) on attitudes and computational scores of developmental mathematics students at two inner city schools with predominantly Black enrollment

This study examined the effects of computer assisted instruction (CAI) 1 hour per week for 18 weeks on changes in computational scores and attitudes of developmental mathematics students at schools with predominantly Black enrollment. Comparisons were made between students using CAI with differing software--PLATO, CSR or both together--and students using traditional instruction (TI) only.^ This study was conducted in the Dade County Public School System from February through June 1991, at two senior high schools. The dependent variables, the State Student Assessment Test (SSAT), and the School Subjects Attitude Scales (SSAS), measured students' computational scores and attitudes toward mathematics in 3 categories: interest, usefulness, and difficulty, respectively.^ Univariate analyses of variance were performed on the least squares mean differences from pretest to posttest for testing main effects and interactions. A t-test measured significant main effects and interactions. Results were interpreted at the.01 level of significance.^ Null hypotheses 1, 2, and 3 compared versions of CAI with the control group, for changes in mathematical computation scores measured with the SSAT. It could not be concluded that changes in standardized mathematics test scores of students using CAI with differing software 1 hour per week for 18 class hours combined with TI were significantly higher than changes in test scores for students receiving TI only.^ Null hypotheses 4, 5, and 6 tested the effects of CAI for attitudes toward mathematics for experimental groups against control groups measured with the SSAS. Changes in attitudes toward mathematics of students using CAI with differing software 1 hour per week for 18 class hours combined with TI were not significantly higher than attitude changes for students receiving TI only.^ Teacher effect on students' computational scores was a more influential variable than CAI. No interaction was found between gender and learning method on standardized mathematics test scores (null hypothesis 7). ^

The Effect of Active Student Responding in Computer-Assisted Instruction on Social-Studies Learning by Students with Learning Disabilities: A Brief Report

An alternating treatment design was used to compare the effects of three student response conditions (Clicking, Repeating, and Listening) during computer-assisted instruction on social-studies facts learning and maintenance. Results showed that all students learned and maintained more social-studies facts taught in the Repeating condition followed by the Clicking condition.

The effect of active student responding in computer -assisted instruction on social -studies learning by students with learning disabilities

The purpose of this study was to compare the effects of three student response conditions during computer-assisted instruction on the acquisition and maintenance of social-studies facts. Two of the conditions required active student responding (ASR), whereas the other required an on-task (OT) response. Participants were five fifth-grade students, with learning disabilities enrolled in a private school. An alternating treatments design with a best treatments phase was used to compare the effects of the response procedures on three major dependent measures: same-day tests, next-day tests, and maintenance tests. ^ Each week for six weeks, participants were provided daily one-to-one instruction on sets of 21 unknown social-studies facts using a hypermedia computer program, with a new set of facts being practiced each week. Each set of 21 facts was divided randomly into three conditions: Clicking-ASR, Repeating-ASR, and Listening-OT. Hypermedia lesson began weekly with the concept introduction lesson, followed by practice and testing. Practice and testing occurred four days per week, per set. During Clicking-ASR, student practice involved the selection of a social-studies response by clicking on an item with the mouse on the hypermedia card. Repeating-ASR instruction required students to orally repeat the social-studies facts when prompted by the computer. During Listening-OT, students listened to the social-studies facts being read by the computer. During weeks seven and eight, instruction occurred with seven unknown facts using only the best treatment. ^ Test results show that all for all 5 students, the Repeating-ASR practice procedure resulted in more social-studies facts stated correctly on same-day tests, next-day tests, and one-and two-week maintenance tests. Clicking-ASR was the next most effective procedure. During the seventh and eighth week of instruction when only the best practice condition was implemented, Repeating-ASR produced higher scores than all conditions (including Repeating-ASR) during the first six weeks of the study. ^ The results lend further support to the growing body of literature that demonstrates the positive relation between ASR and student achievement. Much of the ASR literature has focused on the effects of increased ASR during teacher-led or peer-mediated instruction. This study adds a dimension to that research in that it demonstrated the importance of ASR during computer-assisted instruction and further suggests that the type of ASR used during computer-assisted instruction may influence learning. Future research is needed to investigate the effectiveness of other types of ASR during computer-assisted instruction and to identify other fundamental characteristics of an effective computer-assisted instruction. ^

Operant and respondent procedures to establish social stimuli as reinforcers in children with autism

According to the DSM-IV- TR (American Psychiatric Association, 2000), one of the core deficits in autism is in the impairment of social interaction. Some have suggested that underlying these deficits is the reality that individuals with autism do not find social stimuli to be as reinforcing as other types of stimuli (Dawson, 2008). An interesting and growing body of literature supports the notion that symptoms in autism may be caused by a general reduction in social motivation (Chevallier et al., 2012). A review of the literature suggests that social orienting and social motivation are low in individuals with autism, and including social motivation as a target for therapeutic intervention should be pursued (Helt et al., 2008). Through our understanding of learning processes, researchers in behavior analysis and related fields have been able to use conditioning procedures to change the function of neutral or ineffective stimuli, including tokens (Ayllon & Azrin, 1968), facial expressions (Gewirtz & Pelaez-Nogueras, 1992) and praise (Dozier et al., 2012). The current study aimed to use operant and respondent procedures to condition social stimuli that were empirically shown to not be reinforcing prior to conditioning. Further, this study aimed to compare the two procedures in their effectiveness to condition social stimuli to function as reinforcers, and in their maintenance of effects over time. Using a multiple-baseline, multi-element design, one social stimulus was conditioned under each procedure to compare the different response rates following conditioning. Finally, the study sought to determine if conditioning social stimuli to function as reinforcers had any effect on the social functioning of young children with autism. Six children diagnosed with autism between the ages of 18 months and 3 years participated. Results show that the respondent procedure (pairing) resulted in more robust and enduring effects than the operant procedure (Sd procedure). Results of a social communication assessment (ESCS, Mundy et al., 2003) before and after conditioning demonstrate gains in all areas of social communication, particularly in the areas of initiating and responding to joint attention.^

Operant and Respondent Procedures to Establish Social Stimuli as Reinforcers in Children with Autism

According to the DSM-IV- TR (American Psychiatric Association, 2000), one of the core deficits in autism is in the impairment of social interaction. Some have suggested that underlying these deficits is the reality that individuals with autism do not find social stimuli to be as reinforcing as other types of stimuli (Dawson, 2008). An interesting and growing body of literature supports the notion that symptoms in autism may be caused by a general reduction in social motivation (Chevallier et al., 2012). A review of the literature suggests that social orienting and social motivation are low in individuals with autism, and including social motivation as a target for therapeutic intervention should be pursued (Helt et al., 2008). Through our understanding of learning processes, researchers in behavior analysis and related fields have been able to use conditioning procedures to change the function of neutral or ineffective stimuli, including tokens (Ayllon & Azrin, 1968), facial expressions (Gewirtz & Pelaez-Nogueras, 1992) and praise (Dozier et al., 2012). The current study aimed to use operant and respondent procedures to condition social stimuli that were empirically shown to not be reinforcing prior to conditioning. Further, this study aimed to compare the two procedures in their effectiveness to condition social stimuli to function as reinforcers, and in their maintenance of effects over time. Using a multiple-baseline, multi-element design, one social stimulus was conditioned under each procedure to compare the different response rates following conditioning. Finally, the study sought to determine if conditioning social stimuli to function as reinforcers had any effect on the social functioning of young children with autism. Six children diagnosed with autism between the ages of 18 months and 3 years participated. Results show that the respondent procedure (pairing) resulted in more robust and enduring effects than the operant procedure (Sd procedure). Results of a social communication assessment (ESCS, Mundy et al., 2003) before and after conditioning demonstrate gains in all areas of social communication, particularly in the areas of initiating and responding to joint attention.