Real-time lane detection on suburban streets using visual cue integration

The detection of lane boundaries on suburban streets using images obtained from video constitutes a challenging task. This is mainly due to the difficulties associated with estimating the complex geometric structure of lane boundaries, the quality of lane markings as a result of wear, occlusions by traffic, and shadows caused by road-side trees and structures. Most of the existing techniques for lane boundary detection employ a single visual cue and will only work under certain conditions and where there are clear lane markings. Also, better results are achieved when there are no other onroad objects present. This paper extends our previous work and discusses a novel lane boundary detection algorithm specifically addressing the abovementioned issues through the integration of two visual cues. The first visual cue is based on stripe-like features found on lane lines extracted using a two-dimensional symmetric Gabor filter. The second visual cue is based on a texture characteristic determined using the entropy measure of the predefined neighbourhood around a lane boundary line. The visual cues are then integrated using a rulebased classifier which incorporates a modified sequential covering algorithm to improve robustness. To separate lane boundary lines from other similar features, a road mask is generated using road chromaticity values estimated from CIE L*a*b* colour transformation. Extraneous points around lane boundary lines are then removed by an outlier removal procedure based on studentized residuals. The lane boundary lines are then modelled with Bezier spline curves. To validate the algorithm, extensive experimental evaluation was carried out on suburban streets and the results are presented. 

Color to Gray: Visual Cue Preservation

Both commercial and scientific applications often need to transform color images into gray-scale images, e. g., to reduce the publication cost in printing color images or to help color blind people see visual cues of color images. However, conventional color to gray algorithms are not ready for practical applications because they encounter the following problems: 1) Visual cues are not well defined so it is unclear how to preserve important cues in the transformed gray-scale images; 2) some algorithms have extremely high time cost for computation; and 3) some require human-computer interactions to have a reasonable transformation. To solve or at least reduce these problems, we propose a new algorithm based on a probabilistic graphical model with the assumption that the image is defined over a Markov random field. Thus, color to gray procedure can be regarded as a labeling process to preserve the newly well-defined visual cues of a color image in the transformed gray-scale image. Visual cues are measurements that can be extracted from a color image by a perceiver. They indicate the state of some properties of the image that the perceiver is interested in perceiving. Different people may perceive different cues from the same color image and three cues are defined in this paper, namely, color spatial consistency, image structure information, and color channel perception priority. We cast color to gray as a visual cue preservation procedure based on a probabilistic graphical model and optimize the model based on an integral minimization problem. We apply the new algorithm to both natural color images and artificial pictures, and demonstrate that the proposed approach outperforms representative conventional algorithms in terms of effectiveness and efficiency. In addition, it requires no human-computer interactions.

Improved Biometric Authentication System Using Multimodal Cue Integration

Biometrics is an efficient technology with great possibilities in the area of security system development for official and commercial applications. The biometrics has recently become a significant part of any efficient person authentication solution. The advantage of using biometric traits is that they cannot be stolen, shared or even forgotten. The thesis addresses one of the emerging topics in Authentication System, viz., the implementation of Improved Biometric Authentication System using Multimodal Cue Integration, as the operator assisted identification turns out to be tedious, laborious and time consuming. In order to derive the best performance for the authentication system, an appropriate feature selection criteria has been evolved. It has been seen that the selection of too many features lead to the deterioration in the authentication performance and efficiency. In the work reported in this thesis, various judiciously chosen components of the biometric traits and their feature vectors are used for realizing the newly proposed Biometric Authentication System using Multimodal Cue Integration. The feature vectors so generated from the noisy biometric traits is compared with the feature vectors available in the knowledge base and the most matching pattern is identified for the purpose of user authentication. In an attempt to improve the success rate of the Feature Vector based authentication system, the proposed system has been augmented with the user dependent weighted fusion technique.

The relationship of hearing-impaired children's scores on the Beery Visual-Motor Integration Test and the reading subtests of the American School Achievement Test

This paper discusses visual-motor tests and reading tests for hearing impaired children.

The reliability of two visual motor integration tests used with health adults

Occupational therapists often assess the visual motor integration (VMI) skills of children, adults, and the elderly, which are parts of the Body Functions and Structures of the International Classification of Functioning, Disability and Health. Objective. As it is imperative that therapists use tests and measures with strong psychometric properties, this study aims to examine the reliability of two VMI tests used with adults. Method. Sixty-one healthy adults, 18 males and 43 females, with an average age of 31.82 years, completed the Developmental Test of Visual Motor Integration (DTVMI) and the Full Range Test of Visual Motor Integration (FRTVMI). The Cronbach's alpha coefficient was used to examine the tests’ internal consistency, while the Spearman's rho correlation was used to evaluate the test–retest reliability, intrarater reliability, and interrater reliability of the two VMI tests. Results. The Cronbach's alpha coefficient for the DTVMI and FRTVMI was 0.66 and 0.80, respectively. The test–retest reliability coefficient was 0.77 (p < .001) for the DTVMI and 0.61 (p < .001) for the FRTVMI. The interrater reliability correlation was significant for both DTVMI at 0.79 (p < .001) and FRTVMI at 0.80 (p < .001). The DTVMI intrarater reliability correlation result was 0.95 (p < .001) and the FRTVMI at 0.87 (p < .001). Conclusion. Overall, the DTVMI and the FRTVMI exhibited moderate to high levels of reliability when used with a sample of healthy adults. Both VMI tests appear to exhibit reasonable levels of reliability and are recommended for use with adults and the elderly.

Knowledge on road information in sub-urban lane detection via multiple cue integration

Detection of lane boundaries of a road based on the images or video taken by a video capturing device in a suburban environment is a challenging task. In this paper, a novel lane detection algorithm is proposed without considering camera parameters; which robustly detects lane boundaries in real-time especially for sub-urban roads. Initially, the proposed method fits the CIE L*a*b* transformed road chromaticity values (that is a* and b* values) to a bi-variate Gaussian model followed by the classification of road area based on Mahalanobis distance. Secondly, the classified road area acts as an arbitrary shaped region of interest (AROI) in order to extract blobs resulting from the filtered image by a two dimensional Gabor filter. This is considered as the first cue of images. Thirdly, another cue of images was employed in order to obtain an entropy image. Moreover, results from the color based image cue and entropy image cue were integrated following an outlier removing process. Finally, the correct road lane points are fitted with Bezier splines which act as control points that can form arbitrary shapes. The algorithm was implemented and experiments were carried out on sub-urban roads. The results show the effectiveness of the algorithm in producing more accurate lane boundaries on curvatures and other objects on the road.

Convergent validity of two visual motor integration tests

Background: Occupational therapists often assess visual motor integration (VMI) skills. It is, therefore, imperative that therapists use VMI tests with robust measurement properties.

Objective: This study examined the convergent validity of two VMI tests used to assess children, adolescents and adults.

Method: Three groups of healthy participants (n = 153) completed the Beery-Buktenica Developmental Test of Visual-Motor Integration (DTVMI) and the Full Range Test of Visual Motor Integration (FRTVMI). Seventy-three children aged 5-10 years (37 males and 36 females; mean age 7.5 years, SD = 2.20), 19 adolescents aged 11-17 years (8 males and 11 females; 13.1 years, SD = 2.16), and 61 adults (18 males and 43 females; mean age 31.82 years, SD = 11.20) completed the DTVMI and the FRTVMI. Spearman rho correlation coefficients were used to investigate whether each pair of the VMI test scores for each of the three participant age groups were associated.

Results: The Spearman rho correlation coefficients between all three versions of the DTVMI and FRTVMI were statistically significant. For the child group, the correlation coefficient was rho = 0.70 (p<0.000), while the correlation between the VMI scores obtained by the adolescent group on the two tests was rho = 0.77 (p<0.000). For the adult participant group, the correlation coefficient between the DTVMI and the FRTVMI was rho = 0.70 (p<0.000).

Conclusion: The VMI scores obtained by the three participant age groups on the DTVMI and the FRTVMI were all significantly correlated with each other. Overall, the DTVMI and the FRTVMI exhibited large levels of convergent validity with each other, indicating that the two tests appear to measure similar visual-motor integration constructs.

The reliability of two visual motor integration tests used with children

Occupational therapists often assess the visual motor integration (VMI) skills of children and young people. It is important that therapists use tools with strong psychometric properties. This study aims to examine the reliability of 2 VMI tests. Ninety-two children between the ages of 5 and 17 years (response rate of 31%) completed 2 VMI tests: the Developmental Test of Visual Motor Integration (DTVMI) and the Full Range Test of Visual Motor Integration (FRTVMI). Cronbach's alpha coefficient was used to examine the internal consistency of the 2 VMI tests whereas Spearman's rho correlation was used to evaluate the test–retest reliability, intrarater reliability, and interrater reliability of the 2 VMI tests. The Cronbach's alpha coefficient for the DTVMI was .82 and .72 for the FRTVMI. The test–retest reliability coefficient was .73 (p = .000) for the DTVMI and .49 (p = .05) for the FRTVMI. The interrater correlation was significant for both the DTVMI at .94 (p = .000) and FRTVMI at .68 (p = .001). The DTVMI intrarater reliability correlation result was .90 (p = .000) and the FRTVMI at .85 (p = .000). Overall, the DTVMI exhibited a higher level of reliability than the FRTVMI. Both VMI tests appear to exhibit reasonable levels of reliability and are recommended for use with children and young people.

VISUAL-INERTIAL INTEGRATION FOR HUMAN MOTION TRACKING AND NAVIGATION IN FREE-LIVING ENVIRONMENTS

The aging population has become a burning issue for all modern societies around the world recently. There are two important issues existing now to be solved. One is how to continuously monitor the movements of those people having suffered a stroke in natural living environment for providing more valuable feedback to guide clinical interventions. The other one is how to guide those old people effectively when they are at home or inside other buildings and to make their life easier and convenient. Therefore, human motion tracking and navigation have been active research fields with the increasing number of elderly people. However, motion capture has been extremely challenging to go beyond laboratory environments and obtain accurate measurements of human physical activity especially in free-living environments, and navigation in free-living environments also poses some problems such as the denied GPS signal and the moving objects commonly presented in free-living environments. This thesis seeks to develop new technologies to enable accurate motion tracking and positioning in free-living environments. This thesis comprises three specific goals using our developed IMU board and the camera from the imaging source company: (1) to develop a robust and real-time orientation algorithm using only the measurements from IMU; (2) to develop a robust distance estimation in static free-living environments to estimate people’s position and navigate people in static free-living environments and simultaneously the scale ambiguity problem, usually appearing in the monocular camera tracking, is solved by integrating the data from the visual and inertial sensors; (3) in case of moving objects viewed by the camera existing in free-living environments, to firstly design a robust scene segmentation algorithm and then respectively estimate the motion of the vIMU system and moving objects. To achieve real-time orientation tracking, an Adaptive-Gain Orientation Filter (AGOF) is proposed in this thesis based on the basic theory of deterministic approach and frequency-based approach using only measurements from the newly developed MARG (Magnet, Angular Rate, and Gravity) sensors. To further obtain robust positioning, an adaptive frame-rate vision-aided IMU system is proposed to develop and implement fast vIMU ego-motion estimation algorithms, where the orientation is estimated in real time from MARG sensors in the first step and then used to estimate the position based on the data from visual and inertial sensors. In case of the moving objects viewed by the camera existing in free-living environments, a robust scene segmentation algorithm is firstly proposed to obtain position estimation and simultaneously the 3D motion of moving objects. Finally, corresponding simulations and experiments have been carried out.

Auditory-visual speech integration by prelinguistic infants: Perception of an emergent consonant in the McGurk effect

The McGurk effect, in which auditory [ba] dubbed onto [go] lip movements is perceived as da or tha, was employed in a real-time task to investigate auditory-visual speech perception in prelingual infants. Experiments 1A and 1B established the validity of real-time dubbing for producing the effect. In Experiment 2, 4(1)/(2)-month-olds were tested in a habituation-test paradigm, in which 2 an auditory-visual stimulus was presented contingent upon visual fixation of a live face. The experimental group was habituated to a McGurk stimulus (auditory [ba] visual [ga]), and the control group to matching auditory-visual [ba]. Each group was then presented with three auditory-only test trials, [ba], [da], and [deltaa] (as in then). Visual-fixation durations in test trials showed that the experimental group treated the emergent percept in the McGurk effect, [da] or [deltaa], as familiar (even though they had not heard these sounds previously) and [ba] as novel. For control group infants [da] and [deltaa] were no more familiar than [ba]. These results are consistent with infants'perception of the McGurk effect, and support the conclusion that prelinguistic infants integrate auditory and visual speech information. (C) 2004 Wiley Periodicals, Inc.

Visual-auditory integration during speech imitation in autism

Children with autistic spectrum disorder (ASD) may have poor audio-visual integration, possibly reflecting dysfunctional 'mirror neuron' systems which have been hypothesised to be at the core of the condition. In the present study, a computer program, utilizing speech synthesizer software and a 'virtual' head (Baldi), delivered speech stimuli for identification in auditory, visual or bimodal conditions. Children with ASD were poorer than controls at recognizing stimuli in the unimodal conditions, but once performance on this measure was controlled for, no group difference was found in the bimodal condition. A group of participants with ASD were also trained to develop their speech-reading ability. Training improved visual accuracy and this also improved the children's ability to utilize visual information in their processing of speech. Overall results were compared to predictions from mathematical models based on integration and non-integration, and were most consistent with the integration model. We conclude that, whilst they are less accurate in recognizing stimuli in the unimodal condition, children with ASD show normal integration of visual and auditory speech stimuli. Given that training in recognition of visual speech was effective, children with ASD may benefit from multi-modal approaches in imitative therapy and language training. (C) 2004 Elsevier Ltd. All rights reserved.

Integration of shading and texture cues: testing the linear model

One of the first attempts to develop a formal model of depth cue integration is to be found in Maloney and Landy's (1989) "human depth combination rule". They advocate that the combination of depth cues by the visual sysetem is best described by a weighted linear model. The present experiments tested whether the linear combination rule applies to the integration of texture and shading. As would be predicted by a linear combination rule, the weight assigned to the shading cue did vary as a function of its curvature value. However, the weight assigned to the texture cue varied systematically as a function of the curvature value of both cues. Here we descrive a non-linear model which provides a better fit to the data. Redescribing the stimuli in terms of depth rather than curvature reduced the goodness of fit for all models tested. These results support the hypothesis that the locus of cue integration is a curvature map, rather than a depth map. We conclude that the linear comination rule does not generalize to the integration of shading and texture, and that for these cues it is likely that integration occurs after the recovery of surface curvature.

Stimulus Integration and Parsing in the Primate Auditory Midbrain

Integrating information from multiple sources is a crucial function of the brain. Examples of such integration include multiple stimuli of different modalties, such as visual and auditory, multiple stimuli of the same modality, such as auditory and auditory, and integrating stimuli from the sensory organs (i.e. ears) with stimuli delivered from brain-machine interfaces.

The overall aim of this body of work is to empirically examine stimulus integration in these three domains to inform our broader understanding of how and when the brain combines information from multiple sources.

First, I examine visually-guided auditory, a problem with implications for the general problem in learning of how the brain determines what lesson to learn (and what lessons not to learn). For example, sound localization is a behavior that is partially learned with the aid of vision. This process requires correctly matching a visual location to that of a sound. This is an intrinsically circular problem when sound location is itself uncertain and the visual scene is rife with possible visual matches. Here, we develop a simple paradigm using visual guidance of sound localization to gain insight into how the brain confronts this type of circularity. We tested two competing hypotheses. 1: The brain guides sound location learning based on the synchrony or simultaneity of auditory-visual stimuli, potentially involving a Hebbian associative mechanism. 2: The brain uses a ‘guess and check’ heuristic in which visual feedback that is obtained after an eye movement to a sound alters future performance, perhaps by recruiting the brain’s reward-related circuitry. We assessed the effects of exposure to visual stimuli spatially mismatched from sounds on performance of an interleaved auditory-only saccade task. We found that when humans and monkeys were provided the visual stimulus asynchronously with the sound but as feedback to an auditory-guided saccade, they shifted their subsequent auditory-only performance toward the direction of the visual cue by 1.3-1.7 degrees, or 22-28% of the original 6 degree visual-auditory mismatch. In contrast when the visual stimulus was presented synchronously with the sound but extinguished too quickly to provide this feedback, there was little change in subsequent auditory-only performance. Our results suggest that the outcome of our own actions is vital to localizing sounds correctly. Contrary to previous expectations, visual calibration of auditory space does not appear to require visual-auditory associations based on synchrony/simultaneity.

My next line of research examines how electrical stimulation of the inferior colliculus influences perception of sounds in a nonhuman primate. The central nucleus of the inferior colliculus is the major ascending relay of auditory information before it reaches the forebrain, and thus an ideal target for understanding low-level information processing prior to the forebrain, as almost all auditory signals pass through the central nucleus of the inferior colliculus before reaching the forebrain. Thus, the inferior colliculus is the ideal structure to examine to understand the format of the inputs into the forebrain and, by extension, the processing of auditory scenes that occurs in the brainstem. Therefore, the inferior colliculus was an attractive target for understanding stimulus integration in the ascending auditory pathway.

Moreover, understanding the relationship between the auditory selectivity of neurons and their contribution to perception is critical to the design of effective auditory brain prosthetics. These prosthetics seek to mimic natural activity patterns to achieve desired perceptual outcomes. We measured the contribution of inferior colliculus (IC) sites to perception using combined recording and electrical stimulation. Monkeys performed a frequency-based discrimination task, reporting whether a probe sound was higher or lower in frequency than a reference sound. Stimulation pulses were paired with the probe sound on 50% of trials (0.5-80 µA, 100-300 Hz, n=172 IC locations in 3 rhesus monkeys). Electrical stimulation tended to bias the animals’ judgments in a fashion that was coarsely but significantly correlated with the best frequency of the stimulation site in comparison to the reference frequency employed in the task. Although there was considerable variability in the effects of stimulation (including impairments in performance and shifts in performance away from the direction predicted based on the site’s response properties), the results indicate that stimulation of the IC can evoke percepts correlated with the frequency tuning properties of the IC. Consistent with the implications of recent human studies, the main avenue for improvement for the auditory midbrain implant suggested by our findings is to increase the number and spatial extent of electrodes, to increase the size of the region that can be electrically activated and provide a greater range of evoked percepts.

My next line of research employs a frequency-tagging approach to examine the extent to which multiple sound sources are combined (or segregated) in the nonhuman primate inferior colliculus. In the single-sound case, most inferior colliculus neurons respond and entrain to sounds in a very broad region of space, and many are entirely spatially insensitive, so it is unknown how the neurons will respond to a situation with more than one sound. I use multiple AM stimuli of different frequencies, which the inferior colliculus represents using a spike timing code. This allows me to measure spike timing in the inferior colliculus to determine which sound source is responsible for neural activity in an auditory scene containing multiple sounds. Using this approach, I find that the same neurons that are tuned to broad regions of space in the single sound condition become dramatically more selective in the dual sound condition, preferentially entraining spikes to stimuli from a smaller region of space. I will examine the possibility that there may be a conceptual linkage between this finding and the finding of receptive field shifts in the visual system.

In chapter 5, I will comment on these findings more generally, compare them to existing theoretical models, and discuss what these results tell us about processing in the central nervous system in a multi-stimulus situation. My results suggest that the brain is flexible in its processing and can adapt its integration schema to fit the available cues and the demands of the task.

Animal Control - What constitutes a reliable cue to stop animal movement ?

Controlling free-ranging livestock requires low-stress cues to alter animal behaviour. Recently modulated sound and electric shock were demonstrated to be effective in controlling free-ranging cattle. In this study the behaviour of 60, 300 kg Belmont Red heifers were observed for behavioural changes when presented cues designed to impede their movement through an alley. The heifers were given an overnight drylot shrink off feed but not drinking water prior to being tested. Individual cattle were allowed to move down a 6.5 m wide alley towards a pen of peers and feed located 71 m from their point of release. Each animal was allowed to move through the alley unimpeded five times to establish a basal behavioural pattern. Animals were then randomly assigned to treatments consisting of sound plus shock, vibration plus shock, a visual cue plus shock, shock by itself and a control. The time each animal required to reach the pen of peers and feed was recorded. If the animal was prevented from reaching the pen of peers and feed by not penetrating through the cue barrier at set points along the alley for at least 60 sec the test was stopped and the animal was returned to peers located behind the release pen. Cues and shock were manually applied from a laptop while animals were observed from a 3.5 m tower located outside the alley. Electric shock, sound, vibration and Global Position System (GPS) hardware were housed in a neck collar. Results and implications will be discussed.