Face Detection and Facial Feature Localization for multi-pose faces and complex backgroundimages

The objective of this thesis work, is to propose an algorithm to detect the faces in a digital image with complex background. A lot of work has already been done in the area of face detection, but drawback of some face detection algorithms is the lack of ability to detect faces with closed eyes and open mouth. Thus facial features form an important basis for detection. The current thesis work focuses on detection of faces based on facial objects. The procedure is composed of three different phases: segmentation phase, filtering phase and localization phase. In segmentation phase, the algorithm utilizes color segmentation to isolate human skin color based on its chrominance properties. In filtering phase, Minkowski addition based object removal (Morphological operations) has been used to remove the non-skin regions. In the last phase, Image Processing and Computer Vision methods have been used to find the existence of facial components in the skin regions.This method is effective on detecting a face region with closed eyes, open mouth and a half profile face. The experiment’s results demonstrated that the detection accuracy is around 85.4% and the detection speed is faster when compared to neural network method and other techniques.

Hand Gesture Detection & Recognition System

The project introduces an application using computer vision for Hand gesture recognition. A camera records a live video stream, from which a snapshot is taken with the help of interface. The system is trained for each type of count hand gestures (one, two, three, four, and five) at least once. After that a test gesture is given to it and the system tries to recognize it.A research was carried out on a number of algorithms that could best differentiate a hand gesture. It was found that the diagonal sum algorithm gave the highest accuracy rate. In the preprocessing phase, a self-developed algorithm removes the background of each training gesture. After that the image is converted into a binary image and the sums of all diagonal elements of the picture are taken. This sum helps us in differentiating and classifying different hand gestures.Previous systems have used data gloves or markers for input in the system. I have no such constraints for using the system. The user can give hand gestures in view of the camera naturally. A completely robust hand gesture recognition system is still under heavy research and development; the implemented system serves as an extendible foundation for future work.

First-principle data-driven models for assessment of motor disorders in Parkinson’s disease

Parkinson’s disease (PD) is an increasing neurological disorder in an aging society. The motor and non-motor symptoms of PD advance with the disease progression and occur in varying frequency and duration. In order to affirm the full extent of a patient’s condition, repeated assessments are necessary to adjust medical prescription. In clinical studies, symptoms are assessed using the unified Parkinson’s disease rating scale (UPDRS). On one hand, the subjective rating using UPDRS relies on clinical expertise. On the other hand, it requires the physical presence of patients in clinics which implies high logistical costs. Another limitation of clinical assessment is that the observation in hospital may not accurately represent a patient’s situation at home. For such reasons, the practical frequency of tracking PD symptoms may under-represent the true time scale of PD fluctuations and may result in an overall inaccurate assessment. Current technologies for at-home PD treatment are based on data-driven approaches for which the interpretation and reproduction of results are problematic.  The overall objective of this thesis is to develop and evaluate unobtrusive computer methods for enabling remote monitoring of patients with PD. It investigates first-principle data-driven model based novel signal and image processing techniques for extraction of clinically useful information from audio recordings of speech (in texts read aloud) and video recordings of gait and finger-tapping motor examinations. The aim is to map between PD symptoms severities estimated using novel computer methods and the clinical ratings based on UPDRS part-III (motor examination). A web-based test battery system consisting of self-assessment of symptoms and motor function tests was previously constructed for a touch screen mobile device. A comprehensive speech framework has been developed for this device to analyze text-dependent running speech by: (1) extracting novel signal features that are able to represent PD deficits in each individual component of the speech system, (2) mapping between clinical ratings and feature estimates of speech symptom severity, and (3) classifying between UPDRS part-III severity levels using speech features and statistical machine learning tools. A novel speech processing method called cepstral separation difference showed stronger ability to classify between speech symptom severities as compared to existing features of PD speech. In the case of finger tapping, the recorded videos of rapid finger tapping examination were processed using a novel computer-vision (CV) algorithm that extracts symptom information from video-based tapping signals using motion analysis of the index-finger which incorporates a face detection module for signal calibration. This algorithm was able to discriminate between UPDRS part III severity levels of finger tapping with high classification rates. Further analysis was performed on novel CV based gait features constructed using a standard human model to discriminate between a healthy gait and a Parkinsonian gait. The findings of this study suggest that the symptom severity levels in PD can be discriminated with high accuracies by involving a combination of first-principle (features) and data-driven (classification) approaches. The processing of audio and video recordings on one hand allows remote monitoring of speech, gait and finger-tapping examinations by the clinical staff. On the other hand, the first-principles approach eases the understanding of symptom estimates for clinicians. We have demonstrated that the selected features of speech, gait and finger tapping were able to discriminate between symptom severity levels, as well as, between healthy controls and PD patients with high classification rates. The findings support suitability of these methods to be used as decision support tools in the context of PD assessment.

Motion Cues Analysis for Parkinson Gait Recognition

Background: Previous assessment methods for PG recognition used sensor mechanisms for PG that may cause discomfort. In order to avoid stress of applying wearable sensors, computer vision (CV) based diagnostic systems for PG recognition have been proposed. Main constraints in these methods are the laboratory setup procedures: Novel colored dresses for the patients were specifically designed to segment the test body from a specific colored background. Objective: To develop an image processing tool for home-assessment of Parkinson Gait(PG) by analyzing motion cues extracted during the gait cycles. Methods: The system is based on the idea that a normal body attains equilibrium during the gait by aligning the body posture with the axis of gravity. Due to the rigidity in muscular tone, persons with PD fail to align their bodies with the axis of gravity. The leaned posture of PD patients appears to fall forward. Whereas a normal posture exhibits a constant erect posture throughout the gait. Patients with PD walk with shortened stride angle (less than 15 degrees on average) with high variability in the stride frequency. Whereas a normal gait exhibits a constant stride frequency with an average stride angle of 45 degrees. In order to analyze PG, levodopa-responsive patients and normal controls were videotaped with several gait cycles. First, the test body is segmented in each frame of the gait video based on the pixel contrast from the background to form a silhouette. Next, the center of gravity of this silhouette is calculated. This silhouette is further skeletonized from the video frames to extract the motion cues. Two motion cues were stride frequency based on the cyclic leg motion and the lean frequency based on the angle between the leaned torso tangent and the axis of gravity. The differences in the peaks in stride and lean frequencies between PG and normal gait are calculated using Cosine Similarity measurements. Results: High cosine dissimilarity was observed in the stride and lean frequencies between PG and normal gait. High variations are found in the stride intervals of PG whereas constant stride intervals are found in the normal gait. Conclusions: We propose an algorithm as a source to eliminate laboratory constraints and discomfort during PG analysis. Installing this tool in a home computer with a webcam allows assessment of gait in the home environment.

Monitoring vegetation on railway embankments : supporting maintenance decisions

The national railway administrations in Scandinavia, Germany, and Austria mainly resort to manual inspections to control vegetation growth along railway embankments. Manually inspecting railways is slow and time consuming. A more worrying aspect concerns the fact that human observers are often unable to estimate the true cover of vegetation on railway embankments. Further human observers often tend to disagree with each other when more than one observer is engaged for inspection. Lack of proper techniques to identify the true cover of vegetation even result in the excess usage of herbicides; seriously harming the environment and threating the ecology. Hence work in this study has investigated aspects relevant to human variationand agreement to be able to report better inspection routines. This was studied by mainly carrying out two separate yet relevant investigations.First, thirteen observers were separately asked to estimate the vegetation cover in nine imagesacquired (in nadir view) over the railway tracks. All such estimates were compared relatively and an analysis of variance resulted in a significant difference on the observers’ cover estimates (p<0.05). Bearing in difference between the observers, a second follow-up field-study on the railway tracks was initiated and properly investigated. Two railway segments (strata) representingdifferent levels of vegetationwere carefully selected. Five sample plots (each covering an area of one-by-one meter) were randomizedfrom each stratumalong the rails from the aforementioned segments and ten images were acquired in nadir view. Further three observers (with knowledge in the railway maintenance domain) were separately asked to estimate the plant cover by visually examining theplots. Again an analysis of variance resulted in a significant difference on the observers’ cover estimates (p<0.05) confirming the result from the first investigation.The differences in observations are compared against a computer vision algorithm which detects the "true" cover of vegetation in a given image. The true cover is defined as the amount of greenish pixels in each image as detected by the computer vision algorithm. Results achieved through comparison strongly indicate that inconsistency is prevalent among the estimates reported by the observers. Hence, an automated approach reporting the use of computer vision is suggested, thus transferring the manual inspections into objective monitored inspections

Motion cue analysis for parkinsonian gait recognition

This paper presents a computer-vision based marker-free method for gait-impairment detection in Patients with Parkinson's disease (PWP). The system is based upon the idea that a normal human body attains equilibrium during the gait by aligning the body posture with Axis-of-Gravity (AOG) using feet as the base of support. In contrast, PWP appear to be falling forward as they are less-able to align their body with AOG due to rigid muscular tone. A normal gait exhibits periodic stride-cycles with stride-angle around 45o between the legs, whereas PWP walk with shortened stride-angle with high variability between the stride-cycles. In order to analyze Parkinsonian-gait (PG), subjects were videotaped with several gait-cycles. The subject's body was segmented using a color-segmentation method to form a silhouette. The silhouette was skeletonized for motion cues extraction. The motion cues analyzed were stride-cycles (based on the cyclic leg motion of skeleton) and posture lean (based on the angle between leaned torso of skeleton and AOG). Cosine similarity between an imaginary perfect gait pattern and the subject gait patterns produced 100% recognition rate of PG for 4 normal-controls and 3 PWP. Results suggested that the method is a promising tool to be used for PG assessment in home-environment.

Visual feedback control of a robot in an unknown environment (learning control using neural networks)

In this paper, a visual feedback control approach based on neural networks is presented for a robot with a camera installed on its end-effector to trace an object in an unknown environment. First, the one-to-one mapping relations between the image feature domain of the object to the joint angle domain of the robot are derived. Second, a method is proposed to generate a desired trajectory of the robot by measuring the image feature parameters of the object. Third, a multilayer neural network is used for off-line learning of the mapping relations so as to produce on-line the reference inputs for the robot. Fourth, a learning controller based on a multilayer neural network is designed for realizing the visual feedback control of the robot. Last, the effectiveness of the present approach is verified by tracing a curved line using a 6-degrees-of-freedom robot with a CCD camera installed on its end-effector. The present approach does not necessitate the tedious calibration of the CCD camera and the complicated coordinate transformations.

Automatic age estimation based on facial aging patterns

While recognition of most facial variations, such as identity, expression, and gender, has been extensively studied, automatic age estimation has rarely been explored. In contrast to other facial variations, aging variation presents several unique characteristics which make age estimation a challenging task. This paper proposes an automatic age estimation method named AGES (AGing pattErn Subspace). The basic idea is to model the aging pattern, which is defined as the sequence of a particular individual's face images sorted in time order, by constructing a representative subspace. The proper aging pattern for a previously unseen face image is determined by the projection in the subspace that can reconstruct the face image with minimum reconstruction error, while the position of the face image in that aging pattern will then indicate its age. In the experiments, AGES and its variants are compared with the limited existing age estimation methods (WAS and AAS) and some well-established classification methods (kNN, BP, C4.5, and SVM). Moreover, a comparison with human perception ability on age is conducted. It is interesting to note that the performance of AGES is not only significantly better than that of all the other algorithms, but also comparable to that of the human observers.

Simultaneous estimation of optical flow and object state: A modified approach to optical flow calculation

Differential optical flow methods are widely used within the computer vision community. They are classified as being either local, as in the Lucas-Kanade method, or global, as in the Horn-Schunck technique. As the physical dynamics of an object is inherently coupled into the behavior of its image in the video stream, in this paper, we use such dynamic parameter information in calculating optical flow when tracking a moving object using a video stream. Indeed, we use a modified error function in the minimization that contains physical parameter information. Further, the refined estimates of optical flow is used for better estimation of the physical parameters of the object in the simultaneous estimation of optical flow and object state(SEOS).

Distance-driven fusion of gait and face for human identification in video

Gait and face are two important biometrics for human identification. Complementary properties of these two biometrics suggest fusion of them. The relationship between gait and face in the fusion is affected by the subject-to-camera distance. On the one hand, gait is a suitable biometric trait for human recognition at a distance. On the other hand, face recognition is more reliable when the subject is close to the camera. This paper proposes an adaptive fusion method called distance-driven fusion to combine gait and face for human identification in video. Rather than predefined fixed fusion rules, distance-driven fusion dynamically adjusts its rule according to the subject-to-camera distance in real time. Experimental results show that distance-driven fusion performs better than not only single biometric, but also the conventional
static fusion rules including MEAN, PRODUCT, MIN, and MAX.

Individual discriminative subspace for face recognition under uncontrolled conditions

Most face recognition (FR) algorithms require the face images to satisfy certain restrictions in various aspects like view angle, illumination, occlusion, etc. But what is needed in general is the techniques that can recognize any face images recognizable by human beings. This paper provides one potential solution to this problem. A method named Individual Discriminative Subspace (IDS) is proposed for robust face recognition under uncontrolled conditions. IDS is the subspace where only the images from one particular person converge around the origin while those from others scatter. Each IDS can be used to distinguish one individual from others. There is no restriction on the face images fed into the algorithm, which makes it practical for real-life applications. In the experiments, IDS is tested on two large face databases with extensive variations and performs significantly better than 12 existing FR techniques.

Wavelets/multiwavelets bases and correspondence estimation problem : an analytic study

Correspondence estimation in one of the most active research areas in the field of computer vision and number of techniques has been proposed, possessing both advantages and shortcomings. Among the techniques reported, multiresolution analysis based stereo correspondence estimation has gained lot of research focus in recent years. Although, the most widely employed medium for multiresolution analysis is wavelets and multiwavelets bases, however, relatively little work has been reported in this context. In this work we have tried to address some of the issues regarding the work done in this domain and the inherited shortcomings. In the light of these shortcomings, we propose a new technique to overcome some of the flaws that could have significantly impact on the algorithm performance and has not been addressed in the earlier propositions. Proposed algorithm uses multiresolution analysis enforced with wavelets/multiwavelts transform modulus maxima to establish correspondences between the stereo pair of images. Variety of wavelets and multiwavelets bases, possessing distinct properties such as orthogonality, approximation order, short support and shape are employed to analyse their effect on the performance of correspondence estimation. The idea is to provide knowledge base to understand and establish relationships between wavelets and multiwavelets properties and their effect on the quality of stereo correspondence estimation.

Human action recognition based on 3D SIFT and LDA model

How to recognize human action from videos captured by modern cameras efficiently and effectively is a challenge in real applications. Traditional methods which need professional analysts are facing a bottleneck because of their shortcomings. To cope with the disadvantage, methods based on computer vision techniques, without or with only a few human interventions, have been proposed to analyse human actions in videos automatically. This paper provides a method combining the three dimensional Scale Invariant Feature Transform (SIFT) detector and the Latent Dirichlet Allocation (LDA) model for human motion analysis. To represent videos effectively and robustly, we extract the 3D SIFT descriptor around each interest point, which is sampled densely from 3D Space-time video volumes. After obtaining the representation of each video frame, the LDA model is adopted to discover the underlying structure-the categorization of human actions in the collection of videos. Public available standard datasets are used to test our method. The concluding part discusses the research challenges and future directions.

Human action recognition based on pyramid histogram of oriented gradients

Human action recognition has been attracted lots of interest from computer vision researchers due to its various promising applications. In this paper, we employ Pyramid Histogram of Orientation Gradient (PHOG) to characterize human figures for action recognition. Comparing to silhouette-based features, the PHOG descriptor does not require extraction of human silhouettes or contours. Two state-space models, i.e.; Hidden Markov Model (HMM) and Conditional Random Field (CRF), are adopted to model the dynamic human movement. The proposed PHOG descriptor and the state-space models with respect to different parameters are tested using a standard dataset. We also testify the robustness of the method with respect to various unconstrained conditions and viewpoints. Promising experimental result demonstrates the effectiveness and robustness of our proposed method.

Polar transformation system for offline handwritten character recognition

Offline handwritten recognition is an important automated process in pattern recognition and computer vision field. This paper presents an approach of polar coordinate-based handwritten recognition system involving Support Vector Machines (SVM) classification methodology to achieve high recognition performance. We provide comparison and evaluation for zoning feature extraction methods applied in Polar system. The recognition results we proposed were trained and tested by using SVM with a set of 650 handwritten character images. All the input images are segmented (isolated) handwritten characters. Compared with Cartesian based handwritten recognition system, the recognition rate is more stable and improved up to 86.63%.