Learning feature trajectories using Gabor Filter Bank for human activity segmentation and recognition

We describe a novel method for human activity segmentation and interpretation in surveillance applications based on Gabor filter-bank features. A complex human activity is modeled as a sequence of elementary human actions like walking, running, jogging, boxing, hand-waving etc. Since human silhouette can be modeled by a set of rectangles, the elementary human actions can be modeled as a sequence of a set of rectangles with different orientations and scales. The activity segmentation is based on Gabor filter-bank features and normalized spectral clustering. The feature trajectories of an action category are learnt from training example videos using Dynamic Time Warping. The combined segmentation and the recognition processes are very efficient as both the algorithms share the same framework and Gabor features computed for the former can be used for the later. We have also proposed a simple shadow detection technique to extract good silhouette which is necessary for good accuracy of an action recognition technique. © 2008 IEEE.

Orientation integration in detection and discrimination of contrast-modulated patterns

Orientation detection and discrimination thresholds were measured for Gabor ‘envelopes’ formed from contrast-modulation of luminance ‘carriers’. Consistent with previous research differences between carrier and envelope orientation had no effect on sensitivity to envelopes. Using plaid carriers in which the proportion of contrast modulation ‘carried’ by each plaid component was systematically manipulated, it was shown that this tolerance to carrier-envelope orientation difference reflects linear summation across orientation indicative of a single second-stage channel coding for contrast-defined structure. That contrast envelopes did not exhibit linear summation across spatial-frequency, nor across combinations of orientation and spatial-frequency differences, suggests that these second-order channels operate only within certain spatial scales. Using arrays of Gabor micropatterns as carriers in which the orientation distribution of the carriers was manipulated independently of the difference between envelope orientation and mean carrier orientation, it was further demonstrated that the locus of orientation integration must occur prior to envelope detection. In the context of two-stage models that incorporate a non-linearity between the stages, the pattern of results obtained is consistent with the operation of an orientation pooling process between first-stage and second-stage channels, analogous to having all filters of the first-stage feed into all filters of the second-stage within the same spatial-frequency band.

Image semantic classification by using SVM

There exists an enormous gap between low-level visual feature and high-level semantic information, and the accuracy of content-based image classification and retrieval depends greatly on the description of low-level visual features. Taking this into consideration, a novel texture and edge descriptor is proposed in this paper, which can be represented with a histogram. Furthermore, with the incorporation of the color, texture and edge histograms searnlessly, the images are grouped into semantic classes using a support vector machine (SVM). Experiment results show that the combination descriptor is more discriminative than other feature descriptors such as Gabor texture.

Knowledge on road information in sub-urban lane detection via multiple cue integration

Detection of lane boundaries of a road based on the images or video taken by a video capturing device in a suburban environment is a challenging task. In this paper, a novel lane detection algorithm is proposed without considering camera parameters; which robustly detects lane boundaries in real-time especially for sub-urban roads. Initially, the proposed method fits the CIE L*a*b* transformed road chromaticity values (that is a* and b* values) to a bi-variate Gaussian model followed by the classification of road area based on Mahalanobis distance. Secondly, the classified road area acts as an arbitrary shaped region of interest (AROI) in order to extract blobs resulting from the filtered image by a two dimensional Gabor filter. This is considered as the first cue of images. Thirdly, another cue of images was employed in order to obtain an entropy image. Moreover, results from the color based image cue and entropy image cue were integrated following an outlier removing process. Finally, the correct road lane points are fitted with Bezier splines which act as control points that can form arbitrary shapes. The algorithm was implemented and experiments were carried out on sub-urban roads. The results show the effectiveness of the algorithm in producing more accurate lane boundaries on curvatures and other objects on the road.

The identification of mammalian species through the classification of hair patterns using image pattern recognition

The identification of mammals through the use of their hair is important in the fields of forensics and ecology. The application of computer pattern recognition techniques to this process provides a means of reducing the subjectivity found in the process, as manual techniques rely on the interpretation of a human expert rather than quantitative measures. The first application of image pattern recognition techniques to the classification of African mammalian species using hair patterns is presented. This application uses a 2D Gabor filter-bank and motivates the use of moments to classify hair scale patterns. Application of a 2D Gabor filter-bank to hair scale processing provides results of 52% accuracy when using a filter bank of size four and 72% accuracy when using a filter-bank of size eight. These initial results indicate that 2D Gabor filters produce information that may be successfully

Local energy and pre-envelope

We observe that the local energy is the pre-envelope for analytic function. The maxima and phase of this function can be used to compute and classify visual features such as motion and stereo disparity, texture, etc. We examine the construction of new filters for computing Local Energy, and compare these filters with the Gabor filters and the three-point-filter of Venkatesh and Owens.

Local energy, the pre-envelope, and filter resolution

We examine the construction of new filters for computing local energy, and compare these filters with the Gabor filters and the three-point-filter of Venkatesh [l]. Further, we demonstrate that the effect of convolution with complex Gabor filters is to band-pass (with some differentiating effect) and compute the local energy of the result. The magnitude of the resulting local energy is then used to detect features [2], [3] (step features, texture etc.), and the phase is used to classify the detected features [l], [4] or provide disparity information for stereo [5] and motion work [6], [7]. Each of these types of information can be obtained at multiple resolutions, enabling the use of course to fine strategies for computing disparity, and allowing the discrimination of image textures on the basis of which parts of the Fourier domain they dominate [8], [9].

Real-time lane detection on suburban streets using visual cue integration

The detection of lane boundaries on suburban streets using images obtained from video constitutes a challenging task. This is mainly due to the difficulties associated with estimating the complex geometric structure of lane boundaries, the quality of lane markings as a result of wear, occlusions by traffic, and shadows caused by road-side trees and structures. Most of the existing techniques for lane boundary detection employ a single visual cue and will only work under certain conditions and where there are clear lane markings. Also, better results are achieved when there are no other onroad objects present. This paper extends our previous work and discusses a novel lane boundary detection algorithm specifically addressing the abovementioned issues through the integration of two visual cues. The first visual cue is based on stripe-like features found on lane lines extracted using a two-dimensional symmetric Gabor filter. The second visual cue is based on a texture characteristic determined using the entropy measure of the predefined neighbourhood around a lane boundary line. The visual cues are then integrated using a rulebased classifier which incorporates a modified sequential covering algorithm to improve robustness. To separate lane boundary lines from other similar features, a road mask is generated using road chromaticity values estimated from CIE L*a*b* colour transformation. Extraneous points around lane boundary lines are then removed by an outlier removal procedure based on studentized residuals. The lane boundary lines are then modelled with Bezier spline curves. To validate the algorithm, extensive experimental evaluation was carried out on suburban streets and the results are presented. 

Intelligent facial emotion recognition using a layered encoding cascade optimization model

In this research, we propose a facial expression recognition system with a layered encoding cascade optimization model. Since generating an effective facial representation is a vital step to the success of facial emotion recognition, a modified Local Gabor Binary Pattern operator is first employed to derive a refined initial face representation and we then propose two evolutionary algorithms for feature optimization including (i) direct similarity and (ii) Pareto-based feature selection, under the layered cascade model. The direct similarity feature selection considers characteristics within the same emotion category that give the minimum within-class variation while the Pareto-based feature optimization focuses on features that best represent each expression category and at the same time provide the most distinctions to other expressions. Both a neural network and an ensemble classifier with weighted majority vote are implemented for the recognition of seven expressions based on the selected optimized features. The ensemble model also automatically updates itself with the most recent concepts in the data. Evaluated with the Cohn-Kanade database, our system achieves the best accuracies when the ensemble classifier is applied, and outperforms other research reported in the literature with 96.8% for direct similarity based optimization and 97.4% for the Pareto-based feature selection. Cross-database evaluation with frontal images from the MMI database has also been conducted to further prove system efficiency where it achieves 97.5% for Pareto-based approach and 90.7% for direct similarity-based feature selection and outperforms related research for MMI. When evaluated with 90° side-view images extracted from the videos of the MMI database, the system achieves superior performances with >80% accuracies for both optimization algorithms. Experiments with other weighting and meta-learning combination methods for the construction of ensembles are also explored with our proposed ensemble showing great adpativity to new test data stream for cross-database evaluation. In future work, we aim to incorporate other filtering techniques and evolutionary algorithms into the optimization models to further enhance the recognition performance.

Multi-label approach for human-face classification

Single-label classification models have been widely used for human-face classification. In this paper, we present a multi-label classification approach for human-face classification. Multi-label classification is more appropriate in the real world because a human-face can be associated with multiple labels. Demographic information can be derived and utilized along with facial expression in the field of face classification to assist with multi label classification. Gabor filters; Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) methods, are used to extract and project representative demographic information from facial images. For evaluation, five classification algorithms were used. We evaluate the proposed approach by performing experiments on Yale face images database. Results show the effectiveness of multi-label classification algorithms.

Experimental comparison of approaches for feature extraction of facial attributes

In this paper, we compare the effectiveness of widely used approaches for representation of facial features in face images. Feature extraction is performed on face images for representation of four facial attributes, namely gender, age, race, and expression, by using discrete wavelet transform (DWT), Gabor wavelet, scale-invariant feature transform, local binary pattern (LBP), and Eigenfaces. After feature extraction and dimension reduction, demographic and expression classification is performed to identify the most discriminating techniques for representation of facial features. Extensive experiments are performed using publicly available face databases, namely Yale, Face95 Essex, and Cohn-Kanade (CK+) databases. Experimental results show that DWT, LBP, and Gabor wavelet methods are robust to variations of illumination, facial expression, and geometric transformations. Experimental results also show that race and expression are more difficult to predict than gender and age.