Automatic tracking, super-resolution and recognition of human faces from surveillance video

Identifying an individual from surveillance video is a difficult, time consuming and labour intensive process. The proposed system aims to streamline this process by filtering out unwanted scenes and enhancing an individual's face through super-resolution. An automatic face recognition system is then used to identify the subject or present the human operator with likely matches from a database. A person tracker is used to speed up the subject detection and super-resolution process by tracking moving subjects and cropping a region of interest around the subject's face to reduce the number and size of the image frames to be super-resolved respectively. In this paper, experiments have been conducted to demonstrate how the optical flow super-resolution method used improves surveillance imagery for visual inspection as well as automatic face recognition on an Eigenface and Elastic Bunch Graph Matching system. The optical flow based method has also been benchmarked against the ``hallucination'' algorithm, interpolation methods and the original low-resolution images. Results show that both super-resolution algorithms improved recognition rates significantly. Although the hallucination method resulted in slightly higher recognition rates, the optical flow method produced less artifacts and more visually correct images suitable for human consumption.

Machine recognition of hand-drawn circuit diagrams

An application of image processing techniques to recognition of hand-drawn circuit diagrams is presented. The scanned image of a diagram is pre-processed to remove noise and converted to bilevel. Morphological operations are applied to obtain a clean, connected representation using thinned lines. The diagram comprises of nodes, connections and components. Nodes and components are segmented using appropriate thresholds on a spatially varying object pixel density. Connection paths are traced using a pixel-stack. Nodes are classified using syntactic analysis. Components are classified using a combination of invariant moments, scalar pixel-distribution features, and vector relationships between straight lines in polygonal representations. A node recognition accuracy of 82% and a component recognition accuracy of 86% was achieved on a database comprising 107 nodes and 449 components. This recogniser can be used for layout “beautification” or to generate input code for circuit analysis and simulation packages

Face authentication test on the BANCA database

This work details the results of a face authentication test (FAT2004) (http://www.ee.surrey.ac.uk/banca/icpr2004) held in conjunction with the 17th International Conference on Pattern Recognition. The contest was held on the publicly available BANCA database (http://www.ee.surrey.ac.uk/banca) according to a defined protocol (E. Bailly-Bailliere et al., June 2003). The competition also had a sequestered part in which institutions had to submit their algorithms for independent testing. 13 different verification algorithms from 10 institutions submitted results. Also, a standard set of face recognition software packages from the Internet (http://www.cs.colostate.edu/evalfacerec) were used to provide a baseline performance measure.

Probabilistic matching of image sets for video-based face recognition

We address the problem of face recognition on video by employing the recently proposed probabilistic linear discrimi-nant analysis (PLDA). The PLDA has been shown to be robust against pose and expression in image-based face recognition. In this research, the method is extended and applied to video where image set to image set matching is performed. We investigate two approaches of computing similarities between image sets using the PLDA: the closest pair approach and the holistic sets approach. To better model face appearances in video, we also propose the heteroscedastic version of the PLDA which learns the within-class covariance of each individual separately. Our experi-ments on the VidTIMIT and Honda datasets show that the combination of the heteroscedastic PLDA and the closest pair approach achieves the best performance.

On the application of the probabilistic linear discriminant analysis to face recognition across expression

Facial expression is one of the main issues of face recognition in uncontrolled environments. In this paper, we apply the probabilistic linear discriminant analysis (PLDA) method to recognize faces across expressions. Several PLDA approaches are tested and cross-evaluated on the Cohn-Kanade and JAFFE databases. With less samples per gallery subject, high recognition rates comparable to previous works have been achieved indicating the robustness of the approaches. Among the approaches, the mixture of PLDAs has demonstrated better performances. The experimental results also indicate that facial regions around the cheeks, eyes, and eyebrows are more discriminative than regions around the mouth, jaw, chin, and nose.

Cohort normalization based sparse representation for undersampled face recognition

Abstract. In recent years, sparse representation based classification(SRC) has received much attention in face recognition with multipletraining samples of each subject. However, it cannot be easily applied toa recognition task with insufficient training samples under uncontrolledenvironments. On the other hand, cohort normalization, as a way of mea-suring the degradation effect under challenging environments in relationto a pool of cohort samples, has been widely used in the area of biometricauthentication. In this paper, for the first time, we introduce cohort nor-malization to SRC-based face recognition with insufficient training sam-ples. Specifically, a user-specific cohort set is selected to normalize theraw residual, which is obtained from comparing the test sample with itssparse representations corresponding to the gallery subject, using poly-nomial regression. Experimental results on AR and FERET databases show that cohort normalization can bring SRC much robustness against various forms of degradation factors for undersampled face recognition.

Insect odour perception: recognition of odour components by flower foraging moths

Odours emitted by flowers are complex blends of volatile compounds. These odours are learnt by flower-visiting insect species, improving their recognition of rewarding flowers and thus foraging efficiency. We investigated the flexibility of floral odour learning by testing whether adult moths recognize single compounds common to flowers on which they forage. Dual choice preference tests on Helicoverpa armigera moths allowed free flying moths to forage on one of three flower species; Argyranthemum frutescens (federation daisy), Cajanus cajan (pigeonpea) or Nicotiana tabacum (tobacco). Results showed that, (i) a benzenoid (phenylacetaldehyde) and a monoterpene (linalool) were subsequently recognized after visits to flowers that emitted these volatile constituents, (ii) in a preference test, other monoterpenes in the flowers' odour did not affect the moths' ability to recognize the monoterpene linalool and (iii) relative preferences for two volatiles changed after foraging experience on a single flower species that emitted both volatiles. The importance of using free flying insects and real flowers to understand the mechanisms involved in floral odour learning in nature are discussed in the context of our findings.

Vision-based place recognition : how low can you go?

In this paper we use the algorithm SeqSLAM to address the question, how little and what quality of visual information is needed to localize along a familiar route? We conduct a comprehensive investigation of place recognition performance on seven datasets while varying image resolution (primarily 1 to 512 pixel images), pixel bit depth, field of view, motion blur, image compression and matching sequence length. Results confirm that place recognition using single images or short image sequences is poor, but improves to match or exceed current benchmarks as the matching sequence length increases. We then present place recognition results from two experiments where low-quality imagery is directly caused by sensor limitations; in one, place recognition is achieved along an unlit mountain road by using noisy, long-exposure blurred images, and in the other, two single pixel light sensors are used to localize in an indoor environment. We also show failure modes caused by pose variance and sequence aliasing, and discuss ways in which they may be overcome. By showing how place recognition along a route is feasible even with severely degraded image sequences, we hope to provoke a re-examination of how we develop and test future localization and mapping systems.

Recognition and enforcement of foreign judgments : are the rules appropriate?

The recognition and enforcement of foreign judgments is an aspect of private international law, and concerns situations where a successful party to litigation seeks to rely on a judgment obtained in one court, in a court in another jurisdiction. The most common example where the recognition and enforcement of foreign judgments may arise is where a party who has obtained a favourable judgment in one state or country may seek to recognise and enforce the judgment in another state or country. This occurs because there is no sufficient asset in the state or country where the judgment was rendered to satisfy that judgment. As technological advancements in communications over vast geographical distances have improved exponentially in recent years, there has been an increase in cross-border transactions, as well as litigation arising from these transactions. As a result, the recognition and enforcement of foreign judgments is of increasing importance, since a party who has obtained a judgment in cross-border litigation may wish to recognise and enforce the judgment in another state or country, where the defendant’s assets may be located without having to re-litigate substantive issues that have already been resolved in another court. The purpose of the study is to examine whether the current state of laws for the recognition and enforcement of foreign judgments in Australia, the United States and the European Community are in line with modern-commercial needs. The study is conducted by weighing two competing objectives between the notion of finality of litigation, which encourages courts to recognise and enforce judgments foreign to them, on the one hand, and the adequacy of protection to safeguard the recognition and enforcement proceedings, so that there would be no injustice or unfairness if a foreign judgment is recognised and enforced, on the other. The findings of the study are as follows. In both Australia and the United States, there is a different approach concerning the recognition and enforcement of judgments rendered by courts interstate or in a foreign country. In order to maintain a single and integrated nation, there are constitutional and legislative requirements authorising courts to give conclusive effects to interstate judgments. In contrast, if the recognition and enforcement actions involve judgments rendered by a foreign country’s court, an Australian or a United States court will not recognise and enforce the foreign judgment unless the judgment has satisfied a number of requirements and does not fall under any of the exceptions to justify its non-recognition and non-enforcement. In the European Community, the Brussels I Regulation which governs the recognition and enforcement of judgments among European Union Member States has created a scheme, whereby there is only a minimal requirement that needs to be satisfied for the purposes of recognition and enforcement. Moreover, a judgment that is rendered by a Member State and based on any of the jurisdictional bases set forth in the Brussels I Regulation is entitled to be recognised and enforced in another Member State without further review of its underlying jurisdictional basis. However, there are concerns as to the adequacy of protection available under the Brussels I Regulation to safeguard the judgment-enforcing Member States, as well as those against whom recognition or enforcement is sought. This dissertation concludes by making two recommendations aimed at improving the means by which foreign judgments are recognised and enforced in the selected jurisdictions. The first is for the law in both Australia and the United States to undergo reform, including: adopting the real and substantial connection test as the new jurisdictional basis for the purposes of recognition and enforcement; liberalising the existing defences to safeguard the application of the real and substantial connection test; extending the application of the Foreign Judgments Act 1991 (Cth) in Australia to include at least its important trading partners; and implementing a federal statutory scheme in the United States to govern the recognition and enforcement of foreign judgments. The second recommendation is to introduce a convention on jurisdiction and the recognition and enforcement of foreign judgments. The convention will be a convention double, which provides uniform standards for the rules of jurisdiction a court in a contracting state must exercise when rendering a judgment and a set of provisions for the recognition and enforcement of resulting judgments.

Delirium knowledge and recognition : a randomized controlled trial of a web-based educational intervention for acute care nurses

Delirium is a significant problem for older hospitalized people and is associated with poor outcomes. It is poorly recognized and evidence suggests that a major reason is lack of education. Nurses, who are educated about delirium, can play a significant role in improving delirium recognition. This study evaluated the impact of a delirium specific educational website. A cluster randomized controlled trial, with a pretest/post-test time series design, was conducted to measure delirium knowledge (DK) and delirium recognition (DR) over three time-points. Statistically significant differences were found between the intervention and non-intervention group. The intervention groups' DK scores were higher and the change over time results were statistically significant [T3 and T1 (t=3.78 p=<0.001) and T2 and T1 baseline (t=5.83 p=<0.001)]. Statistically significant improvements were also seen for DR when comparing T2 and T1 results (t=2.56 p=0.011) between both groups but not for changes in DR scores between T3 and T1 (t=1.80 p=0.074). Participants rated the website highly on the visual, functional and content elements. This study supports the concept that web-based delirium learning is an effective and satisfying method of information delivery for registered nurses. Future research is required to investigate clinical outcomes as a result of this web-based education.

Facial expression recognition experiments with data from television broadcasts and the world wide web

Facial expression recognition (FER) systems must ultimately work on real data in uncontrolled environments although most research studies have been conducted on lab-based data with posed or evoked facial expressions obtained in pre-set laboratory environments. It is very difficult to obtain data in real-world situations because privacy laws prevent unauthorized capture and use of video from events such as funerals, birthday parties, marriages etc. It is a challenge to acquire such data on a scale large enough for benchmarking algorithms. Although video obtained from TV or movies or postings on the World Wide Web may also contain ‘acted’ emotions and facial expressions, they may be more ‘realistic’ than lab-based data currently used by most researchers. Or is it? One way of testing this is to compare feature distributions and FER performance. This paper describes a database that has been collected from television broadcasts and the World Wide Web containing a range of environmental and facial variations expected in real conditions and uses it to answer this question. A fully automatic system that uses a fusion based approach for FER on such data is introduced for performance evaluation. Performance improvements arising from the fusion of point-based texture and geometry features, and the robustness to image scale variations are experimentally evaluated on this image and video dataset. Differences in FER performance between lab-based and realistic data, between different feature sets, and between different train-test data splits are investigated.

Local inter-session variability modelling for object classification

Object classification is plagued by the issue of session variation. Session variation describes any variation that makes one instance of an object look different to another, for instance due to pose or illumination variation. Recent work in the challenging task of face verification has shown that session variability modelling provides a mechanism to overcome some of these limitations. However, for computer vision purposes, it has only been applied in the limited setting of face verification. In this paper we propose a local region based intersession variability (ISV) modelling approach, and apply it to challenging real-world data. We propose a region based session variability modelling approach so that local session variations can be modelled, termed Local ISV. We then demonstrate the efficacy of this technique on a challenging real-world fish image database which includes images taken underwater, providing significant real-world session variations. This Local ISV approach provides a relative performance improvement of, on average, 23% on the challenging MOBIO, Multi-PIE and SCface face databases. It also provides a relative performance improvement of 35% on our challenging fish image dataset.

Random Gabor based templates for facial expression recognition in images with facial occlusion

Robust facial expression recognition (FER) under occluded face conditions is challenging. It requires robust algorithms of feature extraction and investigations into the effects of different types of occlusion on the recognition performance to gain insight. Previous FER studies in this area have been limited. They have spanned recovery strategies for loss of local texture information and testing limited to only a few types of occlusion and predominantly a matched train-test strategy. This paper proposes a robust approach that employs a Monte Carlo algorithm to extract a set of Gabor based part-face templates from gallery images and converts these templates into template match distance features. The resulting feature vectors are robust to occlusion because occluded parts are covered by some but not all of the random templates. The method is evaluated using facial images with occluded regions around the eyes and the mouth, randomly placed occlusion patches of different sizes, and near-realistic occlusion of eyes with clear and solid glasses. Both matched and mis-matched train and test strategies are adopted to analyze the effects of such occlusion. Overall recognition performance and the performance for each facial expression are investigated. Experimental results on the Cohn-Kanade and JAFFE databases demonstrate the high robustness and fast processing speed of our approach, and provide useful insight into the effects of occlusion on FER. The results on the parameter sensitivity demonstrate a certain level of robustness of the approach to changes in the orientation and scale of Gabor filters, the size of templates, and occlusions ratios. Performance comparisons with previous approaches show that the proposed method is more robust to occlusion with lower reductions in accuracy from occlusion of eyes or mouth.

Object tracking via non-Euclidean geometry : a Grassmann approach

A robust visual tracking system requires an object appearance model that is able to handle occlusion, pose, and illumination variations in the video stream. This can be difficult to accomplish when the model is trained using only a single image. In this paper, we first propose a tracking approach based on affine subspaces (constructed from several images) which are able to accommodate the abovementioned variations. We use affine subspaces not only to represent the object, but also the candidate areas that the object may occupy. We furthermore propose a novel approach to measure affine subspace-to-subspace distance via the use of non-Euclidean geometry of Grassmann manifolds. The tracking problem is then considered as an inference task in a Markov Chain Monte Carlo framework via particle filtering. Quantitative evaluation on challenging video sequences indicates that the proposed approach obtains considerably better performance than several recent state-of-the-art methods such as Tracking-Learning-Detection and MILtrack.

Semantic context and visual feature effects in object naming : an fMRI study using arterial spin labeling

Previous behavioral studies reported a robust effect of increased naming latencies when objects to be named were blocked within semantic category, compared to items blocked between category. This semantic context effect has been attributed to various mechanisms including inhibition or excitation of lexico-semantic representations and incremental learning of associations between semantic features and names, and is hypothesized to increase demands on verbal self-monitoring during speech production. Objects within categories also share many visual structural features, introducing a potential confound when interpreting the level at which the context effect might occur. Consistent with previous findings, we report a significant increase in response latencies when naming categorically related objects within blocks, an effect associated with increased perfusion fMRI signal bilaterally in the hippocampus and in the left middle to posterior superior temporal cortex. No perfusion changes were observed in the middle section of the left middle temporal cortex, a region associated with retrieval of lexical-semantic information in previous object naming studies. Although a manipulation of visual feature similarity did not influence naming latencies, we observed perfusion increases in the perirhinal cortex for naming objects with similar visual features that interacted with the semantic context in which objects were named. These results provide support for the view that the semantic context effect in object naming occurs due to an incremental learning mechanism, and involves increased demands on verbal self-monitoring.