Automatic tracking, super-resolution and recognition of human faces from surveillance video

Identifying an individual from surveillance video is a difficult, time consuming and labour intensive process. The proposed system aims to streamline this process by filtering out unwanted scenes and enhancing an individual's face through super-resolution. An automatic face recognition system is then used to identify the subject or present the human operator with likely matches from a database. A person tracker is used to speed up the subject detection and super-resolution process by tracking moving subjects and cropping a region of interest around the subject's face to reduce the number and size of the image frames to be super-resolved respectively. In this paper, experiments have been conducted to demonstrate how the optical flow super-resolution method used improves surveillance imagery for visual inspection as well as automatic face recognition on an Eigenface and Elastic Bunch Graph Matching system. The optical flow based method has also been benchmarked against the ``hallucination'' algorithm, interpolation methods and the original low-resolution images. Results show that both super-resolution algorithms improved recognition rates significantly. Although the hallucination method resulted in slightly higher recognition rates, the optical flow method produced less artifacts and more visually correct images suitable for human consumption.

Focus-score weighted super-resolution for uncooperative iris recognition at a distance and on the move

Uncooperative iris identification systems at a distance and on the move often suffer from poor resolution and poor focus of the captured iris images. The lack of pixel resolution and well-focused images significantly degrades the iris recognition performance. This paper proposes a new approach to incorporate the focus score into a reconstruction-based super-resolution process to generate a high resolution iris image from a low resolution and focus inconsistent video sequence of an eye. A reconstruction-based technique, which can incorporate middle and high frequency components from multiple low resolution frames into one desired super-resolved frame without introducing false high frequency components, is used. A new focus assessment approach is proposed for uncooperative iris at a distance and on the move to improve performance for variations in lighting, size and occlusion. A novel fusion scheme is then proposed to incorporate the proposed focus score into the super-resolution process. The experiments conducted on the The Multiple Biometric Grand Challenge portal database shows that our proposed approach achieves an EER of 2.1%, outperforming the existing state-of-the-art averaging signal-level fusion approach by 19.2% and the robust mean super-resolution approach by 8.7%.

Robust mean super-resolution for less cooperative NIR iris recognition at a distance and on the move

Less cooperative iris identification systems at a distance and on the move often suffers from poor resolution. The lack of pixel resolution significantly degrades the iris recognition performance. Super-resolution has been considered to enhance resolution of iris images. This paper proposes a pixelwise super-resolution technique to reconstruct a high resolution iris image from a video sequence of an eye. A novel fusion approach is proposed to incorporate information details from multiple frames using robust mean. Experiments on the MBGC NIR portal database show the validity of the proposed approach in comparison with other resolution enhancement techniques.

Feature-domain super-resolution for IRIS recognition

Uncooperative iris identification systems at a distance suffer from poor resolution of the captured iris images, which significantly degrades iris recognition performance. Superresolution techniques have been employed to enhance the resolution of iris images and improve the recognition performance. However, all existing super-resolution approaches proposed for the iris biometric super-resolve pixel intensity values. This paper considers transferring super-resolution of iris images from the intensity domain to the feature domain. By directly super-resolving only the features essential for recognition, and by incorporating domain specific information from iris models, improved recognition performance compared to pixel domain super-resolution can be achieved. This is the first paper to investigate the possibility of feature domain super-resolution for iris recognition, and experiments confirm the validity of the proposed approach.

Feature-domain super-resolution framework for gabor-based face and iris recognition

The low resolution of images has been one of the major limitations in recognising humans from a distance using their biometric traits, such as face and iris. Superresolution has been employed to improve the resolution and the recognition performance simultaneously, however the majority of techniques employed operate in the pixel domain, such that the biometric feature vectors are extracted from a super-resolved input image. Feature-domain superresolution has been proposed for face and iris, and is shown to further improve recognition performance by capitalising on direct super-resolving the features which are used for recognition. However, current feature-domain superresolution approaches are limited to simple linear features such as Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA), which are not the most discriminant features for biometrics. Gabor-based features have been shown to be one of the most discriminant features for biometrics including face and iris. This paper proposes a framework to conduct super-resolution in the non-linear Gabor feature domain to further improve the recognition performance of biometric systems. Experiments have confirmed the validity of the proposed approach, demonstrating superior performance to existing linear approaches for both face and iris biometrics.

Evaluation of image resolution and super-resolution on face recognition performance

While researchers strive to improve automatic face recognition performance, the relationship between image resolution and face recognition performance has not received much attention. This relationship is examined systematically and a framework is developed such that results from super-resolution techniques can be compared. Three super-resolution techniques are compared with the Eigenface and Elastic Bunch Graph Matching face recognition engines. Parameter ranges over which these techniques provide better recognition performance than interpolated images is determined.

Improved Subject Identification in Surveillance Video using Super Resolution

The time consuming and labour intensive task of identifying individuals in surveillance video is often challenged by poor resolution and the sheer volume of stored video. Faces or identifying marks such as tattoos are often too coarse for direct matching by machine or human vision. Object tracking and super-resolution can then be combined to facilitate the automated detection and enhancement of areas of interest. The object tracking process enables the automatic detection of people of interest, greatly reducing the amount of data for super-resolution. Smaller regions such as faces can also be tracked. A number of instances of such regions can then be utilized to obtain a super-resolved version for matching. Performance improvement from super-resolution is demonstrated using a face verification task. It is shown that there is a consistent improvement of approximately 7% in verification accuracy, using both Eigenface and Elastic Bunch Graph Matching approaches for automatic face verification, starting from faces with an eye to eye distance of 14 pixels. Visual improvement in image fidelity from super-resolved images over low-resolution and interpolated images is demonstrated on a small database. Current research and future directions in this area are also summarized.

Feature-domain super-resolution for iris recognition

Uncooperative iris identification systems at a distance suffer from poor resolution of the acquired iris images, which significantly degrades iris recognition performance. Super-resolution techniques have been employed to enhance the resolution of iris images and improve the recognition performance. However, most existing super-resolution approaches proposed for the iris biometric super-resolve pixel intensity values, rather than the actual features used for recognition. This paper thoroughly investigates transferring super-resolution of iris images from the intensity domain to the feature domain. By directly super-resolving only the features essential for recognition, and by incorporating domain specific information from iris models, improved recognition performance compared to pixel domain super-resolution can be achieved. A framework for applying super-resolution to nonlinear features in the feature-domain is proposed. Based on this framework, a novel feature-domain super-resolution approach for the iris biometric employing 2D Gabor phase-quadrant features is proposed. The approach is shown to outperform its pixel domain counterpart, as well as other feature domain super-resolution approaches and fusion techniques.

Point Spread Function Engineering for Super-Resolution Single-Photon and Multiphoton Fluorescence Microscopy

Super-resolution imaging techniques are of paramount interest for applications in bioimaging and fluorescence microscopy. Recent advances in bioimaging demand application-tailored point spread functions. Here, we present some approaches for generating application-tailored point spread functions along with fast imaging capabilities. Aperture engineering techniques provide interesting solutions for obtaining desired system point spread functions. Specially designed spatial filters—realized by optical mask—are outlined both in a single-lens and 4Pi configuration. Applications include depth imaging, multifocal imaging, and super-resolution imaging. Such an approach is suitable for fruitful integration with most existing state-of-art imaging microscopy modalities.

Spatial filtering nearly eliminates the side-lobes in single- and multi-photon 4pi-type-C super-resolution fluorescence microscopy

Super-resolution microscopy has tremendously progressed our understanding of cellular biophysics and biochemistry. Specifically, 4pi fluorescence microscopy technique stands out because of its axial super-resolution capability. All types of 4pi-microscopy techniques work well in conjugation with deconvolution techniques to get rid of artifacts due to side-lobes. In this regard, we propose a technique based on spatial filter in a 4pi-type-C confocal setup to get rid of these artifacts. Using a special spatial filter, we have reduced the depth-of-focus. Interference of two similar depth-of-focus beams in a 4 pi geometry result in substantial reduction of side-lobes. Studies show a reduction of side-lobes by 46% and 76% for single and two photon variant compared to 4pi - type - C confocal system. This is incredible considering the resolving capability of the existing 4pi - type - C confocal microscopy. Moreover, the main lobe is found to be 150 nm for the proposed spatial filtering technique as compared to 690 nm of the state-of-art confocal system. Reconstruction of experimentally obtained 2PE - 4pi data of green fluorescent protein (GFP)-tagged mitocondrial network shows near elimination of artifacts arising out of side-lobes. Proposed technique may find interesting application in fluorescence microscopy, nano-lithography, and cell biology. (C) 2013 AIP Publishing LLC.

Super-Resolution Reconstruction in Frequency-Domain Optical-Coherence Tomography Using the Finite-Rate-of-Innovation Principle

The standard approach to signal reconstruction in frequency-domain optical-coherence tomography (FDOCT) is to apply the inverse Fourier transform to the measurements. This technique offers limited resolution (due to Heisenberg's uncertainty principle). We propose a new super-resolution reconstruction method based on a parametric representation. We consider multilayer specimens, wherein each layer has a constant refractive index and show that the backscattered signal from such a specimen fits accurately in to the framework of finite-rate-of-innovation (FRI) signal model and is represented by a finite number of free parameters. We deploy the high-resolution Prony method and show that high-quality, super-resolved reconstruction is possible with fewer measurements (about one-fourth of the number required for the standard Fourier technique). To further improve robustness to noise in practical scenarios, we take advantage of an iterated singular-value decomposition algorithm (Cadzow denoiser). We present results of Monte Carlo analyses, and assess statistical efficiency of the reconstruction techniques by comparing their performance against the Cramer-Rao bound. Reconstruction results on experimental data obtained from technical as well as biological specimens show a distinct improvement in resolution and signal-to-reconstruction noise offered by the proposed method in comparison with the standard approach.

Super Resolution Using a Single Image Dictionary

To perform super resolution of low resolution images, state-of-the-art methods are based on learning a pair of lowresolution and high-resolution dictionaries from multiple images. These trained dictionaries are used to replace patches in lowresolution image with appropriate matching patches from the high-resolution dictionary. In this paper we propose using a single common image as dictionary, in conjunction with approximate nearest neighbour fields (ANNF) to perform super resolution (SR). By using a common source image, we are able to bypass the learning phase and also able to reduce the dictionary from a collection of hundreds of images to a single image. By adapting recent developments in ANNF computation, to suit super-resolution, we are able to perform much faster and accurate SR than existing techniques. To establish this claim, we compare the proposed algorithm against various state-of-the-art algorithms, and show that we are able to achieve b etter and faster reconstruction without any training.

MRT Letter: A Unified Accelerated Maximum Likelihood Technique for Widefield, Confocal, and Super-Resolution 4Pi Microscopy

We propose an algorithmic technique for accelerating maximum likelihood (ML) algorithm for image reconstruction in fluorescence microscopy. This is made possible by integrating Biggs-Andrews (BA) method with ML approach. The results on widefield, confocal, and super-resolution 4Pi microscopy reveal substantial improvement in the speed of 3D image reconstruction (the number of iterations has reduced by approximately one-half). Moreover, the quality of reconstruction obtained using accelerated ML closely resembles with nonaccelerated ML method. The proposed technique is a step closer to realize real-time reconstruction in 3D fluorescence microscopy. Microsc. Res. Tech. 78:331-335, 2015. (c) 2015 Wiley Periodicals, Inc.