Automatic tracking, super-resolution and recognition of human faces from surveillance video

Identifying an individual from surveillance video is a difficult, time consuming and labour intensive process. The proposed system aims to streamline this process by filtering out unwanted scenes and enhancing an individual's face through super-resolution. An automatic face recognition system is then used to identify the subject or present the human operator with likely matches from a database. A person tracker is used to speed up the subject detection and super-resolution process by tracking moving subjects and cropping a region of interest around the subject's face to reduce the number and size of the image frames to be super-resolved respectively. In this paper, experiments have been conducted to demonstrate how the optical flow super-resolution method used improves surveillance imagery for visual inspection as well as automatic face recognition on an Eigenface and Elastic Bunch Graph Matching system. The optical flow based method has also been benchmarked against the ``hallucination'' algorithm, interpolation methods and the original low-resolution images. Results show that both super-resolution algorithms improved recognition rates significantly. Although the hallucination method resulted in slightly higher recognition rates, the optical flow method produced less artifacts and more visually correct images suitable for human consumption.

Focus-score weighted super-resolution for uncooperative iris recognition at a distance and on the move

Uncooperative iris identification systems at a distance and on the move often suffer from poor resolution and poor focus of the captured iris images. The lack of pixel resolution and well-focused images significantly degrades the iris recognition performance. This paper proposes a new approach to incorporate the focus score into a reconstruction-based super-resolution process to generate a high resolution iris image from a low resolution and focus inconsistent video sequence of an eye. A reconstruction-based technique, which can incorporate middle and high frequency components from multiple low resolution frames into one desired super-resolved frame without introducing false high frequency components, is used. A new focus assessment approach is proposed for uncooperative iris at a distance and on the move to improve performance for variations in lighting, size and occlusion. A novel fusion scheme is then proposed to incorporate the proposed focus score into the super-resolution process. The experiments conducted on the The Multiple Biometric Grand Challenge portal database shows that our proposed approach achieves an EER of 2.1%, outperforming the existing state-of-the-art averaging signal-level fusion approach by 19.2% and the robust mean super-resolution approach by 8.7%.

Feature-domain super-resolution framework for gabor-based face and iris recognition

The low resolution of images has been one of the major limitations in recognising humans from a distance using their biometric traits, such as face and iris. Superresolution has been employed to improve the resolution and the recognition performance simultaneously, however the majority of techniques employed operate in the pixel domain, such that the biometric feature vectors are extracted from a super-resolved input image. Feature-domain superresolution has been proposed for face and iris, and is shown to further improve recognition performance by capitalising on direct super-resolving the features which are used for recognition. However, current feature-domain superresolution approaches are limited to simple linear features such as Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA), which are not the most discriminant features for biometrics. Gabor-based features have been shown to be one of the most discriminant features for biometrics including face and iris. This paper proposes a framework to conduct super-resolution in the non-linear Gabor feature domain to further improve the recognition performance of biometric systems. Experiments have confirmed the validity of the proposed approach, demonstrating superior performance to existing linear approaches for both face and iris biometrics.

Improved Subject Identification in Surveillance Video using Super Resolution

The time consuming and labour intensive task of identifying individuals in surveillance video is often challenged by poor resolution and the sheer volume of stored video. Faces or identifying marks such as tattoos are often too coarse for direct matching by machine or human vision. Object tracking and super-resolution can then be combined to facilitate the automated detection and enhancement of areas of interest. The object tracking process enables the automatic detection of people of interest, greatly reducing the amount of data for super-resolution. Smaller regions such as faces can also be tracked. A number of instances of such regions can then be utilized to obtain a super-resolved version for matching. Performance improvement from super-resolution is demonstrated using a face verification task. It is shown that there is a consistent improvement of approximately 7% in verification accuracy, using both Eigenface and Elastic Bunch Graph Matching approaches for automatic face verification, starting from faces with an eye to eye distance of 14 pixels. Visual improvement in image fidelity from super-resolved images over low-resolution and interpolated images is demonstrated on a small database. Current research and future directions in this area are also summarized.

Vision-based place recognition : how low can you go?

In this paper we use the algorithm SeqSLAM to address the question, how little and what quality of visual information is needed to localize along a familiar route? We conduct a comprehensive investigation of place recognition performance on seven datasets while varying image resolution (primarily 1 to 512 pixel images), pixel bit depth, field of view, motion blur, image compression and matching sequence length. Results confirm that place recognition using single images or short image sequences is poor, but improves to match or exceed current benchmarks as the matching sequence length increases. We then present place recognition results from two experiments where low-quality imagery is directly caused by sensor limitations; in one, place recognition is achieved along an unlit mountain road by using noisy, long-exposure blurred images, and in the other, two single pixel light sensors are used to localize in an indoor environment. We also show failure modes caused by pose variance and sequence aliasing, and discuss ways in which they may be overcome. By showing how place recognition along a route is feasible even with severely degraded image sequences, we hope to provoke a re-examination of how we develop and test future localization and mapping systems.

Higher-resolution structure of the human insulin receptor ectodomain: Multi-modal inclusion of the insert domain

Insulin receptor (IR) signaling is critical to controlling nutrient uptake and metabolism. However, only a low-resolution (3.8 Å) structure currently exists for the IR ectodomain, with some segments ill-defined or unmodeled due to disorder. Here, we revise this structure using new diffraction data to 3.3 Å resolution that allow improved modeling of the N-linked glycans, the first and third fibronectin type III domains, and the insert domain. A novel haptic interactive molecular dynamics strategy was used to aid fitting to low-resolution electron density maps. The resulting model provides a foundation for investigation of structural transitions in IR upon ligand binding.

X-ray crystallographic and NMR studies of pantothenate synthetase provide insights into the mechanism of homotropic inhibition by pantoate

The structural basis for the homotropic inhibition of pantothenate synthetase by the substrate pantoate was investigated by X-ray crystallography and high-resolution NMR spectroscopic methods. The tertiary structure of the dimeric N-terminal domain of Escherichia coli pantothenate synthetase, determined by X-ray crystallography to a resolution of 1.7 Å, showed a second molecule of pantoate bound in the ATP-binding pocket. Pantoate binding to the ATP-binding site induced large changes in structure, mainly for backbone and side chain atoms of residues in the ATP binding HXGH(34–37) motif. Sequence-specific NMR resonance assignments and solution secondary structure of the dimeric N-terminal domain, obtained using samples enriched in 2H, 13C, and 15N, indicated that the secondary structural elements were conserved in solution. Nitrogen-15 edited two-dimensional solution NMR chemical shift mapping experiments revealed that pantoate, at 10 mm, bound at these two independent sites. The solution NMR studies unambiguously demonstrated that ATP stoichiometrically displaced pantoate from the ATP-binding site. All NMR and X-ray studies were conducted at substrate concentrations used for enzymatic characterization of pantothenate synthetase from different sources [Jonczyk R & Genschel U (2006) J Biol Chem 281, 37435–37446]. As pantoate binding to its canonical site is structurally conserved, these results demonstrate that the observed homotropic effects of pantoate on pantothenate biosynthesis are caused by competitive binding of this substrate to the ATP-binding site. The results presented here have implications for the design and development of potential antibacterial and herbicidal agents.

Structure of Sesbania Mosaic Virus at 4·7 Å Resolution and Partial Sequence of the Coat Protein

Sesbania mosaic virus (SMV) is a plant virus infecting Sesbania grandiflora plants in Andhra Pradesh, India. Amino acid sequence of the tryptic peptides of SMV coat protein were determined using a gas phase sequenator. These sequences showed identical amino acids at 69% of the positions when aligned with the corresponding residues of southern bean mosaic virus (SBMV).Crystals diffracting to better than 3 Å resolution were obtained by precipitating the virus with ammonium sulphate. The crystals belonged to rhombohedral space group R3 with α = 291·4 Å and α = 61·9°. Three-dimensional X-ray diffraction data on these crystals were collected to a resolution of 4·7 Å, using a Siemens-Nicolet area detector system. Self-rotation function studies revealed the icosahedral symmetry of the virus particles, as well as their precise orientation in the unit cell. Cross-rotation function and modelling studies with SBMV showed that it is a valid starting model for SMV structure determination. Low resolution phases computed using a polyalanine model of SBMV were subjected to refinement and extension by real-space electron density averaging and solvent flattening. The final electron density map revealed a polypeptide fold similar to SBMV. The single disulphide bridge of SBMV coat protein is retained in SMV. Four icosahedrally independent cation binding sites have been tentatively identified. Three of these sites, related by a quasi threefold axis, are also found in SBMV. The fourth site is situated on the quasi threefold axis. Aspartic acid residues, which replace Ile218 of SBMV from the quasi threefold-related subunits are suitable ligands to the cation at this site

HIGH-RESOLUTION IMAGING OF THE ATLBS REGIONS: THE RADIO SOURCE COUNTS

The Australia Telescope Low-brightness Survey (ATLBS) regions have been mosaic imaged at a radio frequency of 1.4 GHz with 6 `' angular resolution and 72 mu Jy beam(-1) rms noise. The images (centered at R. A. 00(h)35(m)00(s), decl. -67 degrees 00'00 `' and R. A. 00(h)59(m)17(s), decl. -67.00'00 `', J2000 epoch) cover 8.42 deg(2) sky area and have no artifacts or imaging errors above the image thermal noise. Multi-resolution radio and optical r-band images (made using the 4 m CTIO Blanco telescope) were used to recognize multi-component sources and prepare a source list; the detection threshold was 0.38 mJy in a low-resolution radio image made with beam FWHM of 50 `'. Radio source counts in the flux density range 0.4-8.7 mJy are estimated, with corrections applied for noise bias, effective area correction, and resolution bias. The resolution bias is mitigated using low-resolution radio images, while effects of source confusion are removed by using high-resolution images for identifying blended sources. Below 1 mJy the ATLBS counts are systematically lower than the previous estimates. Showing no evidence for an upturn down to 0.4 mJy, they do not require any changes in the radio source population down to the limit of the survey. The work suggests that automated image analysis for counts may be dependent on the ability of the imaging to reproduce connecting emission with low surface brightness and on the ability of the algorithm to recognize sources, which may require that source finding algorithms effectively work with multi-resolution and multi-wavelength data. The work underscores the importance of using source lists-as opposed to component lists-and correcting for the noise bias in order to precisely estimate counts close to the image noise and determine the upturn at sub-mJy flux density.

The Relationship Between Solar Coronal X-Ray Brightness and Active Region Magnetic Fields: A Study Using High-Resolution Hinode Observations

By using high-resolution observations of nearly co-temporal and co-spatial Solar Optical Telescope spectropolarimeter and X-Ray Telescope coronal X-ray data onboard Hinode, we revisit the problematic relationship between global magnetic quantities and coronal X-ray brightness. Co-aligned vector magnetogram and X-ray data were used for this study. The total X-ray brightness over active regions is well correlated with integrated magnetic quantities such as the total unsigned magnetic flux, the total unsigned vertical current, and the area-integrated square of the vertical and horizontal magnetic fields. On accounting for the inter-dependence of the magnetic quantities, we inferred that the total magnetic flux is the primary determinant of the observed integrated X-ray brightness. Our observations indicate that a stronger coronal X-ray flux is not related to a higher non-potentiality of active-region magnetic fields. The data even suggest a slightly negative correlation between X-ray brightness and a proxy of active-region non-potentiality. Although there are small numerical differences in the established correlations, the main conclusions are qualitatively consistent over two different X-ray filters, the Al-poly and Ti-poly filters, which confirms the strength of our conclusions and validate and extend earlier studies that used low-resolution data. We discuss the implications of our results and the constraints they set on theories of solar coronal heating.

Super Resolution Using a Single Image Dictionary

To perform super resolution of low resolution images, state-of-the-art methods are based on learning a pair of lowresolution and high-resolution dictionaries from multiple images. These trained dictionaries are used to replace patches in lowresolution image with appropriate matching patches from the high-resolution dictionary. In this paper we propose using a single common image as dictionary, in conjunction with approximate nearest neighbour fields (ANNF) to perform super resolution (SR). By using a common source image, we are able to bypass the learning phase and also able to reduce the dictionary from a collection of hundreds of images to a single image. By adapting recent developments in ANNF computation, to suit super-resolution, we are able to perform much faster and accurate SR than existing techniques. To establish this claim, we compare the proposed algorithm against various state-of-the-art algorithms, and show that we are able to achieve b etter and faster reconstruction without any training.

Synthetic image super resolution using FeatureMatch

In this paper, we propose a super resolution (SR) method for synthetic images using FeatureMatch. Existing state-of-the-art super resolution methods are learning based methods, where a pair of low-resolution and high-resolution dictionary pair are trained, and this trained pair is used to replace patches in low-resolution image with appropriate matching patches from the high-resolution dictionary. In this paper, we show that by using Approximate Nearest Neighbour Fields (ANNF), and a common source image, we can by-pass the learning phase, and use a single image for dictionary. Thus, reducing the dictionary from a collection obtained from hundreds of training images, to a single image. We show that by modifying the latest developments in ANNF computation, to suit super resolution, we can perform much faster and more accurate SR than existing techniques. To establish this claim we will compare our algorithm against various state-of-the-art algorithms, and show that we are able to achieve better and faster reconstruction without any training phase.

Toward a High-Resolution Monitoring of Continental Surface Water Extent and Dynamics, at Global Scale: from GIEMS (Global Inundation Extent from Multi-Satellites) to SWOT (Surface Water Ocean Topography)

Up to now, high-resolution mapping of surface water extent from satellites has only been available for a few regions, over limited time periods. The extension of the temporal and spatial coverage was difficult, due to the limitation of the remote sensing technique e.g., the interaction of the radiation with vegetation or cloud for visible observations or the temporal sampling with the synthetic aperture radar (SAR)]. The advantages and the limitations of the various satellite techniques are reviewed. The need to have a global and consistent estimate of the water surfaces over long time periods triggered the development of a multi-satellite methodology to obtain consistent surface water all over the globe, regardless of the environments. The Global Inundation Extent from Multi-satellites (GIEMS) combines the complementary strengths of satellite observations from the visible to the microwave, to produce a low-resolution monthly dataset () of surface water extent and dynamics. Downscaling algorithms are now developed and applied to GIEMS, using high-spatial-resolution information from visible, near-infrared, and synthetic aperture radar (SAR) satellite images, or from digital elevation models. Preliminary products are available down to 500-m spatial resolution. This work bridges the gaps and prepares for the future NASA/CNES Surface Water Ocean Topography (SWOT) mission to be launched in 2020. SWOT will delineate surface water extent estimates and their water storage with an unprecedented spatial resolution and accuracy, thanks to a SAR in an interferometry mode. When available, the SWOT data will be adopted to downscale GIEMS, to produce a long time series of water surfaces at global scale, consistent with the SWOT observations.

A Low-level Geographic Information System for coastal zone management, with applications to Brunei Darussalam: Part I: The concept and its design elements

This contribution is the first part of a four-part series documenting the development of B:RUN, a software program which reads data for common spreadsheets and presents them as low-resolution maps of slates and processes. The program emerged from a need which arose during a project in Brunei Darussalam for a 'low level' approach for researchers to communicate findings as efficiently and expeditiously as possible. Part I provides a overview of the concept and design elements of B:RUN. Part II will highlight results of the economics components of the program evaluating different fishing regimes, sailing distances from ports and fleet operating costs. Environmental aspects will be presented in Part III in the form of overlay maps. Part IV will summarize the implications of B:RUN results to coastal and fishery resources management in Brunei Darussalam and show how this approach can be adapted to other coastlines and used as a teaching and training tool. The following three parts will be published in future editions of Naga, the ICLARM Quarterly. The program is available through ICLARM.

A 1,000 Frames/s Programmable Vision Chip with Variable Resolution and Row-Pixel-Mixed Parallel Image Processors

A programmable vision chip with variable resolution and row-pixel-mixed parallel image processors is presented. The chip consists of a CMOS sensor array, with row-parallel 6-bit Algorithmic ADCs, row-parallel gray-scale image processors, pixel-parallel SIMD Processing Element (PE) array, and instruction controller. The resolution of the image in the chip is variable: high resolution for a focused area and low resolution for general view. It implements gray-scale and binary mathematical morphology algorithms in series to carry out low-level and mid-level image processing and sends out features of the image for various applications. It can perform image processing at over 1,000 frames/s (fps). A prototype chip with 64 x 64 pixels resolution and 6-bit gray-scale image is fabricated in 0.18 mu m Standard CMOS process. The area size of chip is 1.5 mm x 3.5 mm. Each pixel size is 9.5 mu m x 9.5 mu m and each processing element size is 23 mu m x 29 mu m. The experiment results demonstrate that the chip can perform low-level and mid-level image processing and it can be applied in the real-time vision applications, such as high speed target tracking.