A unifying resolution-independent formulation for early vision

We present a model for early vision tasks such as denoising, super-resolution, deblurring, and demosaicing. The model provides a resolution-independent representation of discrete images which admits a truly rotationally invariant prior. The model generalizes several existing approaches: variational methods, finite element methods, and discrete random fields. The primary contribution is a novel energy functional which has not previously been written down, which combines the discrete measurements from pixels with a continuous-domain world viewed through continous-domain point-spread functions. The value of the functional is that simple priors (such as total variation and generalizations) on the continous-domain world become realistic priors on the sampled images. We show that despite its apparent complexity, optimization of this model depends on just a few computational primitives, which although tedious to derive, can now be reused in many domains. We define a set of optimization algorithms which greatly overcome the apparent complexity of this model, and make possible its practical application. New experimental results include infinite-resolution upsampling, and a method for obtaining subpixel superpixels. © 2012 IEEE.

Automated road pavement marking detection from high resolution aerial images based on Multi-resolution image analysis and anisotropic Gaussian filtering

Road features extraction from remote sensed imagery has been a long-term topic of great interest within the photogrammetry and remote sensing communities for over three decades. The majority of the early work only focused on linear feature detection approaches, with restrictive assumption on image resolution and road appearance. The widely available of high resolution digital aerial images makes it possible to extract sub-road features, e.g. road pavement markings. In this paper, we will focus on the automatic extraction of road lane markings, which are required by various lane-based vehicle applications, such as, autonomous vehicle navigation, and lane departure warning. The proposed approach consists of three phases: i) road centerline extraction from low resolution image, ii) road surface detection in the original image, and iii) pavement marking extraction on the generated road surface. The proposed method was tested on the aerial imagery dataset of the Bruce Highway, Queensland, and the results demonstrate the efficiency of our approach.

Super-resolved faces for improved face recognition from surveillance video advances in biometrics

Characteristics of surveillance video generally include low resolution and poor quality due to environmental, storage and processing limitations. It is extremely difficult for computers and human operators to identify individuals from these videos. To overcome this problem, super-resolution can be used in conjunction with an automated face recognition system to enhance the spatial resolution of video frames containing the subject and narrow down the number of manual verifications performed by the human operator by presenting a list of most likely candidates from the database. As the super-resolution reconstruction process is ill-posed, visual artifacts are often generated as a result. These artifacts can be visually distracting to humans and/or affect machine recognition algorithms. While it is intuitive that higher resolution should lead to improved recognition accuracy, the effects of super-resolution and such artifacts on face recognition performance have not been systematically studied. This paper aims to address this gap while illustrating that super-resolution allows more accurate identification of individuals from low-resolution surveillance footage. The proposed optical flow-based super-resolution method is benchmarked against Baker et al.’s hallucination and Schultz et al.’s super-resolution techniques on images from the Terrascope and XM2VTS databases. Ground truth and interpolated images were also tested to provide a baseline for comparison. Results show that a suitable super-resolution system can improve the discriminability of surveillance video and enhance face recognition accuracy. The experiments also show that Schultz et al.’s method fails when dealing surveillance footage due to its assumption of rigid objects in the scene. The hallucination and optical flow-based methods performed comparably, with the optical flow-based method producing less visually distracting artifacts that interfered with human recognition.

Burnt area mapping in insular Southeast Asia using medium resolution satellite imagery

Burnt area mapping in humid tropical insular Southeast Asia using medium resolution (250-500m) satellite imagery is characterized by persisting cloud cover, wide range of land cover types, vast amount of wetland areas and highly varying fire regimes. The objective of this study was to deepen understanding of three major aspects affecting the implementation and limits of medium resolution burnt area mapping in insular Southeast Asia: 1) fire-induced spectral changes, 2) most suitable multitemporal compositing methods and 3) burn scars patterns and size distribution. The results revealed a high variation in fire-induced spectral changes depending on the pre-fire greenness of burnt area. It was concluded that this variation needs to be taken into account in change detection based burnt area mapping algorithms in order to maximize the potential of medium resolution satellite data. Minimum near infrared (MODIS band 2, 0.86μm) compositing method was found to be the most suitable for burnt area mapping purposes using Moderate Resolution Imaging Spectroradiometer (MODIS) data. In general, medium resolution burnt area mapping was found to be usable in the wetlands of insular Southeast Asia, whereas in other areas the usability was seriously jeopardized by the small size of burn scars. The suitability of medium resolution data for burnt area mapping in wetlands is important since recently Southeast Asian wetlands have become a major point of interest in many fields of science due to yearly occurring wild fires that not only degrade these unique ecosystems but also create regional haze problem and release globally significant amounts of carbon into the atmosphere due to burning peat. Finally, super-resolution MODIS images were tested but the test failed to improve the detection of small scars. Therefore, super-resolution technique was not considered to be applicable to regional level burnt area mapping in insular Southeast Asia.

Simultaneous multilayer scanning and detection for multiphoton fluorescence microscopy

Fast three-dimensional (3D) imaging requires parallel optical slicing of a specimen with an efficient detection scheme. The generation of multiple localized dot-like excitation structures solves the problem of simultaneous slicing multiple specimen layers, but an efficient detection scheme is necessary. Confocal theta detection (detection at 90 degrees to the optical axis) provides a suitable detection platform that is capable of cross-talk-free fluorescence detection from each nanodot (axial dimension approximate to 150 nm). Additionally, this technique has the unique feature of imaging a specimen at a large working distance with super-resolution capabilities. Polarization studies show distinct field structures for fixed and fluid samples, indicating a non-negligible field-dipole interaction. The realization of the proposed imaging technique will advance and diversify multiphoton fluorescence microscopy for numerous applications in nanobioimaging and optical engineering.

Low Resolution Face Recognition Across Variations in Pose and Illumination

We propose a completely automatic approach for recognizing low resolution face images captured in uncontrolled environment. The approach uses multidimensional scaling to learn a common transformation matrix for the entire face which simultaneously transforms the facial features of the low resolution and the high resolution training images such that the distance between them approximates the distance had both the images been captured under the same controlled imaging conditions. Stereo matching cost is used to obtain the similarity of two images in the transformed space. Though this gives very good recognition performance, the time taken for computing the stereo matching cost is significant. To overcome this limitation, we propose a reference-based approach in which each face image is represented by its stereo matching cost from a few reference images. Experimental evaluation on the real world challenging databases and comparison with the state-of-the-art super-resolution, classifier based and cross modal synthesis techniques show the effectiveness of the proposed algorithm.

超分辨近场结构中二元共晶合金掩膜的工作机制

超分辨近场结构(super-RENS)技术通过在传统光盘结构中插入掩膜结构而实现近场超分辨,是目前最具实用化前景的超高密度光存储技术之一,其中掩膜层的近场光学特性是决定其光存储性能的关键。利用三维时域有限差分法(3D-FDTD)对合金掩膜的近场光强分布进行了数值仿真和分析,提出二元共晶合金薄膜在激光作用下形成的规则微结构可能是以其作为掩膜层的超分辨近场结构光盘产生较高信噪比(SNR)的原因。

Photon nanojet lens: Design, fabrication and characterization

In this paper, a novel nanolens with super resolution, based on the photon nanojet effect through dielectric nanostructures in visible wavelengths, is proposed. The nanolens is made from plastic SU-8, consisting of parallel semi-cylinders in an array. This paper focuses on the lens designed by numerical simulation with the finite-difference time domain method and nanofabrication of the lens by grayscale electron beam lithography combined with a casting/bonding/lift-off transfer process. Monte Carlo simulation for injected charge distribution and development modeling was applied to define the resultant 3D profile in PMMA as the template for the lens shape. After the casting/bonding/lift-off process, the fabricated nanolens in SU-8 has the desired lens shape, very close to that of PMMA, indicating that the pattern transfer process developed in this work can be reliably applied not only for the fabrication of the lens but also for other 3D nanopatterns in general. The light distribution through the lens near its surface was initially characterized by a scanning near-field optical microscope, showing a well defined focusing image of designed grating lines. Such focusing function supports the great prospects of developing a novel nanolithography based on the photon nanojet effect.

Investigating global tropical cyclone activity with a hierarchy of AGCMs: the role of model resolution

The ability to run General Circulation Models (GCMs) at ever-higher horizontal resolutions has meant that tropical cyclone simulations are increasingly credible. A hierarchy of atmosphere-only GCMs, based on the Hadley Centre Global Environmental Model (HadGEM1), with horizontal resolution increasing from approximately 270km to 60km (at 50N), is used to systematically investigate the impact of spatial resolution on the simulation of global tropical cyclone activity, independent of model formulation. Tropical cyclones are extracted from ensemble simulations and reanalyses of comparable resolutions using a feature-tracking algorithm. Resolution is critical for simulating storm intensity and convergence to observed storm intensities is not achieved with the model hierarchy. Resolution is less critical for simulating the annual number of tropical cyclones and their geographical distribution, which are well captured at resolutions of 135km or higher, particularly for Northern Hemisphere basins. Simulating the interannual variability of storm occurrence requires resolutions of 100km or higher; however, the level of skill is basin dependent. Higher resolution GCMs are increasingly able to capture the interannual variability of the large-scale environmental conditions that contribute to tropical cyclogenesis. Different environmental factors contribute to the interannual variability of tropical cyclones in the different basins: in the North Atlantic basin the vertical wind shear, potential intensity and low-level absolute vorticity are dominant, while in the North Pacific basins mid-level relative humidity and low-level absolute vorticity are dominant. Model resolution is crucial for a realistic simulation of tropical cyclone behaviour, and high-resolution GCMs are found to be valuable tools for investigating the global location and frequency of tropical cyclones.

Development of Multi-modal and Super-resolved Retinal Imaging Systems

Advancements in retinal imaging technologies have drastically improved the quality of eye care in the past couple decades. Scanning laser ophthalmoscopy (SLO) and optical coherence tomography (OCT) are two examples of critical imaging modalities for the diagnosis of retinal pathologies. However current-generation SLO and OCT systems have limitations in diagnostic capability due to the following factors: the use of bulky tabletop systems, monochromatic imaging, and resolution degradation due to ocular aberrations and diffraction.

Bulky tabletop SLO and OCT systems are incapable of imaging patients that are supine, under anesthesia, or otherwise unable to maintain the required posture and fixation. Monochromatic SLO and OCT imaging prevents the identification of various color-specific diagnostic markers visible with color fundus photography like those of neovascular age-related macular degeneration. Resolution degradation due to ocular aberrations and diffraction has prevented the imaging of photoreceptors close to the fovea without the use of adaptive optics (AO), which require bulky and expensive components that limit the potential for widespread clinical use.

In this dissertation, techniques for extending the diagnostic capability of SLO and OCT systems are developed. These techniques include design strategies for miniaturizing and combining SLO and OCT to permit multi-modal, lightweight handheld probes to extend high quality retinal imaging to pediatric eye care. In addition, a method for extending true color retinal imaging to SLO to enable high-contrast, depth-resolved, high-fidelity color fundus imaging is demonstrated using a supercontinuum light source. Finally, the development and combination of SLO with a super-resolution confocal microscopy technique known as optical photon reassignment (OPRA) is demonstrated to enable high-resolution imaging of retinal photoreceptors without the use of adaptive optics.

Investigating the Structure of FtsZ to Understand its Functional Role in Bacterial Cell Division

FtsZ, a bacterial tubulin homologue, is a cytoskeleton protein that plays key roles in cytokinesis of almost all prokaryotes. FtsZ assembles into protofilaments (pfs), one subunit thick, and these pfs assemble further to form a “Z ring” at the center of prokaryotic cells. The Z ring generates a constriction force on the inner membrane, and also serves as a scaffold to recruit cell-wall remodeling proteins for complete cell division in vivo. FtsZ can be subdivided into 3 main functional regions: globular domain, C terminal (Ct) linker, and Ct peptide. The globular domain binds GTP to assembles the pfs. The extreme Ct peptide binds membrane proteins to allow cytoplasmic FtsZ to function at the inner membrane. The Ct linker connects the globular domain and Ct peptide. In the present studies, we used genetic and structural approaches to investigate the function of Escherichia coli (E. coli) FtsZ. We sought to examine three questions: (1) Are lateral bonds between pfs essential for the Z ring? (2) Can we improve direct visualization of FtsZ in vivo by engineering an FtsZ-FP fusion that can function as the sole source of FtsZ for cell division? (3) Is the divergent Ct linker of FtsZ an intrinsically disordered peptide (IDP)?

One model of the Z ring proposes that pfs associate via lateral bonds to form ribbons; however, lateral bonds are still only hypothetical. To explore potential lateral bonding sites, we probed the surface of E. coli FtsZ by inserting either small peptides or whole FPs. Of the four lateral surfaces on FtsZ pfs, we obtained inserts on the front and back surfaces that were functional for cell division. We concluded that these faces are not sites of essential interactions. Inserts at two sites, G124 and R174 located on the left and right surfaces, completely blocked function, and were identified as possible sites for essential lateral interactions. Another goal was to find a location within FtsZ that supported fusion of FP reporter proteins, while allowing the FtsZ-FP to function as the sole source of FtsZ. We discovered one internal site, G55-Q56, where several different FPs could be inserted without impairing function. These FtsZ-FPs may provide advances for imaging Z-ring structure by super-resolution techniques.

The Ct linker is the most divergent region of FtsZ in both sequence and length. In E. coli FtsZ the Ct linker is 50 amino acids (aa), but for other FtsZ it can be as short as 37 aa or as long as 250 aa. The Ct linker has been hypothesized to be an IDP. In the present study, circular dichroism confirmed that isolated Ct linkers of E. coli (50 aa) and C. crescentus (175 aa) are IDPs. Limited trypsin proteolysis followed by mass spectrometry (LC-MS/MS) confirmed Ct linkers of E. coli (50 aa) and B. subtilis (47 aa) as IDPs even when still attached to the globular domain. In addition, we made chimeras, swapping the E. coli Ct linker for other peptides and proteins. Most chimeras allowed for normal cell division in E. coli, suggesting that IDPs with a length of 43 to 95 aa are tolerated, sequence has little importance, and electrostatic charge is unimportant. Several chimeras were purified to confirm the effect they had on pf assembly. We concluded that the Ct linker functions as a flexible tether allowing for force to be transferred from the FtsZ pf to the membrane to constrict the septum for division.

Human identification at a distance using iris and face

This research has successfully applied super-resolution and multiple modality fusion techniques to address the major challenges of human identification at a distance using face and iris. The outcome of the research is useful for security applications.