Temporal Coding of Volumetric Imagery

'Image volumes' refer to realizations of images in other dimensions such as time, spectrum, and focus. Recent advances in scientific, medical, and consumer applications demand improvements in image volume capture. Though image volume acquisition continues to advance, it maintains the same sampling mechanisms that have been used for decades; every voxel must be scanned and is presumed independent of its neighbors. Under these conditions, improving performance comes at the cost of increased system complexity, data rates, and power consumption.

This dissertation explores systems and methods capable of efficiently improving sensitivity and performance for image volume cameras, and specifically proposes several sampling strategies that utilize temporal coding to improve imaging system performance and enhance our awareness for a variety of dynamic applications.

Video cameras and camcorders sample the video volume (x,y,t) at fixed intervals to gain understanding of the volume's temporal evolution. Conventionally, one must reduce the spatial resolution to increase the framerate of such cameras. Using temporal coding via physical translation of an optical element known as a coded aperture, the compressive temporal imaging (CACTI) camera emonstrates a method which which to embed the temporal dimension of the video volume into spatial (x,y) measurements, thereby greatly improving temporal resolution with minimal loss of spatial resolution. This technique, which is among a family of compressive sampling strategies developed at Duke University, temporally codes the exposure readout functions at the pixel level.

Since video cameras nominally integrate the remaining image volume dimensions (e.g. spectrum and focus) at capture time, spectral (x,y,t,\lambda) and focal (x,y,t,z) image volumes are traditionally captured via sequential changes to the spectral and focal state of the system, respectively. The CACTI camera's ability to embed video volumes into images leads to exploration of other information within that video; namely, focal and spectral information. The next part of the thesis demonstrates derivative works of CACTI: compressive extended depth of field and compressive spectral-temporal imaging. These works successfully show the technique's extension of temporal coding to improve sensing performance in these other dimensions.

Geometrical optics-related tradeoffs, such as the classic challenges of wide-field-of-view and high resolution photography, have motivated the development of mulitscale camera arrays. The advent of such designs less than a decade ago heralds a new era of research- and engineering-related challenges. One significant challenge is that of managing the focal volume (x,y,z) over wide fields of view and resolutions. The fourth chapter shows advances on focus and image quality assessment for a class of multiscale gigapixel cameras developed at Duke.

Along the same line of work, we have explored methods for dynamic and adaptive addressing of focus via point spread function engineering. We demonstrate another form of temporal coding in the form of physical translation of the image plane from its nominal focal position. We demonstrate this technique's capability to generate arbitrary point spread functions.

Learning and Exploiting Camera Geometry for Computer Vision

This work explores the use of statistical methods in describing and estimating camera poses, as well as the information feedback loop between camera pose and object detection. Surging development in robotics and computer vision has pushed the need for algorithms that infer, understand, and utilize information about the position and orientation of the sensor platforms when observing and/or interacting with their environment.

The first contribution of this thesis is the development of a set of statistical tools for representing and estimating the uncertainty in object poses. A distribution for representing the joint uncertainty over multiple object positions and orientations is described, called the mirrored normal-Bingham distribution. This distribution generalizes both the normal distribution in Euclidean space, and the Bingham distribution on the unit hypersphere. It is shown to inherit many of the convenient properties of these special cases: it is the maximum-entropy distribution with fixed second moment, and there is a generalized Laplace approximation whose result is the mirrored normal-Bingham distribution. This distribution and approximation method are demonstrated by deriving the analytical approximation to the wrapped-normal distribution. Further, it is shown how these tools can be used to represent the uncertainty in the result of a bundle adjustment problem.

Another application of these methods is illustrated as part of a novel camera pose estimation algorithm based on object detections. The autocalibration task is formulated as a bundle adjustment problem using prior distributions over the 3D points to enforce the objects' structure and their relationship with the scene geometry. This framework is very flexible and enables the use of off-the-shelf computational tools to solve specialized autocalibration problems. Its performance is evaluated using a pedestrian detector to provide head and foot location observations, and it proves much faster and potentially more accurate than existing methods.

Finally, the information feedback loop between object detection and camera pose estimation is closed by utilizing camera pose information to improve object detection in scenarios with significant perspective warping. Methods are presented that allow the inverse perspective mapping traditionally applied to images to be applied instead to features computed from those images. For the special case of HOG-like features, which are used by many modern object detection systems, these methods are shown to provide substantial performance benefits over unadapted detectors while achieving real-time frame rates, orders of magnitude faster than comparable image warping methods.

The statistical tools and algorithms presented here are especially promising for mobile cameras, providing the ability to autocalibrate and adapt to the camera pose in real time. In addition, these methods have wide-ranging potential applications in diverse areas of computer vision, robotics, and imaging.

Validation of Coded Aperture Coherent Scatter Spectral Imaging for Differentiation of Normal and Neoplastic Breast Tissues via Surgical Pathology

This study intends to validate the sensitivity and specificity of coded aperture coherent scatter spectral imaging (CACSSI) by comparison to clinical histological preparation and pathologic analysis methods currently used for the differentiation of normal and neoplastic breast tissues. A composite overlay of the CACSSI rendered image and pathologist interpreted, stained sections validate the ability of coherent scatter imaging to differentiate cancerous tissues from normal, healthy breast structures ex-vivo. Via comparison to the pathologist annotated slides, the CACSSI system may be further optimized to maximized sensitivity and specificity for differentiation of breast carcinomas.

Physical and Genetic Analysis of the CUP1 Tandem Array in the Yeast Saccharomyces cerevisiae

The genomes of many strains of baker’s yeast, Saccharomyces cerevisiae, contain multiple repeats of the copper-binding protein Cup1. Cup1 is a member of the metallothionein family, and is found in a tandem array on chromosome VIII. In this thesis, I describe studies that characterized these tandem arrays and their mechanism of formation across diverse strains of yeast. I show that CUP1 arrays are an illuminating model system for observing recombination in eukaryotes, and describe insights derived from these observations.

In our first study, we analyzed 101 natural isolates of S. cerevisiae in order to examine the diversity of CUP1-containing repeats across different strains. We identified five distinct classes of repeats that contain CUP1. We also showed that some strains have only a single copy of CUP1. By comparing the sequences of all the strains, we were able to elucidate the mechanism of formation of the CUP1 tandem arrays, which involved unequal non-homologous recombination events starting from a strain that had only a single CUP1 gene. Our observation of CUP1 repeat formation allows more general insights about the formation of tandem repeats from single-copy genes in eukaryotes, which is one of the most important mechanisms by which organisms evolve.

In our second study, we delved deeper into our mechanistic investigations by measuring the relative rates of inter-homolog and intra-/inter-sister chromatid recombination in CUP1 tandem arrays. We used a diploid strain that is heterozygous both for insertion of a selectable marker (URA3) inside the tandem array, and also for markers at either end of the array. The intra-/inter-sister chromatid recombination rate turned out to be more than ten-fold greater than the inter-homolog rate. Moreover, we found that loss of the proteins Rad51 and Rad52, which are required for most inter-homolog recombination, did not greatly reduce recombination in the CUP1 tandem repeats. Additionally, we investigated the effects of elevated copper levels on the rate of each type of recombination at the CUP1 locus. Both types of recombination are increased at high concentrations of copper (as is known to be the case for CUP1 transcription). Furthermore, the inter-homolog recombination rate at the CUP1 locus is higher than the average over the genome during mitosis, but is lower than the average during meiosis.

The research described in Chapter 2 is published in 2014.