Learning and Exploiting Camera Geometry for Computer Vision

This work explores the use of statistical methods in describing and estimating camera poses, as well as the information feedback loop between camera pose and object detection. Surging development in robotics and computer vision has pushed the need for algorithms that infer, understand, and utilize information about the position and orientation of the sensor platforms when observing and/or interacting with their environment.

The first contribution of this thesis is the development of a set of statistical tools for representing and estimating the uncertainty in object poses. A distribution for representing the joint uncertainty over multiple object positions and orientations is described, called the mirrored normal-Bingham distribution. This distribution generalizes both the normal distribution in Euclidean space, and the Bingham distribution on the unit hypersphere. It is shown to inherit many of the convenient properties of these special cases: it is the maximum-entropy distribution with fixed second moment, and there is a generalized Laplace approximation whose result is the mirrored normal-Bingham distribution. This distribution and approximation method are demonstrated by deriving the analytical approximation to the wrapped-normal distribution. Further, it is shown how these tools can be used to represent the uncertainty in the result of a bundle adjustment problem.

Another application of these methods is illustrated as part of a novel camera pose estimation algorithm based on object detections. The autocalibration task is formulated as a bundle adjustment problem using prior distributions over the 3D points to enforce the objects' structure and their relationship with the scene geometry. This framework is very flexible and enables the use of off-the-shelf computational tools to solve specialized autocalibration problems. Its performance is evaluated using a pedestrian detector to provide head and foot location observations, and it proves much faster and potentially more accurate than existing methods.

Finally, the information feedback loop between object detection and camera pose estimation is closed by utilizing camera pose information to improve object detection in scenarios with significant perspective warping. Methods are presented that allow the inverse perspective mapping traditionally applied to images to be applied instead to features computed from those images. For the special case of HOG-like features, which are used by many modern object detection systems, these methods are shown to provide substantial performance benefits over unadapted detectors while achieving real-time frame rates, orders of magnitude faster than comparable image warping methods.

The statistical tools and algorithms presented here are especially promising for mobile cameras, providing the ability to autocalibrate and adapt to the camera pose in real time. In addition, these methods have wide-ranging potential applications in diverse areas of computer vision, robotics, and imaging.

Visualizing Rare Watson-Crick-Like Tautomeric and Anionic Mismatches in DNA and RNA

The central dogma of molecular biology relies on the correct Watson-Crick (WC) geometry of canonical deoxyribonucleic acid (DNA) dG•dC and dA•dT base pairs to replicate and transcribe genetic information with speed and an astonishing level of fidelity. In addition, the Watson-Crick geometry of canonical ribonucleic acid (RNA) rG•rC and rA•rU base pairs is highly conserved to ensure that proteins are translated with high fidelity. However, numerous other potential nucleobase tautomeric and ionic configurations are possible that can give rise to entirely new pairing modes between the nucleotide bases. Very early on, James Watson and Francis Crick recognized their importance and in 1953 postulated that if bases adopted one of their less energetically disfavored tautomeric forms (and later ionic forms) during replication it could lead to the formation of a mismatch with a Watson-Crick-like geometry and could give rise to “natural mutations.”

Since this time numerous studies have provided evidence in support of this hypothesis and have expanded upon it; computational studies have addressed the energetic feasibilities of different nucleobases’ tautomeric and ionic forms in siico; crystallographic studies have trapped different mismatches with WC-like geometries in polymerase or ribosome active sites. However, no direct evidence has been given for (i) the direct existence of these WC-like mismatches in canonical DNA duplex, RNA duplexes, or non-coding RNAs; (ii) which, if any, tautomeric or ionic form stabilizes the WC-like geometry. This thesis utilizes nuclear magnetic resonance (NMR) spectroscopy and rotating frame relaxation dispersion (R1ρ RD) in combination with density functional theory (DFT), biochemical assays, and targeted chemical perturbations to show that (i) dG•dT mismatches in DNA duplexes, as well as rG•rU mismatches RNA duplexes and non-coding RNAs, transiently adopt a WC-like geometry that is stabilized by (ii) an interconnected network of rapidly interconverting rare tautomers and anionic bases. These results support Watson and Crick’s tautomer hypothesis, but additionally support subsequent hypotheses invoking anionic mismatches and ultimately tie them together. This dissertation shows that a common mismatch can adopt a Watson-Crick-like geometry globally, in both DNA and RNA, and whose geometry is stabilized by a kinetically linked network of rare tautomeric and anionic bases. The studies herein also provide compelling evidence for their involvement in spontaneous replication and translation errors.