Learning and Exploiting Camera Geometry for Computer Vision

This work explores the use of statistical methods in describing and estimating camera poses, as well as the information feedback loop between camera pose and object detection. Surging development in robotics and computer vision has pushed the need for algorithms that infer, understand, and utilize information about the position and orientation of the sensor platforms when observing and/or interacting with their environment.

The first contribution of this thesis is the development of a set of statistical tools for representing and estimating the uncertainty in object poses. A distribution for representing the joint uncertainty over multiple object positions and orientations is described, called the mirrored normal-Bingham distribution. This distribution generalizes both the normal distribution in Euclidean space, and the Bingham distribution on the unit hypersphere. It is shown to inherit many of the convenient properties of these special cases: it is the maximum-entropy distribution with fixed second moment, and there is a generalized Laplace approximation whose result is the mirrored normal-Bingham distribution. This distribution and approximation method are demonstrated by deriving the analytical approximation to the wrapped-normal distribution. Further, it is shown how these tools can be used to represent the uncertainty in the result of a bundle adjustment problem.

Another application of these methods is illustrated as part of a novel camera pose estimation algorithm based on object detections. The autocalibration task is formulated as a bundle adjustment problem using prior distributions over the 3D points to enforce the objects' structure and their relationship with the scene geometry. This framework is very flexible and enables the use of off-the-shelf computational tools to solve specialized autocalibration problems. Its performance is evaluated using a pedestrian detector to provide head and foot location observations, and it proves much faster and potentially more accurate than existing methods.

Finally, the information feedback loop between object detection and camera pose estimation is closed by utilizing camera pose information to improve object detection in scenarios with significant perspective warping. Methods are presented that allow the inverse perspective mapping traditionally applied to images to be applied instead to features computed from those images. For the special case of HOG-like features, which are used by many modern object detection systems, these methods are shown to provide substantial performance benefits over unadapted detectors while achieving real-time frame rates, orders of magnitude faster than comparable image warping methods.

The statistical tools and algorithms presented here are especially promising for mobile cameras, providing the ability to autocalibrate and adapt to the camera pose in real time. In addition, these methods have wide-ranging potential applications in diverse areas of computer vision, robotics, and imaging.

The role of binocular vision in primate locomotion.

Dissertation

Spatial vision in the purple sea urchin Strongylocentrotus purpuratus (Echinoidea).

Recent evidence that echinoids of the genus Echinometra have moderate visual acuity that appears to be mediated by their spines screening off-axis light suggests that the urchin Strongylocentrotus purpuratus, with its higher spine density, may have even more acute spatial vision. We analyzed the movements of 39 specimens of S. purpuratus after they were placed in the center of a featureless tank containing a round, black target that had an angular diameter of 6.5 deg. or 10 deg. (solid angles of 0.01 sr and 0.024 sr, respectively). An average orientation vector for each urchin was determined by testing the animal four times, with the target placed successively at bearings of 0 deg., 90 deg., 180 deg. and 270 deg. (relative to magnetic east). The urchins showed no significant unimodal or axial orientation relative to any non-target feature of the environment or relative to the changing position of the 6.5 deg. target. However, the urchins were strongly axially oriented relative to the changing position of the 10 deg. target (mean axis from -1 to 179 deg.; 95% confidence interval +/- 12 deg.; P<0.001, Moore's non-parametric Hotelling's test), with 10 of the 20 urchins tested against that target choosing an average bearing within 10 deg. of either the target center or its opposite direction (two would be expected by chance). In addition, the average length of the 20 target-normalized bearings for the 10 deg. target (each the vector sum of the bearings for the four trials) were far higher than would be expected by chance (P<10(-10); Monte Carlo simulation), showing that each urchin, whether it moved towards or away from the target, did so with high consistency. These results strongly suggest that S. purpuratus detected the 10 deg. target, responding either by approaching it or fleeing it. Given that the urchins did not appear to respond to the 6.5 deg. target, it is likely that the 10 deg. target was close to the minimum detectable size for this species. Interestingly, measurements of the spine density of the regions of the test that faced horizontally predicted a similar visual resolution (8.3+/-0.5 deg. for the interambulacrum and 11+/-0.54 deg. for the ambulacrum). The function of this relatively low, but functional, acuity - on par with that of the chambered Nautilus and the horseshoe crab - is unclear but, given the bimodal response, is likely to be related to both shelter seeking and predator avoidance.

Quantifiable biomarkers of normal aging in the Japanese medaka fish (Oryzias latipes).

BACKGROUND: Small laboratory fish share many anatomical and histological characteristics with other vertebrates, yet can be maintained in large numbers at low cost for lifetime studies. Here we characterize biomarkers associated with normal aging in the Japanese medaka (Oryzias latipes), a species that has been widely used in toxicology studies and has potential utility as a model organism for experimental aging research. PRINCIPAL FINDINGS: The median lifespan of medaka was approximately 22 months under laboratory conditions. We performed quantitative histological analysis of tissues from age-grouped individuals representing young adults (6 months old), mature adults (16 months old), and adults that had survived beyond the median lifespan (24 months). Livers of 24-month old individuals showed extensive morphologic changes, including spongiosis hepatis, steatosis, ballooning degeneration, inflammation, and nuclear pyknosis. There were also phagolysosomes, vacuoles, and residual bodies in parenchymal cells and congestion of sinusoidal vessels. Livers of aged individuals were characterized by increases in lipofuscin deposits and in the number of TUNEL-positive apoptotic cells. Some of these degenerative characteristics were seen, to a lesser extent, in the livers of 16-month old individuals, but not in 6-month old individuals. The basal layer of the dermis showed an age-dependent decline in the number of dividing cells and an increase in senescence-associated β-galactosidase. The hearts of aged individuals were characterized by fibrosis and lipofuscin deposition. There was also a loss of pigmented cells from the retinal epithelium. By contrast, age-associated changes were not apparent in skeletal muscle, the ocular lens, or the brain. SIGNIFICANCE: The results provide a set of markers that can be used to trace the process of normal tissue aging in medaka and to evaluate the effect of environmental stressors.

In vivo inhibition of elevated myocardial beta-adrenergic receptor kinase activity in hybrid transgenic mice restores normal beta-adrenergic signaling and function.

BACKGROUND: The clinical syndrome of heart failure (HF) is characterized by an impaired cardiac beta-adrenergic receptor (betaAR) system, which is critical in the regulation of myocardial function. Expression of the betaAR kinase (betaARK1), which phosphorylates and uncouples betaARs, is elevated in human HF; this likely contributes to the abnormal betaAR responsiveness that occurs with beta-agonist administration. We previously showed that transgenic mice with increased myocardial betaARK1 expression had impaired cardiac function in vivo and that inhibiting endogenous betaARK1 activity in the heart led to enhanced myocardial function. METHODS AND RESULTS: We created hybrid transgenic mice with cardiac-specific concomitant overexpression of both betaARK1 and an inhibitor of betaARK1 activity to study the feasibility and functional consequences of the inhibition of elevated betaARK1 activity similar to that present in human HF. Transgenic mice with myocardial overexpression of betaARK1 (3 to 5-fold) have a blunted in vivo contractile response to isoproterenol when compared with non-transgenic control mice. In the hybrid transgenic mice, although myocardial betaARK1 levels remained elevated due to transgene expression, in vitro betaARK1 activity returned to control levels and the percentage of betaARs in the high-affinity state increased to normal wild-type levels. Furthermore, the in vivo left ventricular contractile response to betaAR stimulation was restored to normal in the hybrid double-transgenic mice. CONCLUSIONS: Novel hybrid transgenic mice can be created with concomitant cardiac-specific overexpression of 2 independent transgenes with opposing actions. Elevated myocardial betaARK1 in transgenic mouse hearts (to levels seen in human HF) can be inhibited in vivo by a peptide that can prevent agonist-stimulated desensitization of cardiac betaARs. This may represent a novel strategy to improve myocardial function in the setting of compromised heart function.

Elevated C-peptide and insulin predict increased risk of colorectal adenomas in normal mucosa.

BACKGROUND: Lower concentrations of the insulin-like growth factor binding protein-1 (IGFBP-1) and elevated concentrations of insulin or C-peptide have been associated with an increase in colorectal cancer risk (CRC). However few studies have evaluated IGFBP-1 and C-peptide in relation to adenomatous polyps, the only known precursor for CRC. METHODS: Between November 2001 and December 2002, we examined associations between circulating concentrations of insulin, C-peptide, IGFBP-1 and apoptosis among 190 individuals with one or more adenomatous polyps and 488 with no adenomatous polyps using logistic regression models. RESULTS: Individuals with the highest concentrations of C-peptide were more likely to have adenomas (OR = 2.2, 95% CI 1.4-4.0) than those with the lowest concentrations; associations that appeared to be stronger in men (OR = 4.4, 95% CI 1.7-10.9) than women. Individuals with high insulin concentrations also had a higher risk of adenomas (OR = 3.5, 95% CI 1.7-7.4), whereas higher levels of IGFBP-1 were associated with a reduced risk of adenomas in men only (OR = 0.3, 95% CI 0.1-0.7). Overweight and obese individuals with higher C-peptide levels (>1(st) Q) were at increased risk for lower apoptosis index (OR = 2.5, 95% CI 0.9-7.1), an association that remained strong in overweight and obese men (OR = 6.3, 95% CI 1.0-36.7). Higher levels of IGFBP-1 in overweight and obese individuals were associated with a reduced risk of low apoptosis (OR = 0.3, 95% CI 0.1-1.0). CONCLUSIONS: Associations between these peptides and the apoptosis index in overweight and obese individuals, suggest that the mechanism by which C-peptide could induce adenomas may include its anti-apoptotic properties. This study suggests that hyperinsulinemia and IGF hormones predict adenoma risk, and that outcomes associated with colorectal carcinogenesis maybe modified by gender.

An immunoglobulin C kappa-reactive single chain antibody fusion protein induces tolerance through receptor editing in a normal polyclonal immune system.

Understanding immune tolerance mechanisms is a major goal of immunology research, but mechanistic studies have generally required the use of mouse models carrying untargeted or targeted antigen receptor transgenes, which distort lymphocyte development and therefore preclude analysis of a truly normal immune system. Here we demonstrate an advance in in vivo analysis of immune tolerance that overcomes these shortcomings. We show that custom superantigens generated by single chain antibody technology permit the study of tolerance in a normal, polyclonal immune system. In the present study we generated a membrane-tethered anti-Igkappa-reactive single chain antibody chimeric gene and expressed it as a transgene in mice. B cell tolerance was directly characterized in the transgenic mice and in radiation bone marrow chimeras in which ligand-bearing mice served as recipients of nontransgenic cells. We find that the ubiquitously expressed, Igkappa-reactive ligand induces efficient B cell tolerance primarily or exclusively by receptor editing. We also demonstrate the unique advantages of our model in the genetic and cellular analysis of immune tolerance.

Differential Apaf-1 levels allow cytochrome c to induce apoptosis in brain tumors but not in normal neural tissues.

Brain tumors are typically resistant to conventional chemotherapeutics, most of which initiate apoptosis upstream of mitochondrial cytochrome c release. In this study, we demonstrate that directly activating apoptosis downstream of the mitochondria, with cytosolic cytochrome c, kills brain tumor cells but not normal brain tissue. Specifically, cytosolic cytochrome c is sufficient to induce apoptosis in glioblastoma and medulloblastoma cell lines. In contrast, primary neurons from the cerebellum and cortex are remarkably resistant to cytosolic cytochrome c. Importantly, tumor tissue from mouse models of both high-grade astrocytoma and medulloblastoma display hypersensitivity to cytochrome c when compared with surrounding brain tissue. This differential sensitivity to cytochrome c is attributed to high Apaf-1 levels in the tumor tissue compared with low Apaf-1 levels in the adjacent brain tissue. These differences in Apaf-1 abundance correlate with differences in the levels of E2F1, a previously identified activator of Apaf-1 transcription. ChIP assays reveal that E2F1 binds the Apaf-1 promoter specifically in tumor tissue, suggesting that E2F1 contributes to the expression of Apaf-1 in brain tumors. Together, these results demonstrate an unexpected sensitivity of brain tumors to postmitochondrial induction of apoptosis. Moreover, they raise the possibility that this phenomenon could be exploited therapeutically to selectively kill brain cancer cells while sparing the surrounding brain parenchyma.

Computer vision tools for low-cost and noninvasive measurement of autism-related behaviors in infants.

The early detection of developmental disorders is key to child outcome, allowing interventions to be initiated which promote development and improve prognosis. Research on autism spectrum disorder (ASD) suggests that behavioral signs can be observed late in the first year of life. Many of these studies involve extensive frame-by-frame video observation and analysis of a child's natural behavior. Although nonintrusive, these methods are extremely time-intensive and require a high level of observer training; thus, they are burdensome for clinical and large population research purposes. This work is a first milestone in a long-term project on non-invasive early observation of children in order to aid in risk detection and research of neurodevelopmental disorders. We focus on providing low-cost computer vision tools to measure and identify ASD behavioral signs based on components of the Autism Observation Scale for Infants (AOSI). In particular, we develop algorithms to measure responses to general ASD risk assessment tasks and activities outlined by the AOSI which assess visual attention by tracking facial features. We show results, including comparisons with expert and nonexpert clinicians, which demonstrate that the proposed computer vision tools can capture critical behavioral observations and potentially augment the clinician's behavioral observations obtained from real in-clinic assessments.

Computer vision tools for the non-invasive assessment of autism-related behavioral markers

The early detection of developmental disorders is key to child outcome, allowing interventions to be initiated that promote development and improve prognosis. Research on autism spectrum disorder (ASD) suggests behavioral markers can be observed late in the first year of life. Many of these studies involved extensive frame-by-frame video observation and analysis of a child's natural behavior. Although non-intrusive, these methods are extremely time-intensive and require a high level of observer training; thus, they are impractical for clinical and large population research purposes. Diagnostic measures for ASD are available for infants but are only accurate when used by specialists experienced in early diagnosis. This work is a first milestone in a long-term multidisciplinary project that aims at helping clinicians and general practitioners accomplish this early detection/measurement task automatically. We focus on providing computer vision tools to measure and identify ASD behavioral markers based on components of the Autism Observation Scale for Infants (AOSI). In particular, we develop algorithms to measure three critical AOSI activities that assess visual attention. We augment these AOSI activities with an additional test that analyzes asymmetrical patterns in unsupported gait. The first set of algorithms involves assessing head motion by tracking facial features, while the gait analysis relies on joint foreground segmentation and 2D body pose estimation in video. We show results that provide insightful knowledge to augment the clinician's behavioral observations obtained from real in-clinic assessments.

Huntingtin is required for normal excitatory synapse development in cortical and striatal circuits.

Huntington's disease (HD) is a neurodegenerative disease caused by the expansion of a poly-glutamine (poly-Q) stretch in the huntingtin (Htt) protein. Gain-of-function effects of mutant Htt have been extensively investigated as the major driver of neurodegeneration in HD. However, loss-of-function effects of poly-Q mutations recently emerged as potential drivers of disease pathophysiology. Early synaptic problems in the excitatory cortical and striatal connections have been reported in HD, but the role of Htt protein in synaptic connectivity was unknown. Therefore, we investigated the role of Htt in synaptic connectivity in vivo by conditionally silencing Htt in the developing mouse cortex. When cortical Htt function was silenced, cortical and striatal excitatory synapses formed and matured at an accelerated pace through postnatal day 21 (P21). This exuberant synaptic connectivity was lost over time in the cortex, resulting in the deterioration of synapses by 5 weeks. Synaptic decline in the cortex was accompanied with layer- and region-specific reactive gliosis without cell loss. To determine whether the disease-causing poly-Q mutation in Htt affects synapse development, we next investigated the synaptic connectivity in a full-length knock-in mouse model of HD, the zQ175 mouse. Similar to the cortical conditional knock-outs, we found excessive excitatory synapse formation and maturation in the cortices of P21 zQ175, which was lost by 5 weeks. Together, our findings reveal that cortical Htt is required for the correct establishment of cortical and striatal excitatory circuits, and this function of Htt is lost when the mutant Htt is present.

Genomic and Functional Variation at a Normal Human Centromere

Centromeres are chromosomal loci essential for genome stability. Their malfunction can cause chromosome instability associated with cancer, infertility, and birth defects. This study focused on an intriguing centromere on human chromosome 17, which displays normal functional variation. Centromere identity can be found on either of two large arrays of repetitive DNA. We investigated inter-individual sequence variation on these two arrays and found association between array size, array variation, and centromere function. Our data suggest a functional influence of DNA sequence at this critical epigenetic locus.

The steady-state visual evoked potential in vision research: A review.

Periodic visual stimulation and analysis of the resulting steady-state visual evoked potentials were first introduced over 80 years ago as a means to study visual sensation and perception. From the first single-channel recording of responses to modulated light to the present use of sophisticated digital displays composed of complex visual stimuli and high-density recording arrays, steady-state methods have been applied in a broad range of scientific and applied settings.The purpose of this article is to describe the fundamental stimulation paradigms for steady-state visual evoked potentials and to illustrate these principles through research findings across a range of applications in vision science.

Serum cholesterol levels within the high normal range are associated with better cognitive performance among Chinese elderly

© 2016, Serdi and Springer-Verlag France.Objectives: The association between cognitive function and cholesterol levels is poorly understood and inconsistent results exist among the elderly. The purpose of this study is to investigate the association of cholesterol level with cognitive performance among Chinese elderly. Design: A cross-sectional study was implemented in 2012 and data were analyzed using generalized additive models, linear regression models and logistic regression models. Setting: Community-based setting in eight longevity areas in China. Subjects: A total of 2000 elderly aged 65 years and over (mean 85.8±12.0 years) participated in this study. Measurements: Total cholesterol (TC), triglycerides (TG), low density lipoprotein cholesterol (LDL-C) and high density lipoprotein cholesterol (HDL-C) concentration were determined and cognitive impairment was defined as Mini-Mental State Examination (MMSE) score≤23. Results: There was a significant positive linear association between TC, TG, LDL-C, HDL-C and MMSE score in linear regression models. Each 1 mmol/L increase in TC, TG, LDL-C and HDL-C corresponded to a decreased risk of cognitive impairment in logistic regression models. Compared with the lowest tertile, the highest tertile of TC, LDL-C and HDL-C had a lower risk of cognitive impairment. The adjusted odds ratios and 95% CI were 0.73(0.62–0.84) for TC, 0.81(0.70–0.94) for LDL-C and 0.81(0.70–0.94) for HDL-C. There was no gender difference in the protective effects of high TC and LDL-C levels on cognitive impairment. However, for high HDL-C levels the effect was only observed in women. High TC, LDL-C and HDL-C levels were associated with lower risk of cognitive impairment in the oldest old (aged 80 and older), but not in the younger elderly (aged 65 to 79 years). Conclusions: These findings suggest that cholesterol levels within the high normal range are associated with better cognitive performance in Chinese elderly, specifically in the oldest old. With further validation, low cholesterol may serve a clinical indicator of risk for cognitive impairment in the elderly.