Specialized Mappings Architecture with Applications to Vision-based Estimation of Articulated Body Pose

A fundamental task of vision systems is to infer the state of the world given some form of visual observations. From a computational perspective, this often involves facing an ill-posed problem; e.g., information is lost via projection of the 3D world into a 2D image. Solution of an ill-posed problem requires additional information, usually provided as a model of the underlying process. It is important that the model be both computationally feasible as well as theoretically well-founded. In this thesis, a probabilistic, nonlinear supervised computational learning model is proposed: the Specialized Mappings Architecture (SMA). The SMA framework is demonstrated in a computer vision system that can estimate the articulated pose parameters of a human body or human hands, given images obtained via one or more uncalibrated cameras. The SMA consists of several specialized forward mapping functions that are estimated automatically from training data, and a possibly known feedback function. Each specialized function maps certain domains of the input space (e.g., image features) onto the output space (e.g., articulated body parameters). A probabilistic model for the architecture is first formalized. Solutions to key algorithmic problems are then derived: simultaneous learning of the specialized domains along with the mapping functions, as well as performing inference given inputs and a feedback function. The SMA employs a variant of the Expectation-Maximization algorithm and approximate inference. The approach allows the use of alternative conditional independence assumptions for learning and inference, which are derived from a forward model and a feedback model. Experimental validation of the proposed approach is conducted in the task of estimating articulated body pose from image silhouettes. Accuracy and stability of the SMA framework is tested using artificial data sets, as well as synthetic and real video sequences of human bodies and hands.

Vision Modulates Corticospinal Suppression in a Functionally Specific Manner during Movement of the Opposite Limb

The effect of vision on the excitability of corticospinal projections to the flexor carpi radialis (FCR) and extensor carpi radialis (ECR) muscles of right human forearm was investigated before and during discrete movement of the opposite limb. An external force opposed the initial phase of the movement (wrist flexion) and assisted the reverse phase, so that recruitment of the wrist extensors was minimized. Three conditions were used as follows: viewing the inactive right limb (Vision), viewing the mirror image of the moving left limb (Mirror), and with vision of the right limb occluded (No Vision). Transcranial magnetic stimulation was delivered to the left motor cortex: before, at the onset of, or during the left limb movement to obtain motor evoked potentials (MEPs) in the muscles of the right forearm. At and following movement onset, MEPs obtained in the right FCR were smaller in the Vision condition than in the Mirror and No Vision conditions. A distinct pattern of variation was obtained for the ECR. In all conditions, MEPs in this muscle were elevated upon or following movement of the opposite limb. An additional analysis of ipsilateral silent periods indicated that interhemispheric inhibition plays a role in mediating these effects. Activity-dependent changes in corticospinal output to a resting limb during discrete actions of the opposite limb are thus directly contingent upon where one looks. Furthermore, the extent to which vision exerts an influence upon projections to specific muscles varies in accordance with the functional contribution of their homologs to the intended action.

Cataract extraction and patient vision-related quality of life: a cohort study

Aims: Cataract surgery is one of the most common surgeries performed, but its overuse has been reported. The threshold for cataract surgery has become increasingly lenient; therefore, the selection process and surgical need has been questioned. The aim of this study was to evaluate the changes associated with cataract surgery in patient-reported vision-related quality of life (VR-QoL).

Methods: A prospective cohort study was conducted. Consecutive patients referred to cataract clinics in an NHS unit in Scotland were identified. Those listed for surgery were invited to complete a validated questionnaire (TyPE) to measure VR-QoL pre- and post-operatively. TyPE has five different domains (near vision, distance vision, daytime driving, night-time driving, and glare) and a global score of vision. The influence of pre-operative visual acuity (VA) levels, vision, and lens status of the fellow eye on changes in VR-QoL were explored. 

Results: A total of 320 listed patients were approached, of whom 36 were excluded. Among the 284 enrolled patients, 229 (81%) returned the questionnaire after surgery. Results revealed that the mean overall vision improved, as reported by patients. Improvements were also seen in all sub-domains of the questionnaire.

Conclusion: The majority of patients appear to have improvement in patient-reported VR-QoL, including those with good pre-operative VA and previous surgery to the fellow eye. VA thresholds may not capture the effects of the quality of life on patients. This information can assist clinicians to make more informed decisions when debating over the benefits of listing a patient for cataract extraction.

Review of computer vision in intelligent environment design

This paper discusses and compares the use of vision based and non-vision based technologies in developing intelligent environments. By reviewing the related projects that use vision based techniques in intelligent environment design, the achieved functions, technical issues and drawbacks of those projects are discussed and summarized, and the potential solutions for future improvement are proposed, which leads to the prospective direction of my PhD research.

Emotional well-being and adjustment to vision loss in later life: a meta-synthesis of qualitative studies.

Purpose: To review perceived emotional well-being in older people with visual impairment and perceived factors that inhibit/facilitate psychosocial adjustment to vision loss. Method: The databases of MEDLINE, EMBASE, PsycINFO and CINAHL were searched for studies published from January 1980 to December 2010, which recruited older people with irreversible vision loss, and used qualitative methods for both data collection and analysis. Results sections of the papers were synthesised using a thematic-style analysis to identify the emergent and dominant themes. Results: Seventeen qualitative papers were included in the review, and five main themes emerged from the synthesis: 1) the trauma of an ophthalmic diagnosis, 2) impact of vision loss on daily life, 3) negative impact of visual impairment on psychosocial well-being, 4) factors that inhibit social well-being, and 5) factors that facilitate psychological well-being. We found the response shift model useful for explaining our synthesis. Conclusions: Acquired visual impairment can have a significant impact on older people's well-being and make psychosocial adjustment to the condition a major challenge. Acceptance of the condition and a positive attitude facilitate successful psychosocial adjustment to vision loss as well as social support from family, friends and peers who have successfully adjusted to the condition. [Box: see text].

Vision and night driving abilities of elderly drivers

OBJECTIVE: In this article, we review the impact of vision on older people's night driving abilities. Driving is the preferred and primary mode of transport for older people. It is a complex activity where intact vision is seminal for road safety. Night driving requires mesopic rather than scotopic vision, because there is always some light available when driving at night. Scotopic refers to night vision, photopic refers to vision under well-lit conditions, and mesopic vision is a combination of photopic and scotopic vision in low but not quite dark lighting situations. With increasing age, mesopic vision decreases and glare sensitivity increases, even in the absence of ocular diseases. Because of the increasing number of elderly drivers, more drivers are affected by night vision difficulties. Vision tests, which accurately predict night driving ability, are therefore of great interest. METHODS: We reviewed existing literature on age-related influences on vision and vision tests that correlate or predict night driving ability. RESULTS: We identified several studies that investigated the relationship between vision tests and night driving. These studies found correlations between impaired mesopic vision or increased glare sensitivity and impaired night driving, but no correlation was found among other tests; for example, useful field of view or visual field. The correlation between photopic visual acuity, the most commonly used test when assessing elderly drivers, and night driving ability has not yet been fully clarified. CONCLUSIONS: Photopic visual acuity alone is not a good predictor of night driving ability. Mesopic visual acuity and glare sensitivity seem relevant for night driving. Due to the small number of studies evaluating predictors for night driving ability, further research is needed.

Vision impairment at diagnosis of cancer - choroidal metastasis

Choroidal metastasis represents the most common type of intraocular malignancy and preferably involves the posterior uveal tract. Breast and lung cancer - known or so far undiagnosed - are most frequently identified as the underlying tumor disease. The majority of patients diagnosed with uveal metastasis have additional metastatic manifestations elsewhere, so re-staging before treatment is recommended. The importance of a multidisciplinary approach is obvious. Early diagnosis and timely initiation of treatment are mandatory in case of vision-threatening situations. External beam radiation remains the therapy of choice. Overall survival of patients with uveal metastasis is limited, averaging six to twelve months. The other eye is frequently enough affected as well, justifying regular ophthalmologic follow-up during the further course of the disease.

Photopic and scotopic spatiotemporal tuning of adult zebrafish vision.

Sensitivity to spatial and temporal patterns is a fundamental aspect of vision. Herein, we investigated this sensitivity in adult zebrafish for a wide range of spatial (0.014 to 0.511 cycles/degree [c/d]) and temporal frequencies (0.025 to 6 cycles/s) to better understand their visual system. Measurements were performed at photopic (1.8 log cd m(-2)) and scotopic (-4.5 log cd m(-2)) light levels to assess the optokinetic response (OKR). The resulting spatiotemporal contrast sensitivity (CS) functions revealed that the OKR of zebrafish is tuned to spatial frequency and speed but not to temporal frequencies. Thereby, optimal test parameters for CS measurements were identified. At photopic light levels, a spatial frequency of 0.116 ± 0.01 c/d (mean ± SD) and a grating speed of 8.42 ± 2.15 degrees/second (d/s) was ideal; at scotopic light levels, these values were 0.110 ± 0.02 c/d and 5.45 ± 1.31 d/s, respectively. This study allows to better characterize zebrafish mutants with altered vision and to distinguish between defects of rod and cone photoreceptors as measurements were performed under different light conditions.

Building the View Graph of a Category by Exploiting Image Realism

We propose a weakly supervised method to arrange images of a given category based on the relative pose between the camera and the object in the scene. Relative poses are points on a sphere centered at the object in a given canonical pose, which we call object viewpoints. Our method builds a graph on this sphere by assigning images with similar viewpoint to the same node and by connecting nodes if they are related by a small rotation. The key idea is to exploit a large unlabeled dataset to validate the likelihood of dominant 3D planes of the object geometry. A number of 3D plane hypotheses are evaluated by applying small 3D rotations to each hypothesis and by measuring how well the deformed images match other images in the dataset. Correct hypotheses will result in deformed images that correspond to plausible views of the object, and thus will likely match well other images in the same category. The identified 3D planes are then used to compute affinities between images related by a change of viewpoint. We then use the affinities to build a view graph via a greedy method and the maximum spanning tree.

Local densities orthogonal to beta-sheet amide planes: patterns of packing in globular proteins.

We have investigated the efficiency of packing by calculating intramolecular packing density above and below peptide planes of internal beta-pleated sheet residues in five globular proteins. The orientation of interest was chosen to allow study of regions that are approximately perpendicular to the faces of beta-pleated sheets. In these locations, nonbonded van der Waals packing interactions predominate over hydrogen bonding and solvent interactions. We observed considerable variability in packing densities within these regions, confirming that the interior packing of a protein does not result in uniform occupation of the available space. Patterns of fluctuation in packing density suggest that the regular backbone-to-backbone network of hydrogen bonds is not likely to be interrupted to maximize van der Waals interactions. However, high-density packing tends to occur toward the ends of beta-structure strands where hydrogen bonds are more likely to involve nonpolar side-chain groups or solvent molecules. These features result in internal protein folding with a central low-density core surrounded by a higher-density subsurface shell, consistent with our previous calculations regarding overall protein packing density.

Proinsulin slows retinal degeneration and vision loss in the P23H rat model of retinitis pigmentosa

Proinsulin has been characterized as a neuroprotective molecule. In this work we assess the therapeutic potential of proinsulin on photoreceptor degeneration, synaptic connectivity, and functional activity of the retina in the transgenic P23H rat, an animal model of autosomal dominant retinitis pigmentosa (RP). P23H homozygous rats received an intramuscular injection of an adeno-associated viral vector serotype 1 (AAV1) expressing human proinsulin (hPi+) or AAV1-null vector (hPi−) at P20. Levels of hPi in serum were determined by enzyme-linked immunosorbent assay (ELISA), and visual function was evaluated by electroretinographic (ERG) recording at P30, P60, P90, and P120. Preservation of retinal structure was assessed by immunohistochemistry at P120. Human proinsulin was detected in serum from rats injected with hPi+ at all times tested, with average hPi levels ranging from 1.1 nM (P30) to 1.4 nM (P120). ERG recordings showed an amelioration of vision loss in hPi+ animals. The scotopic b-waves were significantly higher in hPi+ animals than in control rats at P90 and P120. This attenuation of visual deterioration correlated with a delay in photoreceptor degeneration and the preservation of retinal cytoarchitecture. hPi+ animals had 48.7% more photoreceptors than control animals. Presynaptic and postsynaptic elements, as well as the synaptic contacts between photoreceptors and bipolar or horizontal cells, were preserved in hPi+ P23H rats. Furthermore, in hPi+ rat retinas the number of rod bipolar cell bodies was greater than in control rats. Our data demonstrate that hPi expression preserves cone and rod structure and function, together with their contacts with postsynaptic neurons, in the P23H rat. These data strongly support the further development of proinsulin-based therapy to counteract retinitis pigmentosa.

Development and implementation of a rear vision device test protocol. Volume II - simulated-use testing. Final report.

National Highway Traffic Safety Administration, Washington, D.C.

Development and implementation of a rear vision device test protocol. Volume I - physical characteristics measurements. Final report.

National Highway Traffic Safety Administration, Washington, D.C.

The utility of peripheral vision to motor vehicle drivers. Final technical report.

National Highway Traffic Safety Administration, Washington, D.C.

From filters to features:Scale-space analysis of edge and blur coding in human vision

To make vision possible, the visual nervous system must represent the most informative features in the light pattern captured by the eye. Here we use Gaussian scale-space theory to derive a multiscale model for edge analysis and we test it in perceptual experiments. At all scales there are two stages of spatial filtering. An odd-symmetric, Gaussian first derivative filter provides the input to a Gaussian second derivative filter. Crucially, the output at each stage is half-wave rectified before feeding forward to the next. This creates nonlinear channels selectively responsive to one edge polarity while suppressing spurious or "phantom" edges. The two stages have properties analogous to simple and complex cells in the visual cortex. Edges are found as peaks in a scale-space response map that is the output of the second stage. The position and scale of the peak response identify the location and blur of the edge. The model predicts remarkably accurately our results on human perception of edge location and blur for a wide range of luminance profiles, including the surprising finding that blurred edges look sharper when their length is made shorter. The model enhances our understanding of early vision by integrating computational, physiological, and psychophysical approaches. © ARVO.