Direct Estimation of Motion and Extended Scene Structure from a Moving Stereo Rig

We describe a new method for motion estimation and 3D reconstruction from stereo image sequences obtained by a stereo rig moving through a rigid world. We show that given two stereo pairs one can compute the motion of the stereo rig directly from the image derivatives (spatial and temporal). Correspondences are not required. One can then use the images from both pairs combined to compute a dense depth map. The motion estimates between stereo pairs enable us to combine depth maps from all the pairs in the sequence to form an extended scene reconstruction and we show results from a real image sequence. The motion computation is a linear least squares computation using all the pixels in the image. Areas with little or no contrast are implicitly weighted less so one does not have to explicitly apply a confidence measure.

Structure from Stereo and Motion

Stereopsis and motion parallax are two methods for recovering three dimensional shape. Theoretical analyses of each method show that neither alone can recover rigid 3D shapes correctly unless other information, such as perspective, is included. The solutions for recovering rigid structure from motion have a reflection ambiguity; the depth scale of the stereoscopic solution will not be known unless the fixation distance is specified in units of interpupil separation. (Hence the configuration will appear distorted.) However, the correct configuration and the disposition of a rigid 3D shape can be recovered if stereopsis and motion are integrated, for then a unique solution follows from a set of linear equations. The correct interpretation requires only three points and two stereo views.

Learning World Models in Environments with Manifest Causal Structure

This thesis examines the problem of an autonomous agent learning a causal world model of its environment. Previous approaches to learning causal world models have concentrated on environments that are too "easy" (deterministic finite state machines) or too "hard" (containing much hidden state). We describe a new domain --- environments with manifest causal structure --- for learning. In such environments the agent has an abundance of perceptions of its environment. Specifically, it perceives almost all the relevant information it needs to understand the environment. Many environments of interest have manifest causal structure and we show that an agent can learn the manifest aspects of these environments quickly using straightforward learning techniques. We present a new algorithm to learn a rule-based causal world model from observations in the environment. The learning algorithm includes (1) a low level rule-learning algorithm that converges on a good set of specific rules, (2) a concept learning algorithm that learns concepts by finding completely correlated perceptions, and (3) an algorithm that learns general rules. In addition this thesis examines the problem of finding a good expert from a sequence of experts. Each expert has an "error rate"; we wish to find an expert with a low error rate. However, each expert's error rate and the distribution of error rates are unknown. A new expert-finding algorithm is presented and an upper bound on the expected error rate of the expert is derived.

Simultaneous Learning of Nonlinear Manifold and Dynamical Models for High-dimensional Time Series

The goal of this work is to learn a parsimonious and informative representation for high-dimensional time series. Conceptually, this comprises two distinct yet tightly coupled tasks: learning a low-dimensional manifold and modeling the dynamical process. These two tasks have a complementary relationship as the temporal constraints provide valuable neighborhood information for dimensionality reduction and conversely, the low-dimensional space allows dynamics to be learnt efficiently. Solving these two tasks simultaneously allows important information to be exchanged mutually. If nonlinear models are required to capture the rich complexity of time series, then the learning problem becomes harder as the nonlinearities in both tasks are coupled. The proposed solution approximates the nonlinear manifold and dynamics using piecewise linear models. The interactions among the linear models are captured in a graphical model. By exploiting the model structure, efficient inference and learning algorithms are obtained without oversimplifying the model of the underlying dynamical process. Evaluation of the proposed framework with competing approaches is conducted in three sets of experiments: dimensionality reduction and reconstruction using synthetic time series, video synthesis using a dynamic texture database, and human motion synthesis, classification and tracking on a benchmark data set. In all experiments, the proposed approach provides superior performance.

Simultaneous Learning of Non-Linear Manifold and Dynamical Models for High-Dimensional Time Series

The goal of this work is to learn a parsimonious and informative representation for high-dimensional time series. Conceptually, this comprises two distinct yet tightly coupled tasks: learning a low-dimensional manifold and modeling the dynamical process. These two tasks have a complementary relationship as the temporal constraints provide valuable neighborhood information for dimensionality reduction and conversely, the low-dimensional space allows dynamics to be learnt efficiently. Solving these two tasks simultaneously allows important information to be exchanged mutually. If nonlinear models are required to capture the rich complexity of time series, then the learning problem becomes harder as the nonlinearities in both tasks are coupled. The proposed solution approximates the nonlinear manifold and dynamics using piecewise linear models. The interactions among the linear models are captured in a graphical model. The model structure setup and parameter learning are done using a variational Bayesian approach, which enables automatic Bayesian model structure selection, hence solving the problem of over-fitting. By exploiting the model structure, efficient inference and learning algorithms are obtained without oversimplifying the model of the underlying dynamical process. Evaluation of the proposed framework with competing approaches is conducted in three sets of experiments: dimensionality reduction and reconstruction using synthetic time series, video synthesis using a dynamic texture database, and human motion synthesis, classification and tracking on a benchmark data set. In all experiments, the proposed approach provides superior performance.

Task-Irrelevant Perceptual Learning Specific to the Contrast Polarity of Motion Stimuli

Studies of perceptual learning have focused on aspects of learning that are related to early stages of sensory processing. However, conclusions that perceptual learning results in low-level sensory plasticity are of great controversy, largely because such learning can often be attributed to plasticity in later stages of sensory processing or in the decision processes. To address this controversy, we developed a novel random dot motion (RDM) stimulus to target motion cells selective to contrast polarity, by ensuring the motion direction information arises only from signal dot onsets and not their offsets, and used these stimuli in conjunction with the paradigm of task-irrelevant perceptual learning (TIPL). In TIPL, learning is achieved in response to a stimulus by subliminally pairing that stimulus with the targets of an unrelated training task. In this manner, we are able to probe learning for an aspect of motion processing thought to be a function of directional V1 simple cells with a learning procedure that dissociates the learned stimulus from the decision processes relevant to the training task. Our results show learning for the exposed contrast polarity and that this learning does not transfer to the unexposed contrast polarity. These results suggest that TIPL for motion stimuli may occur at the stage of directional V1 simple cells.

Neural Dynamics of Motion Perception, Recognition Learning, and Spatial Attention

Advanced Research Projects Agency (ONR N00014-92-J-4015); National Science Foundation (IRI-90-24877); Office of Naval Research (N00014-91-J-4100)

Nonrigid Structure-From-Motion From 2-D Images Using Markov Chain Monte Carlo

In this paper we present a new method for simultaneously determining three dimensional (3-D) shape and motion of a non-rigid object from uncalibrated two dimensional (2- D) images without assuming the distribution characteristics. A non-rigid motion can be treated as a combination of a rigid rotation and a non-rigid deformation. To seek accurate recovery of deformable structures, we estimate the probability distribution function of the corresponding features through random sampling, incorporating an established probabilistic model. The fitting between the observation and the projection of the estimated 3-D structure will be evaluated using a Markov chain Monte Carlo based expectation maximisation algorithm. Applications of the proposed method to both synthetic and real image sequences are demonstrated with promising results.

A STRUCTURE-FROM-MOTION SCHEME THAT LOOKS FOR PARALLELS, AND ITS IMPLICATIONS FOR APPARENT REVERSALS IN ROTATING TRAPEZIA

Apparent reversals in rotating trapezia have been regarded as evidence that human vision favours methods which are heuristic or form dependent. However, the argument is based on the assumption that general algorithmic methods would avoid the illusion, and that has never been clear. A general algorithm for interpreting moving parallels has been developed to address the issue. It handles a considerable range of stimuli successfully, but finds multiple interpretations in situations which correspond closely to those where apparent reversals occur. This strengthens the hypothesis that apparent reversals may occur when general algorithmic methods fail and heuristics are invoked as a stopgap.

Machine learning methods for quantitative structure-property relationship modeling

Tese de doutoramento, Informática (Bioinformática), Universidade de Lisboa, Faculdade de Ciências, 2014

Quantifying Submerged Fluvial Topography Using Hyperspatial Resolution UAS Imagery and Structure From Motion Photogrammetry.

Quantifying the topography of rivers and their associated bedforms has been a fundamental concern of fluvial geomorphology for decades. Such data, acquired at high temporal and spatial resolutions, are increasingly in demand for process-oriented investigations of flow hydraulics, sediment dynamics and in-stream habitat. In these riverine environments, the most challenging region for topographic measurement is the wetted, submerged channel. Generally, dry bed topography and submerged bathymetry are measured using different methods and technology. This adds to the costs, logistical challenges and data processing requirements of comprehensive river surveys. However, some technologies are capable of measuring the submerged topography. Through-water photogrammetry and bathymetric LiDAR are capable of reasonably accurate measurements of channel beds in clear water. Whilst the cost of bathymetric LiDAR remains high and its resolution relatively coarse, the recent developments in photogrammetry using Structure from Motion (SfM) algorithms promise a fundamental shift in the accessibility of topographic data for a wide range of settings. Here we present results demonstrating the potential of so called SfM-photogrammetry for quantifying both exposed and submerged fluvial topography at the mesohabitat scale. We show that imagery acquired from a rotary-winged Unmanned Aerial System (UAS) can be processed in order to produce digital elevation models (DEMs) with hyperspatial resolutions (c. 0.02 m) for two different river systems over channel lengths of 50-100 m. Errors in submerged areas range from 0.016 m to 0.089 m, which can be reduced to between 0.008 m and 0.053 m with the application of a simple refraction correction. This work therefore demonstrates the potential of UAS platforms and SfM-photogrammetry as a single technique for surveying fluvial topography at the mesoscale (defined as lengths of channel from c.10 m to a few hundred metres). This article is protected by copyright. All rights reserved.

Curso E-learning de sistemas multicorpo (multibody systems) para engenheiros mecânicos

Trabalho Final de Mestrado para obtenção do grau de Mestre em Engenharia Mecânica

UAV - based imagery processing using structure from motion and remote sensing technology

With the recent advances in technology and miniaturization of devices such as GPS or IMU, Unmanned Aerial Vehicles became a feasible platform for a Remote Sensing applications. The use of UAVs compared to the conventional aerial platforms provides a set of advantages such as higher spatial resolution of the derived products. UAV - based imagery obtained by a user grade cameras introduces a set of problems which have to be solved, e. g. rotational or angular differences or unknown or insufficiently precise IO and EO camera parameters. In this work, UAV - based imagery of RGB and CIR type was processed using two different workflows based on PhotoScan and VisualSfM software solutions resulting in the DSM and orthophoto products. Feature detection and matching parameters influence on the result quality as well as a processing time was examined and the optimal parameter setup was presented. Products of the both workflows were compared in terms of a quality and a spatial accuracy. Both workflows were compared by presenting the processing times and quality of the results. Finally, the obtained products were used in order to demonstrate vegetation classification. Contribution of the IHS transformations was examined with respect to the classification accuracy.

Conceptualizing Institutional Capacity Building Through a Learning Process Set in Motion by a Foundation

Employing critical pedagogy and transformative theory as a theoretical framework, I examined a learning process associated with building capacity in community-based organizations (CBOs) through an investigation of the Institutional Capacity Building Program (ICBP) initiated by a Foundation. The study sought to: (a) examine the importance of institutional capacity building for individual and community development; (b) investigate elements of a process associated with a program and characteristics of a learning process for building capacity in CBOs; and (c) analyze the Foundation’s approach to synthesizing, systematizing, and sharing learning. The study used a narrative research design that included 3 one-on-one, hour-long interviews with 2 women having unique vantage points in ICBP: one is a program facilitator working at the Foundation and the other runs a CBO supported by the Foundation. The interviews’ semistructured questions allowed interviewees to share stories regarding their experience with the learning process of ICB and enabled themes to emerge from their day-to-day experience. Through the analysis of this learning process for institutional capacity building, a few lessons can be drawn from the experience of the Foundation.