Synchronization of user-generated videos through trajectory correspondence and a refinement procedure

Temporal synchronization of multiple video recordings of the same dynamic event is a critical task in many computer vision applications e.g. novel view synthesis and 3D reconstruction. Typically this information is implied through the time-stamp information embedded in the video streams. User-generated videos shot using consumer grade equipment do not contain this information; hence, there is a need to temporally synchronize signals using the visual information itself. Previous work in this area has either assumed good quality data with relatively simple dynamic content or the availability of precise camera geometry. Our first contribution is a synchronization technique which tries to establish correspondence between feature trajectories across views in a novel way, and specifically targets the kind of complex content found in consumer generated sports recordings, without assuming precise knowledge of fundamental matrices or homographies. We evaluate performance using a number of real video recordings and show that our method is able to synchronize to within 1 sec, which is significantly better than previous approaches. Our second contribution is a robust and unsupervised view-invariant activity recognition descriptor that exploits recurrence plot theory on spatial tiles. The descriptor is individually shown to better characterize the activities from different views under occlusions than state-of-the-art approaches. We combine this descriptor with our proposed synchronization method and show that it can further refine the synchronization index. © 2013 ACM.

Robust recognition of chess-boards under deformation

Current methods for formation of detected chess-board vertices into a grid structure tend to be weak in situations with a warped grid, and false and missing vertex-features. In this paper we present a highly robust, yet efficient, scheme suitable for inference of regular 2D square mesh structure from vertices recorded both during projection of a chess-board pattern onto 3D objects, and in the more simple case of camera calibration. Examples of the method's performance in a lung function measuring application, observing chess-boards projected on to patients' chests, are given. The method presented is resilient to significant surface deformation, and tolerates inexact vertex-feature detection. This robustness results from the scheme's novel exploitation of feature orientation information. © 2013 IEEE.

SLP: A zero-contact non-invasive method for pulmonary function testing

Structured Light Plethysmography (SLP) is a novel non-invasive method that uses structured light to perform pulmonary function testing that does not require physical contact with a patient. The technique produces an estimate of chest wall volume changes over time. A patient is observed continuously by two cameras and a known pattern of light (i.e. structured light) is projected onto the chest using an off-the-shelf projector. Corner features from the projected light pattern are extracted, tracked and brought into correspondence for both camera views over successive frames. A novel self calibration algorithm recovers the intrinsic and extrinsic camera parameters from these point correspondences. This information is used to reconstruct a surface approximation of the chest wall and several novel ideas for 'cleaning up' the reconstruction are used. The resulting volume and derived statistics (e.g. FVC, FEV) agree very well with data taken with a spirometer. © 2010. The copyright of this document resides with its authors.

Obtaining the shape of a moving object with a specular surface

This paper addresses the basic problem of recovering the 3D surface of an object that is observed in motion by a single camera and under a static but unknown lighting condition. We propose a method to establish pixelwise correspondence between input images by way of depth search by investigating optimal subsets of intensities rather than employing all the relevant pixel values. The thrust of our algorithm is that it is capable of dealing with specularities which appear on the top of shading variance that is caused due to object motion. This is in terms of both stages of finding sparse point correspondence and dense depth search. We also propose that a linearised image basis can be directly computed by the procudure of finding the correspondence. We illustrate the performance of the theoretical propositions using images of real objects. © 2009. The copyright of this document resides with its authors.

Active sensing system with in situ adjustable sensor morphology.

BACKGROUND: Despite the widespread use of sensors in engineering systems like robots and automation systems, the common paradigm is to have fixed sensor morphology tailored to fulfill a specific application. On the other hand, robotic systems are expected to operate in ever more uncertain environments. In order to cope with the challenge, it is worthy of note that biological systems show the importance of suitable sensor morphology and active sensing capability to handle different kinds of sensing tasks with particular requirements. METHODOLOGY: This paper presents a robotics active sensing system which is able to adjust its sensor morphology in situ in order to sense different physical quantities with desirable sensing characteristics. The approach taken is to use thermoplastic adhesive material, i.e. Hot Melt Adhesive (HMA). It will be shown that the thermoplastic and thermoadhesive nature of HMA enables the system to repeatedly fabricate, attach and detach mechanical structures with a variety of shape and size to the robot end effector for sensing purposes. Via active sensing capability, the robotic system utilizes the structure to physically probe an unknown target object with suitable motion and transduce the arising physical stimuli into information usable by a camera as its only built-in sensor. CONCLUSIONS/SIGNIFICANCE: The efficacy of the proposed system is verified based on two results. Firstly, it is confirmed that suitable sensor morphology and active sensing capability enables the system to sense different physical quantities, i.e. softness and temperature, with desirable sensing characteristics. Secondly, given tasks of discriminating two visually indistinguishable objects with respect to softness and temperature, it is confirmed that the proposed robotic system is able to autonomously accomplish them. The way the results motivate new research directions which focus on in situ adjustment of sensor morphology will also be discussed.

Manopt, a matlab toolbox for optimization on manifolds

Optimization on manifolds is a rapidly developing branch of nonlinear optimization. Its focus is on problems where the smooth geometry of the search space can be leveraged to design effcient numerical algorithms. In particular, optimization on manifolds is well-suited to deal with rank and orthogonality constraints. Such structured constraints appear pervasively in machine learning applications, including low-rank matrix completion, sensor network localization, camera network registration, independent component analysis, metric learning, dimensionality reduction and so on. The Manopt toolbox, available at www.manopt.org, is a user-friendly, documented piece of software dedicated to simplify experimenting with state of the art Riemannian optimization algorithms. By dealing internally with most of the differential geometry, the package aims particularly at lowering the entrance barrier. © 2014 Nicolas Boumal.