Warping trajectories for video synchronization

Temporal synchronization of multiple video recordings of the same dynamic event is a critical task in many computer vision applications e.g. novel view synthesis and 3D reconstruction. Typically this information is implied, since recordings are made using the same timebase, or time-stamp information is embedded in the video streams. Recordings using consumer grade equipment do not contain this information; hence, there is a need to temporally synchronize signals using the visual information itself. Previous work in this area has either assumed good quality data with relatively simple dynamic content or the availability of precise camera geometry. In this paper, we propose a technique which exploits feature trajectories across views in a novel way, and specifically targets the kind of complex content found in consumer generated sports recordings, without assuming precise knowledge of fundamental matrices or homographies. Our method automatically selects the moving feature points in the two unsynchronized videos whose 2D trajectories can be best related, thereby helping to infer the synchronization index. We evaluate performance using a number of real recordings and show that synchronization can be achieved to within 1 sec, which is better than previous approaches. Copyright 2013 ACM.

ChESS - Quick and robust detection of chess-board features

Localization of chess-board vertices is a common task in computer vision, underpinning many applications, but relatively little work focusses on designing a specific feature detector that is fast, accurate and robust. In this paper the 'Chess-board Extraction by Subtraction and Summation' (ChESS) feature detector, designed to exclusively respond to chess-board vertices, is presented. The method proposed is robust against noise, poor lighting and poor contrast, requires no prior knowledge of the extent of the chess-board pattern, is computationally very efficient, and provides a strength measure of detected features. Such a detector has significant application both in the key field of camera calibration, as well as in structured light 3D reconstruction. Evidence is presented showing its superior robustness, accuracy, and efficiency in comparison to other commonly used detectors, including Harris & Stephens and SUSAN, both under simulation and in experimental 3D reconstruction of flat plate and cylindrical objects. © 2013 Elsevier Inc. All rights reserved.

Anisotropy preserving DTI processing

Statistical analysis of diffusion tensor imaging (DTI) data requires a computational framework that is both numerically tractable (to account for the high dimensional nature of the data) and geometric (to account for the nonlinear nature of diffusion tensors). Building upon earlier studies exploiting a Riemannian framework to address these challenges, the present paper proposes a novel metric and an accompanying computational framework for DTI data processing. The proposed approach grounds the signal processing operations in interpolating curves. Well-chosen interpolating curves are shown to provide a computational framework that is at the same time tractable and information relevant for DTI processing. In addition, and in contrast to earlier methods, it provides an interpolation method which preserves anisotropy, a central information carried by diffusion tensor data. © 2013 Springer Science+Business Media New York.

Synchronization of user-generated videos through trajectory correspondence and a refinement procedure

Temporal synchronization of multiple video recordings of the same dynamic event is a critical task in many computer vision applications e.g. novel view synthesis and 3D reconstruction. Typically this information is implied through the time-stamp information embedded in the video streams. User-generated videos shot using consumer grade equipment do not contain this information; hence, there is a need to temporally synchronize signals using the visual information itself. Previous work in this area has either assumed good quality data with relatively simple dynamic content or the availability of precise camera geometry. Our first contribution is a synchronization technique which tries to establish correspondence between feature trajectories across views in a novel way, and specifically targets the kind of complex content found in consumer generated sports recordings, without assuming precise knowledge of fundamental matrices or homographies. We evaluate performance using a number of real video recordings and show that our method is able to synchronize to within 1 sec, which is significantly better than previous approaches. Our second contribution is a robust and unsupervised view-invariant activity recognition descriptor that exploits recurrence plot theory on spatial tiles. The descriptor is individually shown to better characterize the activities from different views under occlusions than state-of-the-art approaches. We combine this descriptor with our proposed synchronization method and show that it can further refine the synchronization index. © 2013 ACM.

Semantic transform: Weakly supervised semantic inference for relating visual attributes

Relative (comparative) attributes are promising for thematic ranking of visual entities, which also aids in recognition tasks. However, attribute rank learning often requires a substantial amount of relational supervision, which is highly tedious, and apparently impractical for real-world applications. In this paper, we introduce the Semantic Transform, which under minimal supervision, adaptively finds a semantic feature space along with a class ordering that is related in the best possible way. Such a semantic space is found for every attribute category. To relate the classes under weak supervision, the class ordering needs to be refined according to a cost function in an iterative procedure. This problem is ideally NP-hard, and we thus propose a constrained search tree formulation for the same. Driven by the adaptive semantic feature space representation, our model achieves the best results to date for all of the tasks of relative, absolute and zero-shot classification on two popular datasets. © 2013 IEEE.

A multi-frame image super-resolution method

Multi-frame image super-resolution (SR) aims to utilize information from a set of low-resolution (LR) images to compose a high-resolution (HR) one. As it is desirable or essential in many real applications, recent years have witnessed the growing interest in the problem of multi-frame SR reconstruction. This set of algorithms commonly utilizes a linear observation model to construct the relationship between the recorded LR images to the unknown reconstructed HR image estimates. Recently, regularization-based schemes have been demonstrated to be effective because SR reconstruction is actually an ill-posed problem. Working within this promising framework, this paper first proposes two new regularization items, termed as locally adaptive bilateral total variation and consistency of gradients, to keep edges and flat regions, which are implicitly described in LR images, sharp and smooth, respectively. Thereafter, the combination of the proposed regularization items is superior to existing regularization items because it considers both edges and flat regions while existing ones consider only edges. Thorough experimental results show the effectiveness of the new algorithm for SR reconstruction. (C) 2009 Elsevier B.V. All rights reserved.

A Review of Active Appearance Models

Active appearance model (AAM) is a powerful generative method for modeling deformable objects. The model decouples the shape and the texture variations of objects, which is followed by an efficient gradient-based model fitting method. Due to the flexible and simple framework, AAM has been widely applied in the fields of computer vision. However, difficulties are met when it is applied to various practical issues, which lead to a lot of prominent improvements to the model. Nevertheless, these difficulties and improvements have not been studied systematically. This motivates us to review the recent advances of AAM. This paper focuses on the improvements in the literature in turns of the problems suffered by AAM in practical applications. Therefore, these algorithms are summarized from three aspects, i.e., efficiency, discrimination, and robustness. Additionally, some applications and implementations of AAM are also enumerated. The main purpose of this paper is to serve as a guide for further research.

Image Ratio Features for Facial Expression Recognition Application

Video-based facial expression recognition is a challenging problem in computer vision and human-computer interaction. To target this problem, texture features have been extracted and widely used, because they can capture image intensity changes raised by skin deformation. However, existing texture features encounter problems with albedo and lighting variations. To solve both problems, we propose a new texture feature called image ratio features. Compared with previously proposed texture features, e. g., high gradient component features, image ratio features are more robust to albedo and lighting variations. In addition, to further improve facial expression recognition accuracy based on image ratio features, we combine image ratio features with facial animation parameters (FAPs), which describe the geometric motions of facial feature points. The performance evaluation is based on the Carnegie Mellon University Cohn-Kanade database, our own database, and the Japanese Female Facial Expression database. Experimental results show that the proposed image ratio feature is more robust to albedo and lighting variations, and the combination of image ratio features and FAPs outperforms each feature alone. In addition, we study asymmetric facial expressions based on our own facial expression database and demonstrate the superior performance of our combined expression recognition system.

基于视觉的室内移动机器人自定位技术研究

在关于移动机器人的诸多研究领域中，机器人自定位是十分关键的技术，是实现机器人自主运动和其他任务的基础，而且涉及领域广泛，有很多难点有待解决，因而是一个具有重要研究价值的课题。 本论文以沈阳新松机器人股份有限公司自主研发的家庭服务机器人为研发平台，系统地研究了基于计算机视觉的室内移动机器人自定位问题，成功设计了基于单目视觉人工路标以及粒子滤波的室内移动机器人自定位系统。 本文首先根据室内移动机器人自主导航定位的要求，设计了一种简易美观的新型视觉人工路标，并且研究与实现了该路标的实时准确检测以及不同路标的识别。 其次，在位姿计算方面，本文研究了共面P4P（4点透视）问题的解法及其在位姿计算方面的应用，并分析比较了两种不同P4P解法的优缺点，成功地将两种算法结合起来用于机器人位姿计算。 最后，在机器人自定位方面，本文将单目彩色摄像机作为传感器，在基于贝叶斯滤波理论的自定位理论框架下，利用粒子滤波自定位方法融合视觉信息与码盘信息，实现了自主移动机器人的自定位。 实践证明，本文设计的基于单目视觉人工路标的自定位系统能够成功地应用在室内移动机器人上，具有较高的应用推广价值。

基于几何特征的快速高精度角点检测算法

角点检测应用十分广泛,是许多计算机视觉任务的基础。本文提出了一种快速、高精度的角点检测算法,算法简单新颖,角点条件和角点响应函数设计独特。和以往不同的是:算法在设计上考虑的是角点的局部几何特征,使得处理的数据量大为减少,同时能够很好地保证检测精度等其他性能指标。通过和广泛使用的SUSAN算法、Harris算法在正确率、漏检、精度、抗噪声、计算复杂度等方面进行综合比较,结果表明该算法无论对人工合成图像还是对自然图像均具有良好的性能。

基于OpenGL的交会对接仿真系统

针对基于视觉的空间载体位姿测量很难进行真实实验的特点,提出了一种基于OpenG1的半物理仿真实验方法。该方法通过计算机图形学技术和计算机视觉技术的结合,能够直观、快速地模拟空间载体位姿测量过程。

基于视觉的机器人轨迹误差测量技术

提出了一种基于视觉的机器人轨迹精度测量系统,该系统以计算机视觉为基础,结合激光测量等技术,可实时测量机器人的运动轨迹误差.完成了高精度图像快速采集与处理、系统标定、三维计算及计算结果可视化等关键技术研究及系统研制工作,并在机器人上进行了实验,大量的实验表明,该系统的测量精度和速度均可满足机器人的轨迹测量的需要.