Aerial video geo-registration using terrain models from dense and coherent stereo matching

In the context of aerial imagery, one of the first steps toward a coherent processing of the information contained in multiple images is geo-registration, which consists in assigning geographic 3D coordinates to the pixels of the image. This enables accurate alignment and geo-positioning of multiple images, detection of moving objects and fusion of data acquired from multiple sensors. To solve this problem there are different approaches that require, in addition to a precise characterization of the camera sensor, high resolution referenced images or terrain elevation models, which are usually not publicly available or out of date. Building upon the idea of developing technology that does not need a reference terrain elevation model, we propose a geo-registration technique that applies variational methods to obtain a dense and coherent surface elevation model that is used to replace the reference model. The surface elevation model is built by interpolation of scattered 3D points, which are obtained in a two-step process following a classical stereo pipeline: first, coherent disparity maps between image pairs of a video sequence are estimated and then image point correspondences are back-projected. The proposed variational method enforces continuity of the disparity map not only along epipolar lines (as done by previous geo-registration techniques) but also across them, in the full 2D image domain. In the experiments, aerial images from synthetic video sequences have been used to validate the proposed technique.

Estimating Perceived Video Quality from Objective Parameters in Video over IP Services

In Video over IP services, perceived video quality heavily depends on parameters such as video coding and network Quality of Service. This paper proposes a model for the estimation of perceived video quality in video streaming and broadcasting services that combines the aforementioned parameters with other that depend mainly on the information contents of the video sequences. These fitting parameters are derived from the Spatial and Temporal Information contents of the sequences. This model does not require reference to the original video sequence so it can be used for online, real-time monitoring of perceived video quality in Video over IP services. Furthermore, this paper proposes a measurement workbench designed to acquire both training data for model fitting and test data for model validation. Preliminary results show good correlation between measured and predicted values.

Insertion of impairments in test video sequences for quality assessment based on psychovisual characteristics

Assessing video quality is a complex task. While most pixel-based metrics do not present enough correlation between objective and subjective results, algorithms need to correspond to human perception when analyzing quality in a video sequence. For analyzing the perceived quality derived from concrete video artifacts in determined region of interest we present a novel methodology for generating test sequences which allow the analysis of impact of each individual distortion. Through results obtained after subjective assessment it is possible to create psychovisual models based on weighting pixels belonging to different regions of interest distributed by color, position, motion or content. Interesting results are obtained in subjective assessment which demonstrates the necessity of new metrics adapted to human visual system.

Análisis de imágenes del rostro humano

El objetivo general de este trabajo es el correcto funcionamiento de un sistema de reconocimiento facial compuesto de varios módulos, implementados en distintos lenguajes. Uno de dichos módulos está escrito en Python y se encargarí de determinar el género del rostro o rostros que aparecen en una imagen o en un fotograma de una secuencia de vídeo. El otro módulo, escrito en C++, llevará a cabo el reconocimiento de cada una de las partes de la cara (ojos, nariz, boca) y la orientación hacia la que está posicionada (derecha, izquierda). La primera parte de esta memoria corresponde a la reimplementación de todas las partes de un analizador facial, que constituyen el primer módulo antes mencionado. Estas partes son un analizador, compuesto a su vez por un reconocedor (Tracker) y un procesador (Processor), y una clase visor para poder visualizar los resultados. Por un lado, el reconocedor o "Tracker.es el encargado de encontrar la cara y sus partes, que serán pasadas al procesador o Processor, que analizará la cara obtenida por el reconocedor y determinará su género. Este módulo estaba dise~nado completamente en C y OpenCV 1.0, y ha sido reescrito en Python y OpenCV 2.4. Y en la segunda parte, se explica cómo realizar la comunicación entre el primer módulo escrito en Python y el segundo escrito en C++. Además, se analizarán diferentes herramientas para poder ejecutar código C++ desde programas Python. Dichas herramientas son PyBindGen, Cython y Boost. Dependiendo de las necesidades del programador se contará cuál de ellas es más conveniente utilizar en cada caso. Por último, en el apartado de resultados se puede observar el funcionamiento del sistema con la integración de los dos módulos, y cómo se muestran por pantalla los puntos de interés, el género y la orientación del rostro utilizando imágenes tomadas con una cámara web.---ABSTRACT---The main objective of this document is the proper functioning of a facial recognition system composed of two modules, implemented in diferent languages. One of these modules is written in Python, and his purpose is determining the gender of the face or faces in an image or a frame of a video sequence. The other module is written in C ++ and it will perform the recognition of each of the parts of the face (eyes, nose , mouth), and the head pose (right, left).The first part of this document corresponds to the reimplementacion of all components of a facial analyzer , which constitute the first module that I mentioned before. These parts are an analyzer , composed by a tracke) and a processor, and a viewer to display the results. The tracker function is to find and its parts, which will be passed to the processor, which will analyze the face obtained by the tracker. The processor will determine the face's gender. This module was completely written in C and OpenCV 1.0, and it has been rewritten in Python and OpenCV 2.4. And in the second part, it explains how to comunicate two modules, one of them written in Python and the other one written in C++. Furthermore, it talks about some tools to execute C++ code from Python scripts. The tools are PyBindGen, Cython and Boost. It will tell which one of those tools is better to use depend on the situation. Finally, in the results section it is possible to see how the system works with the integration of the two modules, and how the points of interest, the gender an the head pose are displayed on the screen using images taken from a webcam.

Human-computer interaction based on visual hand-gesture recognition using volumetric spatiograms of local binary patterns

A more natural, intuitive, user-friendly, and less intrusive Human–Computer interface for controlling an application by executing hand gestures is presented. For this purpose, a robust vision-based hand-gesture recognition system has been developed, and a new database has been created to test it. The system is divided into three stages: detection, tracking, and recognition. The detection stage searches in every frame of a video sequence potential hand poses using a binary Support Vector Machine classifier and Local Binary Patterns as feature vectors. These detections are employed as input of a tracker to generate a spatio-temporal trajectory of hand poses. Finally, the recognition stage segments a spatio-temporal volume of data using the obtained trajectories, and compute a video descriptor called Volumetric Spatiograms of Local Binary Patterns (VS-LBP), which is delivered to a bank of SVM classifiers to perform the gesture recognition. The VS-LBP is a novel video descriptor that constitutes one of the most important contributions of the paper, which is able to provide much richer spatio-temporal information than other existing approaches in the state of the art with a manageable computational cost. Excellent results have been obtained outperforming other approaches of the state of the art.

Multiple Hashing Integration for Real-Time Large Scale Part-to-Part Video Matching

A real-time large scale part-to-part video matching algorithm, based on the cross correlation of the intensity of motion curves, is proposed with a view to originality recognition, video database cleansing, copyright enforcement, video tagging or video result re-ranking. Moreover, it is suggested how the most representative hashes and distance functions - strada, discrete cosine transformation, Marr-Hildreth and radial - should be integrated in order for the matching algorithm to be invariant against blur, compression and rotation distortions: (R; _) 2 [1; 20]_[1; 8], from 512_512 to 32_32pixels2 and from 10 to 180_. The DCT hash is invariant against blur and compression up to 64x64 pixels2. Nevertheless, although its performance against rotation is the best, with a success up to 70%, it should be combined with the Marr-Hildreth distance function. With the latter, the image selected by the DCT hash should be at a distance lower than 1.15 times the Marr-Hildreth minimum distance.

Video analysis based vehicle detection and tracking using an MCMC sampling framework

This article presents a probabilistic method for vehicle detection and tracking through the analysis of monocular images obtained from a vehicle-mounted camera. The method is designed to address the main shortcomings of traditional particle filtering approaches, namely Bayesian methods based on importance sampling, for use in traffic environments. These methods do not scale well when the dimensionality of the feature space grows, which creates significant limitations when tracking multiple objects. Alternatively, the proposed method is based on a Markov chain Monte Carlo (MCMC) approach, which allows efficient sampling of the feature space. The method involves important contributions in both the motion and the observation models of the tracker. Indeed, as opposed to particle filter-based tracking methods in the literature, which typically resort to observation models based on appearance or template matching, in this study a likelihood model that combines appearance analysis with information from motion parallax is introduced. Regarding the motion model, a new interaction treatment is defined based on Markov random fields (MRF) that allows for the handling of possible inter-dependencies in vehicle trajectories. As for vehicle detection, the method relies on a supervised classification stage using support vector machines (SVM). The contribution in this field is twofold. First, a new descriptor based on the analysis of gradient orientations in concentric rectangles is dened. This descriptor involves a much smaller feature space compared to traditional descriptors, which are too costly for real-time applications. Second, a new vehicle image database is generated to train the SVM and made public. The proposed vehicle detection and tracking method is proven to outperform existing methods and to successfully handle challenging situations in the test sequences.

Video Prioritization for Unequal Error Protection

We analyze the effect of packet losses in video sequences and propose a lightweight Unequal Error Protection strategy which, by choosing which packet is discarded, reduces strongly the Mean Square Error of the received sequence

Mapping wide row crops with video sequences acquired from a tractor moving at treatment speed

This paper presents a mapping method for wide row crop fields. The resulting map shows the crop rows and weeds present in the inter-row spacing. Because field videos are acquired with a camera mounted on top of an agricultural vehicle, a method for image sequence stabilization was needed and consequently designed and developed. The proposed stabilization method uses the centers of some crop rows in the image sequence as features to be tracked, which compensates for the lateral movement (sway) of the camera and leaves the pitch unchanged. A region of interest is selected using the tracked features, and an inverse perspective technique transforms the selected region into a bird’s-eye view that is centered on the image and that enables map generation. The algorithm developed has been tested on several video sequences of different fields recorded at different times and under different lighting conditions, with good initial results. Indeed, lateral displacements of up to 66% of the inter-row spacing were suppressed through the stabilization process, and crop rows in the resulting maps appear straight

Depth perceptual video coding for free viewpoint video based on H.264/AVC

A novel scheme for depth sequences compression, based on a perceptual coding algorithm, is proposed. A depth sequence describes the object position in the 3D scene, and is used, in Free Viewpoint Video, for the generation of synthetic video sequences. In perceptual video coding the human visual system characteristics are exploited to improve the compression efficiency. As depth sequences are never shown, the perceptual video coding, assessed over them, is not effective. The proposed algorithm is based on a novel perceptual rate distortion optimization process, assessed over the perceptual distortion of the rendered views generated through the encoded depth sequences. The experimental results show the effectiveness of the proposed method, able to obtain a very considerable improvement of the rendered view perceptual quality.

Efficient hybrid monocular-stereo approach to on-board, video-based traffic sign detection and tracking

In this paper we propose an innovative method for the automatic detection and tracking of road traffic signs using an onboard stereo camera. It involves a combination of monocular and stereo analysis strategies to increase the reliability of the detections such that it can boost the performance of any traffic sign recognition scheme. Firstly, an adaptive color and appearance based detection is applied at single camera level to generate a set of traffic sign hypotheses. In turn, stereo information allows for sparse 3D reconstruction of potential traffic signs through a SURF-based matching strategy. Namely, the plane that best fits the cloud of 3D points traced back from feature matches is estimated using a RANSAC based approach to improve robustness to outliers. Temporal consistency of the 3D information is ensured through a Kalman-based tracking stage. This also allows for the generation of a predicted 3D traffic sign model, which is in turn used to enhance the previously mentioned color-based detector through a feedback loop, thus improving detection accuracy. The proposed solution has been tested with real sequences under several illumination conditions and in both urban areas and highways, achieving very high detection rates in challenging environments, including rapid motion and significant perspective distortion