Bayesian scene analysis for multi-camera 3D tracking and camera positioning

Los sistemas de seguimiento mono-cámara han demostrado su notable capacidad para el análisis de trajectorias de objectos móviles y para monitorización de escenas de interés; sin embargo, tanto su robustez como sus posibilidades en cuanto a comprensión semántica de la escena están fuertemente limitadas por su naturaleza local y monocular, lo que los hace insuficientes para aplicaciones realistas de videovigilancia. El objetivo de esta tesis es la extensión de las posibilidades de los sistemas de seguimiento de objetos móviles para lograr un mayor grado de robustez y comprensión de la escena. La extensión propuesta se divide en dos direcciones separadas. La primera puede considerarse local, ya que está orientada a la mejora y enriquecimiento de las posiciones estimadas para los objetos móviles observados directamente por las cámaras del sistema; dicha extensión se logra mediante el desarrollo de un sistema multi-cámara de seguimiento 3D, capaz de proporcionar consistentemente las posiciones 3D de múltiples objetos a partir de las observaciones capturadas por un conjunto de sensores calibrados y con campos de visión solapados. La segunda extensión puede considerarse global, dado que su objetivo consiste en proporcionar un contexto global para relacionar las observaciones locales realizadas por una cámara con una escena de mucho mayor tamaño; para ello se propone un sistema automático de localización de cámaras basado en las trayectorias observadas de varios objetos móviles y en un mapa esquemático de la escena global monitorizada. Ambas líneas de investigación se tratan utilizando, como marco común, técnicas de estimación bayesiana: esta elección está justificada por la versatilidad y flexibilidad proporcionada por dicho marco estadístico, que permite la combinación natural de múltiples fuentes de información sobre los parámetros a estimar, así como un tratamiento riguroso de la incertidumbre asociada a las mismas mediante la inclusión de modelos de observación específicamente diseñados. Además, el marco seleccionado abre grandes posibilidades operacionales, puesto que permite la creación de diferentes métodos numéricos adaptados a las necesidades y características específicas de distintos problemas tratados. El sistema de seguimiento 3D con múltiples cámaras propuesto está específicamente diseñado para permitir descripciones esquemáticas de las medidas realizadas individualmente por cada una de las cámaras del sistema: esta elección de diseño, por tanto, no asume ningún algoritmo específico de detección o seguimiento 2D en ninguno de los sensores de la red, y hace que el sistema propuesto sea aplicable a redes reales de vigilancia con capacidades limitadas tanto en términos de procesamiento como de transmision. La combinación robusta de las observaciones capturadas individualmente por las cámaras, ruidosas, incompletas y probablemente contaminadas por falsas detecciones, se basa en un metodo de asociación bayesiana basado en geometría y color: los resultados de dicha asociación permiten el seguimiento 3D de los objetos de la escena mediante el uso de un filtro de partículas. El sistema de fusión de observaciones propuesto tiene, como principales características, una gran precisión en términos de localización 3D de objetos, y una destacable capacidad de recuperación tras eventuales errores debidos a un número insuficiente de datos de entrada. El sistema automático de localización de cámaras se basa en la observación de múltiples objetos móviles y un mapa esquemático de las áreas transitables del entorno monitorizado para inferir la posición absoluta de dicho sensor. Para este propósito, se propone un novedoso marco bayesiano que combina modelos dinámicos inducidos por el mapa en los objetos móviles presentes en la escena con las trayectorias observadas por la cámara, lo que representa un enfoque nunca utilizado en la literatura existente. El sistema de localización se divide en dos sub-tareas diferenciadas, debido a que cada una de estas tareas requiere del diseño de algoritmos específicos de muestreo para explotar en profundidad las características del marco desarrollado: por un lado, análisis de la ambigüedad del caso específicamente tratado y estimación aproximada de la localización de la cámara, y por otro, refinado de la localización de la cámara. El sistema completo, diseñado y probado para el caso específico de localización de cámaras en entornos de tráfico urbano, podría tener aplicación también en otros entornos y sensores de diferentes modalidades tras ciertas adaptaciones. ABSTRACT Mono-camera tracking systems have proved their capabilities for moving object trajectory analysis and scene monitoring, but their robustness and semantic possibilities are strongly limited by their local and monocular nature and are often insufficient for realistic surveillance applications. This thesis is aimed at extending the possibilities of moving object tracking systems to a higher level of scene understanding. The proposed extension comprises two separate directions. The first one is local, since is aimed at enriching the inferred positions of the moving objects within the area of the monitored scene directly covered by the cameras of the system; this task is achieved through the development of a multi-camera system for robust 3D tracking, able to provide 3D tracking information of multiple simultaneous moving objects from the observations reported by a set of calibrated cameras with semi-overlapping fields of view. The second extension is global, as is aimed at providing local observations performed within the field of view of one camera with a global context relating them to a much larger scene; to this end, an automatic camera positioning system relying only on observed object trajectories and a scene map is designed. The two lines of research in this thesis are addressed using Bayesian estimation as a general unifying framework. Its suitability for these two applications is justified by the flexibility and versatility of that stochastic framework, which allows the combination of multiple sources of information about the parameters to estimate in a natural and elegant way, addressing at the same time the uncertainty associated to those sources through the inclusion of models designed to this end. In addition, it opens multiple possibilities for the creation of different numerical methods for achieving satisfactory and efficient practical solutions to each addressed application. The proposed multi-camera 3D tracking method is specifically designed to work on schematic descriptions of the observations performed by each camera of the system: this choice allows the use of unspecific off-the-shelf 2D detection and/or tracking subsystems running independently at each sensor, and makes the proposal suitable for real surveillance networks with moderate computational and transmission capabilities. The robust combination of such noisy, incomplete and possibly unreliable schematic descriptors relies on a Bayesian association method, based on geometry and color, whose results allow the tracking of the targets in the scene with a particle filter. The main features exhibited by the proposal are, first, a remarkable accuracy in terms of target 3D positioning, and second, a great recovery ability after tracking losses due to insufficient input data. The proposed system for visual-based camera self-positioning uses the observations of moving objects and a schematic map of the passable areas of the environment to infer the absolute sensor position. To this end, a new Bayesian framework combining trajectory observations and map-induced dynamic models for moving objects is designed, which represents an approach to camera positioning never addressed before in the literature. This task is divided into two different sub-tasks, setting ambiguity analysis and approximate position estimation, on the one hand, and position refining, on the other, since they require the design of specific sampling algorithms to correctly exploit the discriminative features of the developed framework. This system, designed for camera positioning and demonstrated in urban traffic environments, can also be applied to different environments and sensors of other modalities after certain required adaptations.

Fluorescent RNA cytochemistry: tracking gene transcripts in living cells

The advent of jellyfish green fluorescent protein and its spectral variants, together with promising new fluorescent proteins from other classes of the Cnidarian phylum (coral and anemones), has greatly enhanced and promises to further boost the detection and localization of proteins in cell biology. It has been less widely appreciated that highly sensitive methods have also recently been developed for detecting the movement and localization in living cells of the very molecules that precede proteins in the gene expression pathway, i.e. RNAs. These approaches include the microinjection of fluorescent RNAs into living cells, the in vivo hybridization of fluorescent oligonucleotides to endogenous RNAs and the expression in cells of fluorescent RNA-binding proteins. This new field of ‘fluorescent RNA cytochemistry’ is summarized in this article, with emphasis on the biological insights it has already provided. These new techniques are likely to soon collaborate with other emerging approaches to advance the investigation of RNA birth, RNA–protein assembly and ribonucleoprotein particle transport in systems such as oocytes, embryos, neurons and other somatic cells, and may even permit the observation of viral replication and transcription pathways as they proceed in living cells, ushering in a new era of nucleic acids research in vivo.

Resolution limits to object tracking with subpixel accuracy

Subpixel methods increase the accuracy and efficiency of image detectors, processing units, and algorithms and provide very cost-effective systems for object tracking. Published methods achieve resolution increases up to three orders of magnitude. In this Letter, we demonstrate that this limit can be theoretically improved by several orders of magnitude, permitting micropixel and submicropixel accuracies. The necessary condition for movement detection is that one single pixel changes its status. We show that an appropriate target design increases the probability of a pixel change for arbitrarily small shifts, thus increasing the detection accuracy of a tracking system. The proposal does not impose severe restriction on the target nor on the sensor, thus allowing easy experimental implementation.

PIV measurements combined with the motion tracking technique to analyze flow around a moving porous structure

To gain a better understanding of the fluid–structure interaction and especially when dealing with a flow around an arbitrarily moving body, it is essential to develop measurement tools enabling the instantaneous detection of moving deformable interface during the flow measurements. A particularly useful application is the determination of unsteady turbulent flow velocity field around a moving porous fishing net structure which is of great interest for selectivity and also for the numerical code validation which needs a realistic database. To do this, a representative piece of fishing net structure is used to investigate both the Turbulent Boundary Layer (TBL) developing over the horizontal porous moving fishing net structure and the turbulent flow passing through the moving porous structure. For such an investigation, Time Resolved PIV measurements are carried out and combined with a motion tracking technique allowing the measurement of the instantaneous motion of the deformable fishing net during PIV measurements. Once the two-dimensional motion of the porous structure is accessed, PIV velocity measurements are analyzed in connection with the detected motion. Finally, the TBL is characterized and the effect of the structure motion on the volumetric flow rate passing though the moving porous structure is clearly demonstrated.

Robust real-time bio-kinematic movement tracking using multiple kinects for tele-rehabilitation

This paper investigates the robust and accurate capture of human joint poses and bio-kinematic movements for exercise monitoring in real-time tele-rehabilitation applications. Recently developed model-based estimation ideas are used to improve the accuracy, robustness, and real-time characteristics considered vital for applications, where affordability and domestic use are the primary focus. We use the spatial diversity of the arbitrarily positioned Microsoft Kinect receivers to improve the reliability and promote the uptake of the concept. The skeleton-based information is fused to enhance accuracy and robustness, critical for biomedical applications. A specific version of a robust Kalman filter (KF) in a linear framework is employed to ensure superior estimator convergence and real-time use, compared to other commonly used filters. The algorithmic development was conducted in a generic form and computer simulations were conducted to verify our assertions. Hardware implementations were carried out to test the viability of the proposed state estimator in terms of the core requirements of reliability, accuracy, and real-time use. Performance of the overall system implemented in an information fusion context was evaluated against the commercially available and industry standard Vicon system for different exercise routines, producing comparable results with much less infrastructure and financial investment.

Maximum power point tracking techniques for photovoltaic systems: a comprehensive review and comparative analysis

Maximum Power Point Tracking (MPPT) is an important concern in Photovoltaic (PV) systems. As PV systems have a high cost of energy it is essential that they are operated to extract the maximum possible power at all times. However, under non-uniform environmental conditions, which frequently arise in the outdoor environment, many MPPT techniques will fail to track the global peak power. This review paper discusses conventional MPPT techniques designed to operate under uniform environmental conditions and highlights why these techniques fail under non-uniform conditions. Following this, techniques designed specifically to operate under non-uniform environmental conditions are analysed and compared. Simulation results which compare the performance of the common Perturb and Observe (P&O) method, the Particle Swarm Optimisation (PSO) and the Simulated Annealing (SA) MPPT approaches under non-uniform environmental conditions are also presented. The research presented in this review indicates that there is no single technique which can achieve reliable global MPPT with low cost and complexity and be easily adapted to different PV systems.

A simulated annealing global maximum power point tracking approach for PV modules under partial shading conditions

This paper proposes a simulated annealing (SA)-based global maximum power point tracking (GMPPT) technique designed for photovoltaic (PV) systems which experience partial shading conditions (PSC). The proposed technique is compared with the common perturb and observe MPPT technique and the particle swarm optimization method for GMPPT. The performance is assessed by considering the time taken to converge and the number of sample cases where the technique converges to the GMPP. Simulation results indicate the improved performance of the SA-based GMPPT algorithm, with arbitrarily selected parameters, in tracking to the global maxima in a multiple module PV system which experiences PSC. Experimental validation of the technique is presented based on PV modules that experience nonuniform environmental conditions. Additionally, studies regarding the influence of the key parameters of the SA-based algorithm are described. Simulation and experimental results verify the effectiveness of the proposed GMPPT method.

Tracking people in 3D using position, size and shape

This paper presents a prototype tracking system for tracking people in enclosed indoor environments where there is a high rate of occlusions. The system uses a stereo camera for acquisition, and is capable of disambiguating occlusions using a combination of depth map analysis, a two step ellipse fitting people detection process, the use of motion models and Kalman filters and a novel fit metric, based on computationally simple object statistics. Testing shows that our fit metric outperforms commonly used position based metrics and histogram based metrics, resulting in more accurate tracking of people.