Simultaneous 3D object tracking and camera parameter estimation by Bayesian methods and transdimensional MCMC sampling

Multi-camera 3D tracking systems with overlapping cameras represent a powerful mean for scene analysis, as they potentially allow greater robustness than monocular systems and provide useful 3D information about object location and movement. However, their performance relies on accurately calibrated camera networks, which is not a realistic assumption in real surveillance environments. Here, we introduce a multi-camera system for tracking the 3D position of a varying number of objects and simultaneously refin-ing the calibration of the network of overlapping cameras. Therefore, we introduce a Bayesian framework that combines Particle Filtering for tracking with recursive Bayesian estimation methods by means of adapted transdimensional MCMC sampling. Addi-tionally, the system has been designed to work on simple motion detection masks, making it suitable for camera networks with low transmission capabilities. Tests show that our approach allows a successful performance even when starting from clearly inaccurate camera calibrations, which would ruin conventional approaches.

2D-Tasks for Cognitive Rehabilitation

Neuropsychological Rehabilitation is a complex clinic process which tries to restore or compensate cognitive and behavioral disorders in people suffering from a central nervous system injury. Information and Communication Technologies (ICTs) in Biomedical Engineering play an essential role in this field, allowing improvement and expansion of present rehabilitation programs. This paper presents a set of cognitive rehabilitation 2D-Tasks for patients with Acquired Brain Injury (ABI). These tasks allow a high degree of personalization and individualization in therapies, based on the opportunities offered by new technologies.

Advances in the Simultaneous Multiple Surface optical design method for imaging and non-imaging applications

Classical imaging optics has been developed over centuries in many areas, such as its paraxial imaging theory and practical design methods like multi-parametric optimization techniques. Although these imaging optical design methods can provide elegant solutions to many traditional optical problems, there are more and more new design problems, like solar concentrator, illumination system, ultra-compact camera, etc., that require maximum energy transfer efficiency, or ultra-compact optical structure. These problems do not have simple solutions from classical imaging design methods, because not only paraxial rays, but also non-paraxial rays should be well considered in the design process. Non-imaging optics is a newly developed optical discipline, which does not aim to form images, but to maximize energy transfer efficiency. One important concept developed from non-imaging optics is the “edge-ray principle”, which states that the energy flow contained in a bundle of rays will be transferred to the target, if all its edge rays are transferred to the target. Based on that concept, many CPC solar concentrators have been developed with efficiency close to the thermodynamic limit. When more than one bundle of edge-rays needs to be considered in the design, one way to obtain solutions is to use SMS method. SMS stands for Simultaneous Multiple Surface, which means several optical surfaces are constructed simultaneously. The SMS method was developed as a design method in Non-imaging optics during the 90s. The method can be considered as an extension to the Cartesian Oval calculation. In the traditional Cartesian Oval calculation, one optical surface is built to transform an input wave-front to an out-put wave-front. The SMS method however, is dedicated to solve more than 1 wave-fronts transformation problem. In the beginning, only 2 input wave-fronts and 2 output wave-fronts transformation problem was considered in the SMS design process for rotational optical systems or free-form optical systems. Usually “SMS 2D” method stands for the SMS procedure developed for rotational optical system, and “SMS 3D” method for the procedure for free-form optical system. Although the SMS method was originally employed in non-imaging optical system designs, it has been found during this thesis that with the improved capability to design more surfaces and control more input and output wave-fronts, the SMS method can also be applied to imaging system designs and possesses great advantage over traditional design methods. In this thesis, one of the main goals to achieve is to further develop the existing SMS-2D method to design with more surfaces and improve the stability of the SMS-2D and SMS-3D algorithms, so that further optimization process can be combined with SMS algorithms. The benefits of SMS plus optimization strategy over traditional optimization strategy will be explained in details for both rotational and free-form imaging optical system designs. Another main goal is to develop novel design concepts and methods suitable for challenging non-imaging applications, e.g. solar concentrator and solar tracker. This thesis comprises 9 chapters and can be grouped into two parts: the first part (chapter 2-5) contains research works in the imaging field, and the second part (chapter 6-8) contains works in the non-imaging field. In the first chapter, an introduction to basic imaging and non-imaging design concepts and theories is given. Chapter 2 presents a basic SMS-2D imaging design procedure using meridian rays. In this chapter, we will set the imaging design problem from the SMS point of view, and try to solve the problem numerically. The stability of this SMS-2D design procedure will also be discussed. The design concepts and procedures developed in this chapter lay the path for further improvement. Chapter 3 presents two improved SMS 3 surfaces’ design procedures using meridian rays (SMS-3M) and skew rays (SMS-1M2S) respectively. The major improvement has been made to the central segments selections, so that the whole SMS procedures become more stable compared to procedures described in Chapter 2. Since these two algorithms represent two types of phase space sampling, their image forming capabilities are compared in a simple objective design. Chapter 4 deals with an ultra-compact SWIR camera design with the SMS-3M method. The difficulties in this wide band camera design is how to maintain high image quality meanwhile reduce the overall system length. This interesting camera design provides a playground for the classical design method and SMS design methods. We will show designs and optical performance from both classical design method and the SMS design method. Tolerance study is also given as the end of the chapter. Chapter 5 develops a two-stage SMS-3D based optimization strategy for a 2 freeform mirrors imaging system. In the first optimization phase, the SMS-3D method is integrated into the optimization process to construct the two mirrors in an accurate way, drastically reducing the unknown parameters to only few system configuration parameters. In the second optimization phase, previous optimized mirrors are parameterized into Qbfs type polynomials and set up in code V. Code V optimization results demonstrates the effectiveness of this design strategy in this 2-mirror system design. Chapter 6 shows an etendue-squeezing condenser optics, which were prepared for the 2010 IODC illumination contest. This interesting design employs many non-imaging techniques such as the SMS method, etendue-squeezing tessellation, and groove surface design. This device has theoretical efficiency limit as high as 91.9%. Chapter 7 presents a freeform mirror-type solar concentrator with uniform irradiance on the solar cell. Traditional parabolic mirror concentrator has many drawbacks like hot-pot irradiance on the center of the cell, insufficient use of active cell area due to its rotational irradiance pattern and small acceptance angle. In order to conquer these limitations, a novel irradiance homogenization concept is developed, which lead to a free-form mirror design. Simulation results show that the free-form mirror reflector has rectangular irradiance pattern, uniform irradiance distribution and large acceptance angle, which confirm the viability of the design concept. Chapter 8 presents a novel beam-steering array optics design strategy. The goal of the design is to track large angle parallel rays by only moving optical arrays laterally, and convert it to small angle parallel output rays. The design concept is developed as an extended SMS method. Potential applications of this beam-steering device are: skylights to provide steerable natural illumination, building integrated CPV systems, and steerable LED illumination. Conclusion and future lines of work are given in Chapter 9. Resumen La óptica de formación de imagen clásica se ha ido desarrollando durante siglos, dando lugar tanto a la teoría de óptica paraxial y los métodos de diseño prácticos como a técnicas de optimización multiparamétricas. Aunque estos métodos de diseño óptico para formación de imagen puede aportar soluciones elegantes a muchos problemas convencionales, siguen apareciendo nuevos problemas de diseño óptico, concentradores solares, sistemas de iluminación, cámaras ultracompactas, etc. que requieren máxima transferencia de energía o dimensiones ultracompactas. Este tipo de problemas no se pueden resolver fácilmente con métodos clásicos de diseño porque durante el proceso de diseño no solamente se deben considerar los rayos paraxiales sino también los rayos no paraxiales. La óptica anidólica o no formadora de imagen es una disciplina que ha evolucionado en gran medida recientemente. Su objetivo no es formar imagen, es maximazar la eficiencia de transferencia de energía. Un concepto importante de la óptica anidólica son los “rayos marginales”, que se pueden utilizar para el diseño de sistemas ya que si todos los rayos marginales llegan a nuestra área del receptor, todos los rayos interiores también llegarán al receptor. Haciendo uso de este principio, se han diseñado muchos concentradores solares que funcionan cerca del límite teórico que marca la termodinámica. Cuando consideramos más de un haz de rayos marginales en nuestro diseño, una posible solución es usar el método SMS (Simultaneous Multiple Surface), el cuál diseña simultáneamente varias superficies ópticas. El SMS nació como un método de diseño para óptica anidólica durante los años 90. El método puede ser considerado como una extensión del cálculo del óvalo cartesiano. En el método del óvalo cartesiano convencional, se calcula una superficie para transformar un frente de onda entrante a otro frente de onda saliente. El método SMS permite transformar varios frentes de onda de entrada en frentes de onda de salida. Inicialmente, sólo era posible transformar dos frentes de onda con dos superficies con simetría de rotación y sin simetría de rotación, pero esta limitación ha sido superada recientemente. Nos referimos a “SMS 2D” como el método orientado a construir superficies con simetría de rotación y llamamos “SMS 3D” al método para construir superficies sin simetría de rotación o free-form. Aunque el método originalmente fue aplicado en el diseño de sistemas anidólicos, se ha observado que gracias a su capacidad para diseñar más superficies y controlar más frentes de onda de entrada y de salida, el SMS también es posible aplicarlo a sistemas de formación de imagen proporcionando una gran ventaja sobre los métodos de diseño tradicionales. Uno de los principales objetivos de la presente tesis es extender el método SMS-2D para permitir el diseño de sistemas con mayor número de superficies y mejorar la estabilidad de los algoritmos del SMS-2D y SMS-3D, haciendo posible combinar la optimización con los algoritmos. Los beneficios de combinar SMS y optimización comparado con el proceso de optimización tradicional se explican en detalle para sistemas con simetría de rotación y sin simetría de rotación. Otro objetivo importante de la tesis es el desarrollo de nuevos conceptos de diseño y nuevos métodos en el área de la concentración solar fotovoltaica. La tesis está estructurada en 9 capítulos que están agrupados en dos partes: la primera de ellas (capítulos 2-5) se centra en la óptica formadora de imagen mientras que en la segunda parte (capítulos 6-8) se presenta el trabajo del área de la óptica anidólica. El primer capítulo consta de una breve introducción de los conceptos básicos de la óptica anidólica y la óptica en formación de imagen. El capítulo 2 describe un proceso de diseño SMS-2D sencillo basado en los rayos meridianos. En este capítulo se presenta el problema de diseñar un sistema formador de imagen desde el punto de vista del SMS y se intenta obtener una solución de manera numérica. La estabilidad de este proceso se analiza con detalle. Los conceptos de diseño y los algoritmos desarrollados en este capítulo sientan la base sobre la cual se realizarán mejoras. El capítulo 3 presenta dos procedimientos para el diseño de un sistema con 3 superficies SMS, el primero basado en rayos meridianos (SMS-3M) y el segundo basado en rayos oblicuos (SMS-1M2S). La mejora más destacable recae en la selección de los segmentos centrales, que hacen más estable todo el proceso de diseño comparado con el presentado en el capítulo 2. Estos dos algoritmos representan dos tipos de muestreo del espacio de fases, su capacidad para formar imagen se compara diseñando un objetivo simple con cada uno de ellos. En el capítulo 4 se presenta un diseño ultra-compacto de una cámara SWIR diseñada usando el método SMS-3M. La dificultad del diseño de esta cámara de espectro ancho radica en mantener una alta calidad de imagen y al mismo tiempo reducir drásticamente sus dimensiones. Esta cámara es muy interesante para comparar el método de diseño clásico y el método de SMS. En este capítulo se presentan ambos diseños y se analizan sus características ópticas. En el capítulo 5 se describe la estrategia de optimización basada en el método SMS-3D. El método SMS-3D calcula las superficies ópticas de manera precisa, dejando sólo unos pocos parámetros libres para decidir la configuración del sistema. Modificando el valor de estos parámetros se genera cada vez mediante SMS-3D un sistema completo diferente. La optimización se lleva a cabo variando los mencionados parámetros y analizando el sistema generado. Los resultados muestran que esta estrategia de diseño es muy eficaz y eficiente para un sistema formado por dos espejos. En el capítulo 6 se describe un sistema de compresión de la Etendue, que fue presentado en el concurso de iluminación del IODC en 2010. Este interesante diseño hace uso de técnicas propias de la óptica anidólica, como el método SMS, el teselado de las lentes y el diseño mediante grooves. Este dispositivo tiene un límite teórica en la eficiencia del 91.9%. El capítulo 7 presenta un concentrador solar basado en un espejo free-form con irradiancia uniforme sobre la célula. Los concentradores parabólicos tienen numerosas desventajas como los puntos calientes en la zona central de la célula, uso no eficiente del área de la célula al ser ésta cuadrada y además tienen ángulos de aceptancia de reducido. Para poder superar estas limitaciones se propone un novedoso concepto de homogeneización de la irrandancia que se materializa en un diseño con espejo free-form. El análisis mediante simulación demuestra que la irradiancia es homogénea en una región rectangular y con mayor ángulo de aceptancia, lo que confirma la viabilidad del concepto de diseño. En el capítulo 8 se presenta un novedoso concepto para el diseño de sistemas afocales dinámicos. El objetivo del diseño es realizar un sistema cuyo haz de rayos de entrada pueda llegar con ángulos entre ±45º mientras que el haz de rayos a la salida sea siempre perpendicular al sistema, variando únicamente la posición de los elementos ópticos lateralmente. Las aplicaciones potenciales de este dispositivo son varias: tragaluces que proporcionan iluminación natural, sistemas de concentración fotovoltaica integrados en los edificios o iluminación direccionable con LEDs. Finalmente, el último capítulo contiene las conclusiones y las líneas de investigación futura.

Response threshold models, stochastic learning automata and ant colony optimization-based decentralized self-coordination algorithms for heterogeneous multi-tasks distribution in multi-robot systems

In recent decades, there has been an increasing interest in systems comprised of several autonomous mobile robots, and as a result, there has been a substantial amount of development in the eld of Articial Intelligence, especially in Robotics. There are several studies in the literature by some researchers from the scientic community that focus on the creation of intelligent machines and devices capable to imitate the functions and movements of living beings. Multi-Robot Systems (MRS) can often deal with tasks that are dicult, if not impossible, to be accomplished by a single robot. In the context of MRS, one of the main challenges is the need to control, coordinate and synchronize the operation of multiple robots to perform a specic task. This requires the development of new strategies and methods which allow us to obtain the desired system behavior in a formal and concise way. This PhD thesis aims to study the coordination of multi-robot systems, in particular, addresses the problem of the distribution of heterogeneous multi-tasks. The main interest in these systems is to understand how from simple rules inspired by the division of labor in social insects, a group of robots can perform tasks in an organized and coordinated way. We are mainly interested on truly distributed or decentralized solutions in which the robots themselves, autonomously and in an individual manner, select a particular task so that all tasks are optimally distributed. In general, to perform the multi-tasks distribution among a team of robots, they have to synchronize their actions and exchange information. Under this approach we can speak of multi-tasks selection instead of multi-tasks assignment, which means, that the agents or robots select the tasks instead of being assigned a task by a central controller. The key element in these algorithms is the estimation ix of the stimuli and the adaptive update of the thresholds. This means that each robot performs this estimate locally depending on the load or the number of pending tasks to be performed. In addition, it is very interesting the evaluation of the results in function in each approach, comparing the results obtained by the introducing noise in the number of pending loads, with the purpose of simulate the robot's error in estimating the real number of pending tasks. The main contribution of this thesis can be found in the approach based on self-organization and division of labor in social insects. An experimental scenario for the coordination problem among multiple robots, the robustness of the approaches and the generation of dynamic tasks have been presented and discussed. The particular issues studied are: Threshold models: It presents the experiments conducted to test the response threshold model with the objective to analyze the system performance index, for the problem of the distribution of heterogeneous multitasks in multi-robot systems; also has been introduced additive noise in the number of pending loads and has been generated dynamic tasks over time. Learning automata methods: It describes the experiments to test the learning automata-based probabilistic algorithms. The approach was tested to evaluate the system performance index with additive noise and with dynamic tasks generation for the same problem of the distribution of heterogeneous multi-tasks in multi-robot systems. Ant colony optimization: The goal of the experiments presented is to test the ant colony optimization-based deterministic algorithms, to achieve the distribution of heterogeneous multi-tasks in multi-robot systems. In the experiments performed, the system performance index is evaluated by introducing additive noise and dynamic tasks generation over time.

System Integration in a UAS design. An example of undergraduate student's tasks.

Systems integration is the origin of most major difficulties found in the engineering design of aeronautical vehicles. The whole design team must assure that each subsystem accomplishes its particular goals and that, together with the rest of the systems, they all meet the general aircraft requirements.Design and building of UAS is a field of actuation to which leading Universities, research Centers and Aeronautical designers have dedicated a lot of effort. In recent years, a team of students, lecturers and professors at the Escuela Universitaria de Ingeniería Técnica Aeronáutica (EUITA) have been working on the design and building of a UAS for civil observation. The design of multi-mission Unmanned Aerial Vehicles (UAVs) has seen a rapid progress in the last years. A wide variety of designs and applications, some of them really ingenious, have been proposed. The project, which has been going on as a teamwork experience for the last ten years, consists of the design and building of a UAV, and its peculiarity is that it has been carried out entirely by undergraduate students, as part of their Final Research Project. The students face a challenge that includes all the features and stages of an authentic engineering project. We present the current moment of evolution in the process, together with a description of the main difficulties the project has undergone, as a global experience in engineering design and development.

Detection and Isolation of Simultaneous Additive and Parametric Faults in Nonlinear Stochastic Dynamical Systems

This paper presents a new fault detection and isolation scheme for dealing with simultaneous additive and parametric faults. The new design integrates a system for additive fault detection based on Castillo and Zufiria, 2009 and a new parametric fault detection and isolation scheme inspired in Munz and Zufiria, 2008 . It is shown that the so far existing schemes do not behave correctly when both additive and parametric faults occur simultaneously; to solve the problem a new integrated scheme is proposed. Computer simulation results are presented to confirm the theoretical studies.

Brain rhythms alterations and their effect in functional networks: A simultaneous EEG/fMRI study in Fixation-off epilepsy

Fixation-off sensitivity (FOS) denotes the forms of EEG abnormalities, which are elicited by elimination of central vision or fixation. The phenomenon seems to depend on variables that modulate the alpha rhythm, however, the cerebral mechanisms underlying FOS remain unclear [1]. The scarce previous fMRI findings related to FOS have shown activation in extrastriate cortex [2] and also in frontal areas [3][4]. On the other hand, simultaneous EEG-fMRI technique has been used to assess the relationship between spontaneous power fluctuations of electrical rhythms and associated fMRI signal modulations. These studies have identified that lateral frontoparietal networks show a negative correlation with alpha band in healthy subjects. This neuroanatomical pattern is related to attentional processes and cognitive resources. Moreover, a sub-beta band (17-23 Hz) has been identified with posterior cingulate, temporoparietal junction and dorso-medial prefrontal cortex activations, which correspond to the DMN [5][6].

Decentralized multi-tasks distribution in heterogeneous robot teams by means of ant colony optimization and learning automata

This paper focuses on the general problem of coordinating multiple robots. More specifically, it addresses the self-election of heterogeneous specialized tasks by autonomous robots. In this paper we focus on a specifically distributed or decentralized approach as we are particularly interested on decentralized solution where the robots themselves autonomously and in an individual manner, are responsible of selecting a particular task so that all the existing tasks are optimally distributed and executed. In this regard, we have established an experimental scenario to solve the corresponding multi-tasks distribution problem and we propose a solution using two different approaches by applying Ant Colony Optimization-based deterministic algorithms as well as Learning Automata-based probabilistic algorithms. We have evaluated the robustness of the algorithm, perturbing the number of pending loads to simulate the robot’s error in estimating the real number of pending tasks and also the dynamic generation of loads through time. The paper ends with a critical discussion of experimental results.

Application of self-organizing techniques for the distribution of heterogeneous multi-tasks in multi-robot systems

This paper focuses on the general problem of coordinating of multi-robot systems, more specifically, it addresses the self-election of heterogeneous and specialized tasks by autonomous robots. In this regard, it has proposed experimenting with two different techniques based chiefly on selforganization and emergence biologically inspired, by applying response threshold models as well as ant colony optimization. Under this approach it can speak of multi-tasks selection instead of multi-tasks allocation, that means, as the agents or robots select the tasks instead of being assigned a task by a central controller. The key element in these algorithms is the estimation of the stimuli and the adaptive update of the thresholds. This means that each robot performs this estimate locally depending on the load or the number of pending tasks to be performed. It has evaluated the robustness of the algorithms, perturbing the number of pending loads to simulate the robot’s error in estimating the real number of pending tasks and also the dynamic generation of loads through time. The paper ends with a critical discussion of experimental results.

Video-based tasks for emotional processing rehabilitation in schizophrenia

Schizophrenia is a mental disorder characterized by a breakdown of cognitive processes and by a deficit of typi-cal emotional responses. Effectiveness of computerized task has been demonstrated in the field of cognitive rehabilitation. However, current rehabilitation programs based on virtual environments normally focus on higher cognitive functions, not covering social cognition training. This paper presents a set of video-based tasks specifically designed for the rehabilita-tion of emotional processing deficits in patients in early stages of schizophrenia or schizoaffective disorders. These tasks are part of the Mental Health program of Guttmann NeuroPer-sonalTrainer® cognitive tele-rehabilitation platform, and entail innovation both from a clinical and technological per-spective in relation with former traditional therapeutic con-tents.

A vision-based strategy for autonomous aerial refueling tasks

Autonomous aerial refueling is a key enabling technology for both manned and unmanned aircraft where extended flight duration or range are required. The results presented within this paper offer one potential vision-based sensing solution, together with a unique test environment. A hierarchical visual tracking algorithm based on direct methods is proposed and developed for the purposes of tracking a drogue during the capture stage of autonomous aerial refueling, and of estimating its 3D position. Intended to be applied in real time to a video stream from a single monocular camera mounted on the receiver aircraft, the algorithm is shown to be highly robust, and capable of tracking large, rapid drogue motions within the frame of reference. The proposed strategy has been tested using a complex robotic testbed and with actual flight hardware consisting of a full size probe and drogue. Results show that the vision tracking algorithm can detect and track the drogue at real-time frame rates of more than thirty frames per second, obtaining a robust position estimation even with strong motions and multiple occlusions of the drogue.

Robust control of underactuated wheeled mobile manipulators using GPI disturbance observers

This article describes the design of a linear observer–linear controller-based robust output feedback scheme for output reference trajectory tracking tasks in the case of nonlinear, multivariable, nonholonomic underactuated mobile manipulators. The proposed linear feedback scheme is based on the use of a classical linear feedback controller and suitably extended, high-gain, linear Generalized Proportional Integral (GPI) observers, thus aiding the linear feedback controllers to provide an accurate simultaneous estimation of each flat output associated phase variables and of the exogenous and perturbation inputs. This information is used in the proposed feedback controller in (a) approximate, yet close, cancelations, as lumped unstructured time-varying terms, of the influence of the highly coupled nonlinearities, and (b) the devising of proper linear output feedback control laws based on the approximate estimates of the string of phase variables associated with the flat outputs simultaneously provided by the disturbance observers. Simulations reveal the effectiveness of the proposed approach.

Bayesian scene analysis for multi-camera 3D tracking and camera positioning

Los sistemas de seguimiento mono-cámara han demostrado su notable capacidad para el análisis de trajectorias de objectos móviles y para monitorización de escenas de interés; sin embargo, tanto su robustez como sus posibilidades en cuanto a comprensión semántica de la escena están fuertemente limitadas por su naturaleza local y monocular, lo que los hace insuficientes para aplicaciones realistas de videovigilancia. El objetivo de esta tesis es la extensión de las posibilidades de los sistemas de seguimiento de objetos móviles para lograr un mayor grado de robustez y comprensión de la escena. La extensión propuesta se divide en dos direcciones separadas. La primera puede considerarse local, ya que está orientada a la mejora y enriquecimiento de las posiciones estimadas para los objetos móviles observados directamente por las cámaras del sistema; dicha extensión se logra mediante el desarrollo de un sistema multi-cámara de seguimiento 3D, capaz de proporcionar consistentemente las posiciones 3D de múltiples objetos a partir de las observaciones capturadas por un conjunto de sensores calibrados y con campos de visión solapados. La segunda extensión puede considerarse global, dado que su objetivo consiste en proporcionar un contexto global para relacionar las observaciones locales realizadas por una cámara con una escena de mucho mayor tamaño; para ello se propone un sistema automático de localización de cámaras basado en las trayectorias observadas de varios objetos móviles y en un mapa esquemático de la escena global monitorizada. Ambas líneas de investigación se tratan utilizando, como marco común, técnicas de estimación bayesiana: esta elección está justificada por la versatilidad y flexibilidad proporcionada por dicho marco estadístico, que permite la combinación natural de múltiples fuentes de información sobre los parámetros a estimar, así como un tratamiento riguroso de la incertidumbre asociada a las mismas mediante la inclusión de modelos de observación específicamente diseñados. Además, el marco seleccionado abre grandes posibilidades operacionales, puesto que permite la creación de diferentes métodos numéricos adaptados a las necesidades y características específicas de distintos problemas tratados. El sistema de seguimiento 3D con múltiples cámaras propuesto está específicamente diseñado para permitir descripciones esquemáticas de las medidas realizadas individualmente por cada una de las cámaras del sistema: esta elección de diseño, por tanto, no asume ningún algoritmo específico de detección o seguimiento 2D en ninguno de los sensores de la red, y hace que el sistema propuesto sea aplicable a redes reales de vigilancia con capacidades limitadas tanto en términos de procesamiento como de transmision. La combinación robusta de las observaciones capturadas individualmente por las cámaras, ruidosas, incompletas y probablemente contaminadas por falsas detecciones, se basa en un metodo de asociación bayesiana basado en geometría y color: los resultados de dicha asociación permiten el seguimiento 3D de los objetos de la escena mediante el uso de un filtro de partículas. El sistema de fusión de observaciones propuesto tiene, como principales características, una gran precisión en términos de localización 3D de objetos, y una destacable capacidad de recuperación tras eventuales errores debidos a un número insuficiente de datos de entrada. El sistema automático de localización de cámaras se basa en la observación de múltiples objetos móviles y un mapa esquemático de las áreas transitables del entorno monitorizado para inferir la posición absoluta de dicho sensor. Para este propósito, se propone un novedoso marco bayesiano que combina modelos dinámicos inducidos por el mapa en los objetos móviles presentes en la escena con las trayectorias observadas por la cámara, lo que representa un enfoque nunca utilizado en la literatura existente. El sistema de localización se divide en dos sub-tareas diferenciadas, debido a que cada una de estas tareas requiere del diseño de algoritmos específicos de muestreo para explotar en profundidad las características del marco desarrollado: por un lado, análisis de la ambigüedad del caso específicamente tratado y estimación aproximada de la localización de la cámara, y por otro, refinado de la localización de la cámara. El sistema completo, diseñado y probado para el caso específico de localización de cámaras en entornos de tráfico urbano, podría tener aplicación también en otros entornos y sensores de diferentes modalidades tras ciertas adaptaciones. ABSTRACT Mono-camera tracking systems have proved their capabilities for moving object trajectory analysis and scene monitoring, but their robustness and semantic possibilities are strongly limited by their local and monocular nature and are often insufficient for realistic surveillance applications. This thesis is aimed at extending the possibilities of moving object tracking systems to a higher level of scene understanding. The proposed extension comprises two separate directions. The first one is local, since is aimed at enriching the inferred positions of the moving objects within the area of the monitored scene directly covered by the cameras of the system; this task is achieved through the development of a multi-camera system for robust 3D tracking, able to provide 3D tracking information of multiple simultaneous moving objects from the observations reported by a set of calibrated cameras with semi-overlapping fields of view. The second extension is global, as is aimed at providing local observations performed within the field of view of one camera with a global context relating them to a much larger scene; to this end, an automatic camera positioning system relying only on observed object trajectories and a scene map is designed. The two lines of research in this thesis are addressed using Bayesian estimation as a general unifying framework. Its suitability for these two applications is justified by the flexibility and versatility of that stochastic framework, which allows the combination of multiple sources of information about the parameters to estimate in a natural and elegant way, addressing at the same time the uncertainty associated to those sources through the inclusion of models designed to this end. In addition, it opens multiple possibilities for the creation of different numerical methods for achieving satisfactory and efficient practical solutions to each addressed application. The proposed multi-camera 3D tracking method is specifically designed to work on schematic descriptions of the observations performed by each camera of the system: this choice allows the use of unspecific off-the-shelf 2D detection and/or tracking subsystems running independently at each sensor, and makes the proposal suitable for real surveillance networks with moderate computational and transmission capabilities. The robust combination of such noisy, incomplete and possibly unreliable schematic descriptors relies on a Bayesian association method, based on geometry and color, whose results allow the tracking of the targets in the scene with a particle filter. The main features exhibited by the proposal are, first, a remarkable accuracy in terms of target 3D positioning, and second, a great recovery ability after tracking losses due to insufficient input data. The proposed system for visual-based camera self-positioning uses the observations of moving objects and a schematic map of the passable areas of the environment to infer the absolute sensor position. To this end, a new Bayesian framework combining trajectory observations and map-induced dynamic models for moving objects is designed, which represents an approach to camera positioning never addressed before in the literature. This task is divided into two different sub-tasks, setting ambiguity analysis and approximate position estimation, on the one hand, and position refining, on the other, since they require the design of specific sampling algorithms to correctly exploit the discriminative features of the developed framework. This system, designed for camera positioning and demonstrated in urban traffic environments, can also be applied to different environments and sensors of other modalities after certain required adaptations.

Bayesian scene analysis for multi-camera 3D tracking and camera positioning

Los sistemas de seguimiento mono-cámara han demostrado su notable capacidad para el análisis de trajectorias de objectos móviles y para monitorización de escenas de interés; sin embargo, tanto su robustez como sus posibilidades en cuanto a comprensión semántica de la escena están fuertemente limitadas por su naturaleza local y monocular, lo que los hace insuficientes para aplicaciones realistas de videovigilancia. El objetivo de esta tesis es la extensión de las posibilidades de los sistemas de seguimiento de objetos móviles para lograr un mayor grado de robustez y comprensión de la escena. La extensión propuesta se divide en dos direcciones separadas. La primera puede considerarse local, ya que está orientada a la mejora y enriquecimiento de las posiciones estimadas para los objetos móviles observados directamente por las cámaras del sistema; dicha extensión se logra mediante el desarrollo de un sistema multi-cámara de seguimiento 3D, capaz de proporcionar consistentemente las posiciones 3D de múltiples objetos a partir de las observaciones capturadas por un conjunto de sensores calibrados y con campos de visión solapados. La segunda extensión puede considerarse global, dado que su objetivo consiste en proporcionar un contexto global para relacionar las observaciones locales realizadas por una cámara con una escena de mucho mayor tamaño; para ello se propone un sistema automático de localización de cámaras basado en las trayectorias observadas de varios objetos móviles y en un mapa esquemático de la escena global monitorizada. Ambas líneas de investigación se tratan utilizando, como marco común, técnicas de estimación bayesiana: esta elección está justificada por la versatilidad y flexibilidad proporcionada por dicho marco estadístico, que permite la combinación natural de múltiples fuentes de información sobre los parámetros a estimar, así como un tratamiento riguroso de la incertidumbre asociada a las mismas mediante la inclusión de modelos de observación específicamente diseñados. Además, el marco seleccionado abre grandes posibilidades operacionales, puesto que permite la creación de diferentes métodos numéricos adaptados a las necesidades y características específicas de distintos problemas tratados. El sistema de seguimiento 3D con múltiples cámaras propuesto está específicamente diseñado para permitir descripciones esquemáticas de las medidas realizadas individualmente por cada una de las cámaras del sistema: esta elección de diseño, por tanto, no asume ningún algoritmo específico de detección o seguimiento 2D en ninguno de los sensores de la red, y hace que el sistema propuesto sea aplicable a redes reales de vigilancia con capacidades limitadas tanto en términos de procesamiento como de transmision. La combinación robusta de las observaciones capturadas individualmente por las cámaras, ruidosas, incompletas y probablemente contaminadas por falsas detecciones, se basa en un metodo de asociación bayesiana basado en geometría y color: los resultados de dicha asociación permiten el seguimiento 3D de los objetos de la escena mediante el uso de un filtro de partículas. El sistema de fusión de observaciones propuesto tiene, como principales características, una gran precisión en términos de localización 3D de objetos, y una destacable capacidad de recuperación tras eventuales errores debidos a un número insuficiente de datos de entrada. El sistema automático de localización de cámaras se basa en la observación de múltiples objetos móviles y un mapa esquemático de las áreas transitables del entorno monitorizado para inferir la posición absoluta de dicho sensor. Para este propósito, se propone un novedoso marco bayesiano que combina modelos dinámicos inducidos por el mapa en los objetos móviles presentes en la escena con las trayectorias observadas por la cámara, lo que representa un enfoque nunca utilizado en la literatura existente. El sistema de localización se divide en dos sub-tareas diferenciadas, debido a que cada una de estas tareas requiere del diseño de algoritmos específicos de muestreo para explotar en profundidad las características del marco desarrollado: por un lado, análisis de la ambigüedad del caso específicamente tratado y estimación aproximada de la localización de la cámara, y por otro, refinado de la localización de la cámara. El sistema completo, diseñado y probado para el caso específico de localización de cámaras en entornos de tráfico urbano, podría tener aplicación también en otros entornos y sensores de diferentes modalidades tras ciertas adaptaciones. ABSTRACT Mono-camera tracking systems have proved their capabilities for moving object trajectory analysis and scene monitoring, but their robustness and semantic possibilities are strongly limited by their local and monocular nature and are often insufficient for realistic surveillance applications. This thesis is aimed at extending the possibilities of moving object tracking systems to a higher level of scene understanding. The proposed extension comprises two separate directions. The first one is local, since is aimed at enriching the inferred positions of the moving objects within the area of the monitored scene directly covered by the cameras of the system; this task is achieved through the development of a multi-camera system for robust 3D tracking, able to provide 3D tracking information of multiple simultaneous moving objects from the observations reported by a set of calibrated cameras with semi-overlapping fields of view. The second extension is global, as is aimed at providing local observations performed within the field of view of one camera with a global context relating them to a much larger scene; to this end, an automatic camera positioning system relying only on observed object trajectories and a scene map is designed. The two lines of research in this thesis are addressed using Bayesian estimation as a general unifying framework. Its suitability for these two applications is justified by the flexibility and versatility of that stochastic framework, which allows the combination of multiple sources of information about the parameters to estimate in a natural and elegant way, addressing at the same time the uncertainty associated to those sources through the inclusion of models designed to this end. In addition, it opens multiple possibilities for the creation of different numerical methods for achieving satisfactory and efficient practical solutions to each addressed application. The proposed multi-camera 3D tracking method is specifically designed to work on schematic descriptions of the observations performed by each camera of the system: this choice allows the use of unspecific off-the-shelf 2D detection and/or tracking subsystems running independently at each sensor, and makes the proposal suitable for real surveillance networks with moderate computational and transmission capabilities. The robust combination of such noisy, incomplete and possibly unreliable schematic descriptors relies on a Bayesian association method, based on geometry and color, whose results allow the tracking of the targets in the scene with a particle filter. The main features exhibited by the proposal are, first, a remarkable accuracy in terms of target 3D positioning, and second, a great recovery ability after tracking losses due to insufficient input data. The proposed system for visual-based camera self-positioning uses the observations of moving objects and a schematic map of the passable areas of the environment to infer the absolute sensor position. To this end, a new Bayesian framework combining trajectory observations and map-induced dynamic models for moving objects is designed, which represents an approach to camera positioning never addressed before in the literature. This task is divided into two different sub-tasks, setting ambiguity analysis and approximate position estimation, on the one hand, and position refining, on the other, since they require the design of specific sampling algorithms to correctly exploit the discriminative features of the developed framework. This system, designed for camera positioning and demonstrated in urban traffic environments, can also be applied to different environments and sensors of other modalities after certain required adaptations.

Non-invasive user monitoring in daily tasks for cognitive rehabilitation

El uso de técnicas para la monitorización del movimiento humano generalmente permite a los investigadores analizar la cinemática y especialmente las capacidades motoras en aquellas actividades de la vida cotidiana que persiguen un objetivo concreto como pueden ser la preparación de bebidas y comida, e incluso en tareas de aseo. Adicionalmente, la evaluación del movimiento y el comportamiento humanos en el campo de la rehabilitación cognitiva es esencial para profundizar en las dificultades que algunas personas encuentran en la ejecución de actividades diarias después de accidentes cerebro-vasculares. Estas dificultades están principalmente asociadas a la realización de pasos secuenciales y al reconocimiento del uso de herramientas y objetos. La interpretación de los datos sobre la actitud de este tipo de pacientes para reconocer y determinar el nivel de éxito en la ejecución de las acciones, y para ampliar el conocimiento en las enfermedades cerebrales, sus consecuencias y severidad, depende totalmente de los dispositivos usados para la captura de esos datos y de la calidad de los mismos. Más aún, existe una necesidad real de mejorar las técnicas actuales de rehabilitación cognitiva contribuyendo al diseño de sistemas automáticos para crear una especie de terapeuta virtual que asegure una vida más independiente de estos pacientes y reduzca la carga de trabajo de los terapeutas. Con este objetivo, el uso de sensores y dispositivos para obtener datos en tiempo real de la ejecución y estado de la tarea de rehabilitación es esencial para también contribuir al diseño y entrenamiento de futuros algoritmos que pudieran reconocer errores automáticamente para informar al paciente acerca de ellos mediante distintos tipos de pistas como pueden ser imágenes, mensajes auditivos o incluso videos. La tecnología y soluciones existentes en este campo no ofrecen una manera totalmente robusta y efectiva para obtener datos en tiempo real, por un lado, porque pueden influir en el movimiento del propio paciente en caso de las plataformas basadas en el uso de marcadores que necesitan sensores pegados en la piel; y por otro lado, debido a la complejidad o alto coste de implantación lo que hace difícil pensar en la idea de instalar un sistema en el hospital o incluso en la casa del paciente. Esta tesis presenta la investigación realizada en el campo de la monitorización del movimiento de pacientes para proporcionar un paso adelante en términos de detección, seguimiento y reconocimiento del comportamiento de manos, gestos y cara mediante una manera no invasiva la cual puede mejorar la técnicas actuales de rehabilitación cognitiva para la adquisición en tiempo real de datos sobre el comportamiento del paciente y la ejecución de la tarea. Para entender la importancia del marco de esta tesis, inicialmente se presenta un resumen de las principales enfermedades cognitivas y se introducen las consecuencias que tienen en la ejecución de tareas de la vida diaria. Más aún, se investiga sobre las metodologías actuales de rehabilitación cognitiva. Teniendo en cuenta que las manos son la principal parte del cuerpo para la ejecución de tareas manuales de la vida cotidiana, también se resumen las tecnologías existentes para la captura de movimiento de manos. Una de las principales contribuciones de esta tesis está relacionada con el diseño y evaluación de una solución no invasiva para detectar y seguir las manos durante la ejecución de tareas manuales de la vida cotidiana que a su vez involucran la manipulación de objetos. Esta solución la cual no necesita marcadores adicionales y está basada en una cámara de profundidad de bajo coste, es robusta, precisa y fácil de instalar. Otra contribución presentada se centra en el reconocimiento de gestos para detectar el agarre de objetos basado en un sensor infrarrojo de última generación, y también complementado con una cámara de profundidad. Esta nueva técnica, y también no invasiva, sincroniza ambos sensores para seguir objetos específicos además de reconocer eventos concretos relacionados con tareas de aseo. Más aún, se realiza una evaluación preliminar del reconocimiento de expresiones faciales para analizar si es adecuado para el reconocimiento del estado de ánimo durante la tarea. Por su parte, todos los componentes y algoritmos desarrollados son integrados en un prototipo simple para ser usado como plataforma de monitorización. Se realiza una evaluación técnica del funcionamiento de cada dispositivo para analizar si es adecuada para adquirir datos en tiempo real durante la ejecución de tareas cotidianas reales. Finalmente, se estudia la interacción con pacientes reales para obtener información del nivel de usabilidad del prototipo. Dicha información es esencial y útil para considerar una rehabilitación cognitiva basada en la idea de instalación del sistema en la propia casa del paciente al igual que en el hospital correspondiente. ABSTRACT The use of human motion monitoring techniques usually let researchers to analyse kinematics, especially in motor strategies for goal-oriented activities of daily living, such as the preparation of drinks and food, and even grooming tasks. Additionally, the evaluation of human movements and behaviour in the field of cognitive rehabilitation is essential to deep into the difficulties some people find in common activities after stroke. This difficulties are mainly associated with sequence actions and the recognition of tools usage. The interpretation of attitude data of this kind of patients in order to recognize and determine the level of success of the execution of actions, and to broaden the knowledge in brain diseases, consequences and severity, depends totally on the devices used for the capture of that data and the quality of it. Moreover, there is a real need of improving the current cognitive rehabilitation techniques by contributing to the design of automatic systems to create a kind of virtual therapist for the improvement of the independent life of these stroke patients and to reduce the workload of the occupational therapists currently in charge of them. For this purpose, the use of sensors and devices to obtain real time data of the execution and state of the rehabilitation task is essential to also contribute to the design and training of future smart algorithms which may recognise errors to automatically provide multimodal feedback through different types of cues such as still images, auditory messages or even videos. The technology and solutions currently adopted in the field don't offer a totally robust and effective way for obtaining real time data, on the one hand, because they may influence the patient's movement in case of marker-based platforms which need sensors attached to the skin; and on the other hand, because of the complexity or high cost of implementation, which make difficult the idea of installing a system at the hospital or even patient's home. This thesis presents the research done in the field of user monitoring to provide a step forward in terms of detection, tracking and recognition of hand movements, gestures and face via a non-invasive way which could improve current techniques for cognitive rehabilitation for real time data acquisition of patient's behaviour and execution of the task. In order to understand the importance of the scope of the thesis, initially, a summary of the main cognitive diseases that require for rehabilitation and an introduction of the consequences on the execution of daily tasks are presented. Moreover, research is done about the actual methodology to provide cognitive rehabilitation. Considering that the main body members involved in the completion of a handmade daily task are the hands, the current technologies for human hands movements capture are also highlighted. One of the main contributions of this thesis is related to the design and evaluation of a non-invasive approach to detect and track user's hands during the execution of handmade activities of daily living which involve the manipulation of objects. This approach does not need the inclusion of any additional markers. In addition, it is only based on a low-cost depth camera, it is robust, accurate and easy to install. Another contribution presented is focused on the hand gesture recognition for detecting object grasping based on a brand new infrared sensor, and also complemented with a depth camera. This new, and also non-invasive, solution which synchronizes both sensors to track specific tools as well as recognize specific events related to grooming is evaluated. Moreover, a preliminary assessment of the recognition of facial expressions is carried out to analyse if it is adequate for recognizing mood during the execution of task. Meanwhile, all the corresponding hardware and software developed are integrated in a simple prototype with the purpose of being used as a platform for monitoring the execution of the rehabilitation task. Technical evaluation of the performance of each device is carried out in order to analyze its suitability to acquire real time data during the execution of real daily tasks. Finally, a kind of healthcare evaluation is also presented to obtain feedback about the usability of the system proposed paying special attention to the interaction with real users and stroke patients. This feedback is quite useful to consider the idea of a home-based cognitive rehabilitation as well as a possible hospital installation of the prototype.