A Ground-Truth Video Dataset for the Development and Evaluation of Vision-based Sense-and-Avoid systems

The importance of vision-based systems for Sense-and-Avoid is increasing nowadays as remotely piloted and autonomous UAVs become part of the non-segregated airspace. The development and evaluation of these systems demand flight scenario images which are expensive and risky to obtain. Currently Augmented Reality techniques allow the compositing of real flight scenario images with 3D aircraft models to produce useful realistic images for system development and benchmarking purposes at a much lower cost and risk. With the techniques presented in this paper, 3D aircraft models are positioned firstly in a simulated 3D scene with controlled illumination and rendering parameters. Realistic simulated images are then obtained using an image processing algorithm which fuses the images obtained from the 3D scene with images from real UAV flights taking into account on board camera vibrations. Since the intruder and camera poses are user-defined, ground truth data is available. These ground truth annotations allow to develop and quantitatively evaluate aircraft detection and tracking algorithms. This paper presents the software developed to create a public dataset of 24 videos together with their annotations and some tracking application results.

Real-Time Adaptive Multi-Classifier Multi-Resolution Visual Tracking Framework for Unmanned Aerial Vehicles

This paper presents a novel robust visual tracking framework, based on discriminative method, for Unmanned Aerial Vehicles (UAVs) to track an arbitrary 2D/3D target at real-time frame rates, that is called the Adaptive Multi-Classifier Multi-Resolution (AMCMR) framework. In this framework, adaptive Multiple Classifiers (MC) are updated in the (k-1)th frame-based Multiple Resolutions (MR) structure with compressed positive and negative samples, and then applied them in the kth frame-based Multiple Resolutions (MR) structure to detect the current target. The sample importance has been integrated into this framework to improve the tracking stability and accuracy. The performance of this framework was evaluated with the Ground Truth (GT) in different types of public image databases and real flight-based aerial image datasets firstly, then the framework has been applied in the UAV to inspect the Offshore Floating Platform (OFP). The evaluation and application results show that this framework is more robust, efficient and accurate against the existing state-of-art trackers, overcoming the problems generated by the challenging situations such as obvious appearance change, variant illumination, partial/full target occlusion, blur motion, rapid pose variation and onboard mechanical vibration, among others. To our best knowledge, this is the first work to present this framework for solving the online learning and tracking freewill 2D/3D target problems, and applied it in the UAVs.

Using the Cross-Entropy Method for Control Optimization: A Case Study of See-and-Avoid on Unmanned Aerial Vehicles

This paper presents an adaptation of the Cross-Entropy (CE) method to optimize fuzzy logic controllers. The CE is a recently developed optimization method based on a general Monte-Carlo approach to combinatorial and continuous multi-extremal optimization and importance sampling. This work shows the application of this optimization method to optimize the inputs gains, the location and size of the different membership functions' sets of each variable, as well as the weight of each rule from the rule's base of a fuzzy logic controller (FLC). The control system approach presented in this work was designed to command the orientation of an unmanned aerial vehicle (UAV) to modify its trajectory for avoiding collisions. An onboard looking forward camera was used to sense the environment of the UAV. The information extracted by the image processing algorithm is the only input of the fuzzy control approach to avoid the collision with a predefined object. Real tests with a quadrotor have been done to corroborate the improved behavior of the optimized controllers at different stages of the optimization process.

Definition of masks related to psychovisual features for video quality assessment

Video Quality Assessment needs to correspond to human perception. Pixel-based metrics (PSNR or MSE) fail in many circumstances for not taking into account the spatio-temporal property of human's visual perception. In this paper we propose a new pixel-weighted method to improve video quality metrics for artifacts evaluation. The method applies a psychovisual model based on motion, level of detail, pixel location and the appearance of human faces, which approximate the quality to the human eye's response. Subjective tests were developed to adjust the psychovisual model for demonstrating the noticeable improvement of an algorithm when weighting the pixels according to the factors analyzed instead of treating them equally. The analysis developed demonstrates the necessity of models adapted to the specific visualization of contents and the model presents an advance in quality to be applied over sequences when a determined artifact is analyzed.

Analysis of air flow past and through the 2415-3S airfoil for an unmanned aerial vehicle with internal propulsion system

This paper deals with the prediction of velocity fields on the 2415-3S airfoil which will be used for an unmanned aerial vehicle with internal propulsion system and in this way analyze the air flow through an internal duct of the airfoil using computational fluid dynamics. The main objective is to evaluate the effect of the internal air flow past the airfoil and how this affects the aerodynamic performance by means of lift and drag forces. For this purpose, three different designs of the internal duct were studied; starting from the base 2415-3S airfoil developed in previous investigation, basing on the hypothesis of decreasing the flow separation produced when the propulsive airflow merges the external flow, and in this way obtaining the best configuration. For that purpose, an exhaustive study of the mesh sensitivity was performed. It was used a non-structured mesh since the computational domain is three-dimensional and complex. The selected mesh contains approximately 12.5 million elements. Both the computational domain and the numerical solution were made with commercial CAD and CFD software, respectively. Air, incompressible and steady was analyzed. The boundary conditions are in concordance with experimental setup in the AF 6109 wind tunnel. The k-e model is utilized to describe the turbulent flow process as followed in references. Results allowed obtaining velocity contours as well as lift and drag coefficients and also the location of separation and reattachment regions in some cases for zero degrees of angle of attack on the internal and external surfaces of the airfoil. Finally, the selection of the configuration with the best aerodynamic performance was made, selecting the option without curved baffles.

Numerical analysis of air flow past the 2415-3S airfoil for an unmanned aerial vehicle with internal propulsion system

This paper deals with the prediction of pressure and velocity fields on the 2415-3S airfoil which will be used for and unmanned aerial vehicle with internal propulsion system and in this way analyze the air flow through an internal duct of the airfoil using computational fluid dynamics. The main objective is to evaluate the effect of the internal air flow past the airfoil and how this affects the aerodynamic performance by means of lift and drag forces. For this purpose, three different designs of the internal duct were studied; starting from the base 2415-3S airfoil developed in previous investigation, basing on the hypothesis of decreasing the flow separation produced when the propulsive airflow merges the external flow, and in this way obtaining the best configuration. For that purpose, an exhaustive study of the mesh sensitivity was performed. It was used a non-structured mesh since the computational domain is tridimensional and complex. The selected mesh contains approximately 12.5 million elements. Both the computational domain and the numerical solution were made with commercial CAD and CFD software respectively. Air, incompressible and steady was analyzed. The boundary conditions are in concordance with experimental setup in the AF 6109 wind tunnel. The k-ε model is utilized to describe the turbulent flow process as followed in references. Results allowed obtaining pressure and velocity contours as well as lift and drag coefficients and also the location of separation and reattachment regions in some cases for zero degrees of angle of attack on the internal and external surfaces of the airfoil. Finally, the selection of the configuration with the best aerodynamic performance was made, selecting the option without curved baffles.

Analysis of quality of experience in 3D video systems

Esta tesis presenta un estudio exhaustivo sobre la evaluación de la calidad de experiencia (QoE, del inglés Quality of Experience) percibida por los usuarios de sistemas de vídeo 3D, analizando el impacto de los efectos introducidos por todos los elementos de la cadena de procesamiento de vídeo 3D. Por lo tanto, se presentan varias pruebas de evaluación subjetiva específicamente diseñadas para evaluar los sistemas considerados, teniendo en cuenta todos los factores perceptuales relacionados con la experiencia visual tridimensional, tales como la percepción de profundidad y la molestia visual. Concretamente, se describe un test subjetivo basado en la evaluación de degradaciones típicas que pueden aparecer en el proceso de creación de contenidos de vídeo 3D, por ejemplo debidas a calibraciones incorrectas de las cámaras o a algoritmos de procesamiento de la señal de vídeo (p. ej., conversión de 2D a 3D). Además, se presenta el proceso de generación de una base de datos de vídeos estereoscópicos de alta calidad, disponible gratuitamente para la comunidad investigadora y que ha sido utilizada ampliamente en diferentes trabajos relacionados con vídeo 3D. Asimismo, se presenta otro estudio subjetivo, realizado entre varios laboratorios, con el que se analiza el impacto de degradaciones causadas por la codificación de vídeo, así como diversos formatos de representación de vídeo 3D. Igualmente, se describen tres pruebas subjetivas centradas en el estudio de posibles efectos causados por la transmisión de vídeo 3D a través de redes de televisión sobre IP (IPTV, del inglés Internet Protocol Television) y de sistemas de streaming adaptativo de vídeo. Para estos casos, se ha propuesto una innovadora metodología de evaluación subjetiva de calidad vídeo, denominada Content-Immersive Evaluation of Transmission Impairments (CIETI), diseñada específicamente para evaluar eventos de transmisión simulando condiciones realistas de visualización de vídeo en ámbitos domésticos, con el fin de obtener conclusiones más representativas sobre la experiencia visual de los usuarios finales. Finalmente, se exponen dos experimentos subjetivos comparando varias tecnologías actuales de televisores 3D disponibles en el mercado de consumo y evaluando factores perceptuales de sistemas Super Multiview Video (SMV), previstos a ser la tecnología futura de televisores 3D de consumo, gracias a una prometedora visualización de contenido 3D sin necesidad de gafas específicas. El trabajo presentado en esta tesis ha permitido entender los factores perceptuales y técnicos relacionados con el procesamiento y visualización de contenidos de vídeo 3D, que pueden ser de utilidad en el desarrollo de nuevas tecnologías y técnicas de evaluación de la QoE, tanto metodologías subjetivas como métricas objetivas. ABSTRACT This thesis presents a comprehensive study of the evaluation of the Quality of Experience (QoE) perceived by the users of 3D video systems, analyzing the impact of effects introduced by all the elements of the 3D video processing chain. Therefore, various subjective assessment tests are presented, particularly designed to evaluate the systems under consideration, and taking into account all the perceptual factors related to the 3D visual experience, such as depth perception and visual discomfort. In particular, a subjective test is presented, based on evaluating typical degradations that may appear during the content creation, for instance due to incorrect camera calibration or video processing algorithms (e.g., 2D to 3D conversion). Moreover, the process of generation of a high-quality dataset of 3D stereoscopic videos is described, which is freely available for the research community, and has been already widely used in different works related with 3D video. In addition, another inter-laboratory subjective study is presented analyzing the impact of coding impairments and representation formats of stereoscopic video. Also, three subjective tests are presented studying the effects of transmission events that take place in Internet Protocol Television (IPTV) networks and adaptive streaming scenarios for 3D video. For these cases, a novel subjective evaluation methodology, called Content-Immersive Evaluation of Transmission Impairments (CIETI), was proposed, which was especially designed to evaluate transmission events simulating realistic home-viewing conditions, to obtain more representative conclusions about the visual experience of the end users. Finally, two subjective experiments are exposed comparing various current 3D displays available in the consumer market, and evaluating perceptual factors of Super Multiview Video (SMV) systems, expected to be the future technology for consumer 3D displays thanks to a promising visualization of 3D content without specific glasses. The work presented in this thesis has allowed to understand perceptual and technical factors related to the processing and visualization of 3D video content, which may be useful in the development of new technologies and approaches for QoE evaluation, both subjective methodologies and objective metrics.

Visual attention and perception models for assessing quality in 2D and 3D stereoscopic video

La medida de calidad de vídeo sigue siendo necesaria para definir los criterios que caracterizan una señal que cumpla los requisitos de visionado impuestos por el usuario. Las nuevas tecnologías, como el vídeo 3D estereoscópico o formatos más allá de la alta definición, imponen nuevos criterios que deben ser analizadas para obtener la mayor satisfacción posible del usuario. Entre los problemas detectados durante el desarrollo de esta tesis doctoral se han determinado fenómenos que afectan a distintas fases de la cadena de producción audiovisual y tipo de contenido variado. En primer lugar, el proceso de generación de contenidos debe encontrarse controlado mediante parámetros que eviten que se produzca el disconfort visual y, consecuentemente, fatiga visual, especialmente en lo relativo a contenidos de 3D estereoscópico, tanto de animación como de acción real. Por otro lado, la medida de calidad relativa a la fase de compresión de vídeo emplea métricas que en ocasiones no se encuentran adaptadas a la percepción del usuario. El empleo de modelos psicovisuales y diagramas de atención visual permitirían ponderar las áreas de la imagen de manera que se preste mayor importancia a los píxeles que el usuario enfocará con mayor probabilidad. Estos dos bloques se relacionan a través de la definición del término saliencia. Saliencia es la capacidad del sistema visual para caracterizar una imagen visualizada ponderando las áreas que más atractivas resultan al ojo humano. La saliencia en generación de contenidos estereoscópicos se refiere principalmente a la profundidad simulada mediante la ilusión óptica, medida en términos de distancia del objeto virtual al ojo humano. Sin embargo, en vídeo bidimensional, la saliencia no se basa en la profundidad, sino en otros elementos adicionales, como el movimiento, el nivel de detalle, la posición de los píxeles o la aparición de caras, que serán los factores básicos que compondrán el modelo de atención visual desarrollado. Con el objetivo de detectar las características de una secuencia de vídeo estereoscópico que, con mayor probabilidad, pueden generar disconfort visual, se consultó la extensa literatura relativa a este tema y se realizaron unas pruebas subjetivas preliminares con usuarios. De esta forma, se llegó a la conclusión de que se producía disconfort en los casos en que se producía un cambio abrupto en la distribución de profundidades simuladas de la imagen, aparte de otras degradaciones como la denominada “violación de ventana”. A través de nuevas pruebas subjetivas centradas en analizar estos efectos con diferentes distribuciones de profundidades, se trataron de concretar los parámetros que definían esta imagen. Los resultados de las pruebas demuestran que los cambios abruptos en imágenes se producen en entornos con movimientos y disparidades negativas elevadas que producen interferencias en los procesos de acomodación y vergencia del ojo humano, así como una necesidad en el aumento de los tiempos de enfoque del cristalino. En la mejora de las métricas de calidad a través de modelos que se adaptan al sistema visual humano, se realizaron también pruebas subjetivas que ayudaron a determinar la importancia de cada uno de los factores a la hora de enmascarar una determinada degradación. Los resultados demuestran una ligera mejora en los resultados obtenidos al aplicar máscaras de ponderación y atención visual, los cuales aproximan los parámetros de calidad objetiva a la respuesta del ojo humano. ABSTRACT Video quality assessment is still a necessary tool for defining the criteria to characterize a signal with the viewing requirements imposed by the final user. New technologies, such as 3D stereoscopic video and formats of HD and beyond HD oblige to develop new analysis of video features for obtaining the highest user’s satisfaction. Among the problems detected during the process of this doctoral thesis, it has been determined that some phenomena affect to different phases in the audiovisual production chain, apart from the type of content. On first instance, the generation of contents process should be enough controlled through parameters that avoid the occurrence of visual discomfort in observer’s eye, and consequently, visual fatigue. It is especially necessary controlling sequences of stereoscopic 3D, with both animation and live-action contents. On the other hand, video quality assessment, related to compression processes, should be improved because some objective metrics are adapted to user’s perception. The use of psychovisual models and visual attention diagrams allow the weighting of image regions of interest, giving more importance to the areas which the user will focus most probably. These two work fields are related together through the definition of the term saliency. Saliency is the capacity of human visual system for characterizing an image, highlighting the areas which result more attractive to the human eye. Saliency in generation of 3DTV contents refers mainly to the simulated depth of the optic illusion, i.e. the distance from the virtual object to the human eye. On the other hand, saliency is not based on virtual depth, but on other features, such as motion, level of detail, position of pixels in the frame or face detection, which are the basic features that are part of the developed visual attention model, as demonstrated with tests. Extensive literature involving visual comfort assessment was looked up, and the development of new preliminary subjective assessment with users was performed, in order to detect the features that increase the probability of discomfort to occur. With this methodology, the conclusions drawn confirmed that one common source of visual discomfort was when an abrupt change of disparity happened in video transitions, apart from other degradations, such as window violation. New quality assessment was performed to quantify the distribution of disparities over different sequences. The results confirmed that abrupt changes in negative parallax environment produce accommodation-vergence mismatches derived from the increasing time for human crystalline to focus the virtual objects. On the other side, for developing metrics that adapt to human visual system, additional subjective tests were developed to determine the importance of each factor, which masks a concrete distortion. Results demonstrated slight improvement after applying visual attention to objective metrics. This process of weighing pixels approximates the quality results to human eye’s response.

Estudio, ensamblaje, caracterización y ensayos de dos modelos reales de RPA

Un dron o un RPA (del inglés, Remote Piloted Aircraft) es un vehículo aéreo no tripulado capaz de despegar, volar y aterrizar de forma autónoma, semiautónoma o manual, siempre con control remoto. Además, toda aeronave de estas características debe ser capaz de mantener un nivel de vuelo controlado y sostenido. A lo largo de los años, estos aparatos han ido evolución tanto en aplicaciones como en su estética y características físicas, siempre impulsado por los requerimientos militares en cada momento. Gracias a este desarrollo, hoy en día los drones son uno más en la sociedad y desempeñan tareas que para cualquier ser humano serían peligrosas o difíciles de llevar a cabo. Debido a la reciente proliferación de los RPA, los gobiernos de los distintos países se han visto obligados a redactar nuevas leyes y/o modificar las ya existentes en relación a los diferentes usos del espacio aéreo para promover la convivencia de estas aeronaves con el resto de vehículos aéreos. El objeto principal de este proyecto es ensamblar, caracterizar y configurar dos modelos reales de dron: el DJI F450 y el TAROT t810. Para conseguir un montaje apropiado a las aplicaciones posteriores que se les va a dar, antes de su construcción se ha realizado un estudio individualizado en detalle de cada una de las partes y módulos que componen estos vehículos. Adicionalmente, se ha investigado acerca de los distintos tipos de sistemas de transmisión de control remoto, vídeo y telemetría, sin dejar de lado las baterías que impulsarán al aparato durante sus vuelos. De este modo, es sabido que los RPA están compuestos por distintos módulos operativos: los principales, todo aquel módulo para que el aparato pueda volar, y los complementarios, que son aquellos que dotan a cada aeronave de características adicionales y personalizadas que lo hacen apto para diferentes usos. A lo largo de este proyecto se han instalado y probado diferentes módulos adicionales en cada uno de los drones, además de estar ambos constituidos por distintos bloques principales, incluyendo el controlador principal: NAZA-M Lite instalado en el dron DJI F450 y NAZA-M V2 incorporado en el TAROT t810. De esta forma se ha podido establecer una comparativa real acerca del comportamiento de éstos, tanto de forma conjunta como de ambos controladores individualmente. Tras la evaluación experimental obtenida tras diversas pruebas de vuelo, se puede concluir que ambos modelos de controladores se ajustan a las necesidades demandadas por el proyecto y sus futuras aplicaciones, siendo más apropiada la elección del modelo M Lite por motivos estrictamente económicos, ya que su comportamiento en entornos recreativos es similar al modelo M V2. ABSTRACT. A drone or RPA (Remote Piloted Aircraft) is an unmanned aerial vehicle that is able to take off, to fly and to land autonomously, semi-autonomously or manually, always connected via remote control. In addition, these aircrafts must be able to keep a controlled and sustained flight level. Over the years, the applications for these devices have evolved as much as their aesthetics and physical features both boosted by the military needs along time. Thanks to this development, nowadays drones are part of our society, executing tasks potentially dangerous or difficult to complete by humans. Due to the recent proliferation of RPA, governments worldwide have been forced to draft legislation and/or modify the existing ones about the different uses of the aerial space to promote the cohabitation of these aircrafts with the rest of the aerial vehicles. The main objective of this project is to assemble, to characterize and to set-up two real drone models: DJI F450 and TAROT t810. Before constructing the vehicles, a detailed study of each part and module that composes them has been carried out, in order to get an appropriate structure for their expected uses. Additionally, the different kinds of remote control, video and telemetry transmission systems have been investigated, including the batteries that will power the aircrafts during their flights. RPA are made of several operative modules: main modules, i.e. those which make the aircraft fly, and complementary modules, that customize each aircraft and equip them with additional features, making them suitable for a particular use. Along this project, several complementary modules for each drone have been installed and tested. Furthermore, both are built from different main units, including the main controller: NAZA-M Lite installed on DJI F450 and NAZA-M V2 on board of TAROT t810. This way, it has been possible to establish an accurate comparison, related to the performance of both models, not only jointly but individually as well. After several flight tests and an experimental evaluation, it can be concluded that both main controller models are suitable for the requirements fixed for the project and the future applications, being more appropriate to choose the M Lite model strictly due to economic reasons, as its performance in recreational environment is similar to the M V2.

Quality of experience of adaptive video streaming: investigation in service parameters and subjective quality assessment methodology

The usage of HTTP adaptive streaming (HAS) has become widely spread in multimedia services. Because it allows the service providers to improve the network resource utilization and user׳s Quality of Experience (QoE). Using this technology, the video playback interruption is reduced since the network and server status in addition to capability of user device, all are taken into account by HAS client to adapt the quality to the current condition. Adaptation can be done using different strategies. In order to provide optimal QoE, the perceptual impact of adaptation strategies from point of view of the user should be studied. However, the time-varying video quality due to the adaptation which usually takes place in a long interval introduces a new type of impairment making the subjective evaluation of adaptive streaming system challenging. The contribution of this paper is two-fold: first, it investigates the testing methodology to evaluate HAS QoE by comparing the subjective experimental outcomes obtained from ACR standardized method and a semi-continuous method developed to evaluate the long sequences. In addition, influence of using audiovisual stimuli to evaluate the video-related impairment is inquired. Second, impact of some of the adaptation technical factors including the quality switching amplitude and chunk size in combination with high range of commercial content type is investigated. The results of this study provide a good insight toward achieving appropriate testing method to evaluate HAS QoE, in addition to designing switching strategies with optimal visual quality.

Time-Lapse Video Microscopy of Gliding Motility in Toxoplasma gondii Reveals a Novel, Biphasic Mechanism of Cell LocomotionV⃞

Toxoplasma gondii is a member of the phylum Apicomplexa, a diverse group of intracellular parasites that share a unique form of gliding motility. Gliding is substrate dependent and occurs without apparent changes in cell shape and in the absence of traditional locomotory organelles. Here, we demonstrate that gliding is characterized by three distinct forms of motility: circular gliding, upright twirling, and helical rotation. Circular gliding commences while the crescent-shaped parasite lies on its right side, from where it moves in a counterclockwise manner at a rate of ∼1.5 μm/s. Twirling occurs when the parasite rights itself vertically, remaining attached to the substrate by its posterior end and spinning clockwise. Helical gliding is similar to twirling except that it occurs while the parasite is positioned horizontally, resulting in forward movement that follows the path of a corkscrew. The parasite begins lying on its left side (where the convex side is defined as dorsal) and initiates a clockwise revolution along the long axis of the crescent-shaped body. Time-lapse video analyses indicated that helical gliding is a biphasic process. During the first 180o of the turn, the parasite moves forward one body length at a rate of ∼1–3 μm/s. In the second phase, the parasite flips onto its left side, in the process undergoing little net forward motion. All three forms of motility were disrupted by inhibitors of actin filaments (cytochalasin D) and myosin ATPase (butanedione monoxime), indicating that they rely on an actinomyosin motor in the parasite. Gliding motility likely provides the force for active penetration of the host cell and may participate in dissemination within the host and thus is of both fundamental and practical interest.