Subjective quality assessment of an adaptive video streaming model

With the recent increased popularity and high usage of HTTP Adaptive Streaming (HAS) techniques, various studies have been carried out in this area which generally focused on the technical enhancement of HAS technology and applications. However, a lack of common HAS standard led to multiple proprietary approaches which have been developed by major Internet companies. In the emerging MPEG-DASH standard the packagings of the video content and HTTP syntax have been standardized; but all the details of the adaptation behavior are left to the client implementation. Nevertheless, to design an adaptation algorithm which optimizes the viewing experience of the enduser, the multimedia service providers need to know about the Quality of Experience (QoE) of different adaptation schemes. Taking this into account, the objective of this experiment was to study the QoE of a HAS-based video broadcast model. The experiment has been carried out through a subjective study of the end user response to various possible clients’ behavior for changing the video quality taking different QoE-influence factors into account. The experimental conclusions have made a good insight into the QoE of different adaptation schemes which can be exploited by HAS clients for designing the adaptation algorithms.

Perceptual distortion modeling for side-by-side 3D video delivery

A frame-level distortion model based on perceptual features of the human visual system is proposed to improve the performance of unequal error protection strategies and provide better quality of experience to users in Side-by-Side 3D video delivery systems.

Fast 2D to 3D conversion using a clustering-based hierarchical search in a machine learning framework

Automatic 2D-to-3D conversion is an important application for filling the gap between the increasing number of 3D displays and the still scant 3D content. However, existing approaches have an excessive computational cost that complicates its practical application. In this paper, a fast automatic 2D-to-3D conversion technique is proposed, which uses a machine learning framework to infer the 3D structure of a query color image from a training database with color and depth images. Assuming that photometrically similar images have analogous 3D structures, a depth map is estimated by searching the most similar color images in the database, and fusing the corresponding depth maps. Large databases are desirable to achieve better results, but the computational cost also increases. A clustering-based hierarchical search using compact SURF descriptors to characterize images is proposed to drastically reduce search times. A significant computational time improvement has been obtained regarding other state-of-the-art approaches, maintaining the quality results.

Hammer: an Android based application for end-user industrial robot programming

This paper presents a novel tablet based end-user interface for industrial robot programming (called Hammer). This application makes easier to program tasks for industrial robots like polishing, milling or grinding. It is based on the Scratch programming language, but specifically design and created for Android OS. It is a visual programming concept that allows non-skilled programmer operators to create programs. The application also allows to monitor the tasks while it is being executed by overlapping real time information through augmented reality. The application includes a teach pendant screen that can be customized according to the operator needs at every moment.

Hardware timestamping for image acquisition system based on FlexRIO and IEEE 1588 v2 Standard

Current fusion devices consist of multiple diagnostics and hundreds or even thousands of signals. This situation forces on multiple occasions to use distributed data acquisition systems as the best approach. In this type of distributed systems, one of the most important issues is the synchronization between signals, so that it is possible to have a temporal correlation as accurate as possible between the acquired samples of all channels. In last decades, many fusion devices use different types of video cameras to provide inside views of the vessel during operations and to monitor plasma behavior. The synchronization between each video frame and the rest of the different signals acquired from any other diagnostics is essential in order to know correctly the plasma evolution, since it is possible to analyze jointly all the information having accurate knowledge of their temporal correlation. The developed system described in this paper allows timestamping image frames in a real-time acquisition and processing system using 1588 clock distribution. The system has been implemented using FPGA based devices together with a 1588 synchronized timing card (see Fig.1). The solution is based on a previous system [1] that allows image acquisition and real-time image processing based on PXIe technology. This architecture is fully compatible with the ITER Fast Controllers [2] and offers integration with EPICS to control and monitor the entire system. However, this set-up is not able to timestamp the frames acquired since the frame grabber module does not present any type of timing input (IRIG-B, GPS, PTP). To solve this lack, an IEEE1588 PXI timing device its used to provide an accurate way to synchronize distributed data acquisition systems using the Precision Time Protocol (PTP) IEEE 1588 2008 standard. This local timing device can be connected to a master clock device for global synchronization. The timing device has a buffer timestamp for each PXI trigger line and requires tha- a software application assigns each frame the corresponding timestamp. The previous action is critical and cannot be achieved if the frame rate is high. To solve this problem, it has been designed a solution that distributes the clock from the IEEE 1588 timing card to all FlexRIO devices [3]. This solution uses two PXI trigger lines that provide the capacity to assign timestamps to every frame acquired and register events by hardware in a deterministic way. The system provides a solution for timestamping frames to synchronize them with the rest of the different signals.

3D simulation of complex shading affecting PV systems taking benefit from the power of graphics cards developed for the video game industry

Shading reduces the power output of a photovoltaic (PV) system. The design engineering of PV systems requires modeling and evaluating shading losses. Some PV systems are affected by complex shading scenes whose resulting PV energy losses are very difficult to evaluate with current modeling tools. Several specialized PV design and simulation software include the possibility to evaluate shading losses. They generally possess a Graphical User Interface (GUI) through which the user can draw a 3D shading scene, and then evaluate its corresponding PV energy losses. The complexity of the objects that these tools can handle is relatively limited. We have created a software solution, 3DPV, which allows evaluating the energy losses induced by complex 3D scenes on PV generators. The 3D objects can be imported from specialized 3D modeling software or from a 3D object library. The shadows cast by this 3D scene on the PV generator are then directly evaluated from the Graphics Processing Unit (GPU). Thanks to the recent development of GPUs for the video game industry, the shadows can be evaluated with a very high spatial resolution that reaches well beyond the PV cell level, in very short calculation times. A PV simulation model then translates the geometrical shading into PV energy output losses. 3DPV has been implemented using WebGL, which allows it to run directly from a Web browser, without requiring any local installation from the user. This also allows taken full benefits from the information already available from Internet, such as the 3D object libraries. This contribution describes, step by step, the method that allows 3DPV to evaluate the PV energy losses caused by complex shading. We then illustrate the results of this methodology to several application cases that are encountered in the world of PV systems design. Keywords: 3D, modeling, simulation, GPU, shading, losses, shadow mapping, solar, photovoltaic, PV, WebGL

Methodology based on FBG sensors for monitoring of FRP-strengthened beams

Advanced composite materials are increasingly used in the strengthening of reinforced concrete (RC) structures. The use of externally bonded strips made of fibre-reinforced plastics (FRP) as strengthening method has gained widespread acceptance in recent years since it has many advantages over the traditional techniques. However, unfortunately, this strengthening method is often associated with a brittle and sudden failure caused by some form of FRP bond failure, originated at the termination of the FRP material or at intermediate areas in the vicinity of flexural cracks in the RC beam. Up to date, little effort in the early prediction of the debonding in its initial instants even though this effect is not noticeable by simple visual observation. An early detection of this phenomenon might help in taking actions to prevent future catastrophes. Fibre-optic Bragg grating (FBG) sensors are able to measure strains locally with high resolution and accuracy. Furthermore, as their physical size is extremely small compared with other strain measuring components, it enables to be embedded at the concrete-FRP interface for determining the strain distribution without influencing the mechanical properties of the host materials. This paper shows the development of a debonding identification methodology based on strains experimentally measured. For, it a simplified model is implemented to simulate the behaviour of FRP-strengthened reinforced concrete beams. This model is taken as a basis to. develop an model updating procedure able to detect minor debonding at the concrete-FRP interface from experimental strains obtained by using FBG sensors embedded at the interface

A simplified spectral approachfor impedance-based damage identification of frp-strengthened rc beams

Hoy en día, el refuerzo y reparación de estructuras de hormigón armado mediante el pegado de bandas de polímeros reforzados con fibras (FRP) se emplea cada vez con más frecuencia a causa de sus numerosas ventajas. Sin embargo, las vigas reforzadas con esta técnica pueden experimentar un modo de fallo frágil a causa del despegue repentino de la banda de FRP a partir de una fisura intermedia. A pesar de su importancia, el número de trabajos que abordan el estudio de este mecanismo de fallo y su monitorización es muy limitado. Por ello, el desarrollo de metodologías capaces de monitorizar a largo plazo la adherencia de este refuerzo a las estructuras de hormigón e identificar cuándo se inicia el despegue de la banda constituyen un importante desafío a abordar. El principal objetivo de esta tesis es la implementación de una metodología fiable y efectiva, capaz de detectar el despegue de una banda de FRP en una viga de hormigón armado a partir de una fisura intermedia. Para alcanzar este objetivo se ha implementado un procedimiento de calibración numérica a partir de ensayos experimentales. Para ello, en primer lugar, se ha desarrollado un modelo numérico unidimensional simple y no costoso representativo del comportamiento de este tipo vigas de hormigón reforzadas con FRP, basado en un modelo de fisura discreta para el hormigón y el método de elementos espectrales. La formación progresiva de fisuras a flexion y el consiguiente despegue en la interface entre el hormigón y el FRP se formulan mediante la introducción de un nuevo elemento capaz de representar ambos fenómenos simultáneamente sin afectar al procedimiento numérico. Además, con el modelo propuesto, se puede obtener de una forma sencilla la respuesta dinámica en altas frecuencias de este tipo de estructuras, lo cual puede hacer muy útil su uso como herramienta de diagnosis y detección del despegue en su fase inicial mediante una monitorización de la variación de las características dinámicas locales de la estructura. Un método de evaluación no destructivo muy prometedor para la monitorización local de las estructuras es el método de la impedancia usando sensores-actuadores piezoeléctricos (PZT). La impedancia eléctrica de los sensores PZT se puede relacionar con la impedancia mecánica de las estructuras donde se encuentran adheridos Ya que la impedancia mecánica de una estructura se verá afectada por su deterioro, se pueden implementar indicadores de daño mediante una comparación del espectro de admitancia (inversa de la impedancia) a lo largo de distintas etapas durante el periodo de servicio de una estructura. Cualquier cambio en el espectro se podría interpretar como una variación en la integridad de la estructura. La impedancia eléctrica se mide a altas frecuencias con lo cual esta metodología debería ser muy sensible a la detección de estados de daño incipiente local, tal como se desea en la aplicación de este trabajo. Se ha implementado un elemento espectral PZT-FRP como extensión del modelo previamente desarrollado, con el objetivo de poder calcular numéricamente la impedancia eléctrica de sensores PZT adheridos a bandas de FRP sobre una viga de hormigón armado. El modelo, combinado con medidas experimentales captadas mediante sensores PZT, se implementa en el marco de una metodología de calibración de modelos para detectar cuantitativamente el despegue en la interfase entre una banda de FRP y una viga de hormigón. El procedimiento de optimización se resuelve empleando el método del enjambre cooperativo con un algoritmo bagging. Los resultados muestran una gran aproximación en la estimación del daño para el problema propuesto. Adicionalmente, se ha desarrollado también un método adaptativo para el mallado de elementos espectrales con el objetivo de localizar las zonas dañadas a partir de los resultados experimentales, el cual contribuye a aumentar la robustez y efectividad del método propuesto a la hora de identificar daños incipientes en su aparición inicial. Finalmente, se ha llevado a cabo un procedimiento de optimización multi-objetivo para detectar el despegue inicial en una viga de hormigón a escala real reforzada con FRP a partir de las impedancias captadas con una red de sensores PZT instrumentada a lo largo de la longitud de la viga. Cada sensor aporta los datos para definir cada una de las funciones objetivo que definen el procedimiento. Combinando el modelo previo de elementos espectrales con un algoritmo PSO multi-objetivo el procedimiento de detección de daño resultante proporciona resultados satisfactorios considerando la escala de la estructura y todas las incertidumbres características ligadas a este proceso. Los resultados obtenidos prueban la viabilidad y capacidad de los métodos antes mencionados y también su potencial en aplicaciones reales. Abstract Nowadays, the external bonding of fibre reinforced polymer (FRP) plates or sheets is increasingly used for the strengthening and retrofitting of reinforced concrete (RC) structures due to its numerous advantages. However, this kind of strengthening often leads to brittle failure modes being the most dominant failure mode the debonding induced by an intermediate crack (IC). In spite of its importance, the number of studies regarding the IC debonding mechanism and bond health monitoring is very limited. Methodologies able to monitor the long-term efficiency of bonding and successfully identify the initiation of FRP debonding constitute a challenge to be met. The main purpose of this thesisis the implementation of a reliable and effective methodology of damage identification able to detect intermediate crack debonding in FRP-strengthened RC beams. To achieve this goal, a model updating procedure based on numerical simulations and experimental tests has been implemented. For it, firstly, a simple and non-expensive one-dimensional model based on the discrete crack approach for concrete and the spectral element method has been developed. The progressive formation of flexural cracks and subsequent concrete-FRP interfacial debonding is formulated by the introduction of a new element able to represent both phenomena simultaneously without perturbing the numerical procedure. Furthermore, with the proposed model, high frequency dynamic response for these kinds of structures can also be obtained in a very simple and non-expensive way, which makes this procedure very useful as a tool for diagnoses and detection of debonding in its initial stage by monitoring the change in local dynamic characteristics. One very promising active non-destructive evaluation method for local monitoring is impedance-based structural health monitoring(SHM)using piezoelectric ceramic (PZT) sensor-actuators. The electrical impedance of the PZT can be directly related to the mechanical impedance of the host structural component where the PZT transducers are attached. Since the structural mechanical impedance will be affected by the presence of structural damage, comparisons of admittance (inverse of impedance) spectra at various times during the service period of the structure can be used as damage indicator. Any change in the spectra might be an indication of a change in the structural integrity. The electrical impedance is measured at high frequencies with which this methodology appears to be very sensitive to incipient damage in structural systems as desired for our application. Abonded-PZT-FRP spectral beam element approach based on an extension of the previous discrete crack approach is implemented in the calculation of the electrical impedance of the PZT transducer bonded to the FRP plates of a RC beam. This approach in conjunction with the experimental measurements of PZT actuator-sensors mounted on the structure is used to present an updating methodology to quantitatively detect interfacial debonding between a FRP strip and the host RC structure. The updating procedure is solved by using an ensemble particle swarm optimization approach with abagging algorithm, and the results demonstrate a big improvement for the performance and accuracy of the damage detection in the proposed problem. Additionally, an adaptive strategy of spectral element mesh has been also developed to detect damage location with experimental results, which shows the robustness and effectiveness of the proposed method to identify initial and incipient damages at its early stage. Lastly, multi-objective optimization has been carried out to detect debonding damage in a real scale FRP-strengthened RC beam by using impedance signatures. A net of PZT sensors is distributed along the beam to construct impedance-based multiple objectives under gradually induced damage scenario. By combining the spectral element model presented previously and an ensemble multi-objective PSO algorithm, the implemented damage detection process yields satisfactory predictions considering the scale and uncertainties of the structure. The obtained results prove the feasibility and capability of the aforementioned methods and also their potentials in real engineering applications.

Foreground segmentation in depth imagery using depth and spatial dynamic models for video surveillance applications

Low-cost systems that can obtain a high-quality foreground segmentation almostindependently of the existing illumination conditions for indoor environments are verydesirable, especially for security and surveillance applications. In this paper, a novelforeground segmentation algorithm that uses only a Kinect depth sensor is proposedto satisfy the aforementioned system characteristics. This is achieved by combininga mixture of Gaussians-based background subtraction algorithm with a new Bayesiannetwork that robustly predicts the foreground/background regions between consecutivetime steps. The Bayesian network explicitly exploits the intrinsic characteristics ofthe depth data by means of two dynamic models that estimate the spatial and depthevolution of the foreground/background regions. The most remarkable contribution is thedepth-based dynamic model that predicts the changes in the foreground depth distributionbetween consecutive time steps. This is a key difference with regard to visible imagery,where the color/gray distribution of the foreground is typically assumed to be constant.Experiments carried out on two different depth-based databases demonstrate that theproposed combination of algorithms is able to obtain a more accurate segmentation of theforeground/background than other state-of-the art approaches.

A study of feature combination for vehicle detection based on image processing

Video analytics play a critical role in most recent traffic monitoring and driver assistance systems. In this context, the correct detection and classification of surrounding vehicles through image analysis has been the focus of extensive research in the last years. Most of the pieces of work reported for image-based vehicle verification make use of supervised classification approaches and resort to techniques, such as histograms of oriented gradients (HOG), principal component analysis (PCA), and Gabor filters, among others. Unfortunately, existing approaches are lacking in two respects: first, comparison between methods using a common body of work has not been addressed; second, no study of the combination potentiality of popular features for vehicle classification has been reported. In this study the performance of the different techniques is first reviewed and compared using a common public database. Then, the combination capabilities of these techniques are explored and a methodology is presented for the fusion of classifiers built upon them, taking into account also the vehicle pose. The study unveils the limitations of single-feature based classification and makes clear that fusion of classifiers is highly beneficial for vehicle verification.

Online Learning-based Robust Visual Tracking for Autonomous Landing of Unmanned Aerial Vehicles

Autonomous landing is a challenging and important technology for both military and civilian applications of Unmanned Aerial Vehicles (UAVs). In this paper, we present a novel online adaptive visual tracking algorithm for UAVs to land on an arbitrary field (that can be used as the helipad) autonomously at real-time frame rates of more than twenty frames per second. The integration of low-dimensional subspace representation method, online incremental learning approach and hierarchical tracking strategy allows the autolanding task to overcome the problems generated by the challenging situations such as significant appearance change, variant surrounding illumination, partial helipad occlusion, rapid pose variation, onboard mechanical vibration (no video stabilization), low computational capacity and delayed information communication between UAV and Ground Control Station (GCS). The tracking performance of this presented algorithm is evaluated with aerial images from real autolanding flights using manually- labelled ground truth database. The evaluation results show that this new algorithm is highly robust to track the helipad and accurate enough for closing the vision-based control loop.

Online Learning-based Robust Visual Tracking for Autonomous Landing of Unmanned Aerial Vehicles

Autonomous landing is a challenging and important technology for both military and civilian applications of Unmanned Aerial Vehicles (UAVs). In this paper, we present a novel online adaptive visual tracking algorithm for UAVs to land on an arbitrary field (that can be used as the helipad) autonomously at real-time frame rates of more than twenty frames per second. The integration of low-dimensional subspace representation method, online incremental learning approach and hierarchical tracking strategy allows the autolanding task to overcome the problems generated by the challenging situations such as significant appearance change, variant surrounding illumination, partial helipad occlusion, rapid pose variation, onboard mechanical vibration (no video stabilization), low computational capacity and delayed information communication between UAV and Ground Control Station (GCS). The tracking performance of this presented algorithm is evaluated with aerial images from real autolanding flights using manually- labelled ground truth database. The evaluation results show that this new algorithm is highly robust to track the helipad and accurate enough for closing the vision-based control loop.

Use of Biological Motion based Cues and Ecological Sounds in the Neurorehabilitation of Apraxia

Technological progress in the area of informatics and human interface platforms create a window of opportunities for the neurorehablitation of patients with motor impairments. The CogWatch project (www.cogwatch.eu) aims to create an intelligent assistance system to improve motor planning and execution in patients with apraxia during their daily activities. Due to the brain damage caused by cardiovascular incident these patients suffer from impairments in the ability to use tools, and to sequence actions during daily tasks (such as making breakfast). Based on the common coding theory (Hommel et al., 2001) and mirror neuron primate research (Rizzolatti et al., 2001) we aim to explore use of cues, which incorporate aspects of biological motion from healthy adults performing everyday tasks requiring tool use and ecological sounds linked to the action goal. We hypothesize that patients with apraxia will benefit from supplementary sensory information relevant to the task, which will reinforce the selection of the appropriate motor plan. Findings from this study determine the type of sensory guidance in the CogWatch interface. Rationale for the experimental design is presented and the relevant literature is discussed.

Definition of masks related to psychovisual features for video quality assessment

Video Quality Assessment needs to correspond to human perception. Pixel-based metrics (PSNR or MSE) fail in many circumstances for not taking into account the spatio-temporal property of human's visual perception. In this paper we propose a new pixel-weighted method to improve video quality metrics for artifacts evaluation. The method applies a psychovisual model based on motion, level of detail, pixel location and the appearance of human faces, which approximate the quality to the human eye's response. Subjective tests were developed to adjust the psychovisual model for demonstrating the noticeable improvement of an algorithm when weighting the pixels according to the factors analyzed instead of treating them equally. The analysis developed demonstrates the necessity of models adapted to the specific visualization of contents and the model presents an advance in quality to be applied over sequences when a determined artifact is analyzed.

Analysis of quality of experience in 3D video systems

Esta tesis presenta un estudio exhaustivo sobre la evaluación de la calidad de experiencia (QoE, del inglés Quality of Experience) percibida por los usuarios de sistemas de vídeo 3D, analizando el impacto de los efectos introducidos por todos los elementos de la cadena de procesamiento de vídeo 3D. Por lo tanto, se presentan varias pruebas de evaluación subjetiva específicamente diseñadas para evaluar los sistemas considerados, teniendo en cuenta todos los factores perceptuales relacionados con la experiencia visual tridimensional, tales como la percepción de profundidad y la molestia visual. Concretamente, se describe un test subjetivo basado en la evaluación de degradaciones típicas que pueden aparecer en el proceso de creación de contenidos de vídeo 3D, por ejemplo debidas a calibraciones incorrectas de las cámaras o a algoritmos de procesamiento de la señal de vídeo (p. ej., conversión de 2D a 3D). Además, se presenta el proceso de generación de una base de datos de vídeos estereoscópicos de alta calidad, disponible gratuitamente para la comunidad investigadora y que ha sido utilizada ampliamente en diferentes trabajos relacionados con vídeo 3D. Asimismo, se presenta otro estudio subjetivo, realizado entre varios laboratorios, con el que se analiza el impacto de degradaciones causadas por la codificación de vídeo, así como diversos formatos de representación de vídeo 3D. Igualmente, se describen tres pruebas subjetivas centradas en el estudio de posibles efectos causados por la transmisión de vídeo 3D a través de redes de televisión sobre IP (IPTV, del inglés Internet Protocol Television) y de sistemas de streaming adaptativo de vídeo. Para estos casos, se ha propuesto una innovadora metodología de evaluación subjetiva de calidad vídeo, denominada Content-Immersive Evaluation of Transmission Impairments (CIETI), diseñada específicamente para evaluar eventos de transmisión simulando condiciones realistas de visualización de vídeo en ámbitos domésticos, con el fin de obtener conclusiones más representativas sobre la experiencia visual de los usuarios finales. Finalmente, se exponen dos experimentos subjetivos comparando varias tecnologías actuales de televisores 3D disponibles en el mercado de consumo y evaluando factores perceptuales de sistemas Super Multiview Video (SMV), previstos a ser la tecnología futura de televisores 3D de consumo, gracias a una prometedora visualización de contenido 3D sin necesidad de gafas específicas. El trabajo presentado en esta tesis ha permitido entender los factores perceptuales y técnicos relacionados con el procesamiento y visualización de contenidos de vídeo 3D, que pueden ser de utilidad en el desarrollo de nuevas tecnologías y técnicas de evaluación de la QoE, tanto metodologías subjetivas como métricas objetivas. ABSTRACT This thesis presents a comprehensive study of the evaluation of the Quality of Experience (QoE) perceived by the users of 3D video systems, analyzing the impact of effects introduced by all the elements of the 3D video processing chain. Therefore, various subjective assessment tests are presented, particularly designed to evaluate the systems under consideration, and taking into account all the perceptual factors related to the 3D visual experience, such as depth perception and visual discomfort. In particular, a subjective test is presented, based on evaluating typical degradations that may appear during the content creation, for instance due to incorrect camera calibration or video processing algorithms (e.g., 2D to 3D conversion). Moreover, the process of generation of a high-quality dataset of 3D stereoscopic videos is described, which is freely available for the research community, and has been already widely used in different works related with 3D video. In addition, another inter-laboratory subjective study is presented analyzing the impact of coding impairments and representation formats of stereoscopic video. Also, three subjective tests are presented studying the effects of transmission events that take place in Internet Protocol Television (IPTV) networks and adaptive streaming scenarios for 3D video. For these cases, a novel subjective evaluation methodology, called Content-Immersive Evaluation of Transmission Impairments (CIETI), was proposed, which was especially designed to evaluate transmission events simulating realistic home-viewing conditions, to obtain more representative conclusions about the visual experience of the end users. Finally, two subjective experiments are exposed comparing various current 3D displays available in the consumer market, and evaluating perceptual factors of Super Multiview Video (SMV) systems, expected to be the future technology for consumer 3D displays thanks to a promising visualization of 3D content without specific glasses. The work presented in this thesis has allowed to understand perceptual and technical factors related to the processing and visualization of 3D video content, which may be useful in the development of new technologies and approaches for QoE evaluation, both subjective methodologies and objective metrics.