Temporal segmentation tool for high-quality real time video editing software

The increasing use of video editing software requires faster and more efficient editing tools. As a first step, these tools perform a temporal segmentation in shots that allows a later building of indexes describing the video content. Here, we propose a novel real-time high-quality shot detection strategy, suitable for the last generation of video editing software requiring both low computational cost and high quality results. While abrupt transitions are detected through a very fast pixel-based analysis, gradual transitions are obtained from an efficient edge-based analysis. Both analyses are reinforced with a motion analysis that helps to detect and discard false detections. This motion analysis is carried out exclusively over a reduced set of candidate transitions, thus maintaining the computational requirements demanded by new applications to fulfill user needs.

Application of NMR techniques for the evaluation of fruit internal quality and physical properties of food

Los alimentos son sistemas complejos, formados por diversas estructuras a diferentes escalas: macroscópica y microscópica. Muchas propiedades de los alimentos, que son importantes para su procesamiento, calidad y tratamiento postcosecha, están relacionados con su microestructura. La presente tesis doctoral propone una metodología completa para la determinación de la estructura de alimentos desde un punto de vista multi-escala, basándose en métodos de Resonancia Magnética Nuclear (NMR). Las técnicas de NMR son no invasivas y no destructivas y permiten el estudio tanto de macro- como de microestructura. Se han utilizado distintos procedimientos de NMR dependiendo del nivel que se desea estudiar. Para el nivel macroestructural, la Imagen de Resonancia Magnética (MRI) ha resultado ser muy útil para la caracterización de alimentos. Para el estudio microestructural, la MRI requiere altos tiempos de adquisición, lo que hace muy difícil la transferencia de esta técnica a aplicaciones en industria. Por tanto, la optimización de procedimientos de NMR basados en secuencias relaxometría 2D T1/T2 ha resultado ser una estrategia primordial en esta tesis. Estos protocolos de NMR se han implementado satisfactoriamente por primera vez en alto campo magnético. Se ha caracterizado la microestructura de productos alimentarios enteros por primera vez utilizando este tipo de protocolos. Como muestras, se han utilizado dos tipos de productos: modelos de alimentos y alimentos reales (manzanas). Además, como primer paso para su posterior implementación en la industria agroalimentaria, se ha mejorado una línea transportadora, especialmente diseñada para trabajar bajo condiciones de NMR en trabajos anteriores del grupo LPF-TAGRALIA. Se han estudiado y seleccionado las secuencias más rápidas y óptimas para la detección de dos tipos de desórdenes internos en manzanas: vitrescencia y roturas internas. La corrección de las imágenes en movimiento se realiza en tiempo real. Asimismo, se han utilizado protocolos de visión artificial para la clasificación automática de manzanas potencialmente afectadas por vitrescencia. El presente documento está dividido en diferentes capítulos: el Capítulo 2 explica los antecedentes de la presente tesis y el marco del proyecto en el que se ha desarrollado. El Capítulo 3 recoge el estado del arte. El Capítulo 4 establece los objetivos de esta tesis doctoral. Los resultados se dividen en cinco sub-secciones (dentro del Capítulo 5) que corresponden con los trabajos publicados bien en revistas revisadas por pares, bien en congresos internacionales o bien como capítulos de libros revisados por pares. La Sección 5.1. es un estudio del desarrollo de la vitrescencia en manzanas mediante MRI y lo relaciona con la posición de la fruta dentro de la copa del árbol. La Sección 5.2 presenta un trabajo sobre macro- y microestructura en modelos de alimentos. La Sección 5.3 es un artículo en revisión en una revista revisada por pares, en el que se hace un estudio microestrcutural no destructivo mediante relaxometría 2D T1/T2. la Sección 5.4, hace una comparación entre manzanas afectadas por vitrescencia mediante dos técnicas: tomografía de rayos X e MRI, en manzana. Por último, en la Sección 5.5 se muestra un trabajo en el que se hace un estudio de secuencias de MRI en línea para la evaluación de calidad interna en manzanas. Los siguientes capítulos ofrecen una discusión y conclusiones (Capítulo 6 y 7 respectivamente) de todos los capítulos de esta tesis doctoral. Finalmente, se han añadido tres apéndices: el primero con una introducción de los principios básicos de resonancia magnética nuclear (NMR) y en los otros dos, se presentan sendos estudios sobre el efecto de las fibras en la rehidratación de cereales de desayuno extrusionados, mediante diversas técnicas. Ambos trabajos se presentaron en un congreso internacional. Los resultados más relevantes de la presente tesis doctoral, se pueden dividir en tres grandes bloques: resultados sobre macroestructura, resultados sobre microestructura y resultados sobre MRI en línea. Resultados sobre macroestructura: - La imagen de resonancia magnética (MRI) se aplicó satisfactoriamente para la caracterización de macroestructura. En particular, la reconstrucción 3D de imágenes de resonancia magnética permitió identificar y caracterizar dos tipos distintos de vitrescencia en manzanas: central y radial, que se caracterizan por el porcentaje de daño y la conectividad (número de Euler). - La MRI proveía un mejor contraste para manzanas afectadas por vitrescencia que las imágenes de tomografía de rayos X (X-Ray CT), como se pudo verificar en muestras idénticas de manzana. Además, el tiempo de adquisición de la tomografía de rayos X fue alrededor de 12 veces mayor (25 minutos) que la adquisición de las imágenes de resonancia magnética (2 minutos 2 segundos). Resultados sobre microestructura: - Para el estudio de microestructura (nivel subcelular) se utilizaron con éxito secuencias de relaxometría 2D T1/T2. Estas secuencias se usaron por primera vez en alto campo y sobre piezas de alimento completo, convirtiéndose en una forma no destructiva de llevar a cabo estudios de microestructura. - El uso de MRI junto con relaxometría 2D T1/T2 permite realizar estudios multiescala en alimentos de forma no destructiva. Resultados sobre MRI en línea: - El uso de imagen de resonancia magnética en línea fue factible para la identificación de dos tipos de desórdenes internos en manzanas: vitrescencia y podredumbre interna. Las secuencias de imagen tipo FLASH resultaron adecuadas para la identificación en línea de vitrescencia en manzanas. Se realizó sin selección de corte, debido a que la vitrescencia puede desarrollarse en cualquier punto del volumen de la manzana. Se consiguió reducir el tiempo de adquisición, de modo que se llegaron a adquirir 1.3 frutos por segundos (758 ms por fruto). Las secuencias de imagen tipo UFLARE fueron adecuadas para la detección en línea de la podredumbre interna en manzanas. En este caso, se utilizó selección de corte, ya que se trata de un desorden que se suele localizar en la parte central del volumen de la manzana. Se consiguió reducir el tiempo de adquisicón hasta 0.67 frutos por segundo (1475 ms por fruto). En ambos casos (FLASH y UFLARE) fueron necesarios algoritmos para la corrección del movimiento de las imágenes en tiempo real. ABSTRACT Food is a complex system formed by several structures at different scales: macroscopic and microscopic. Many properties of foods that are relevant to process engineering or quality and postharvest treatments are related to their microstructure. This Ph.D Thesis proposes a complete methodology for food structure determination, in a multiscale way, based on the Nuclear Magnetic Resonance (NMR) phenomenon since NMR techniques are non-invasive and non-destructive, and allow both, macro- and micro-structure study. Different NMR procedures are used depending on the structure level under study. For the macrostructure level, Magnetic Resonance Imaging (MRI) revealed its usefulness for food characterization. For microstructure insight, MRI required high acquisition times, which is a hindrance for transference to industry applications. Therefore, optimization of NMR procedures based on T1/T2 relaxometry sequences was a key strategy in this Thesis. These NMR relaxometry protocols, are successfully implemented in high magnetic field. Microstructure of entire food products have been characterized for the first time using these protocols. Two different types of food products have been studied: food models and actual food (apples). Furthermore, as a first step for the food industry implementation, a grading line system, specially designed for working under NMR conditions in previous works of the LPF-TAGRALIA group, is improved. The study and selection of the most suitable rapid sequence to detect two different types of disorders in apples (watercore and internal breakdown) is performed and the real time image motion correction is applied. In addition, artificial vision protocols for the automatic classification of apples potentially affected by watercore are applied. This document is divided into seven different chapters: Chapter 2 explains the thesis background and the framework of the project in which it has been worked. Chapter 3 comprises the state of the art. Chapter 4 establishes de objectives of this Ph.D thesis. The results are divided into five different sections (in Chapter 5) that correspond to published peered reviewed works. Section 5.1 assesses the watercore development in apples with MRI and studies the effect of fruit location in the canopy. Section 5.2 is an MRI and 2D relaxometry study for macro- and microstructure assessment in food models. Section 5.3 is a non-destructive microstructural study using 2D T1/T2 relaxometry on watercore affected apples. Section 5.4 makes a comparison of X-ray CT and MRI on watercore disorder of different apple cultivars. Section 5.5, that is a study of online MRI sequences for the evaluation of apple internal quality. The subsequent chapters offer a general discussion and conclusions (Chapter 6 and Chapter 7 respectively) of all the works performed in the frame of this Ph.D thesis (two peer reviewed journals, one book chapter and one international congress).Finally, three appendices are included in which an introduction to NMR principles is offered and two published proceedings regarding the effect of fiber on the rehydration of extruded breakfast cereal are displayed. The most relevant results can be summarized into three sections: results on macrostructure, results on microstructure and results on on-line MRI. Results on macrostructure: - MRI was successfully used for macrostructure characterization. Indeed, 3D reconstruction of MRI in apples allows to identify two different types of watercore (radial and block), which are characterized by the percentage of damage and the connectivity (Euler number). - MRI provides better contrast for watercore than X-Ray CT as verified on identical samples. Furthermore, X-Ray CT images acquisition time was around 12 times higher (25 minutes) than MRI acquisition time (2 minutes 2 seconds). Results on microstructure: - 2D T1/T2 relaxometry were successfully applied for microstructure (subcellular level) characterization. 2D T1/T2 relaxometry sequences have been applied for the first time on high field for entire food pieces, being a non-destructive way to achieve microstructure study. - The use of MRI together with 2D T1/T2 relaxometry sequences allows a non-destructive multiscale study of food. Results on on-line MRI: - The use of on-line MRI was successful for the identification of two different internal disorders in apples: watercore and internal breakdown. FLASH imaging was a suitable technique for the on-line detection of watercore disorder in apples, with no slice selection, since watercore is a physiological disorder that may be developed anywhere in the apple volume. 1.3 fruits were imaged per second (768 ms per fruit). UFLARE imaging is a suitable sequence for the on-line detection of internal breakdown disorder in apples. Slice selection was used, as internal breakdown is usually located in the central slice of the apple volume. 0.67 fruits were imaged per second (1475 ms per fruit). In both cases (FLASH and UFLARE) motion correction was performed in real time, during the acquisition of the images.

Development of a human movement monitoring system based on wearable devices

It is essential to remotely and continuously monitor the movements of individuals in many social areas, for example, taking care of aging people, physical therapy, athletic training etc. Many methods have been used, such as video record, motion analysis or sensor-based methods. Due to the limitations in remote communication, power consumption, portability and so on, most of them are not able to fulfill the requirements. The development of wearable technology and cloud computing provides a new efficient way to achieve this goal. This paper presents an intelligent human movement monitoring system based on a smartwatch, an Android smartphone and a distributed data management engine. This system includes advantages of wide adaptability, remote and long-term monitoring capacity, high portability and flexibility. The structure of the system and its principle are introduced. Four experiments are designed to prove the feasibility of the system. The results of the experiments demonstrate the system is able to detect different actions of individuals with adequate accuracy.

Automatic motion compensation of free breathing acquired myocardial perfusion data by using independent component analysis

Images acquired during free breathing using first-pass gadolinium-enhanced myocardial perfusion magnetic resonance imaging (MRI) exhibit a quasiperiodic motion pattern that needs to be compensated for if a further automatic analysis of the perfusion is to be executed. In this work, we present a method to compensate this movement by combining independent component analysis (ICA) and image registration: First, we use ICA and a time?frequency analysis to identify the motion and separate it from the intensity change induced by the contrast agent. Then, synthetic reference images are created by recombining all the independent components but the one related to the motion. Therefore, the resulting image series does not exhibit motion and its images have intensities similar to those of their original counterparts. Motion compensation is then achieved by using a multi-pass image registration procedure. We tested our method on 39 image series acquired from 13 patients, covering the basal, mid and apical areas of the left heart ventricle and consisting of 58 perfusion images each. We validated our method by comparing manually tracked intensity profiles of the myocardial sections to automatically generated ones before and after registration of 13 patient data sets (39 distinct slices). We compared linear, non-linear, and combined ICA based registration approaches and previously published motion compensation schemes. Considering run-time and accuracy, a two-step ICA based motion compensation scheme that first optimizes a translation and then for non-linear transformation performed best and achieves registration of the whole series in 32 ± 12 s on a recent workstation. The proposed scheme improves the Pearsons correlation coefficient between manually and automatically obtained time?intensity curves from .84 ± .19 before registration to .96 ± .06 after registration

MIA - a free and open source software for gray scale medical image analysis

Background Gray scale images make the bulk of data in bio-medical image analysis, and hence, the main focus of many image processing tasks lies in the processing of these monochrome images. With ever improving acquisition devices, spatial and temporal image resolution increases, and data sets become very large. Various image processing frameworks exists that make the development of new algorithms easy by using high level programming languages or visual programming. These frameworks are also accessable to researchers that have no background or little in software development because they take care of otherwise complex tasks. Specifically, the management of working memory is taken care of automatically, usually at the price of requiring more it. As a result, processing large data sets with these tools becomes increasingly difficult on work station class computers. One alternative to using these high level processing tools is the development of new algorithms in a languages like C++, that gives the developer full control over how memory is handled, but the resulting workflow for the prototyping of new algorithms is rather time intensive, and also not appropriate for a researcher with little or no knowledge in software development. Another alternative is in using command line tools that run image processing tasks, use the hard disk to store intermediate results, and provide automation by using shell scripts. Although not as convenient as, e.g. visual programming, this approach is still accessable to researchers without a background in computer science. However, only few tools exist that provide this kind of processing interface, they are usually quite task specific, and don’t provide an clear approach when one wants to shape a new command line tool from a prototype shell script. Results The proposed framework, MIA, provides a combination of command line tools, plug-ins, and libraries that make it possible to run image processing tasks interactively in a command shell and to prototype by using the according shell scripting language. Since the hard disk becomes the temporal storage memory management is usually a non-issue in the prototyping phase. By using string-based descriptions for filters, optimizers, and the likes, the transition from shell scripts to full fledged programs implemented in C++ is also made easy. In addition, its design based on atomic plug-ins and single tasks command line tools makes it easy to extend MIA, usually without the requirement to touch or recompile existing code. Conclusion In this article, we describe the general design of MIA, a general purpouse framework for gray scale image processing. We demonstrated the applicability of the software with example applications from three different research scenarios, namely motion compensation in myocardial perfusion imaging, the processing of high resolution image data that arises in virtual anthropology, and retrospective analysis of treatment outcome in orthognathic surgery. With MIA prototyping algorithms by using shell scripts that combine small, single-task command line tools is a viable alternative to the use of high level languages, an approach that is especially useful when large data sets need to be processed.

Video analysis based vehicle detection and tracking using an MCMC sampling framework

This article presents a probabilistic method for vehicle detection and tracking through the analysis of monocular images obtained from a vehicle-mounted camera. The method is designed to address the main shortcomings of traditional particle filtering approaches, namely Bayesian methods based on importance sampling, for use in traffic environments. These methods do not scale well when the dimensionality of the feature space grows, which creates significant limitations when tracking multiple objects. Alternatively, the proposed method is based on a Markov chain Monte Carlo (MCMC) approach, which allows efficient sampling of the feature space. The method involves important contributions in both the motion and the observation models of the tracker. Indeed, as opposed to particle filter-based tracking methods in the literature, which typically resort to observation models based on appearance or template matching, in this study a likelihood model that combines appearance analysis with information from motion parallax is introduced. Regarding the motion model, a new interaction treatment is defined based on Markov random fields (MRF) that allows for the handling of possible inter-dependencies in vehicle trajectories. As for vehicle detection, the method relies on a supervised classification stage using support vector machines (SVM). The contribution in this field is twofold. First, a new descriptor based on the analysis of gradient orientations in concentric rectangles is dened. This descriptor involves a much smaller feature space compared to traditional descriptors, which are too costly for real-time applications. Second, a new vehicle image database is generated to train the SVM and made public. The proposed vehicle detection and tracking method is proven to outperform existing methods and to successfully handle challenging situations in the test sequences.

Nonrigid Motion Compensation of Free Breathing Acquired Myocardial Perfusion Data

In this work, we present a novel method to compensate the movement in images acquired during free breathing using first-pass gadolinium enhanced, myocardial perfusion magnetic resonance imaging (MRI). First, we use independent component analysis (ICA) to identify the optimal number of independent components (ICs) that separate the breathing motion from the intensity change induced by the contrast agent. Then, synthetic images are created by recombining the ICs, but other then in previously published work (Milles et al. 2008), we omit the component related to motion, and therefore, the resulting reference image series is free of motion. Motion compensation is then achieved by using a multi-pass non-rigid image registration scheme. We tested our method on 15 distinct image series (5 patients) consisting of 58 images each and we validated our method by comparing manually tracked intensity profiles of the myocardial sections to automatically generated ones before and after registration. The average correlation to the manually obtained curves before registration 0:89 0:11 was increased to 0:98 0:02

A New Fast Motion Estimation and Mode Decision algorithm for H.264 Depth Maps encoding in Free Viewpoint TV

In this paper, we consider a scenario where 3D scenes are modeled through a View+Depth representation. This representation is to be used at the rendering side to generate synthetic views for free viewpoint video. The encoding of both type of data (view and depth) is carried out using two H.264/AVC encoders. In this scenario we address the reduction of the encoding complexity of depth data. Firstly, an analysis of the Mode Decision and Motion Estimation processes has been conducted for both view and depth sequences, in order to capture the correlation between them. Taking advantage of this correlation, we propose a fast mode decision and motion estimation algorithm for the depth encoding. Results show that the proposed algorithm reduces the computational burden with a negligible loss in terms of quality of the rendered synthetic views. Quality measurements have been conducted using the Video Quality Metric.

3D + t Morphological Processing: Applications to Embryogenesis Image Analysis

We propose to directly process 3D + t image sequences with mathematical morphology operators, using a new classi?cation of the 3D+t structuring elements. Several methods (?ltering, tracking, segmentation) dedicated to the analysis of 3D + t datasets of zebra?sh embryogenesis are introduced and validated through a synthetic dataset. Then, we illustrate the application of these methods to the analysis of datasets of zebra?sh early development acquired with various microscopy techniques. This processing paradigm produces spatio-temporal coherent results as it bene?ts from the intrinsic redundancy of the temporal dimension, and minimizes the needs for human intervention in semi-automatic algorithms.

Assembling models of embryo development: Image analysis and the construction of digital atlases

Digital atlases of animal development provide a quantitative description of morphogenesis, opening the path toward processes modeling. Prototypic atlases offer a data integration framework where to gather information from cohorts of individuals with phenotypic variability. Relevant information for further theoretical reconstruction includes measurements in time and space for cell behaviors and gene expression. The latter as well as data integration in a prototypic model, rely on image processing strategies. Developing the tools to integrate and analyze biological multidimensional data are highly relevant for assessing chemical toxicity or performing drugs preclinical testing. This article surveys some of the most prominent efforts to assemble these prototypes, categorizes them according to salient criteria and discusses the key questions in the field and the future challenges toward the reconstruction of multiscale dynamics in model organisms.

EVA: Laparoscopic instrument tracking based on endoscopic video analysis for psychomotor skills assessment

INTRODUCTION: The EVA (Endoscopic Video Analysis) tracking system a new tracking system for extracting motions of laparoscopic instruments based on non-obtrusive video tracking was developed. The feasibility of using EVA in laparoscopic settings has been tested in a box trainer setup. METHODS: EVA makes use of an algorithm that employs information of the laparoscopic instrument's shaft edges in the image, the instrument's insertion point, and the camera's optical centre to track the 3D position of the instrument tip. A validation study of EVA comprised a comparison of the measurements achieved with EVA and the TrEndo tracking system. To this end, 42 participants (16 novices, 22 residents, and 4 experts) were asked to perform a peg transfer task in a box trainer. Ten motion-based metrics were used to assess their performance. RESULTS: Construct validation of the EVA has been obtained for seven motion-based metrics. Concurrent validation revealed that there is a strong correlation between the results obtained by EVA and the TrEndo for metrics such as path length (p=0,97), average speed (p=0,94) or economy of volume (p=0,85), proving the viability of EVA. CONCLUSIONS: EVA has been successfully used in the training setup showing potential of endoscopic video analysis to assess laparoscopic psychomotor skills. The results encourage further implementation of video tracking in training setups and in image guided surgery.

Object-based image analysis for mapping Natura 2000 habitats.

Clasificación de una imagen de alta resolución "Quickbird" con la técnica de análisis de imágenes en base a objetos.

Objects-based Image Analysis for Mapping Natura 2000 Habitats to Improve Forest Management.

Clasificación de una imagen de alta resolución "Quickbird" con la técnica de análisis de imágenes en base a objetos

Seed image analysis and taxonomy of Diplotaxis DC. (Brassicaceae, Brassiceae)

The genus Diplotaxis, comprising 32 or 34 species, plus several additional infraspecific taxa, displays a considerable degree of heterogeneity in the morphology, molecular markers, chromosome numbers and geographical amplitude of the species. The taxonomic relationships within the genus Diplotaxis were investigated by phenetic characterisation of germplasm belonging to 27 taxa of the genus, because there is an increasing interest in Diplotaxis, since some of its species (D. tenuifolia, D. muralis) are gathered or cultivated for human consumption, whereas others are frequent arable weeds (D. erucoides) in many European vineyards. Using a computer-aided vision system, 33 morpho-colorimetric features of seeds were electronically measured. The data were used to implement a statistical classifier, which is able to discriminate the taxa within the genus Diplotaxis, in order to compare the resulting species grouping with the current infrageneric systematics of this genus. Despite the high heterogeneity of the samples, due to the great intra-population variability, the stepwise Linear Discriminant Analysis method, applied to distinguish the groups, was able to reach over 80% correct identification. The results obtained allowed us to confirm the current taxonomic position of most taxa and suggested the taxonomic position of others for reconsideration.

Image processing methods for human brain connectivity analysis from in-vivo diffusion MRI

In this PhD Thesis proposal, the principles of diffusion MRI (dMRI) in its application to the human brain mapping of connectivity are reviewed. The background section covers the fundamentals of dMRI, with special focus on those related to the distortions caused by susceptibility inhomogeneity across tissues. Also, a deep survey of available correction methodologies for this common artifact of dMRI is presented. Two methodological approaches to improved correction are introduced. Finally, the PhD proposal describes its objectives, the research plan, and the necessary resources.