Robot Motion Vision by Fixation

In many motion-vision scenarios, a camera (mounted on a moving vehicle) takes images of an environment to find the "motion'' and shape. We introduce a direct-method called fixation for solving this motion-vision problem in its general case. Fixation uses neither feature-correspondence nor optical-flow. Instead, spatio-temporal brightness gradients are used directly. In contrast to previous direct methods, fixation does not restrict the motion or the environment. Moreover, fixation method neither requires tracked images as its input nor uses mechanical tracking for obtaining fixated images. The experimental results on real images are presented and the implementation issues and techniques are discussed.

Learning-Based Approach to Real Time Tracking and Analysis of Faces

This paper describes a trainable system capable of tracking faces and facialsfeatures like eyes and nostrils and estimating basic mouth features such as sdegrees of openness and smile in real time. In developing this system, we have addressed the twin issues of image representation and algorithms for learning. We have used the invariance properties of image representations based on Haar wavelets to robustly capture various facial features. Similarly, unlike previous approaches this system is entirely trained using examples and does not rely on a priori (hand-crafted) models of facial features based on optical flow or facial musculature. The system works in several stages that begin with face detection, followed by localization of facial features and estimation of mouth parameters. Each of these stages is formulated as a problem in supervised learning from examples. We apply the new and robust technique of support vector machines (SVM) for classification in the stage of skin segmentation, face detection and eye detection. Estimation of mouth parameters is modeled as a regression from a sparse subset of coefficients (basis functions) of an overcomplete dictionary of Haar wavelets.

Visual Speech Synthesis by Morphing Visemes

We present MikeTalk, a text-to-audiovisual speech synthesizer which converts input text into an audiovisual speech stream. MikeTalk is built using visemes, which are a small set of images spanning a large range of mouth shapes. The visemes are acquired from a recorded visual corpus of a human subject which is specifically designed to elicit one instantiation of each viseme. Using optical flow methods, correspondence from every viseme to every other viseme is computed automatically. By morphing along this correspondence, a smooth transition between viseme images may be generated. A complete visual utterance is constructed by concatenating viseme transitions. Finally, phoneme and timing information extracted from a text-to-speech synthesizer is exploited to determine which viseme transitions to use, and the rate at which the morphing process should occur. In this manner, we are able to synchronize the visual speech stream with the audio speech stream, and hence give the impression of a photorealistic talking face.

A swarm intelligence method for feature tracking in AMV derivation

Feature tracking is a key step in the derivation of Atmospheric Motion Vectors (AMV). Most operational derivation processes use some template matching technique, such as Euclidean distance or cross-correlation, for the tracking step. As this step is very expensive computationally, often shortrange forecasts generated by Numerical Weather Prediction (NWP) systems are used to reduce the search area. Alternatives, such as optical flow methods, have been explored, with the aim of improving the number and quality of the vectors generated and the computational efficiency of the process. This paper will present the research carried out to apply Stochastic Diffusion Search, a generic search technique in the Swarm Intelligence family, to feature tracking in the context of AMV derivation. The method will be described, and we will present initial results, with Euclidean distance as reference.

Estimation of visual motion parameters used for bio-inspired navigation

Visual motion cues play an important role in animal and humans locomotion without the need to extract actual ego-motion information. This paper demonstrates a method for estimating the visual motion parameters, namely the Time-To-Contact (TTC), Focus of Expansion (FOE), and image angular velocities, from a sparse optical flow estimation registered from a downward looking camera. The presented method is capable of estimating the visual motion parameters in a complicated 6 degrees of freedom motion and in real time with suitable accuracy for mobile robots visual navigation.

Caminhamento fotogramétrico utilizando o fluxo óptico filtrado

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Estabilização postural do olhar: efeitos da distância do objeto

The aim of the present study was to analyze the effects of looking at targets located at different distances on body oscillation during tasks of distinct difficulties. In Experiment 1, ten participants in quiet stance fixated targets in three conditions: No object-far (fixation on far-target without near-target), Object-near (fixation on near target with fartarget), and Object-far (fixation on far-target with near-target). Mean oscillations of trunk in anterior-posterior axis were smallest in the Object-near condition; the No object-far and Object-far conditions were similar. In Experiment 2, seven participants in kiba-dachi, a karate stance, were submitted to three conditions: Blindfolded, No object-far, and Object-near. Mean oscillations of head and trunk in anterior-posterior axis were smaller in the Object-near as compared to blindfolded condition; trunk oscillated more during No object-far than Object-near condition. The results support the notion that a simple posture is not automatically regulated by the optical flow, but different amounts of visual instability may be tolerated according to the fixation distance, regardless the presence of non-fixated objects; the control of a more difficult posture may also accommodate the effects of fixation distance.

Compact and autonomous multiwavelength microanalyzer for in-line and in situ colorimetric determinations

Nowadays, the attainment of microsystems that integrate most of the stages involved in an analytical process has raised an enormous interest in several research fields. This approach provides experimental set-ups of increased robustness and reliability, which simplify their application to in-line and continuous biomedical and environmental monitoring. In this work, a novel, compact and autonomous microanalyzer aimed at multiwavelength colorimetric determinations is presented. It integrates the microfluidics (a three-dimensional mixer and a 25 mm length "Z-shape" optical flow-cell), a highly versatile multiwavelength optical detection system and the associated electronics for signal processing and drive, all in the same device. The flexibility provided by its design allows the microanalyzer to be operated either in single fixed mode to provide a dedicated photometer or in multiple wavelength mode to obtain discrete pseudospectra. To increase its reliability, automate its operation and allow it to work under unattended conditions, a multicommutation sub-system was developed and integrated with the experimental set-up. The device was initially evaluated in the absence of chemical reactions using four acidochromic dyes and later applied to determine some key environmental parameters such as phenol index, chromium(VI) and nitrite ions. Results were comparable with those obtained with commercial instrumentation and allowed to demonstrate the versatility of the proposed microanalyzer as an autonomous and portable device able to be applied to other analytical methodologies based on colorimetric determinations.

Estimación del movimiento de objetos en secuencias de imágenes de intensidad y profundidad

Máster Universitario en Sistemas Inteligentes y Aplicaciones Numéricas en Ingeniería (SIANI)

3D reconstruction from a vascular tree model

[EN] In this paper, we present a vascular tree model made with synthetic materials and which allows us to obtain images to make a 3D reconstruction.We have used PVC tubes of several diameters and lengths that will let us evaluate the accuracy of our 3D reconstruction. In order to calibrate the camera we have used a corner detector. Also we have used Optical Flow techniques to follow the points through the images going and going back. We describe two general techniques to extract a sequence of corresponding points from multiple views of an object. The resulting sequence of points will be used later to reconstruct a set of 3D points representing the object surfaces on the scene. We have made the 3D reconstruction choosing by chance a couple of images and we have calculated the projection error. After several repetitions, we have found the best 3D location for the point.

Model of vessels for the 3D reconstruction

[EN] In this paper, we present a vascular tree model made with synthetic materials and which allows us to obtain images to make a 3D reconstruction. In order to create this model, we have used PVC tubes of several diameters and lengths that will let us evaluate the accuracy of our 3D reconstruction. We have made the 3D reconstruction from a series of images that we have from our model and after we have calibrated the camera. In order to calibrate it we have used a corner detector. Also we have used Optical Flow techniques to follow the points through the images going and going back. Once we have the set of images where we have located a point, we have made the 3D reconstruction choosing by chance a couple of images and we have calculated the projection error. After several repetitions, we have found the best 3D location for the point.

Studio ed implementazione di algoritmi per la navigazione inerziale di velivoli autonomi ad ala rotante

I sistemi di navigazione inerziale, denominati INS, e quelli di navigazione inerziale assistita, ovvero che sfruttano anche sensori di tipo non inerziale come ad esempio il GPS, denominati in questo caso INS/GPS, hanno visto un forte incremento del loro utilizzo soprattutto negli ultimi anni. I filtri complementari sfruttano segnali in ingresso che presentano caratteristiche complementari in termine di banda. Con questo lavoro di tesi mi sono inserito nel contesto del progetto SHERPA (Smart collaboration between Humans and ground-aErial Robots for imProving rescuing activities in Alpine environments), un progetto europeo, coordinato dall'Università di Bologna, che prevede di mettere a punto una piattaforma robotica in grado di aiutare i soccorritori che operano in ambienti ostili, come quelli del soccorso alpino, le guardie forestali, la protezione civile. In particolare è prevista la possibilità di lanciare i droni direttamente da un elicottero di supporto, per cui potrebbe essere necessario effettuare l'avvio del sistema in volo. Ciò comporta che il sistema di navigazione dovrà essere in grado di convergere allo stato reale del sistema partendo da un grande errore iniziale, dal momento che la fase di inizializzazione funziona bene solo in condizioni di velivolo fermo. Si sono quindi ricercati, in special modo, schemi che garantissero la convergenza globale. Gli algoritmi implementati sono alla base della navigazione inerziale, assistita da GPS ed Optical Flow, della prima piattaforma aerea sviluppata per il progetto SHERPA, soprannominata DreamDroneOne, che include una grande varietà di hardware appositamente studiati per il progetto, come il laser scanner, la camera termica, ecc. Dopo una panoramica dell'architettura del sistema di Guida, Navigazione e Controllo (GNC) in cui mi sono inserito, si danno alcuni cenni sulle diverse terne di riferimento e trasformazioni, si descrivono i diversi sensori utilizzati per la navigazione, si introducono gli AHRS (Attitude Heading Rference System), per la determinazione del solo assetto sfruttando la IMU ed i magnetometri, si analizza l'AHRS basato su Extended Kalman Filter. Si analizzano, di seguito, un algoritmo non lineare per la stima dell'assetto molto recente, e il sistema INS/GPS basato su EKF, si presenta un filtro complementare molto recente per la stima di posizione ed assetto, si presenta un filtro complementare per la stima di posizione e velocità, si analizza inoltre l'uso di un predittore GPS. Infine viene presentata la piattaforma hardware utilizzata per l'implementazione e la validazione, si descrive il processo di prototipazione software nelle sue fasi e si mostrano i risultati sperimentali.

Passive-Active Geometric Calibration for View-Dependent Projections onto Arbitrary Surfaces

In this paper we present a hybrid technique for correcting distortions that appear when projecting images onto geometrically complex, colored and textured surfaces. It analyzes the optical flow that results from perspective distortions during motions of the observer and tries to use this information for computing the correct image warping. If this fails due to an unreliable optical flow, an accurate -but slower and visiblestructured light projection is automatically triggered. Together with an appropriate radiometric compensation, view-dependent content can be projected onto arbitrary everyday surfaces. An implementation mainly on the GPU ensures fast frame rates.

High Resolution Image Correspondences for Video Post-Production

We present an algorithm for estimating dense image correspondences. Our versatile approach lends itself to various tasks typical for video post-processing, including image morphing, optical flow estimation, stereo rectification, disparity/depth reconstruction, and baseline adjustment. We incorporate recent advances in feature matching, energy minimization, stereo vision, and data clustering into our approach. At the core of our correspondence estimation we use Efficient Belief Propagation for energy minimization. While state-of-the-art algorithms only work on thumbnail-sized images, our novel feature downsampling scheme in combination with a simple, yet efficient data term compression, can cope with high-resolution data. The incorporation of SIFT (Scale-Invariant Feature Transform) features into data term computation further resolves matching ambiguities, making long-range correspondence estimation possible. We detect occluded areas by evaluating the correspondence symmetry, we further apply Geodesic matting to automatically determine plausible values in these regions.

Mejoras en el procesamiento de imágenes de TC para el tratamiento con radioterapia adaptativa del cáncer de próstata

El cáncer de próstata es el tipo de cáncer con mayor prevalencia entre los hombres del mundo occidental y, pese a tener una alta tasa de supervivencia relativa, es la segunda mayor causa de muerte por cáncer en este sector de la población. El tratamiento de elección frente al cáncer de próstata es, en la mayoría de los casos, la radioterapia externa. Las técnicas más modernas de radioterapia externa, como la radioterapia modulada en intensidad, permiten incrementar la dosis en el tumor mientras se reduce la dosis en el tejido sano. Sin embargo, la localización del volumen objetivo varía con el día de tratamiento, y se requieren movimientos muy pequeños de los órganos para sacar partes del volumen objetivo fuera de la región terapéutica, o para introducir tejidos sanos críticos dentro. Para evitar esto se han desarrollado técnicas más avanzadas, como la radioterapia guiada por imagen, que se define por un manejo más preciso de los movimientos internos mediante una adaptación de la planificación del tratamiento basada en la información anatómica obtenida de imágenes de tomografía computarizada (TC) previas a la sesión terapéutica. Además, la radioterapia adaptativa añade la información dosimétrica de las fracciones previas a la información anatómica. Uno de los fundamentos de la radioterapia adaptativa es el registro deformable de imágenes, de gran utilidad a la hora de modelar los desplazamientos y deformaciones de los órganos internos. Sin embargo, su utilización conlleva nuevos retos científico-tecnológicos en el procesamiento de imágenes, principalmente asociados a la variabilidad de los órganos, tanto en localización como en apariencia. El objetivo de esta tesis doctoral es mejorar los procesos clínicos de delineación automática de contornos y de cálculo de dosis acumulada para la planificación y monitorización de tratamientos con radioterapia adaptativa, a partir de nuevos métodos de procesamiento de imágenes de TC (1) en presencia de contrastes variables, y (2) cambios de apariencia del recto. Además, se pretende (3) proveer de herramientas para la evaluación de la calidad de los contornos obtenidos en el caso del gross tumor volumen (GTV). Las principales contribuciones de esta tesis doctoral son las siguientes: _ 1. La adaptación, implementación y evaluación de un algoritmo de registro basado en el flujo óptico de la fase de la imagen como herramienta para el cálculo de transformaciones no-rígidas en presencia de cambios de intensidad, y su aplicabilidad a tratamientos de radioterapia adaptativa en cáncer de próstata con uso de agentes de contraste radiológico. Los resultados demuestran que el algoritmo seleccionado presenta mejores resultados cualitativos en presencia de contraste radiológico en la vejiga, y no distorsiona la imagen forzando deformaciones poco realistas. 2. La definición, desarrollo y validación de un nuevo método de enmascaramiento de los contenidos del recto (MER), y la evaluación de su influencia en el procedimiento de radioterapia adaptativa en cáncer de próstata. Las segmentaciones obtenidas mediante el MER para la creación de máscaras homogéneas en las imágenes de sesión permiten mejorar sensiblemente los resultados de los algoritmos de registro en la región rectal. Así, el uso de la metodología propuesta incrementa el índice de volumen solapado entre los contornos manuales y automáticos del recto hasta un valor del 89%, cercano a los resultados obtenidos usando máscaras manuales para el registro de las dos imágenes. De esta manera se pueden corregir tanto el cálculo de los nuevos contornos como el cálculo de la dosis acumulada. 3. La definición de una metodología de evaluación de la calidad de los contornos del GTV, que permite la representación de la distribución espacial del error, adaptándola a volúmenes no-convexos como el formado por la próstata y las vesículas seminales. Dicha metodología de evaluación, basada en un nuevo algoritmo de reconstrucción tridimensional y una nueva métrica de cuantificación, presenta resultados precisos con una gran resolución espacial en un tiempo despreciable frente al tiempo de registro. Esta nueva metodología puede ser una herramienta útil para la comparación de distintos algoritmos de registro deformable orientados a la radioterapia adaptativa en cáncer de próstata. En conclusión, el trabajo realizado en esta tesis doctoral corrobora las hipótesis de investigación postuladas, y pretende servir como cimiento de futuros avances en el procesamiento de imagen médica en los tratamientos de radioterapia adaptativa en cáncer de próstata. Asimismo, se siguen abriendo nuevas líneas de aplicación futura de métodos de procesamiento de imágenes médicas con el fin de mejorar los procesos de radioterapia adaptativa en presencia de cambios de apariencia de los órganos, e incrementar la seguridad del paciente. I.2 Inglés Prostate cancer is the most prevalent cancer amongst men in the Western world and, despite having a relatively high survival rate, is the second leading cause of cancer death in this sector of the population. The treatment of choice against prostate cancer is, in most cases, external beam radiation therapy. The most modern techniques of external radiotherapy, as intensity modulated radiotherapy, allow increasing the dose to the tumor whilst reducing the dose to healthy tissue. However, the location of the target volume varies with the day of treatment, and very small movements of the organs are required to pull out parts of the target volume outside the therapeutic region, or to introduce critical healthy tissues inside. Advanced techniques, such as the image-guided radiotherapy (IGRT), have been developed to avoid this. IGRT is defined by more precise handling of internal movements by adapting treatment planning based on the anatomical information obtained from computed tomography (CT) images prior to the therapy session. Moreover, the adaptive radiotherapy adds dosimetric information of previous fractions to the anatomical information. One of the fundamentals of adaptive radiotherapy is deformable image registration, very useful when modeling the displacements and deformations of the internal organs. However, its use brings new scientific and technological challenges in image processing, mainly associated to the variability of the organs, both in location and appearance. The aim of this thesis is to improve clinical processes of automatic contour delineation and cumulative dose calculation for planning and monitoring of adaptive radiotherapy treatments, based on new methods of CT image processing (1) in the presence of varying contrasts, and (2) rectum appearance changes. It also aims (3) to provide tools for assessing the quality of contours obtained in the case of gross tumor volume (GTV). The main contributions of this PhD thesis are as follows: 1. The adaptation, implementation and evaluation of a registration algorithm based on the optical flow of the image phase as a tool for the calculation of non-rigid transformations in the presence of intensity changes, and its applicability to adaptive radiotherapy treatment in prostate cancer with use of radiological contrast agents. The results demonstrate that the selected algorithm shows better qualitative results in the presence of radiological contrast agents in the urinary bladder, and does not distort the image forcing unrealistic deformations. 2. The definition, development and validation of a new method for masking the contents of the rectum (MER, Spanish acronym), and assessing their impact on the process of adaptive radiotherapy in prostate cancer. The segmentations obtained by the MER for the creation of homogenous masks in the session CT images can improve significantly the results of registration algorithms in the rectal region. Thus, the use of the proposed methodology increases the volume overlap index between manual and automatic contours of the rectum to a value of 89%, close to the results obtained using manual masks for both images. In this way, both the calculation of new contours and the calculation of the accumulated dose can be corrected. 3. The definition of a methodology for assessing the quality of the contours of the GTV, which allows the representation of the spatial distribution of the error, adapting it to non-convex volumes such as that formed by the prostate and seminal vesicles. Said evaluation methodology, based on a new three-dimensional reconstruction algorithm and a new quantification metric, presents accurate results with high spatial resolution in a time negligible compared to the registration time. This new approach may be a useful tool to compare different deformable registration algorithms oriented to adaptive radiotherapy in prostate cancer In conclusion, this PhD thesis corroborates the postulated research hypotheses, and is intended to serve as a foundation for future advances in medical image processing in adaptive radiotherapy treatment in prostate cancer. In addition, it opens new future applications for medical image processing methods aimed at improving the adaptive radiotherapy processes in the presence of organ’s appearance changes, and increase the patient safety.