A Multiresolution Image Alignment Technique Based on Direct Methods for Pose Estimation of Aerial Vehicles

In this paper, we seek to expand the use of direct methods in real-time applications by proposing a vision-based strategy for pose estimation of aerial vehicles. The vast majority of approaches make use of features to estimate motion. Conversely, the strategy we propose is based on a MR (Multi-Resolution) implementation of an image registration technique (Inverse Compositional Image Alignment ICIA) using direct methods. An on-board camera in a downwards-looking configuration, and the assumption of planar scenes, are the bases of the algorithm. The motion between frames (rotation and translation) is recovered by decomposing the frame-to-frame homography obtained by the ICIA algorithm applied to a patch that covers around the 80% of the image. When the visual estimation is required (e.g. GPS drop-out), this motion is integrated with the previous known estimation of the vehicles' state, obtained from the on-board sensors (GPS/IMU), and the subsequent estimations are based only on the vision-based motion estimations. The proposed strategy is tested with real flight data in representative stages of a flight: cruise, landing, and take-off, being two of those stages considered critical: take-off and landing. The performance of the pose estimation strategy is analyzed by comparing it with the GPS/IMU estimations. Results show correlation between the visual estimation obtained with the MR-ICIA and the GPS/IMU data, that demonstrate that the visual estimation can be used to provide a good approximation of the vehicle's state when it is required (e.g. GPS drop-outs). In terms of performance, the proposed strategy is able to maintain an estimation of the vehicle's state for more than one minute, at real-time frame rates based, only on visual information.

Methodology for registration of distended recutms in pelvic CT studies

Purpose: Accurate delineation of the rectum is of high importance in off-line adaptive radiation therapy since it is a major dose-limiting organ in prostate cancer radiotherapy. The intensity-based deformable image registration (DIR) methods cannot create a correct spatial transformation if there is no correspondence between the template and the target images. The variation of rectal filling, gas, or feces, creates a noncorrespondence in image intensities that becomes a great obstacle for intensity-based DIR. Methods: In this study the authors have designed and implemented a semiautomatic method to create a rectum mask in pelvic computed tomography (CT) images. The method, that includes a DIR based on the demons algorithm, has been tested in 13 prostate cancer cases, each comprising of two CT scans, for a total of 26 CT scans. Results: The use of the manual segmentation in the planning image and the proposed rectum mask method (RMM) method in the daily image leads to an improvement in the DIR performance in pelvic CT images, obtaining a mean value of overlap volume index = 0.89, close to the values obtained using the manual segmentations in both images. Conclusions: The application of the RMM method in the daily image and the manual segmentations in the planning image during prostate cancer treatments increases the performance of the registration in presence of rectal fillings, obtaining very good agreement with a physician's manual contours.

Methodology for registration of distended rectums in pelvic CT studies

Purpose: Accurate delineation of the rectum is of high importance in off-line adaptive radiation therapy since it is a major dose-limiting organ in prostate cancer radiotherapy. The intensity-based deformable image registration (DIR) methods cannot create a correct spatial transformation if there is no correspondence between the template and the target images. The variation of rectal filling, gas, or feces, creates a noncorrespondence in image intensities that becomes a great obstacle for intensity-based DIR. Methods: In this study the authors have designed and implemented a semiautomatic method to create a rectum mask in pelvic computed tomography (CT) images. The method, that includes a DIR based on the demons algorithm, has been tested in 13 prostate cancer cases, each comprising of two CT scans, for a total of 26 CT scans. Results: The use of the manual segmentation in the planning image and the proposed rectum mask method (RMM) method in the daily image leads to an improvement in the DIR performance in pelvic CT images, obtaining a mean value of overlap volume index = 0.89, close to the values obtained using the manual segmentations in both images. Conclusions: The application of the RMM method in the daily image and the manual segmentations in the planning image during prostate cancer treatments increases the performance of the registration in presence of rectal fillings, obtaining very good agreement with a physician's manual contours.

MRI Registration for Human Hip Kinematics

Measurement of joint kinematics can provide knowledge to help improve joint prosthesis design, as well as identify joint motion patterns that may lead to joint degeneration or injury. More investigation into how the hip translates in live human subjects during high amplitude motions is needed. This work presents a design of a non-invasive method using the registration between images from conventional Magnetic Resonance Imaging (MRI) and open MRI to calculate three dimensional hip joint kinematics. The method was tested on a single healthy subject in three different poses. MRI protocols for the conventional gantry, high-resolution MRI and the open gantry, lowresolution MRI were developed. The scan time for the low-resolution protocol was just under 6 minutes. High-resolution meshes and low resolution contours were derived from segmentation of the high-resolution and low-resolution images, respectively. Low-resolution contours described the poses as scanned, whereas the meshes described the bones’ geometries. The meshes and contours were registered to each other, and joint kinematics were calculated. The segmentation and registration were performed for both cortical and sub-cortical bone surfaces. A repeatability study was performed by comparing the kinematic results derived from three users’ segmentations of the sub-cortical bone surfaces from a low-resolution scan. The root mean squared error of all registrations was below 1.92mm. The maximum range between segmenters in translation magnitude was 0.95mm, and the maximum deviation from the average of all orientations was 1.27◦. This work demonstrated that this method for non-invasive measurement of hip kinematics is promising for measuring high-range-of-motion hip motions in vivo.

Open-source virtual bronchoscopy for image guided navigation

This thesis describes the development of an open-source system for virtual bronchoscopy used in combination with electromagnetic instrument tracking. The end application is virtual navigation of the lung for biopsy of early stage cancer nodules. The open-source platform 3D Slicer was used for creating freely available algorithms for virtual bronchscopy. Firstly, the development of an open-source semi-automatic algorithm for prediction of solitary pulmonary nodule malignancy is presented. This approach may help the physician decide whether to proceed with biopsy of the nodule. The user-selected nodule is segmented in order to extract radiological characteristics (i.e., size, location, edge smoothness, calcification presence, cavity wall thickness) which are combined with patient information to calculate likelihood of malignancy. The overall accuracy of the algorithm is shown to be high compared to independent experts' assessment of malignancy. The algorithm is also compared with two different predictors, and our approach is shown to provide the best overall prediction accuracy. The development of an airway segmentation algorithm which extracts the airway tree from surrounding structures on chest Computed Tomography (CT) images is then described. This represents the first fundamental step toward the creation of a virtual bronchoscopy system. Clinical and ex-vivo images are used to evaluate performance of the algorithm. Different CT scan parameters are investigated and parameters for successful airway segmentation are optimized. Slice thickness is the most affecting parameter, while variation of reconstruction kernel and radiation dose is shown to be less critical. Airway segmentation is used to create a 3D rendered model of the airway tree for virtual navigation. Finally, the first open-source virtual bronchoscopy system was combined with electromagnetic tracking of the bronchoscope for the development of a GPS-like system for navigating within the lungs. Tools for pre-procedural planning and for helping with navigation are provided. Registration between the lungs of the patient and the virtually reconstructed airway tree is achieved using a landmark-based approach. In an attempt to reduce difficulties with registration errors, we also implemented a landmark-free registration method based on a balanced airway survey. In-vitro and in-vivo testing showed good accuracy for this registration approach. The centreline of the 3D airway model is extracted and used to compensate for possible registration errors. Tools are provided to select a target for biopsy on the patient CT image, and pathways from the trachea towards the selected targets are automatically created. The pathways guide the physician during navigation, while distance to target information is updated in real-time and presented to the user. During navigation, video from the bronchoscope is streamed and presented to the physician next to the 3D rendered image. The electromagnetic tracking is implemented with 5 DOF sensing that does not provide roll rotation information. An intensity-based image registration approach is implemented to rotate the virtual image according to the bronchoscope's rotations. The virtual bronchoscopy system is shown to be easy to use and accurate in replicating the clinical setting, as demonstrated in the pre-clinical environment of a breathing lung method. Animal studies were performed to evaluate the overall system performance.

MRI Registration for Human Hip Kinematics

Measurement of joint kinematics can provide knowledge to help improve joint prosthesis design, as well as identify joint motion patterns that may lead to joint degeneration or injury. More investigation into how the hip translates in live human subjects during high amplitude motions is needed. This work presents a design of a non-invasive method using the registration between images from conventional Magnetic Resonance Imaging (MRI) and open MRI to calculate three dimensional hip joint kinematics. The method was tested on a single healthy subject in three different poses. MRI protocols for the conventional gantry, high-resolution MRI and the open gantry, lowresolution MRI were developed. The scan time for the low-resolution protocol was just under 6 minutes. High-resolution meshes and low resolution contours were derived from segmentation of the high-resolution and low-resolution images, respectively. Low-resolution contours described the poses as scanned, whereas the meshes described the bones’ geometries. The meshes and contours were registered to each other, and joint kinematics were calculated. The segmentation and registration were performed for both cortical and sub-cortical bone surfaces. A repeatability study was performed by comparing the kinematic results derived from three users’ segmentations of the sub-cortical bone surfaces from a low-resolution scan. The root mean squared error of all registrations was below 1.92mm. The maximum range between segmenters in translation magnitude was 0.95mm, and the maximum deviation from the average of all orientations was 1.27◦. This work demonstrated that this method for non-invasive measurement of hip kinematics is promising for measuring high-range-of-motion hip motions in vivo.

Registration-by-regression of coronary CTA and X-ray angiography

We evaluate the integration of 3D preoperative computed tomography angiography of the coronary arteries with intraoperative 2D X-ray angiographies by a recently proposed novel registration-by-regression method. The method relates image features of 2D projection images to the transformation parameters of the 3D image. We compared different sets of features and studied the influence of preprocessing the training set. For the registration evaluation, a gold standard was developed from eight X-ray angiography sequences from six different patients. The alignment quality was measured using the 3D mean target registration error (mTRE). The registration-by-regression method achieved moderate accuracy (median mTRE of 15 mm) on real images. It does therefore not provide yet a complete solution to the 3D–2D registration problem but it could be used as an initialisation method to eliminate the need for manual initialisation.

Image Processing Algorithms applied to Radio SLAM using MIMO Radars

Radio Simultaneous Location and Mapping (SLAM) consists of the simultaneous tracking of the target and estimation of the surrounding environment, to build a map and estimate the target movements within it. It is an increasingly exploited technique for automotive applications, in order to improve the localization of obstacles and the target relative movement with respect to them, for emergency situations, for example when it is necessary to explore (with a drone or a robot) environments with a limited visibility, or for personal radar applications, thanks to its versatility and cheapness. Until today, these systems were based on light detection and ranging (lidar) or visual cameras, high-accuracy and expensive approaches that are limited to specific environments and weather conditions. Instead, in case of smoke, fog or simply darkness, radar-based systems can operate exactly in the same way. In this thesis activity, the Fourier-Mellin algorithm is analyzed and implemented, to verify the applicability to Radio SLAM, in which the radar frames can be treated as images and the radar motion between consecutive frames can be covered with registration. Furthermore, a simplified version of that algorithm is proposed, in order to solve the problems of the Fourier-Mellin algorithm when working with real radar images and improve the performance. The INRAS RBK2, a MIMO 2x16 mmWave radar, is used for experimental acquisitions, consisting of multiple tests performed in Lab-E of the Cesena Campus, University of Bologna. The different performances of Fourier-Mellin and its simplified version are compared also with the MatchScan algorithm, a classic algorithm for SLAM systems.

White matter microstructure underlying default mode network connectivity in the human brain

Resting state functional magnetic resonance imaging (fMRI) reveals a distinct network of correlated brain function representing a default mode state of the human brain The underlying structural basis of this functional connectivity pattern is still widely unexplored We combined fractional anisotropy measures of fiber tract integrity derived from diffusion tensor imaging (DTI) and resting state fMRI data obtained at 3 Tesla from 20 healthy elderly subjects (56 to 83 years of age) to determine white matter microstructure e 7 underlying default mode connectivity We hypothesized that the functional connectivity between the posterior cingulate and hippocampus from resting state fMRI data Would be associated with the white matter microstructure in the cingulate bundle and fiber tracts connecting posterior cingulate gyrus With lateral temporal lobes, medial temporal lobes, and precuneus This was demonstrated at the p<0001 level using a voxel-based multivariate analysis of covariance (MANCOVA) approach In addition, we used a data-driven technique of joint independent component analysis (ICA) that uncovers spatial pattern that are linked across modalities. It revealed a pattern of white matter tracts including cingulate bundle and associated fiber tracts resembling the findings from the hypothesis-driven analysis and was linked to the pattern of default mode network (DMN) connectivity in the resting state fMRI data Out findings support the notion that the functional connectivity between the posterior cingulate and hippocampus and the functional connectivity across the entire DMN is based oil distinct pattern of anatomical connectivity within the cerebral white matter (C) 2009 Elsevier Inc All rights reserved

Three-dimensional space-borne synthetic aperture radar (SAR) imaging with multiple pass processing

Three-dimensional (3D) synthetic aperture radar (SAR) imaging via multiple-pass processing is an extension of interferometric SAR imaging. It exploits more than two flight passes to achieve a desired resolution in elevation. In this paper, a novel approach is developed to reconstruct a 3D space-borne SAR image with multiple-pass processing. It involves image registration, phase correction and elevational imaging. An image model matching is developed for multiple image registration, an eigenvector method is proposed for the phase correction and the elevational imaging is conducted using a Fourier transform or a super-resolution method for enhancement of elevational resolution. 3D SAR images are obtained by processing simulated data and real data from the first European Remote Sensing satellite (ERS-1) with the proposed approaches.

Sistema de visão para percepção e navegação de um UAV Quadrotor

Mestrado em Engenharia Electrotécnica e de Computadores

Comportamento da tibiotársica na sequência de movimento de sentar para de pé em crianças prematuras

Introdução: A prematuridade constitui um fator de risco para a ocorrência de lesões ao nível do sistema nervoso central, sendo que uma idade gestacional inferior a 36 semanas potencia esse mesmo risco, nomeadamente para a paralisia cerebral (PC) do tipo diplegia espástica. A sequência de movimento de sentado para de pé (SPP), sendo uma das aprendizagens motoras que exige um controlo postural (CP) ao nível da tibiotársica, parece ser uma tarefa funcional frequentemente comprometida em crianças prematuras com e sem PC. Objetivo(s): Descrever o comportamento dos músculos da tibiotársica, tibial anterior (TA) e solear (SOL), no que diz respeito ao timing de ativação, magnitude e co-ativação muscular durante a fase I e início da fase II na sequência de movimento de SPP realizada por cinco crianças prematuras com PC do tipo diplegia espástica e cinco crianças prematuras sem diagnóstico de alteração neuromotoras, sendo as primeiras sujeitas a um programa de intervenção baseado nos princípios do conceito de Bobath – Tratamento do Neurodesenvolvimento (TND). Métodos: Foram avaliadas 10 crianças prematuras, cinco com PC e cinco sem diagnóstico de alterações neuromotoras, tendo-se recorrido à eletromiografia de superfície para registar parâmetros musculares, nomeadamente timings, magnitudes e valores de co-ativação dos músculos TA e SOL, associados à fase I e inico da fase II da sequência de movimento de SPP. Procedeu-se ao registo de imagem de modo a facilitar a avaliação dos componentes de movimento associados a esta tarefa. Estes procedimentos foram realizados num único momento, no caso das crianças sem diagnóstico de alterações neuromotoras e em dois momentos, antes e após a aplicação de um programa de intervenção segundo o Conceito de Bobath – TND no caso das crianças com PC. A estas foi ainda aplicado o Teste da Medida das Funções Motoras (TMFM–88) e a Classificação Internacional da Funcionalidade Incapacidade e Saúde – crianças e jovens (CIF-CJ). Resultados: Através da eletromiografia constatou-se que ambos os grupos apresentaram timings de ativação afastados da janela temporal considerada como ajustes posturais antecipatórios (APAs), níveis elevados de co-ativação, em alguns casos com inversão na ordem de recrutamento muscular o que foi possível modificar nas crianças com PC após o período de intervenção. Nestas, verificou-se ainda que, a sequência de movimento de SPP foi realizada com menor número de compensações e com melhor relação entre estruturas proximais e distais compatível com o aumento do score final do TMFM-88 e modificação positiva nos itens de atividade e participação da CIF-CJ. Conclusão: As crianças prematuras com e sem PC apresentaram alterações no CP da tibiotársica e níveis elevados de co-ativação muscular. Após o período de intervenção as crianças com PC apresentaram modificações positivas no timing e co-ativação muscular, com impacto funcional evidenciado no aumento do score final da TMFM-88 e modificações positivas na CIF-CJ.

Large-Area Photo-Mosaics Using Global Alignment and Navigation Data

Seafloor imagery is a rich source of data for the study of biological and geological processes. Among several applications, still images of the ocean floor can be used to build image composites referred to as photo-mosaics. Photo-mosaics provide a wide-area visual representation of the benthos, and enable applications as diverse as geological surveys, mapping and detection of temporal changes in the morphology of biodiversity. We present an approach for creating globally aligned photo-mosaics using 3D position estimates provided by navigation sensors available in deep water surveys. Without image registration, such navigation data does not provide enough accuracy to produce useful composite images. Results from a challenging data set of the Lucky Strike vent field at the Mid Atlantic Ridge are reported

A system to evaluate the accuracy of a visual mosaicking methodology

When underwater vehicles navigate close to the ocean floor, computer vision techniques can be applied to obtain motion estimates. A complete system to create visual mosaics of the seabed is described in this paper. Unfortunately, the accuracy of the constructed mosaic is difficult to evaluate. The use of a laboratory setup to obtain an accurate error measurement is proposed. The system consists on a robot arm carrying a downward looking camera. A pattern formed by a white background and a matrix of black dots uniformly distributed along the surveyed scene is used to find the exact image registration parameters. When the robot executes a trajectory (simulating the motion of a submersible), an image sequence is acquired by the camera. The estimated motion computed from the encoders of the robot is refined by detecting, to subpixel accuracy, the black dots of the image sequence, and computing the 2D projective transform which relates two consecutive images. The pattern is then substituted by a poster of the sea floor and the trajectory is executed again, acquiring the image sequence used to test the accuracy of the mosaicking system

3-d residual eddy current field characterisation: applied to diffusion weighted magnetic resonance imaging.

Clinical use of the Stejskal-Tanner diffusion weighted images is hampered by the geometric distortions that result from the large residual 3-D eddy current field induced. In this work, we aimed to predict, using linear response theory, the residual 3-D eddy current field required for geometric distortion correction based on phantom eddy current field measurements. The predicted 3-D eddy current field induced by the diffusion-weighting gradients was able to reduce the root mean square error of the residual eddy current field to ~1 Hz. The model's performance was tested on diffusion weighted images of four normal volunteers, following distortion correction, the quality of the Stejskal-Tanner diffusion-weighted images was found to have comparable quality to image registration based corrections (FSL) at low b-values. Unlike registration techniques the correction was not hindered by low SNR at high b-values, and results in improved image quality relative to FSL. Characterization of the 3-D eddy current field with linear response theory enables the prediction of the 3-D eddy current field required to correct eddy current induced geometric distortions for a wide range of clinical and high b-value protocols.