Reliable estimation of dense optical flow fields with large displacements

[EN] In this paper we show that a classic optical flow technique by Nagel and Enkelmann can be regarded as an early anisotropic diffusion method with a diffusion tensor. We introduce three improvements into the model formulation that avoid inconsistencies caused by centering the brightness term and the smoothness term in different images use a linear scale-space focusing strategy from coarse to fine scales for avoiding convergence to physically irrelevant local minima, and create an energy functional that is invariant under linear brightness changes.  Applying a gradient descent method to the resulting energy functional leads to a system of diffusion-reaction equations. We prove that this system has a unique solution under realistic assumptions on the initial data, and we present an efficient linear implicit numerical scheme in detail. Our method creates flow fields with 100% density over the entire image domain, it is robust under a large range of parameter variations, and it can recover displacement fields that are far beyond the typical one-pixel limits which are characteristic for many differential methods for determining optical flow. We show that it performs better than the classic optical flow methods with 100%  density that are evaluated by Barron et al. (1994). Our software is available from the Internet.

A PDE model for computing the optical flow

[EN] In this paper we present a new model for optical flow calculation using a variational formulation which preserves discontinuities of the flow much better than classical methods. We study the Euler-Lagrange equations asociated to the variational problem. In the case of quadratic energy, we show the existence and uniqueness of the corresponding evolution problem. Since our method avoid linearization in the optical flow constraint, it can recover large displacement in the scene. We avoid convergence to irrelevant local minima by embedding our method into a linear scale-space framework and using a focusing strategy from coarse to fine scales.

Simultaneous Multiscale Polyaffine Registration by Incorporating Deformation Statistics

Locally affine (polyaffine) image registration methods capture intersubject non-linear deformations with a low number of parameters, while providing an intuitive interpretation for clinicians. Considering the mandible bone, anatomical shape differences can be found at different scales, e.g. left or right side, teeth, etc. Classically, sequential coarse to fine registration are used to handle multiscale deformations, instead we propose a simultaneous optimization of all scales. To avoid local minima we incorporate a prior on the polyaffine transformations. This kind of groupwise registration approach is natural in a polyaffine context, if we assume one configuration of regions that describes an entire group of images, with varying transformations for each region. In this paper, we reformulate polyaffine deformations in a generative statistical model, which enables us to incorporate deformation statistics as a prior in a Bayesian setting. We find optimal transformations by optimizing the maximum a posteriori probability. We assume that the polyaffine transformations follow a normal distribution with mean and concentration matrix. Parameters of the prior are estimated from an initial coarse to fine registration. Knowing the region structure, we develop a blockwise pseudoinverse to obtain the concentration matrix. To our knowledge, we are the first to introduce simultaneous multiscale optimization through groupwise polyaffine registration. We show results on 42 mandible CT images.

Quantitative assessment of oral orbicular muscle deformation after cleft lip reconstruction: an ultrasound elastography study

Reconstruction of a cleft lip leads inevitably to scar tissue formation. Scar tissue within the restored oral orbicular muscle might be assessed by quantification of the local contractility of this muscle. Furthermore, information about the contraction capability of the oral orbicular muscle is crucial for planning the revision surgery of an individual patient. We used ultrasound elastography to determine the local deformation (strain) of the upper lip and to differentiate contracting muscle from passive scar tissue. Raw ultrasound data (radio-frequency format; rf-) were acquired, while the lips were brought from normal state into a pout condition and back in normal state, in three patients and three normal individuals. During this movement, the oral orbicular muscle contracts and, consequently, thickens in contrast to scar tissue that will not contract, or even expand. An iterative coarse-to-fine strain estimation method was used to calculate the local tissue strain. Analysis of the raw ultrasound data allows estimation of tissue strain with a high precision. The minimum strain that can be assessed reproducibly is 0.1%. In normal individuals, strain of the orbicular oral muscle was in the order of 20%. Also, a uniform strain distribution in the oral orbicular muscle was found. However, in patients deviating values were found in the region of the reconstruction and the muscle tissue surrounding that. In two patients with a successful reconstruction, strain was reduced by 6% in the reconstructed region with respect to the normal parts of the muscle (from 22% to 16% and from 25% to 19%). In a patient with severe aesthetical and functional disability, strain decreased from 30% in the normal region to 5% in the reconstructed region. With ultrasound elastography, the strain of the oral orbicular muscle can be quantified. In healthy subjects, the strain profiles and maximum strain values in all parts of the muscle were similar. The maximum strain of the muscle during pout was 20% +/- 1%. In surgically repaired cleft lips, decreased deformation was observed.

Fully Automatic Segmentation of Hip CT Images

Automatic segmentation of the hip joint with pelvis and proximal femur surfaces from CT images is essential for orthopedic diagnosis and surgery. It remains challenging due to the narrowness of hip joint space, where the adjacent surfaces of acetabulum and femoral head are hardly distinguished from each other. This chapter presents a fully automatic method to segment pelvic and proximal femoral surfaces from hip CT images. A coarse-to-fine strategy was proposed to combine multi-atlas segmentation with graph-based surface detection. The multi-atlas segmentation step seeks to coarsely extract the entire hip joint region. It uses automatically detected anatomical landmarks to initialize and select the atlas and accelerate the segmentation. The graph based surface detection is to refine the coarsely segmented hip joint region. It aims at completely and efficiently separate the adjacent surfaces of the acetabulum and the femoral head while preserving the hip joint structure. The proposed strategy was evaluated on 30 hip CT images and provided an average accuracy of 0.55, 0.54, and 0.50 mm for segmenting the pelvis, the left and right proximal femurs, respectively.

Radioisotope stratigraphy, sedimentology and geochemistry of Late Quaternary sediments from the Eastern Arctic Ocean

To reconstruct Recent and past sedimentary environments, marine sediments of Upper Pleistocene and Holocene ages from the eastern Arctic Ocean and especially from the Nansen-Gakkel Ridge (NGR) were investigated by means of radioisotopic, geochemical and sedimentological methods. In combination with mass physical property data and lithological analysis these investigations allow clearly to characterize the depositional environments. Age dating by using the radioisotope 230Th gives evidence that the investigated sediments from the NGR are younger than 250,000 years. Identical lithological sediment sequences within and between sediment cores from the NGR can be related to sedimentary processes which are clearly controlled by palaeoclimate. The sediments consist predominantly of siliciclastic, terrigenous ice-rafted detritus (IRD) deriving from assorted and redeposited sediments from the Siberian shelfs. By their geochemical composition the sediments are similar to mudstone, graywacke and arcose. Sea-ice as well as icebergs play a major roll in marine arctic sedimentation. In the NGR area rapid change in sedimentary conditions can be detected 128,000 years ago. This was due to drastic change in the kind of ice cover, resulting from rapid climatic change within only hundreds of years. So icebergs, deriving mostly from Siberian shelfs, vanished and sea-ice became dominant in the eastern Arctic Ocean. At least three short-period retreats of the shelf ice between 186,000 and 128,000 years are responsible for the change of coarse to fine-grained sediments in the NGR area. These warmer stages lasted between 1,000 and 3,000 years. By monitoring and comparing the distribution patterns of sedimentologic, mass physical and geochemical properties with 230Th ex activity distribution patterns in the sediment cores from the NGR, there is clear evidence that sediment dilution is responsible for high 230Th ex activity variations. Thus sedimentation rate is the controlling factor of 230Th ex activity variations. The 230Th flux density in sediments from the NGR seems to be highly dependent On topographic Position. The distribution patterns of chemical elements in sediment cores are in general governed by lithology. The derivation of a method for dry bulk density determination gave the opportunity to establish a high resolution stratigraphy on sediment cores from the eastern Arctic Ocean, based on 230Thex activity analyses. For the first time sedimentation and accumulation rates were determined for recent sediments in the eastern Arctic Ocean by 230Th ex analyses. Bulk accumulation rates are highly variable in space and time, ranging between 0.2 and 30 g/cm**2/ka. In the sediments from the NGR highly variable accumulation rates are related to the kind of ice cover. There is evidence for hydrothermal input into the sediments of the NGR. Hydrothermal activity probably also influences surficial sediments in the Sofia Basin. High contents of As are typical for surficial sediments from the NGR. In particular SL 370-20 from the bottom of the rift valley has As contents exceeding in parts 300 ppm. Hydrothermal activity can be traced back to at least 130,000 years. Recent to subrecent tectonic activity is documented by the rock debris in KAL 370 from the NGR. In four other sediment cores from the NGR rift valley area tectonically induced movements can be dated to about 130,000 years ago, related most probably to the rapid climate change. Processes of early diagenesis in sediments from the NGR caused the aobilization and redeposition of Fe, Mn and Mo. These diagenetic processes probably took place during the last 130,000 years. In sediment cores from the NGR high amounts of kaolinite are related to coarse grained siliciclastic material, probably indicating reworking and redeposition of siberian sandstones with kaolinitic binding material. In contrast to kaolinite, illite is correlated to total clay and 232Th contents. Aragonite, associated with serpentinites in the rift valley area of the NGR, was precipitated under cold bottom-water conditions. Preliminary data result in a time of formation about 60 - 80 ka ago. Manganese precipitates with high Ni contents, which can be related to the ultrabasic rocks, are of similar age.

CaCO3 size distribution in surface sediments

Lysocline reconstructions play an important role in scenarios purporting to explain the lowered atmospheric CO2 content of glacial time. These reconstructions are based on indicators such as the CaCO3 content, the percent of coarse fraction, the ratio of fragments to whole foraminifera shells, the ratio of solution-susceptible to solution-resistant species, and the ratio of coarse to fine CaCO3. All assume that changes with time in the composition of the input material do not bias the result. However, as the composition of the input material does depend on climate, none of these indicators provides an absolute measure of the extent of dissolution. In this paper we evaluate the reliability of the ratio of >63 µm CaCO3 to total CaCO3 as a dissolution indicator. We present here results that suggest that in today's tropics this ratio appears to be determined solely by CO3= ion concentration and water depth (i.e., the saturation state of bottom waters). This finding offers the possibility that the size fraction index can be used to reconstruct CO3= ion concentrations for the late Quaternary ocean to an accuracy of ±5 µmol/kg.

Vision-Based Tracking, Odometry and Control for UAV Autonomy

El principal objetivo de este trabajo es proporcionar una solución en tiempo real basada en visión estéreo o monocular precisa y robusta para que un vehículo aéreo no tripulado (UAV) sea autónomo en varios tipos de aplicaciones UAV, especialmente en entornos abarrotados sin señal GPS. Este trabajo principalmente consiste en tres temas de investigación de UAV basados en técnicas de visión por computador: (I) visual tracking, proporciona soluciones efectivas para localizar visualmente objetos de interés estáticos o en movimiento durante el tiempo que dura el vuelo del UAV mediante una aproximación adaptativa online y una estrategia de múltiple resolución, de este modo superamos los problemas generados por las diferentes situaciones desafiantes, tales como cambios significativos de aspecto, iluminación del entorno variante, fondo del tracking embarullado, oclusión parcial o total de objetos, variaciones rápidas de posición y vibraciones mecánicas a bordo. La solución ha sido utilizada en aterrizajes autónomos, inspección de plataformas mar adentro o tracking de aviones en pleno vuelo para su detección y evasión; (II) odometría visual: proporciona una solución eficiente al UAV para estimar la posición con 6 grados de libertad (6D) usando únicamente la entrada de una cámara estéreo a bordo del UAV. Un método Semi-Global Blocking Matching (SGBM) eficiente basado en una estrategia grueso-a-fino ha sido implementada para una rápida y profunda estimación del plano. Además, la solución toma provecho eficazmente de la información 2D y 3D para estimar la posición 6D, resolviendo de esta manera la limitación de un punto de referencia fijo en la cámara estéreo. Una robusta aproximación volumétrica de mapping basada en el framework Octomap ha sido utilizada para reconstruir entornos cerrados y al aire libre bastante abarrotados en 3D con memoria y errores correlacionados espacialmente o temporalmente; (III) visual control, ofrece soluciones de control prácticas para la navegación de un UAV usando Fuzzy Logic Controller (FLC) con la estimación visual. Y el framework de Cross-Entropy Optimization (CEO) ha sido usado para optimizar el factor de escala y la función de pertenencia en FLC. Todas las soluciones basadas en visión en este trabajo han sido probadas en test reales. Y los conjuntos de datos de imágenes reales grabados en estos test o disponibles para la comunidad pública han sido utilizados para evaluar el rendimiento de estas soluciones basadas en visión con ground truth. Además, las soluciones de visión presentadas han sido comparadas con algoritmos de visión del estado del arte. Los test reales y los resultados de evaluación muestran que las soluciones basadas en visión proporcionadas han obtenido rendimientos en tiempo real precisos y robustos, o han alcanzado un mejor rendimiento que aquellos algoritmos del estado del arte. La estimación basada en visión ha ganado un rol muy importante en controlar un UAV típico para alcanzar autonomía en aplicaciones UAV. ABSTRACT The main objective of this dissertation is providing real-time accurate robust monocular or stereo vision-based solution for Unmanned Aerial Vehicle (UAV) to achieve the autonomy in various types of UAV applications, especially in GPS-denied dynamic cluttered environments. This dissertation mainly consists of three UAV research topics based on computer vision technique: (I) visual tracking, it supplys effective solutions to visually locate interesting static or moving object over time during UAV flight with on-line adaptivity approach and multiple-resolution strategy, thereby overcoming the problems generated by the different challenging situations, such as significant appearance change, variant surrounding illumination, cluttered tracking background, partial or full object occlusion, rapid pose variation and onboard mechanical vibration. The solutions have been utilized in autonomous landing, offshore floating platform inspection and midair aircraft tracking for sense-and-avoid; (II) visual odometry: it provides the efficient solution for UAV to estimate the 6 Degree-of-freedom (6D) pose using only the input of stereo camera onboard UAV. An efficient Semi-Global Blocking Matching (SGBM) method based on a coarse-to-fine strategy has been implemented for fast depth map estimation. In addition, the solution effectively takes advantage of both 2D and 3D information to estimate the 6D pose, thereby solving the limitation of a fixed small baseline in the stereo camera. A robust volumetric occupancy mapping approach based on the Octomap framework has been utilized to reconstruct indoor and outdoor large-scale cluttered environments in 3D with less temporally or spatially correlated measurement errors and memory; (III) visual control, it offers practical control solutions to navigate UAV using Fuzzy Logic Controller (FLC) with the visual estimation. And the Cross-Entropy Optimization (CEO) framework has been used to optimize the scaling factor and the membership function in FLC. All the vision-based solutions in this dissertation have been tested in real tests. And the real image datasets recorded from these tests or available from public community have been utilized to evaluate the performance of these vision-based solutions with ground truth. Additionally, the presented vision solutions have compared with the state-of-art visual algorithms. Real tests and evaluation results show that the provided vision-based solutions have obtained real-time accurate robust performances, or gained better performance than those state-of-art visual algorithms. The vision-based estimation has played a critically important role for controlling a typical UAV to achieve autonomy in the UAV application.

Contextual visual localization: cascaded submap classification, optimized saliency detection, and fast view matching

In this paper, we present a novel coarse-to-fine visual localization approach: contextual visual localization. This approach relies on three elements: (i) a minimal-complexity classifier for performing fast coarse localization (submap classification); (ii) an optimized saliency detector which exploits the visual statistics of the submap; and (iii) a fast view-matching algorithm which filters initial matchings with a structural criterion. The latter algorithm yields fine localization. Our experiments show that these elements have been successfully integrated for solving the global localization problem. Context, that is, the awareness of being in a particular submap, is defined by a supervised classifier tuned for a minimal set of features. Visual context is exploited both for tuning (optimizing) the saliency detection process, and to select potential matching views in the visual database, close enough to the query view.

Mineral, chemical and isotope compositions of clay subfractions from formations of the Srednii Peninsula, Murmansk coast of the Barents Sea

Fine-grained clay subfractions (SFs) with particle size of <0.1, 0.1-0.2, 0.2-0.3, 0.3-0.6, 0.6-2.0, and 2-5 µm separated from claystone of Upper Precambrian Pumanskaya and Poropelonskaya formations on the Srednii Peninsula were studied by transmission electron microscopy, X-ray diffraction, and Rb-Sr methods. All subfractions consist of low-temperature illite and chlorite, and contribution of chlorite decreases with diminishing particle size. The crystallinity index and I002/I001 ratio increase from coarse- to fine-grained SFs. Leaching by ammonium acetate solution and Rb-Sr systematics in combination with mineralogical and morphological data indicate that illite in Upper Proterozoic claystone from the Srednii Peninsula formed during three time intervals: 810-830, 610-620, and about 570 Ma ago. The first generation of this mineral with low Rb/Sr ratio dominates in coarse-grained SFs while the second and third generations with a high Rb/Sr ratio prevail in fine-grained SFs. All of three generations are known in Poropelon claystone, whereas Puman claystone contains only illite of the first and second generations. Geological processes responsible for multistage illite evolution in claystones are discussed.

High performance shift invariant motion estimation and compensation in wavelet domain video compression

The contributions of this dissertation are in the development of two new interrelated approaches to video data compression: (1) A level-refined motion estimation and subband compensation method for the effective motion estimation and motion compensation. (2) A shift-invariant sub-decimation decomposition method in order to overcome the deficiency of the decimation process in estimating motion due to its shift-invariant property of wavelet transform. ^ The enormous data generated by digital videos call for an intense need of efficient video compression techniques to conserve storage space and minimize bandwidth utilization. The main idea of video compression is to reduce the interpixel redundancies inside and between the video frames by applying motion estimation and motion compensation (MEMO) in combination with spatial transform coding. To locate the global minimum of the matching criterion function reasonably, hierarchical motion estimation by coarse to fine resolution refinements using discrete wavelet transform is applied due to its intrinsic multiresolution and scalability natures. ^ Due to the fact that most of the energies are concentrated in the low resolution subbands while decreased in the high resolution subbands, a new approach called level-refined motion estimation and subband compensation (LRSC) method is proposed. It realizes the possible intrablocks in the subbands for lower entropy coding while keeping the low computational loads of motion estimation as the level-refined method, thus to achieve both temporal compression quality and computational simplicity. ^ Since circular convolution is applied in wavelet transform to obtain the decomposed subframes without coefficient expansion, symmetric-extended wavelet transform is designed on the finite length frame signals for more accurate motion estimation without discontinuous boundary distortions. ^ Although wavelet transformed coefficients still contain spatial domain information, motion estimation in wavelet domain is not as straightforward as in spatial domain due to the shift variance property of the decimation process of the wavelet transform. A new approach called sub-decimation decomposition method is proposed, which maintains the motion consistency between the original frame and the decomposed subframes, improving as a consequence the wavelet domain video compressions by shift invariant motion estimation and compensation. ^

Le décours temporel de l'utilisation des fréquences spatiales dans les troubles du spectre autistique

Notre système visuel extrait d'ordinaire l'information en basses fréquences spatiales (FS) avant celles en hautes FS. L'information globale extraite tôt peut ainsi activer des hypothèses sur l'identité de l'objet et guider l'extraction d'information plus fine spécifique par la suite. Dans les troubles du spectre autistique (TSA), toutefois, la perception des FS est atypique. De plus, la perception des individus atteints de TSA semble être moins influencée par leurs a priori et connaissances antérieures. Dans l'étude décrite dans le corps de ce mémoire, nous avions pour but de vérifier si l'a priori de traiter l'information des basses aux hautes FS était présent chez les individus atteints de TSA. Nous avons comparé le décours temporel de l'utilisation des FS chez des sujets neurotypiques et atteints de TSA en échantillonnant aléatoirement et exhaustivement l'espace temps x FS. Les sujets neurotypiques extrayaient les basses FS avant les plus hautes: nous avons ainsi pu répliquer le résultat de plusieurs études antérieures, tout en le caractérisant avec plus de précision que jamais auparavant. Les sujets atteints de TSA, quant à eux, extrayaient toutes les FS utiles, basses et hautes, dès le début, indiquant qu'ils ne possédaient pas l'a priori présent chez les neurotypiques. Il semblerait ainsi que les individus atteints de TSA extraient les FS de manière purement ascendante, l'extraction n'étant pas guidée par l'activation d'hypothèses.

Le décours temporel de l'utilisation des fréquences spatiales dans les troubles du spectre autistique

Notre système visuel extrait d'ordinaire l'information en basses fréquences spatiales (FS) avant celles en hautes FS. L'information globale extraite tôt peut ainsi activer des hypothèses sur l'identité de l'objet et guider l'extraction d'information plus fine spécifique par la suite. Dans les troubles du spectre autistique (TSA), toutefois, la perception des FS est atypique. De plus, la perception des individus atteints de TSA semble être moins influencée par leurs a priori et connaissances antérieures. Dans l'étude décrite dans le corps de ce mémoire, nous avions pour but de vérifier si l'a priori de traiter l'information des basses aux hautes FS était présent chez les individus atteints de TSA. Nous avons comparé le décours temporel de l'utilisation des FS chez des sujets neurotypiques et atteints de TSA en échantillonnant aléatoirement et exhaustivement l'espace temps x FS. Les sujets neurotypiques extrayaient les basses FS avant les plus hautes: nous avons ainsi pu répliquer le résultat de plusieurs études antérieures, tout en le caractérisant avec plus de précision que jamais auparavant. Les sujets atteints de TSA, quant à eux, extrayaient toutes les FS utiles, basses et hautes, dès le début, indiquant qu'ils ne possédaient pas l'a priori présent chez les neurotypiques. Il semblerait ainsi que les individus atteints de TSA extraient les FS de manière purement ascendante, l'extraction n'étant pas guidée par l'activation d'hypothèses.

Planar Odometry from a Radial Laser Scanner. A Range Flow-based Approach

In this paper we present a fast and precise method to estimate the planar motion of a lidar from consecutive range scans. For every scanned point we formulate the range flow constraint equation in terms of the sensor velocity, and minimize a robust function of the resulting geometric constraints to obtain the motion estimate. Conversely to traditional approaches, this method does not search for correspondences but performs dense scan alignment based on the scan gradients, in the fashion of dense 3D visual odometry. The minimization problem is solved in a coarse-to-fine scheme to cope with large displacements, and a smooth filter based on the covariance of the estimate is employed to handle uncertainty in unconstraint scenarios (e.g. corridors). Simulated and real experiments have been performed to compare our approach with two prominent scan matchers and with wheel odometry. Quantitative and qualitative results demonstrate the superior performance of our approach which, along with its very low computational cost (0.9 milliseconds on a single CPU core), makes it suitable for those robotic applications that require planar odometry. For this purpose, we also provide the code so that the robotics community can benefit from it.

An investigation on the use of SNR distributions for the optimisation of coarse-fine spectrum sensing for cognitive radio

This thesis investigates the optimisation of Coarse-Fine (CF) spectrum sensing architectures under a distribution of SNRs for Dynamic Spectrum Access (DSA). Three different detector architectures are investigated: the Coarse-Sorting Fine Detector (CSFD), the Coarse-Deciding Fine Detector (CDFD) and the Hybrid Coarse-Fine Detector (HCFD). To date, the majority of the work on coarse-fine spectrum sensing for cognitive radio has focused on a single value for the SNR. This approach overlooks the key advantage that CF sensing has to offer, namely that high powered signals can be easily detected without extra signal processing. By considering a range of SNR values, the detector can be optimised more effectively and greater performance gains realised. This work considers the optimisation of CF spectrum sensing schemes where the security and performance are treated separately. Instead of optimising system performance at a single, constant, low SNR value, the system instead is optimised for the average operating conditions. The security is still provided such that at the low SNR values the safety specifications are met. By decoupling the security and performance, the system’s average performance increases whilst maintaining the protection of licensed users from harmful interference. The different architectures considered in this thesis are investigated in theory, simulation and physical implementation to provide a complete overview of the performance of each system. This thesis provides a method for estimating SNR distributions which is quick, accurate and relatively low cost. The CSFD is modelled and the characteristic equations are found for the CDFD scheme. The HCFD is introduced and optimisation schemes for all three architectures are proposed. Finally, using the Implementing Radio In Software (IRIS) test-bed to confirm simulation results, CF spectrum sensing is shown to be significantly quicker than naive methods, whilst still meeting the required interference probability rates and not requiring substantial receiver complexity increases.