Neurogeometry of stereo vision

This work aims to develop a neurogeometric model of stereo vision, based on cortical architectures involved in the problem of 3D perception and neural mechanisms generated by retinal disparities. First, we provide a sub-Riemannian geometry for stereo vision, inspired by the work on the stereo problem by Zucker (2006), and using sub-Riemannian tools introduced by Citti-Sarti (2006) for monocular vision. We present a mathematical interpretation of the neural mechanisms underlying the behavior of binocular cells, that integrate monocular inputs. The natural compatibility between stereo geometry and neurophysiological models shows that these binocular cells are sensitive to position and orientation. Therefore, we model their action in the space R3xS2 equipped with a sub-Riemannian metric. Integral curves of the sub-Riemannian structure model neural connectivity and can be related to the 3D analog of the psychophysical association fields for the 3D process of regular contour formation. Then, we identify 3D perceptual units in the visual scene: they emerge as a consequence of the random cortico-cortical connection of binocular cells. Considering an opportune stochastic version of the integral curves, we generate a family of kernels. These kernels represent the probability of interaction between binocular cells, and they are implemented as facilitation patterns to define the evolution in time of neural population activity at a point. This activity is usually modeled through a mean field equation: steady stable solutions lead to consider the associated eigenvalue problem. We show that three-dimensional perceptual units naturally arise from the discrete version of the eigenvalue problem associated to the integro-differential equation of the population activity.

First steps in underwater photogrammetry with the support of an unmanned underwater vehicle

The evolution of modern and increasingly sensitive image sensors, the increasingly compact design of the cameras, and the recent emergence of low-cost cameras allowed the Underwater Photogrammetry to become an infallible and irreplaceable technique used to estimate the structure of the seabed with high accuracy. Within this context, the main topic of this work is the Underwater Photogrammetry from a geomatic point of view and all the issues associated with its implementation, in particular with the support of Unmanned Underwater Vehicles. Questions such as: how does the technique work, what is needed to deal with a proper survey, what tools are available to apply this technique, and how to resolve uncertainties in measurement will be the subject of this thesis. The study conducted can be divided into two major parts: one devoted to several ad-hoc surveys and tests, thus a practical part, another supported by the bibliographical research. However the main contributions are related to the experimental section, in which two practical case studies are carried out in order to improve the quality of the underwater survey of some calibration platforms. The results obtained from these two experiments showed that, the refractive effects due to water and underwater housing can be compensated by the distortion coefficients in the camera model, but if the aim is to achieve high accuracy then a model that takes into account the configuration of the underwater housing, based on ray tracing, must also be coupled. The major contributions that this work brought are: an overview of the practical issues when performing surveys exploiting an UUV prototype, a method to reach a reliable accuracy in the 3D reconstructions without the use of an underwater local geodetic network, a guide for who addresses underwater photogrammetry topics for the first time, and the use of open-source environments.

Metodologie di Corrispondenza Stereo Basate su Deep Learning per Superfici Altamente Riflettenti e Trasparenti: Dataset e Architettura

Nell’ambito della Stereo Vision, settore della Computer Vision, partendo da coppie di immagini RGB, si cerca di ricostruire la profondità della scena. La maggior parte degli algoritmi utilizzati per questo compito ipotizzano che tutte le superfici presenti nella scena siano lambertiane. Quando sono presenti superfici non lambertiane (riflettenti o trasparenti), gli algoritmi stereo esistenti sbagliano la predizione della profondità. Per risolvere questo problema, durante l’esperienza di tirocinio, si è realizzato un dataset contenente oggetti trasparenti e riflettenti che sono la base per l’allenamento della rete. Agli oggetti presenti nelle scene sono associate annotazioni 3D usate per allenare la rete. Invece, nel seguente lavoro di tesi, utilizzando l’algoritmo RAFT-Stereo [1], rete allo stato dell’arte per la stereo vision, si analizza come la rete modifica le sue prestazioni (predizione della disparità) se al suo interno viene inserito un modulo per la segmentazione semantica degli oggetti. Si introduce questo layer aggiuntivo perché, trovare la corrispondenza tra due punti appartenenti a superfici lambertiane, risulta essere molto complesso per una normale rete. Si vuole utilizzare l’informazione semantica per riconoscere questi tipi di superfici e così migliorarne la disparità. È stata scelta questa architettura neurale in quanto, durante l’esperienza di tirocinio riguardante la creazione del dataset Booster [2], è risultata la migliore su questo dataset. L’obiettivo ultimo di questo lavoro è vedere se il riconoscimento di superfici non lambertiane, da parte del modulo semantico, influenza la predizione della disparità migliorandola. Nell’ambito della stereo vision, gli elementi riflettenti e trasparenti risultano estremamente complessi da analizzare, ma restano tuttora oggetto di studio dati gli svariati settori di applicazione come la guida autonoma e la robotica.

Depth Estimation in Stereo Biomedical Images via Proxy-Supervised Deep Learning

In order to estimate depth through supervised deep learning-based stereo methods, it is necessary to have access to precise ground truth depth data. While the gathering of precise labels is commonly tackled by deploying depth sensors, this is not always a viable solution. For instance, in many applications in the biomedical domain, the choice of sensors capable of sensing depth at small distances with high precision on difficult surfaces (that present non-Lambertian properties) is very limited. It is therefore necessary to find alternative techniques to gather ground truth data without having to rely on external sensors. In this thesis, two different approaches have been tested to produce supervision data for biomedical images. The first aims to obtain input stereo image pairs and disparities through simulation in a virtual environment, while the second relies on a non-learned disparity estimation algorithm in order to produce noisy disparities, which are then filtered by means of hand-crafted confidence measures to create noisy labels for a subset of pixels. Among the two, the second approach, which is referred in literature as proxy-labeling, has shown the best results and has even outperformed the non-learned disparity estimation algorithm used for supervision.

Mitigating non-Lambertian surfaces issues in Stereo Matching with Neural Radiance Fields

Depth estimation from images has long been regarded as a preferable alternative compared to expensive and intrusive active sensors, such as LiDAR and ToF. The topic has attracted the attention of an increasingly wide audience thanks to the great amount of application domains, such as autonomous driving, robotic navigation and 3D reconstruction. Among the various techniques employed for depth estimation, stereo matching is one of the most widespread, owing to its robustness, speed and simplicity in setup. Recent developments has been aided by the abundance of annotated stereo images, which granted to deep learning the opportunity to thrive in a research area where deep networks can reach state-of-the-art sub-pixel precision in most cases. Despite the recent findings, stereo matching still begets many open challenges, two among them being finding pixel correspondences in presence of objects that exhibits a non-Lambertian behaviour and processing high-resolution images. Recently, a novel dataset named Booster, which contains high-resolution stereo pairs featuring a large collection of labeled non-Lambertian objects, has been released. The work shown that training state-of-the-art deep neural network on such data improves the generalization capabilities of these networks also in presence of non-Lambertian surfaces. Regardless being a further step to tackle the aforementioned challenge, Booster includes a rather small number of annotated images, and thus cannot satisfy the intensive training requirements of deep learning. This thesis work aims to investigate novel view synthesis techniques to augment the Booster dataset, with ultimate goal of improving stereo matching reliability in presence of high-resolution images that displays non-Lambertian surfaces.

Stereo matching su GPU

La Stereo Vision è un popolare argomento di ricerca nel campo della Visione Artificiale; esso consiste nell’usare due immagini di una stessa scena,prodotte da due fotocamere diverse, per estrarre informazioni in 3D. L’idea di base della Stereo Vision è la simulazione della visione binoculare umana:le due fotocamere sono disposte in orizzontale per fungere da “occhi” che guardano la scena in 3D. Confrontando le due immagini ottenute, si possono ottenere informazioni riguardo alle posizioni degli oggetti della scena.In questa relazione presenteremo un algoritmo di Stereo Vision: si tratta di un algoritmo parallelo che ha come obiettivo di tracciare le linee di livello di un area geografica. L’algoritmo in origine era stato implementato per la Connection Machine CM-2, un supercomputer sviluppato negli anni 80, ed era espresso in *Lisp, un linguaggio derivato dal Lisp e ideato per la macchina stessa. Questa relazione tratta anche la traduzione e l’implementazione dell’algoritmo in CUDA, ovvero un’architettura hardware per l’elaborazione pa- rallela sviluppata da NVIDIA, che consente di eseguire codice parallelo su GPU. Si darà inoltre uno sguardo alle difficoltà che sono state riscontrate nella traduzione da *Lisp a CUDA.

Assessing the relationship between bee/flower diversity and vegetation structural heterogeneity through UAV photogrammetry

The ecosystem services provided by bees are very important. Factors as habitat fragmentation, intensive agriculture and climate change are contributing to the decline of bee populations. The use of remote sensing could be a useful tool for the recognition of sites with a high diversity, before performing a more expensive survey in the field. In this study the ability of Unmanned Aerial Vehicles (UAV) images to estimate biodiversity at local scale has been analysed testing the concept of the Height Variation Hypothesis (HVH). This approach states that, the higher the vegetation height heterogeneity (HH) measured by remote sensing information, the higher the vertical complexity and the higher vegetation species diversity. In this thesis the concept has been brought to a higher level, in order to understand if the vegetation HH can be considered a proxy also of bee species diversity and abundance. We tested this approach collecting field data on bees/flowers and RGB images through an UAV campaign in 30 grasslands in the South of the Netherlands. The Canopy Height Model (CHM) were derived through the photogrammetry technique "Structure from Motion" (SfM) with resolutions of 10cm, 25cm, 50cm. Successively, the HH assessed on the CHM using the Rao's Q heterogeneity index was correlated to the field data (bee abundance, diversity and bee/flower species richness). The correlations were all positive and significant. The highest R2 values were found when the HH was calculated at 10cm and correlated to bee species richness (R2 = 0.41) and Shannon’s H index (R2 = 0.38). Using a lower spatial resolution the goodness of fit slightly decreases. For flower species richness the R2 ranged between 0.36 to 0.39. Our results suggest that methods based on the concept behind the HVH, in this case deriving information of HH from UAV data, can be developed into valuable tools for large-scale, standardized and cost-effective monitoring of flower diversity and of the habitat quality for bees.

The Geometric Curvature Of The Spine Of Runners During Maximal Incremental Effort Test.

This study sought to analyse the behaviour of the average spinal posture using a novel investigative procedure in a maximal incremental effort test performed on a treadmill. Spine motion was collected via stereo-photogrammetric analysis in thirteen amateur athletes. At each time percentage of the gait cycle, the reconstructed spine points were projected onto the sagittal and frontal planes of the trunk. On each plane, a polynomial was fitted to the data, and the two-dimensional geometric curvature along the longitudinal axis of the trunk was calculated to quantify the geometric shape of the spine. The average posture presented at the gait cycle defined the spine Neutral Curve. This method enabled the lateral deviations, lordosis, and kyphosis of the spine to be quantified noninvasively and in detail. The similarity between each two volunteers was a maximum of 19% on the sagittal plane and 13% on the frontal (p<0.01). The data collected in this study can be considered preliminary evidence that there are subject-specific characteristics in spinal curvatures during running. Changes induced by increases in speed were not sufficient for the Neutral Curve to lose its individual characteristics, instead behaving like a postural signature. The data showed the descriptive capability of a new method to analyse spinal postures during locomotion; however, additional studies, and with larger sample sizes, are necessary for extracting more general information from this novel methodology.

Remotely sensed estimates of vegetation structural characteristics in restored wetlands, Southern California

Traditional field sampling approaches for ecological studies of restored habitat can only cover small areas in detail, con be time consuming, and are often invasive and destructive. Spatially extensive and non-invasive remotely sensed data can make field sampling more focused and efficient. The objective of this work was to investigate the feasibility and accuracy of hand-held and airborne remotely sensed data to estimate vegetation structural parameters for an indicator plant species in a restored wetland. High spatial resolution, digital, multispectral camera images were captured from an aircraft over Sweetwater Marsh (San Diego County, California) during each growing season between 1992-1996. Field data were collected concurrently, which included plant heights, proportional ground cover and canopy architecture type, and spectral radiometer measurements. Spartina foliosa (Pacific cordgrass) is the indicator species for the restoration monitoring. A conceptual model summarizing the controls on the spectral reflectance properties of Pacific cordgrass was established. Empirical models were developed relating the stem length, density, and canopy architecture of cordgrass to normalized-difference-vegetation-index values. The most promising results were obtained from empirical estimates of total ground cover using image data that had been stratified into high, middle, and low marsh zones. As part of on-going restoration monitoring activities, this model is being used to provide maps of estimated vegetation cover.

Determining forest structural attributes using an inverted geometric-optical model in mixed eucalypt forests, Southeast Queensland, Australia

The Montreal Process indicators are intended to provide a common framework for assessing and reviewing progress toward sustainable forest management. The potential of a combined geometrical-optical/spectral mixture analysis model was assessed for mapping the Montreal Process age class and successional age indicators at a regional scale using Landsat Thematic data. The project location is an area of eucalyptus forest in Emu Creek State Forest, Southeast Queensland, Australia. A quantitative model relating the spectral reflectance of a forest to the illumination geometry, slope, and aspect of the terrain surface and the size, shape, and density, and canopy size. Inversion of this model necessitated the use of spectral mixture analysis to recover subpixel information on the fractional extent of ground scene elements (such as sunlit canopy, shaded canopy, sunlit background, and shaded background). Results obtained fron a sensitivity analysis allowed improved allocation of resources to maximize the predictive accuracy of the model. It was found that modeled estimates of crown cover projection, canopy size, and tree densities had significant agreement with field and air photo-interpreted estimates. However, the accuracy of the successional stage classification was limited. The results obtained highlight the potential for future integration of high and moderate spatial resolution-imaging sensors for monitoring forest structure and condition. (C) Elsevier Science Inc., 2000.

Monitoring growth in rapidly urbanizing areas using remotely sensed data

Urbanization and the ability to manage for a sustainable future present numerous challenges for geographers and planners in metropolitan regions. Remotely sensed data are inherently suited to provide information on urban land cover characteristics, and their change over time, at various spatial and temporal scales. Data models for establishing the range of urban land cover types and their biophysical composition (vegetation, soil, and impervious surfaces) are integrated to provide a hierarchical approach to classifying land cover within urban environments. These data also provide an essential component for current simulation models of urban growth patterns, as both calibration and validation data. The first stages of the approach have been applied to examine urban growth between 1988 and 1995 for a rapidly developing area in southeast Queensland, Australia. Landsat Thematic Mapper image data provided accurate (83% adjusted overall accuracy) classification of broad land cover types and their change over time. The combination of commonly available remotely sensed data, image processing methods, and emerging urban growth models highlights an important application for current and next generation moderate spatial resolution image data in studies of urban environments.

Molecular diversity through sugar scaffolds

Monosaccharides provide an excellent platform to tailor molecular diversity by appending desired substituents at selected positions around the sugar scaffold. The presence of five functionalized and stereo-controlled centres on the sugar scaffolds gives the chemist plenty of scope to custom design molecules to a pharmacophore model. This review focuses on the peptidomimetic developments in this area, as well as the concept of tailoring structural and functional diversity in a library using carbohydrate scaffolds and how this can lead to increased hit rates and rapid identification of leads, which has promising prospects for drug development.

Spatial and temporal variation in distribution of mangroves in Moreton Bay, subtropical Australia: a comparison of pattern metrics and change detection analyses based on aerial photographs

An assessment of the changes in the distribution and extent of mangroves within Moreton Bay, southeast Queensland, Australia, was carried out. Two assessment methods were evaluated: spatial and temporal pattern metrics analysis, and change detection analysis. Currently, about 15,000 ha of mangroves are present in Moreton Bay. These mangroves are important ecosystems, but are subject to disturbance from a number of sources. Over the past 25 years, there has been a loss of more than 3800 ha, as a result of natural losses and mangrove clearing (e.g. for urban and industrial development, agriculture and aquaculture). However, areas of new mangroves have become established over the same time period, offsetting these losses to create a net loss of about 200 ha. These new mangroves have mainly appeared in the southern bay region and the bay islands, particularly on the landward edge of existing mangroves. In addition, spatial patterns and species composition of mangrove patches have changed. The pattern metrics analysis provided an overview of mangrove distribution and change in the form of single metric values, while the change detection analysis gave a more detailed and spatially explicit description of change. An analysis of the effects of spatial scales on the pattern metrics indicated that they were relatively insensitive to scale at spatial resolutions less than 50 m, but that most metrics became sensitive at coarser resolutions, a finding which has implications for mapping of mangroves based on remotely sensed data. (C) 2003 Elsevier Science B.V. All rights reserved.

Detecting sugarcane 'orange rust' disease using EO-1 Hyperion hyperspectral imagery

This Letter evaluates several narrow-band indices from EO-1 Hyperion imagery in discriminating sugarcane areas affected by 'orange rust' ( Puccinia kuehnii ) disease. Forty spectral vegetation indices (SVIs), focusing on bands related to leaf pigments, leaf internal structure, and leaf water content, were generated from an image acquired over Mackay, Queensland, Australia. Discriminant function analysis was used to select an optimum set of indices based on their correlations with the discriminant function. The predictive ability of each index was also assessed based on the accuracy of classification. Results demonstrated that Hyperion imagery can be used to detect orange rust disease in sugarcane crops. While some indices that only used visible near-infrared (VNIR) bands (e.g. SIPI and R800/R680) offer separability, the combination of VNIR bands with the moisture-sensitive band (1660 nm) yielded increased separability of rust-affected areas. The newly formulated 'Disease-Water Stress Indices' (DWSI-1=R800/R1660; DSWI-2=R1660/R550; DWSI-5=(R800+R550)/(R1660+R680)) produced the largest correlations, indicating their superior ability to discriminate sugarcane areas affected by orange rust disease.