Computational Tools for Stereo and Light Field Photography

This thesis covers a broad part of the field of computational photography, including video stabilization and image warping techniques, introductions to light field photography and the conversion of monocular images and videos into stereoscopic 3D content. We present a user assisted technique for stereoscopic 3D conversion from 2D images. Our approach exploits the geometric structure of perspective images including vanishing points. We allow a user to indicate lines, planes, and vanishing points in the input image, and directly employ these as guides of an image warp that produces a stereo image pair. Our method is most suitable for scenes with large scale structures such as buildings and is able to skip the step of constructing a depth map. Further, we propose a method to acquire 3D light fields using a hand-held camera, and describe several computational photography applications facilitated by our approach. As the input we take an image sequence from a camera translating along an approximately linear path with limited camera rotations. Users can acquire such data easily in a few seconds by moving a hand-held camera. We convert the input into a regularly sampled 3D light field by resampling and aligning them in the spatio-temporal domain. We also present a novel technique for high-quality disparity estimation from light fields. Finally, we show applications including digital refocusing and synthetic aperture blur, foreground removal, selective colorization, and others.

Estimación de la Reflectividad del Multitrayecto en Entornos Urbanos en Radares de Apertura Sintética

Synthetic Aperture Radar (SAR) images a target region reflectivity function in the multi-dimensional spatial domain of range and cross-range. SAR synthesizes a large aperture radar in order to achieve finer azimuth resolution than the one provided by any on-board real antenna. Conventional SAR techniques assume a single reflection of transmitted waveforms from targets. Nevertheless, today¿s new scenes force SAR systems to work in urban environments. Consequently, multiple-bounce returns are added to direct-scatter echoes. We refer to these as ghost images, since they obscure true target image and lead to poor resolution. By analyzing the quadratic phase error (QPE), this paper demonstrates that Earth¿s curvature influences the defocusing degree of multipath returns. In addition to the QPE, other parameters such as integrated sidelobe ratio (ISLR), peak sidelobe ratio (PSLR), contrast and entropy provide us with the tools to identify direct-scatter echoes in images containing undesired returns coming from multipath.

Técnica de Inversión Temporal para la Mejora del Enfocado de Imágenes SAR en Entornos Urbanos

Conventional SAR (Synthetic Aperture Radar) techniques only consider a single reflection of transmitted waveforms from targets. Nevertheless, today?s new applications force SAR systems to work in much more complex scenes such as urban environments. As a result, multiple-bounce returns are additionally superposed to direct echoes. We refer to these as ghost images, since they obscure true target image and lead to poor resolution. By applying Time Reversal concept to SAR imaging (TR-SAR), it is possible to reduce considerably ?or almost mitigate? ghosting artifacts, recovering the lost resolution due to multipath effects. Furthermore, some focusing indicators such as entropy (E), contrast (C) and Rényi entropy (RE) provide us a good focusing criterion when using TR-SAR.

New Technique for Static Target Detection in Dense-Multipath Urban Environments

Synthetic Aperture Radar (SAR) images a target region reflectivity function in the multi-dimensional spatial domain of range and cross-range with a finer azimuth resolution than the one provided by any on-board real antenna. Conventional SAR techniques assume a single reflection of transmitted waveforms from targets. Nevertheless, new uses of Unmanned Aerial Vehicles (UAVs) for civilian-security applications force SAR systems to work in much more complex scenes such as urban environments. Consequently, multiple-bounce returns are additionally superposed to direct-scatter echoes. They are known as ghost images, since they obscure true target image and lead to poor resolution. All this may involve a significant problem in applications related to surveillance and security. In this work, an innovative multipath mitigation technique is presented in which Time Reversal (TR) concept is applied to SAR images when the target is concealed in clutter, leading to TR-SAR technique. This way, the effect of multipath is considerably reduced ?or even removed?, recovering the lost resolution due to multipath propagation. Furthermore, some focusing indicators such as entropy (E), contrast (C) and Rényi entropy (RE) provide us with a good focusing criterion when using TR-SAR.

Study of the effects of moving targets in SAR images for their detection.

Synthetic Aperture Radar’s (SAR) are systems designed in the early 50’s that are capable of obtaining images of the ground using electromagnetic signals. Thus, its activity is not interrupted by adverse meteorological conditions or during the night, as it occurs in optical systems. The name of the system comes from the creation of a synthetic aperture, larger than the real one, by moving the platform that carries the radar (typically a plane or a satellite). It provides the same resolution as a static radar equipped with a larger antenna. As it moves, the radar keeps emitting pulses every 1/PRF seconds —the PRF is the pulse repetition frequency—, whose echoes are stored and processed to obtain the image of the ground. To carry out this process, the algorithm needs to make the assumption that the targets in the illuminated scene are not moving. If that is the case, the algorithm is able to extract a focused image from the signal. However, if the targets are moving, they get unfocused and/or shifted from their position in the final image. There are applications in which it is especially useful to have information about moving targets (military, rescue tasks,studyoftheflowsofwater,surveillanceofmaritimeroutes...).Thisfeatureiscalled Ground Moving Target Indicator (GMTI). That is why the study and the development of techniques capable of detecting these targets and placing them correctly in the scene is convenient. In this document, some of the principal GMTI algorithms used in SAR systems are detailed. A simulator has been created to test the features of each implemented algorithm on a general situation with moving targets. Finally Monte Carlo tests have been performed, allowing us to extract conclusions and statistics of each algorithm.

Spatiotemporal characteristics of the Huangtupo landslide in the Three Gorges region (China) constrained by radar interferometry

The Huangtupo landslide is one of the largest in the Three Gorges region, China. The county-seat town of Badong, located on the south shore between the Xiling and Wu gorges of the Yangtze River, was moved to this unstable slope prior to the construction of the Three Gorges Project, since the new Three Gorges reservoir completely submerged the location of the old city. The instability of the slope is affecting the new town by causing residential safety problems. The Huangtupo landslide provides scientists an opportunity to understand landslide response to fluctuating river water level and heavy rainfall episodes, which is essential to decide upon appropriate remediation measures. Interferometric Synthetic Aperture Radar (InSAR) techniques provide a very useful tool for the study of superficial and spatially variable displacement phenomena. In this paper, three sets of radar data have been processed to investigate the Huangtupo landslide. Results show that maximum displacements are affecting the northwest zone of the slope corresponding to Riverside slumping mass I#. The other main landslide bodies (i.e. Riverside slumping mass II#, Substation landslide and Garden Spot landslide) exhibit a stable behaviour in agreement with in situ data, although some active areas have been recognized in the foot of the Substation landslide and Garden Spot landslide. InSAR has allowed us to study the kinematic behaviour of the landslide and to identify its active boundaries. Furthermore, the analysis of the InSAR displacement time-series has helped recognize the different displacement patterns on the slope and their relationships with various triggering factors. For those persistent scatterers, which exhibit long-term displacements, they can be decomposed into a creep model (controlled by geological conditions) and a superimposed recoverable term (dependent on external factors), which appears closely correlated with reservoir water level changes close to the river's edge. These results, combined with in situ data, provide a comprehensive analysis of the Huangtupo landslide, which is essential for its management.

Monitoring an earthfill dam using differential SAR interferometry: La Pedrera dam, Alicante, Spain

Surface displacement at the dykes of La Pedrera reservoir (SE Spain) has been measured by satellite differential Synthetic Aperture Radar (SAR) interferometry. At the main dyke, a displacement of about 13 cm along the satellite line of sight has been estimated between August 1995 and May 2010, from a dataset composed by ERS-1, ERS-2 and Envisat-ASAR images. Two independent short-term processing tasks were also carried out with ERS-2/Envisat-ASAR (from June 2008 to May 2010) and TerraSAR-X (from August 2008 to June 2010) images which have shown similar spatial and temporal displacement patterns. The joint analysis of historical instrument surveys and DInSAR-derived data has allowed the identification of a long-term deformation process which is reflected at the dam's surface and is also clearly recognizable in the inspection gallery. The plausible causes of the displacements measured by DInSAR are also discussed in the paper. Finally, DInSAR data have been used to compute the long-term settlement of La Pedrera dam, showing a good agreement with external studies. Consequently, this work demonstrates the integration of DInSAR with in-situ techniques which helps provide a complete spatial vision of the displacements in the dam thereby helping to differentiate the causal mechanisms.

Twenty-year advanced DInSAR analysis of severe land subsidence: The Alto Guadalentín Basin (Spain) case study

A twenty-year period of severe land subsidence evolution in the Alto Guadalentín Basin (southeast Spain) is monitored using multi-sensor SAR images, processed by advanced differential interferometric synthetic aperture radar (DInSAR) techniques. The SAR images used in this study consist of four datasets acquired by ERS-1/2, ENVISAT, ALOS and COSMO-SkyMed satellites between 1992 and 2012. The integration of ground surface displacement maps retrieved for different time periods allows us to quantify up to 2.50 m of cumulated displacements that occurred between 1992 and 2012 in the Alto Guadalentín Basin. DInSAR results were locally compared with global positioning system (GPS) data available for two continuous stations located in the study area, demonstrating the high consistency of local vertical motion measurements between the two different surveying techniques. An average absolute error of 4.6 ± 4 mm for the ALOS data and of 4.8 ± 3.5 mm for the COSMO-SkyMed data confirmed the reliability of the analysis. The spatial analysis of DInSAR ground surface displacement reveals a direct correlation with the thickness of the compressible alluvial deposits. Detected ground subsidence in the past 20 years is most likely a consequence of a 100–200 m groundwater level drop that has persisted since the 1970s due to the overexploitation of the Alto Guadalentín aquifer system. The negative gradient of the pore pressure is responsible for the extremely slow consolidation of a very thick (> 100 m) layer of fine-grained silt and clay layers with low vertical hydraulic permeability (approximately 50 mm/h) wherein the maximum settlement has still not been reached.

Subsidence damage assessment of a Gothic church using differential interferometry and field data

The Santas Justa and Rufina Gothic church (fourteenth century) has suffered several physical, mechanical, chemical, and biochemical types of pathologies along its history: rock alveolization, efflorescence, biological activity, and capillary ascent of groundwater. However, during the last two decades, a new phenomenon has seriously affected the church: ground subsidence caused by aquifer overexploitation. Subsidence is a process that affects the whole Vega Baja of the Segura River basin and consists of gradual sinking in the ground surface caused by soil consolidation due to a pore pressure decrease. This phenomenon has been studied by differential synthetic aperture radar interferometry techniques, which illustrate settlements up to 100 mm for the 1993–2009 period for the whole Orihuela city. Although no differential synthetic aperture radar interferometry information is available for the church due to the loss of interferometric coherence, the spatial analysis of nearby deformation combined with fieldwork has advanced the current understanding on the mechanisms that affect the Santas Justa and Rufina church. These results show the potential interest and the limitations of using this remote sensing technique as a complementary tool for the forensic analysis of building structures.

Magnetoencephalography:technical improvements in image co-registration and studies of visual cortical oscillations

The work presented in this thesis is divided into two distinct sections. In the first, the functional neuroimaging technique of Magnetoencephalography (MEG) is described and a new technique is introduced for accurate combination of MEG and MRI co-ordinate systems. In the second part of this thesis, MEG and the analysis technique of SAM are used to investigate responses of the visual system in the context of functional specialisation within the visual cortex. In chapter one, the sources of MEG signals are described, followed by a brief description of the necessary instrumentation for accurate MEG recordings. This chapter is concluded by introducing the forward and inverse problems of MEG, techniques to solve the inverse problem, and a comparison of MEG with other neuroimaging techniques. Chapter two provides an important contribution to the field of research with MEG. Firstly, it is described how MEG and MRI co-ordinate systems are combined for localisation and visualisation of activated brain regions. A previously used co-registration methods is then described, and a new technique is introduced. In a series of experiments, it is demonstrated that using fixed fiducial points provides a considerable improvement in the accuracy and reliability of co-registration. Chapter three introduces the visual system starting from the retina and ending with the higher visual rates. The functions of the magnocellular and the parvocellular pathways are described and it is shown how the parallel visual pathways remain segregated throughout the visual system. The structural and functional organisation of the visual cortex is then described. Chapter four presents strong evidence in favour of the link between conscious experience and synchronised brain activity. The spatiotemporal responses of the visual cortex are measured in response to specific gratings. It is shown that stimuli that induce visual discomfort and visual illusions share their physical properties with those that induce highly synchronised gamma frequency oscillations in the primary visual cortex. Finally chapter five is concerned with localization of colour in the visual cortex. In this first ever use of Synthetic Aperture Magnetometry to investigate colour processing in the visual cortex, it is shown that in response to isoluminant chromatic gratings, the highest magnitude of cortical activity arise from area V2.

Investigation of the neural correlates of recognition memory using magnetoencephalography

Neuroimaging literature has identified several regions involved in encoding and recognition processes. A review of the literature illustrated considerable variations in the precise location and mechanisms of these processes, and it was these variations that were investigated in the studies in this thesis. Magnetoencephalography (MEG) was used as the neuroimaging tool and a preliminary study identified Synthetic Aperture Magnetometry (SAM) and not a traditional dipole fitting technique, as an appropriate tool for identifying the multiple cortical regions involved in recognition memory. It has been suggested that there is hemispheric asymmetry in encoding and recognition processes. There are two main hypotheses: the first suggesting that there is task-specificity, the second that this specificity is determined by stimulus modality. A series of experiments was completed with two main aims: first to produce consistent and complementary recognition memory data with MEG, and second to determine whether there exists any hemispheric asymmetry in recognition memory. The results obtained from five experiments demonstrated activation of prefrontal and middle temporal structures, which were consistent with those reported in previous neuroimaging studies. It was suggested that this diverse activation may be explained by the involvement of a semantic network during recognition memory processes. In support of this, a subsequent study involving a semantic encoding task demonstrated that category-specific differences in cortical activation also existed in the recognition memory phase. Controlling for the involvement of such semantic processes produced predominantly bilateral activation. It was suggested that the apparent hemispheric asymmetry findings reported in the literature may be due to the 'coarse' temporal analysis available with earlier imaging techniques, which over-simplified the networks reported by being unable to recognise the early complex processes associated with semantic processing which these MEG studies were able to identify. The importance of frequency-specific activations, specifically theta synchronisation and alpha desynchronisation, in memory processes was also investigated.

Effective electromagnetic noise cancellation with beamformers and synthetic gradiometry in shielded and partly shielded environments

The major challenge of MEG, the inverse problem, is to estimate the very weak primary neuronal currents from the measurements of extracranial magnetic fields. The non-uniqueness of this inverse solution is compounded by the fact that MEG signals contain large environmental and physiological noise that further complicates the problem. In this paper, we evaluate the effectiveness of magnetic noise cancellation by synthetic gradiometers and the beamformer analysis method of synthetic aperture magnetometry (SAM) for source localisation in the presence of large stimulus-generated noise. We demonstrate that activation of primary somatosensory cortex can be accurately identified using SAM despite the presence of significant stimulus-related magnetic interference. This interference was generated by a contact heat evoked potential stimulator (CHEPS), recently developed for thermal pain research, but which to date has not been used in a MEG environment. We also show that in a reduced shielding environment the use of higher order synthetic gradiometry is sufficient to obtain signal-to-noise ratios (SNRs) that allow for accurate localisation of cortical sensory function.

Design and implementation of a non-destructive defect detection technique based on UWB-SAR imaging

In the last twenty years aerospace and automotive industries started working widely with composite materials, which are not easy to test using classic Non-Destructive Inspection (NDI) techniques. Pairwise, the development of safety regulations sets higher and higher standards for the qualification and certification of those materials. In this thesis a new concept of a Non-Destructive defect detection technique is proposed, based on Ultrawide-Band (UWB) Synthetic Aperture Radar (SAR) imaging. Similar SAR methods are yet applied either in minefield [22] and head stroke [14] detection. Moreover feasibility studies have already demonstrated the validity of defect detection by means of UWB radars [12, 13]. The system was designed using a cheap commercial off-the-shelf radar device by Novelda and several tests of the developed system have been performed both on metallic specimen (aluminum plate) and on composite coupon (carbon fiber). The obtained results confirm the feasibility of the method and highlight the good performance of the developed system considered the radar resolution. In particular, the system is capable of discerning healthy coupons from damaged ones, and correctly reconstruct the reflectivity image of the tested defects, namely a 8 x 8 mm square bulge and a 5 mm drilled holes on metal specimen and a 5 mm drilled hole on composite coupon.

Tile-Based Two-Dimensional Phase Unwrapping for Digital Holography Using a Modular Framework

A variety of physical and biomedical imaging techniques, such as digital holography, interferometric synthetic aperture radar (InSAR), or magnetic resonance imaging (MRI) enable measurement of the phase of a physical quantity additionally to its amplitude. However, the phase can commonly only be measured modulo 2π, as a so called wrapped phase map. Phase unwrapping is the process of obtaining the underlying physical phase map from the wrapped phase. Tile-based phase unwrapping algorithms operate by first tessellating the phase map, then unwrapping individual tiles, and finally merging them to a continuous phase map. They can be implemented computationally efficiently and are robust to noise. However, they are prone to failure in the presence of phase residues or erroneous unwraps of single tiles. We tried to overcome these shortcomings by creating novel tile unwrapping and merging algorithms as well as creating a framework that allows to combine them in modular fashion. To increase the robustness of the tile unwrapping step, we implemented a model-based algorithm that makes efficient use of linear algebra to unwrap individual tiles. Furthermore, we adapted an established pixel-based unwrapping algorithm to create a quality guided tile merger. These original algorithms as well as previously existing ones were implemented in a modular phase unwrapping C++ framework. By examining different combinations of unwrapping and merging algorithms we compared our method to existing approaches. We could show that the appropriate choice of unwrapping and merging algorithms can significantly improve the unwrapped result in the presence of phase residues and noise. Beyond that, our modular framework allows for efficient design and test of new tile-based phase unwrapping algorithms. The software developed in this study is freely available.

Landscape Patterns of Permafrost Disturbance and Degradation in the Canadian High Arctic

Intensification of permafrost disturbances such as active layer detachments (ALDs) and retrogressive thaw slumps (RTS) have been observed across the circumpolar Arctic. These features are indicators of unstable conditions stemming from recent climate warming and permafrost degradation. In order to understand the processes interacting to give rise to these features, a multidisciplinary approach is required; i.e., interactions between geomorphology, hydrology, vegetation and ground thermal conditions. The goal of this research is to detect and map permafrost disturbance, predict landscape controls over disturbance and determine approaches for monitoring disturbance, all with the goal of contributing to the mitigation of permafrost hazards. Permafrost disturbance inventories were created by applying semi-automatic change detection techniques to IKONOS satellite imagery collected at the Cape Bounty Arctic Watershed Observatory (CBAWO). These methods provide a means to estimate the spatial distribution of permafrost disturbances for a given area for use as an input in susceptibility modelling. Permafrost disturbance susceptibility models were then developed using generalized additive and generalized linear models (GAM, GLM) fitted to disturbed and undisturbed locations and relevant GIS-derived predictor variables (slope, potential solar radiation, elevation). These models successfully delineated areas across the landscape that were susceptible to disturbances locally and regionally when transferred to an independent validation location. Permafrost disturbance susceptibility models are a first-order assessment of landscape susceptibility and are promising for designing land management strategies for remote permafrost regions. Additionally, geomorphic patterns associated with higher susceptibility provide important knowledge about processes associated with the initiation of disturbances. Permafrost degradation was analyzed at the CBAWO using differential interferometric synthetic aperture radar (DInSAR). Active-layer dynamics were interpreted using inter-seasonal and intra-seasonal displacement measurements and highlight the importance of hydroclimatic factors on active layer change. Collectively, these research approaches contribute to permafrost monitoring and the assessment of landscape-scale vulnerability in order to develop permafrost disturbance mitigation strategies.