Interactive Augmentation of Live Images using a HDR Stereo Camera

Adding virtual objects to real environments plays an important role in todays computer graphics: Typical examples are virtual furniture in a real room and virtual characters in real movies. For a believable appearance, consistent lighting of the virtual objects is required. We present an augmented reality system that displays virtual objects with consistent illumination and shadows in the image of a simple webcam. We use two high dynamic range video cameras with fisheye lenses permanently recording the environment illumination. A sampling algorithm selects a few bright parts in one of the wide angle images and the corresponding points in the second camera image. The 3D position can then be calculated using epipolar geometry. Finally, the selected point lights are used in a multi pass algorithm to draw the virtual object with shadows. To validate our approach, we compare the appearance and shadows of the synthetic objects with real objects.

Sharpness Matching in Stereo Images

When stereo images are captured under less than ideal conditions, there may be inconsistencies between the two images in brightness, contrast, blurring, etc. When stereo matching is performed between the images, these variations can greatly reduce the quality of the resulting depth map. In this paper we propose a method for correcting sharpness variations in stereo image pairs which is performed as a pre-processing step to stereo matching. Our method is based on scaling the 2D discrete cosine transform (DCT) coefficients of both images so that the two images have the same amount of energy in each of a set of frequency bands. Experiments show that applying the proposed correction method can greatly improve the disparity map quality when one image in a stereo pair is more blurred than the other.

A Video Database for the Development of Stereo-3D Post-Production Algorithms

This paper introduces a database of freely available stereo-3D content designed to facilitate research in stereo post-production. It describes the structure and content of the database and provides some details about how the material was gathered. The database includes examples of many of the scenarios characteristic to broadcast footage. Material was gathered at different locations including a studio with controlled lighting and both indoor and outdoor on-location sites with more restricted lighting control. The database also includes video sequences with accompanying 3D audio data recorded in an Ambisonics format. An intended consequence of gathering the material is that the database contains examples of degradations that would be commonly present in real-world scenarios. This paper describes one such artefact caused by uneven exposure in the stereo views, causing saturation in the over-exposed view. An algorithm for the restoration of this artefact is proposed in order to highlight the usefuiness of the database.

Photometric properties of Mars soils analogs

We have measured the bidirectional reflectance of analogs of dry, wet, and frozen Martian soils over a wide range of phase angles in the visible spectral range. All samples were produced from two geologic samples: the standard JSC Mars-1 soil simulant and Hawaiian basaltic sand. In a first step, experiments were conducted with the dry samples to investigate the effects of surface texture. Comparisons with results independently obtained by different teams with similar samples showed a satisfying reproducibility of the photometric measurements as well as a noticeable influence of surface textures resulting from different sample preparation procedures. In a second step, water was introduced to produce wet and frozen samples and their photometry investigated. Optical microscope images of the samples provided information about their microtexture. Liquid water, even in relatively low amount, resulted in the disappearance of the backscattering peak and the appearance of a forward-scattering peak whose intensity increases with the amount of water. Specular reflections only appeared when water was present in an amount large enough to allow water to form a film at the surface of the sample. Icy samples showed a wide variability of photometric properties depending on the physical properties of the water ice. We discuss the implications of these measurements in terms of the expected photometric behavior of the Martian surface, from equatorial to circum-polar regions. In particular, we propose some simple photometric criteria to improve the identification of wet and/or icy soils from multiple observations under different geometries.

Observations of the northern seasonal polar cap on Mars II: HiRISE photometric analysis of evolution of northern polar dunes in spring

We present an overview of our analyses of HiRISE observations of spring evolution of selected dune areas of the north polar erg. The north polar erg is covered annually by seasonal volatile ice layer, a mixture of CO2 and H2O with mineral dust contamination. In spring, this layer sublimes creating visually enigmatic phenomena, e.g. dark and bright fan-shaped deposits, dark–bright–dark bandings, dark down-slope streaks, and seasonal polygonal cracks. Similar phenomena in southern polar areas are believed to be related to the specific process of solid-state greenhouse effect. In the north, it is currently unclear if the solid-state greenhouse effect is able to explain all the observed phenomena especially because the increased influence of H2O on the time scales of this process has not yet been quantified. HiRISE observations of our selected locations show that the ground exhibits a temporal behaviour similar to the one observed in the southern polar areas: a brightening phase starting close to the spring equinox with a subsequent darkening towards summer solstice. The resolution of HiRISE enabled us to study dunes and substrate individually and even distinguish between different developments on windward and slip face sides of single dunes. Differences in the seasonal evolution between steep slip faces and flatter substrate and windward sides of dunes have been identified and compared to CRISM data of CO2 and H2O distributions on dunes. We also observe small scale dark blotches that appear in early observations and tend to sustain a low reflectivity throughout the spring. These blotches can be regarded as the analogue of dark fan deposits in southern polar areas, leading us to the conclusion that both martian polar areas follow similar spring evolutions.

Rapid Volume Loss from Two East Greenland Outlet Glaciers Quantified Using Repeat Stereo Satellite Imagery

The coastal portions of Kangerdlugssuaq and Helheim glaciers in southeast Greenland lost at least 51 +/- 8 km(-3) yr(-1) of ice between 2001-2006 due to thinning and retreat, according to an analysis of sequential digital elevation models (DEMs) derived from stereo ASTER satellite imagery. The dominant contribution to this ice loss was dynamic thinning caused by the acceleration in flow of both glaciers. Peak rates of change, including thinning rates of similar to 90 m yr(-1), coincided with the rapid increases in flow speed. Extrapolation of the measured data to the ice divides yields an estimated combined catchment volume loss of similar to 122 +/- 30 km(-3) yr(-1), which accounts for half the total mass loss from the ice sheet reported in recent studies. These catchment-wide volume losses contributed similar to 0.31 +/- 0.07 mm yr(-1) to global sea level rise over the 5-year observation period with the coastal regions alone contributing at least 0.1 +/- 0.02 mm yr(-1).

Optimizing stereo-to-multiview conversion for autostereoscopic displays

We present a novel stereo-to-multiview video conversion method for glasses-free multiview displays. Different from previous stereo-to-multiview approaches, our mapping algorithm utilizes the limited depth range of autostereoscopic displays optimally and strives to preserve the scene's artistic composition and perceived depth even under strong depth compression. We first present an investigation of how perceived image quality relates to spatial frequency and disparity. The outcome of this study is utilized in a two-step mapping algorithm, where we (i) compress the scene depth using a non-linear global function to the depth range of an autostereoscopic display and (ii) enhance the depth gradients of salient objects to restore the perceived depth and salient scene structure. Finally, an adapted image domain warping algorithm is proposed to generate the multiview output, which enables overall disparity range extension.

Computational Tools for Stereo and Light Field Photography

This thesis covers a broad part of the field of computational photography, including video stabilization and image warping techniques, introductions to light field photography and the conversion of monocular images and videos into stereoscopic 3D content. We present a user assisted technique for stereoscopic 3D conversion from 2D images. Our approach exploits the geometric structure of perspective images including vanishing points. We allow a user to indicate lines, planes, and vanishing points in the input image, and directly employ these as guides of an image warp that produces a stereo image pair. Our method is most suitable for scenes with large scale structures such as buildings and is able to skip the step of constructing a depth map. Further, we propose a method to acquire 3D light fields using a hand-held camera, and describe several computational photography applications facilitated by our approach. As the input we take an image sequence from a camera translating along an approximately linear path with limited camera rotations. Users can acquire such data easily in a few seconds by moving a hand-held camera. We convert the input into a regularly sampled 3D light field by resampling and aligning them in the spatio-temporal domain. We also present a novel technique for high-quality disparity estimation from light fields. Finally, we show applications including digital refocusing and synthetic aperture blur, foreground removal, selective colorization, and others.

Photometric monitoring of non-resolved space debris and databases of optical light curves.

The population of space debris increased drastically during the last years. Collisions involving massive objects may produce large number of fragments leading to significantly growth of the space debris population. An effective remediation measure in order to stabilize the population in LEO, is therefore the removal of large, massive space debris. To remove these objects, not only precise orbits, but also more detailed information about their attitude states will be required. One important property of an object targeted for removal is its spin period and spin axis orientation. If we observe a rotating object, the observer sees different surface areas of the object which leads to changes in the measured intensity. Rotating objects will produce periodic brightness vari ations with frequencies which are related to the spin periods. Photometric monitoring is the real tool for remote diagnostics of the satellite rotation around its center of mass. This information is also useful, for example, in case of contingency. Moreover, it is also important to take into account the orientation of non-spherical body (e.g. space debris) in the numerical integration of its motion when a close approach with the another spacecr aft is predicted. We introduce the two databases of light curves: the AIUB data base, which contains about a thousand light curves of LEO, MEO and high-altitude debris objects (including a few functional objects) obtained over more than seven years, and the data base of the Astronomical Observatory of Odessa University (Ukraine), which contains the results of more than 10 years of photometric monitoring of functioning satellites and large space debris objects in low Earth orbit. AIUB used its 1m ZIMLAT telescope for all light curves. For tracking low-orbit satellites, the Astronomical Observatory of Odessa used the KT-50 telescope, which has an alt-azimuth mount and allows tracking objects moving at a high angular velocity. The diameter of the KT-50 main mirror is 0.5 m, and the focal length is 3 m. The Odessa's Atlas of light curves includes almost 5,5 thousand light curves for ~500 correlated objects from a time period of 2005-2014. The processing of light curves and the determination of the rotation period in the inertial frame is challenging. Extracted frequencies and reconstructed phases for some interesting targets, e.g. GLONASS satellites, for which also SLR data were available for confirmation, will be presented. The rotation of the Envisat satellite after its sudden failure will be analyzed. The deceleration of its rotation rate within 3 years is studied together with the attempt to determine the orientation of the rotation axis.

Sublimation of ice–tholins mixtures: A morphological and spectro-photometric study

Sublimation, the direct transition from solid to gas phase, is a process responsible for shaping and changing the reflectance properties of many Solar System surfaces. In this study, we have characterized the evolution of the structure/texture and of the visible and near-infrared (VIS–NIR) spectral reflectance of surfaces made of water ice mixed with analogues of complex extraterrestrial organic matter, named tholins, under low temperature (<-70° C) and pressure (10-⁵mbar) conditions. The experiments were carried out in the SCITEAS simulation setup recently built as part of the Laboratory for Outflow Studies of Sublimating Materials (LOSSy) at the University of Bern (Pommerol, A. et al. [2015a]. Planet. Space Sci. 109–110, 106–122). As the water ice sublimated, we observed in situ the formation of a sublimation lag deposit made of a water-free porous (>90% porosity) network of organic filaments on top of the ice. The temporal evolution of the tholins and water ice spectral features (reflectance at the absorption bands wavelengths, red slope, from 0.40 to 1.90lm) are analyzed throughout the sublimation of the samples. We studied how different mixtures of tholins with water (0.1 wt.% tholins as coating or inclusions within the water particles), and different ice particle sizes (4.5 ± 2.5 or 67 ± 31lm) influence the morphological and spectral evolutions of the samples. The sublimation of the ice below the mantle produces a gas flow responsible for the ejection of mm to cm-sized fragments of the deposit in outbursts-like events. The results show remarkable differences between these samples in term of mantle structure, speed of mantle building, rates and surface area of mantle ejections. These data provide useful references for interpreting remote-sensing observations of icy Solar System surfaces, in particular the activity of comet nuclei where sublimation of organic-rich ices and deposition of organic-dust particles likely play a major role. Consequently, the data presented here could be of high interest for the interpretation of Rosetta, and also New Horizons, observations.

Geomorphology and spectrophotometry of Philae's landing site on comet 67P/Churyumov-Gerasimenko

Context. On 12 November 2014 the European mission Rosetta succeeded in delivering a lander, named Philae, on the surface of one of the smallest, low-gravity and most primitive bodies of the solar system, the comet 67P/Churyumov-Gerasimenko (67P). Aims. The aim of this paper is to provide a comprehensive geomorphological and spectrophotometric analysis of Philae's landing site (Agilkia) to give an essential framework for the interpretation of its in situ measurements. Methods. OSIRIS images, coupled with gravitational slopes derived from the 3D shape model based on stereo-photogrammetry were used to interpret the geomorphology of the site. We adopted the Hapke model, using previously derived parameters, to photometrically correct the images in orange filter (649.2 nm). The best approximation to the Hapke model, given by the Akimov parameter-less function, was used to correct the reflectance for the effects of viewing and illumination conditions in the other filters. Spectral analyses on coregistered color cubes were used to retrieve spectrophotometric properties. Results. The landing site shows an average normal albedo of 6.7% in the orange filter with variations of similar to 15% and a global featureless spectrum with an average red spectral slope of 15.2%/100 nm between 480.7 nm (blue filter) and 882.1 nm (near-IR filter). The spatial analysis shows a well-established correlation between the geomorphological units and the photometric characteristics of the surface. In particular, smooth deposits have the highest reflectance a bluer spectrum than the outcropping material across the area. Conclusions. The featureless spectrum and the redness of the material are compatible with the results by other instruments that have suggested an organic composition. The observed small spectral variegation could be due to grain size effects. However, the combination of photometric and spectral variegation suggests that a compositional differentiation is more likely. This might be tentatively interpreted as the effect of the efficient dust-transport processes acting on 67P. High-activity regions might be the original sources for smooth fine-grained materials that then covered Agilkia as a consequence of airfall of residual material. More observations performed by OSIRIS as the comet approaches the Sun would help interpreting the processes that work at shaping the landing site and the overall nucleus.

Shape model, reference system definition, and cartographic mapping standards for comet 67P/Churyumov-Gerasimenko Stereo-photogrammetric analysis of Rosetta/OSIRIS image data

We analyzed more than 200 OSIRIS NAC images with a pixel scale of 0.9-2.4 m/pixel of comet 67P/Churyumov-Gerasimenko (67P) that have been acquired from onboard the Rosetta spacecraft in August and September 2014 using stereo-photogrammetric methods (SPG). We derived improved spacecraft position and pointing data for the OSIRIS images and a high-resolution shape model that consists of about 16 million facets (2 m horizontal sampling) and a typical vertical accuracy at the decimeter scale. From this model, we derive a volume for the northern hemisphere of 9.35 km(3) +/- 0.1 km(3). With the assumption of a homogeneous density distribution and taking into account the current uncertainty of the position of the comet's center-of-mass, we extrapolated this value to an overall volume of 18.7 km(3) +/- 1.2 km(3), and, with a current best estimate of 1.0 X 10(13) kg for the mass, we derive a bulk density of 535 kg/m(3) +/- 35 kg/m(3). Furthermore, we used SPG methods to analyze the rotational elements of 67P. The rotational period for August and September 2014 was determined to be 12.4041 +/- 0.0004 h. For the orientation of the rotational axis (z-axis of the body-fixed reference frame) we derived a precession model with a half-cone angle of 0.14 degrees, a cone center position at 69.54 degrees/64.11 degrees (RA/Dec J2000 equatorial coordinates), and a precession period of 10.7 days. For the definition of zero longitude (x-axis orientation), we finally selected the boulder-like Cheops feature on the big lobe of 67P and fixed its spherical coordinates to 142.35 degrees right-hand-rule eastern longitude and -0.28 degrees latitude. This completes the definition of the new Cheops reference frame for 67P. Finally, we defined cartographic mapping standards for common use and combined analyses of scientific results that have been obtained not only within the OSIRIS team, but also within other groups of the Rosetta mission.