Flash Photography Enhancement via Intrinsic Relighting

We enhance photographs shot in dark environments by combining a picture taken with the available light and one taken with the flash. We preserve the ambiance of the original lighting and insert the sharpness from the flash image. We use the bilateral filter to decompose the images into detail and large scale. We reconstruct the image using the large scale of the available lighting and the detail of the flash. We detect and correct flash shadows. This combines the advantages of available illumination and flash photography.

A Survey of Image-based Relighting Techniques

Image-based Relighting (IBRL) has recently attracted a lot of research interest for its ability to relight real objects or scenes, from novel illuminations captured in natural/synthetic environments. Complex lighting effects such as subsurface scattering, interreflection, shadowing, mesostructural self-occlusion, refraction and other relevant phenomena can be generated using IBRL. The main advantage of image-based graphics is that the rendering time is independent of scene complexity as the rendering is actually a process of manipulating image pixels, instead of simulating light transport. The goal of this paper is to provide a complete and systematic overview of the research in Imagebased Relighting. We observe that essentially all IBRL techniques can be broadly classified into three categories (Fig. 9), based on how the scene/illumination information is captured: Reflectance function-based, Basis function-based and Plenoptic function-based. We discuss the characteristics of each of these categories and their representative methods. We also discuss about the sampling density and types of light source(s), relevant issues of IBRL.

Reflectance Transfer for Material Editing and Relighting

We present a new approach to diffuse reflectance estimation for dynamic scenes. Non-parametric image statistics are used to transfer reflectance properties from a static example set to a dynamic image sequence. The approach allows diffuse reflectance estimation for surface materials with inhomogeneous appearance, such as those which commonly occur with patterned or textured clothing. Material editing is also possible by transferring edited reflectance properties. Material reflectance properties are initially estimated from static images of the subject under multiple directional illuminations using photometric stereo. The estimated reflectance together with the corresponding image under uniform ambient illumination form a prior set of reference material observations. Material reflectance properties are then estimated for video sequences of a moving person captured under uniform ambient illumination by matching the observed local image statistics to the reference observations. Results demonstrate that the transfer of reflectance properties enables estimation of the dynamic surface normals and subsequent relighting combined with material editing. This approach overcomes limitations of previous work on material transfer and relighting of dynamic scenes which was limited to surfaces with regions of homogeneous reflectance. We evaluate our approach for relighting 3D model sequences reconstructed from multiple view video. Comparison to previous model relighting demonstrates improved reproduction of detailed texture and shape dynamics.

Cosine Lobe Based Relighting from Gradient Illumination Photographs

We present an image-based method for relighting a scene by analytically fitting cosine lobes to the reflectance function at each pixel, based on gradient illumination photographs. Realistic relighting results for many materials are obtained using a single per-pixel cosine lobe obtained from just two color photographs: one under uniform white illumination and the other under colored gradient illumination. For materials with wavelength-dependent scattering, a better fit can be obtained using independent cosine lobes for the red, green, and blue channels, obtained from three achromatic gradient illumination conditions instead of the colored gradient condition. We explore two cosine lobe reflectance functions, both of which allow an analytic fit to the gradient conditions. One is non-zero over half the sphere of lighting directions, which works well for diffuse and specular materials, but fails for materials with broader scattering such as fur. The other is non-zero everywhere, which works well for broadly scattering materials and still produces visually plausible results for diffuse and specular materials. We also perform an approximate diffuse/specular separation of the reflectance, and estimate scene geometry from the recovered photometric normals to produce hard shadows cast by the geometry, while still reconstructing the input photographs exactly.

Neural Radiance Fields for Relighting and View Synthesis

Neural scene representation and neural rendering are new computer vision techniques that enable the reconstruction and implicit representation of real 3D scenes from a set of 2D captured images, by fitting a deep neural network. The trained network can then be used to render novel views of the scene. A recent work in this field, Neural Radiance Fields (NeRF), presented a state-of-the-art approach, which uses a simple Multilayer Perceptron (MLP) to generate photo-realistic RGB images of a scene from arbitrary viewpoints. However, NeRF does not model any light interaction with the fitted scene; therefore, despite producing compelling results for the view synthesis task, it does not provide a solution for relighting. In this work, we propose a new architecture to enable relighting capabilities in NeRF-based representations and we introduce a new real-world dataset to train and evaluate such a model. Our method demonstrates the ability to perform realistic rendering of novel views under arbitrary lighting conditions.

MixAR mobile prototype: visualizing virtually reconstructed ancient structures in situ

Archeology and related areas have a special interest on cultural heritage sites since they provide valuable information about past civilizations. However, the ancient buildings present in these sites are commonly found in an advanced state of degradation which difficult the professional/expert analysis. Virtual reconstructions of such buildings aim to provide a digital insight of how these historical places could have been in ancient times. Moreover, the visualization of such models has been explored by some Augmented Reality (AR) systems capable of providing support to experts. Their compelling and appealing environments have also been applied to promote the social and cultural participation of general public. The existing AR solutions regarding this thematic rarely explore the potential of realism, due to the following lacks: the exploration of mixed environments is usually only supported for indoors or outdoors, not both in the same system; the adaptation of the illumination conditions to the reconstructed structures is rarely addressed causing a decrease of credibility. MixAR [1] is a system concerned with those challenges, aiming to provide the visualization of virtual buildings augmented upon real ruins, allowing soft transitions among its interiors and exteriors and using relighting techniques for a faithful interior illumination, while the user freely moves in a given cultural heritage site, carrying a mobile unit. Regarding the focus of this paper, we intend to report the current state of MixAR mobile unit prototype, which allows visualizing virtual buildings – properly aligned with real-world structures – based on user's location, during outdoor navigation. In order to evaluate the prototype performance, a set of tests were made using virtual models with different complexities.

The Plutarch’s Motive of "Descensus Animae" in "Nag Hammadi and the Corpus Hermeticum"

Al evaluar los contactos de Plutarco con otras culturas contemporáneas, los investigadores todavía no han llegado a un consenso acerca de la relación entre el queronense y la literatura cristiano-primitiva. Un buen ejemplo de esto aparece al atender al motivo de la creación del alma humana. La intención de las próximas páginas es, tras un análisis de los textos plutarqueos, atender a estos posibles contactos con NHC, los heresiólogos y el Corpus Hermeticum a fin de dilucidar sus similitudes y diferencias.