Image Space Adaptive Rendering

In this thesis, we develop an adaptive framework for Monte Carlo rendering, and more specifically for Monte Carlo Path Tracing (MCPT) and its derivatives. MCPT is attractive because it can handle a wide variety of light transport effects, such as depth of field, motion blur, indirect illumination, participating media, and others, in an elegant and unified framework. However, MCPT is a sampling-based approach, and is only guaranteed to converge in the limit, as the sampling rate grows to infinity. At finite sampling rates, MCPT renderings are often plagued by noise artifacts that can be visually distracting. The adaptive framework developed in this thesis leverages two core strategies to address noise artifacts in renderings: adaptive sampling and adaptive reconstruction. Adaptive sampling consists in increasing the sampling rate on a per pixel basis, to ensure that each pixel value is below a predefined error threshold. Adaptive reconstruction leverages the available samples on a per pixel basis, in an attempt to have an optimal trade-off between minimizing the residual noise artifacts and preserving the edges in the image. In our framework, we greedily minimize the relative Mean Squared Error (rMSE) of the rendering by iterating over sampling and reconstruction steps. Given an initial set of samples, the reconstruction step aims at producing the rendering with the lowest rMSE on a per pixel basis, and the next sampling step then further reduces the rMSE by distributing additional samples according to the magnitude of the residual rMSE of the reconstruction. This iterative approach tightly couples the adaptive sampling and adaptive reconstruction strategies, by ensuring that we only sample densely regions of the image where adaptive reconstruction cannot properly resolve the noise. In a first implementation of our framework, we demonstrate the usefulness of our greedy error minimization using a simple reconstruction scheme leveraging a filterbank of isotropic Gaussian filters. In a second implementation, we integrate a powerful edge aware filter that can adapt to the anisotropy of the image. Finally, in a third implementation, we leverage auxiliary feature buffers that encode scene information (such as surface normals, position, or texture), to improve the robustness of the reconstruction in the presence of strong noise.

Denoising Your Monte Carlo Renders: Recent Advances in Image Space-Adaptive Sampling and Reconstruction

With the ongoing shift in the computer graphics industry toward Monte Carlo rendering, there is a need for effective, practical noise-reduction techniques that are applicable to a wide range of rendering effects and easily integrated into existing production pipelines. This course surveys recent advances in image-space adaptive sampling and reconstruction algorithms for noise reduction, which have proven very effective at reducing the computational cost of Monte Carlo techniques in practice. These approaches leverage advanced image-filtering techniques with statistical methods for error estimation. They are attractive because they can be integrated easily into conventional Monte Carlo rendering frameworks, they are applicable to most rendering effects, and their computational overhead is modest.

Three-Dimensional Semiautomatic Road Extraction From a High-Resolution Aerial Image by Dynamic-Programming Optimization in the Object Space

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Exploring Object Perception with Random Image Structure Evolution

We have developed a technique called RISE (Random Image Structure Evolution), by which one may systematically sample continuous paths in a high-dimensional image space. A basic RISE sequence depicts the evolution of an object's image from a random field, along with the reverse sequence which depicts the transformation of this image back into randomness. The processing steps are designed to ensure that important low-level image attributes such as the frequency spectrum and luminance are held constant throughout a RISE sequence. Experiments based on the RISE paradigm can be used to address some key open issues in object perception. These include determining the neural substrates underlying object perception, the role of prior knowledge and expectation in object perception, and the developmental changes in object perception skills from infancy to adulthood.

A recursive approach to space resection using straight lines

An approach using straight lines as features to solve the photogrammetric space resection problem is presented. An explicit mathematical model relating straight lines, in both object and image space, is used. Based on this model, Kalman Filtering is applied to solve the space resection problem. The recursive property of the filter is used in an iterative process which uses the sequentially estimated camera location parameters to feedback to the feature extraction process in the image. This feedback process leads to a gradual reduction of the image space for feature searching, and consequently eliminates the bottleneck due to the high computational cost of the image segmentation phase. It also enables feature extraction and the determination of feature correspondence in image and object space in an automatic way, i.e., without operator interference. Results obtained from simulated and real data show that highly accurate space resection parameters are obtained as well as a progressive processing time reduction. The obtained accuracy, the automatic correspondence process, and the short related processing time show that the proposed approach can be used in many real-time machine vision systems, making possible the implementation of applications not feasible until now.

Control of Redundant Joint Structures Using Image Information During the Tracking of Non-Smooth Trajectories

Visual information is increasingly being used in a great number of applications in order to perform the guidance of joint structures. This paper proposes an image-based controller which allows the joint structure guidance when its number of degrees of freedom is greater than the required for the developed task. In this case, the controller solves the redundancy combining two different tasks: the primary task allows the correct guidance using image information, and the secondary task determines the most adequate joint structure posture solving the possible joint redundancy regarding the performed task in the image space. The method proposed to guide the joint structure also employs a smoothing Kalman filter not only to determine the moment when abrupt changes occur in the tracked trajectory, but also to estimate and compensate these changes using the proposed filter. Furthermore, a direct visual control approach is proposed which integrates the visual information provided by this smoothing Kalman filter. This last aspect permits the correct tracking when noisy measurements are obtained. All the contributions are integrated in an application which requires the tracking of the faces of Asperger children.

Entre a plasticidade do corpo e da imagem : contributos para uma viagem interactiva pelos territórios incertos da arte contemporânea

Este trabalho de investigação começou por ser estruturado em torno de quatro grandes capítulos (quatro grandes linhas de orientação temática), todos eles amplamente desenvolvidos no sentido de podermos cartografar alguns dos principais territórios e sintomas da arte contemporânea, sendo certo também, que cada um deles assenta precisamente nos princípios de uma estrutura maleável que, para todos os efeitos, se encontra em processo de construção (work in progress), neste caso, graças à plasticidade do corpo, do espaço, da imagem e do uso criativo das tecnologias digitais, no âmbito das quais, aliás, tudo se parece produzir, transformar e disseminar hoje em dia à nossa volta (quase como se de uma autêntica viagem interactiva se tratasse). Por isso, a partir daqui, todo o esforço que se segue procurará ensaiar uma hipótese de trabalho (desenvolver uma investigação) que, porventura, nos permita desbravar alguns caminhos em direcção aos intermináveis túneis do futuro, sempre na expectativa de podermos dar forma, função e sentido a um desejo irreprimível de liberdade criativa, pois, a arte contemporânea tem essa extraordinária capacidade de nos transportar para muitos outros lugares do mundo, tão reais e imaginários como a nossa própria vida. Assim sendo, há que sumariar algumas das principais etapas a desenvolver ao longo desta investigação. Ora, num primeiro momento, começaremos por reflectir sobre o conceito alargado de «crise» (a crise da modernidade), para logo de seguida podermos abordar a questão da crise das antigas categorias estéticas, questionando assim, para todos os efeitos, quer o conceito de «belo» (Platão) e de «gosto» (Kant), quer ainda o conceito de «forma» (Foccilon), não só no sentido de tentarmos compreender algumas das principais razões que terão estado na origem do chamado «fim da arte» (Hegel), mas também algumas daquelas que terão conduzido à estetização generalizada da experiência contemporânea e à sua respectiva disseminação pelas mais variadas plataformas digitais. Num segundo momento, procuraremos reflectir sobre alguns dos principais problemas da inquietante história das imagens, nomeadamente para tentarmos perceber como é que todas estas transformações técnicas (ligadas ao aparecimento da fotografia, do cinema, do vídeo, do computador e da internet) terão contribuído para o processo de instauração e respectivo alargamento daquilo que todos nós ficaríamos a conhecer como a nova «era da imagem», ou a imagem na «era da sua própria reprodutibilidade técnica» (Benjamin), pois, só assim é que conseguiremos interrogar este imparável processo de movimentação, fragmentação, disseminação, simulação e interacção das mais variadas «formas de vida» (Nietzsche, Agamben). Entretanto, chegados ao terceiro grande momento, interessa-nos percepcionar a arte contemporânea como uma espécie de plataforma interactiva que, por sua vez, nos levará a interpelar alguns dos principais dispositivos metafóricos e experimentais da viagem, neste caso, da viagem enquanto linha facilitadora de acesso à arte, à cultura e à vida contemporânea em geral, ou seja, todo um processo de reflexão que nos incitará a cartografar alguns dos mais atractivos sintomas provenientes da estética do flâneur (na perspectiva de Rimbaud, Baudelaire, Long e Benjamin) e, consequentemente, a convocar algumas das principais sensações decorrentes da experiência altamente sedutora daqueles que vivem mergulhados na órbita interactiva do ciberespaço (na condição de ciberflâneurs), quase como se o mundo inteiro, agora, fosse tão somente um espaço poético «inteiramente navegável» (Manovich). Por fim, no quarto e último momento, procuraremos fazer uma profunda reflexão sobre a inquietante história do corpo, principalmente com o objectivo de reforçar a ideia de que apesar das suas inúmeras fragilidades biológicas (um ser que adoece e morre), o corpo continua a ser uma das «categorias mais persistentes de toda a cultura ocidental» (Ieda Tucherman), não só porque ele resistiu a todas as transformações que lhe foram impostas historicamente, mas também porque ele se soube reinventar e readaptar pacientemente face a todas essas transformações históricas. Sinal evidente de que a sua plasticidade lhe iria conferir, principalmente a partir do século XX («o século do corpo») um estatuto teórico e performativo verdadeiramente especial. Tão especial, aliás, que basta termos uma noção, mesmo que breve, da sua inquietante história para percebermos imediatamente a extraordinária importância dalgumas das suas mais variadas transformações, atracções, ligações e exibições ao longo das últimas décadas, nomeadamente sob o efeito criativo das tecnologias digitais (no âmbito das quais se processam algumas das mais interessantes operações de dinamização cultural e artística do nosso tempo). Em suma, esperamos sinceramente que este trabalho de investigação possa vir a contribuir para o processo de alargamento das fronteiras cada vez mais incertas, dinâmicas e interactivas do conhecimento daquilo que parece constituir, hoje em dia, o jogo fundamental da nossa contemporaneidade.

Parallel imaging with phase scrambling.

PURPOSE: Most existing methods for accelerated parallel imaging in MRI require additional data, which are used to derive information about the sensitivity profile of each radiofrequency (RF) channel. In this work, a method is presented to avoid the acquisition of separate coil calibration data for accelerated Cartesian trajectories. METHODS: Quadratic phase is imparted to the image to spread the signals in k-space (aka phase scrambling). By rewriting the Fourier transform as a convolution operation, a window can be introduced to the convolved chirp function, allowing a low-resolution image to be reconstructed from phase-scrambled data without prominent aliasing. This image (for each RF channel) can be used to derive coil sensitivities to drive existing parallel imaging techniques. As a proof of concept, the quadratic phase was applied by introducing an offset to the x(2) - y(2) shim and the data were reconstructed using adapted versions of the image space-based sensitivity encoding and GeneRalized Autocalibrating Partially Parallel Acquisitions algorithms. RESULTS: The method is demonstrated in a phantom (1 × 2, 1 × 3, and 2 × 2 acceleration) and in vivo (2 × 2 acceleration) using a 3D gradient echo acquisition. CONCLUSION: Phase scrambling can be used to perform parallel imaging acceleration without acquisition of separate coil calibration data, demonstrated here for a 3D-Cartesian trajectory. Further research is required to prove the applicability to other 2D and 3D sampling schemes. Magn Reson Med, 2014. © 2014 Wiley Periodicals, Inc.

Controle de Manipulador Redundante Utilizando Realimentação Visual

In conventional robot manipulator control, the desired path is specified in cartesian space and converted to joint space through inverse kinematics mapping. The joint references generated by this mapping are utilized for dynamic control in joint space. Thus, the end-effector position is, in fact, controlled indirectly, in open-loop, and the accuracy of grip position control directly depends on the accuracy of the available kinematic model. In this report, a new scheme for redundant manipulator kinematic control, based on visual servoing is proposed. In the proposed system, a robot image acquired through a CCD camera is processed in order to compute the position and orientation of each link of the robot arm. The robot task is specified as a temporal sequence of reference images of the robot arm. Thus, both the measured pose and the reference pose are specified in the same image space, and its difference is utilized to generate a cartesian space error for kinematic control purposes. The proposed control scheme was applied in a four degree-of-freedom planar redundant robot arm, experimental results are shown

DETERMINATION of THE INDIRECT ORIENTATION of ORBITAL PUSHBROOM IMAGES USING CONTROL STRAIGHT LINES

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Uso de dados fotogramétricos no refinamento geométrico de contornos de telhados de edifícios extraídos de dados laser

In this paper, a methodology is proposed for the geometric refinement of laser scanning building roof contours using high-resolution aerial images and Markov Random Field (MRF) models. The proposed methodology takes for granted that the 3D description of each building roof reconstructed from the laser scanning data (i.e., a polyhedron) is topologically correct and that it is only necessary to improve its accuracy. Since roof ridges are accurately extracted from laser scanning data, our main objective is to use high-resolution aerial images to improve the accuracy of roof outlines. In order to meet this goal, the available roof contours are first projected onto the image-space. After that, the projected polygons and the straight lines extracted from the image are used to establish an MRF description, which is based on relations ( relative length, proximity, and orientation) between the two sets of straight lines. The energy function associated with the MRF is minimized by using a modified version of the brute force algorithm, resulting in the grouping of straight lines for each roof object. Finally, each grouping of straight lines is topologically reconstructed based on the topology of the corresponding laser scanning polygon projected onto the image-space. The preliminary results showed that the proposed methodology is promising, since most sides of the refined polygons are geometrically better than corresponding projected laser scanning straight lines.

Extração semi-automática de rodovia baseada em um par estéreo de imagens aéreas e em otimização por programação dinâmica no espaço-objeto

This article proposes a method for 3D road extraction from a stereopair of aerial images. The dynamic programming (DP) algorithm is used to carry out the optimization process in the object-space, instead of usually doing it in the image-space such as the DP traditional methodologies. This means that road centerlines are directly traced in the object-space, implying that a mathematical relationship is necessary to connect road points in object and image-space. This allows the integration of radiometric information from images into the associate mathematical road model. As the approach depends on an initial approximation of each road, it is necessary a few seed points to coarsely describe the road. Usually, the proposed method allows good results to be obtained, but large anomalies along the road can disturb its performance. Therefore, the method can be used for practical application, although it is expected some kind of local manual edition of the extracted road centerline.

Orientação indireta de imagens com uso de apoio fotogramétrico extraído a partir da integração de dados derivados do sistema lidar e fotogrametria

In this work a method is proposed to allow the indirect orientation of images using photogrammetric control extracted through integration of data derived from Photogrammetry and Light Detection and Ranging (LiDAR) system. The photogrammetric control is obtained by using an inverse photogrammetric model, which allows the projection of image space straight lines onto the object space. This mathematical model is developed based on the intersection between the collinearity-based straight line and a DSM of region, derived from LiDAR data. The mathematical model used in the indirect orientation of the image is known as the model of equivalent t planes. This mathematical model is based on the equivalence between the vector normal to the projection plane in the image space and to the vector normal to the rotated projection plane in the object space. The goal of this work is to verify the quality, efficiency and potential of photogrammetric control straight lines obtained with proposed method applied to the indirect orientation of images. The quality of generated photogrammetric control was statistically available and the results showed that proposed method is promising and it has potential for the indirect orientation of images.

Extração semiautomática de rodovias no espaço-objeto: uso integrado de um estereopar de imagens aéreas e um MDT

Neste artigo é proposto um método semiautomático para extração de rodovias combinando um estereopar de imagens aéreas de baixa resolução com um poliedro gerado a partir de um modelo digital do terreno (MDT). O problema é formulado no espaço-objeto através de uma função objetivo que modela o objeto 'rodovia' como uma curva suave e pertencente a uma superfície poliédrica. A função objetivo proposta depende também de informações radiométricas, que são acessadas no espaço-imagem via relação de colinearidade entre pontos da rodovia no espaço-objeto e os correspondentes nos espaços imagem do estereopar. A linha poligonal que melhor modela a rodovia selecionada é obtida por otimização no espaço-objeto da função objetivo, tendo por base o algoritmo de programação dinâmica. O processo de otimização é iterativo e dependente do fornecimento por um operador de uma aproximação inicial para a rodovia selecionada. Os resultados obtidos mostraram que o método é robusto frente a anomalias existentes ao longo das rodovias, tais como obstruções causadas por sombras e árvores.

Automatic absolute orientation of scanned aerial photographs

Image orientation is a basic problem in Digital Photogrammetry. While interior and relative orientations were succesfully automated, the same can not be said about absolute orientation. This process can be automated by using an approach based on relational matching and a heuristic that uses the analytical relation between straight features in the object space and its homologous in the image space. A build-in self-diagnosis is also used in this method, that is based on the implementation of data snooping statistic test in the process of spatial resection, using the Iterated Extended Kalman Filtering (IEKF). The aim of this paper is to present the basic principles of the proposed approach and results based on real data.