Looking through the eyes of the painter: from visual perception to non-photorealistic rendering

In this paper we present a brief overview of the processing in the primary visual cortex, the multi-scale line/edge and keypoint representations, and a model of brightness perception. This model, which is being extended from 1D to 2D, is based on a symbolic line and edge interpretation: lines are represented by scaled Gaussians and edges by scaled, Gaussian-windowed error functions. We show that this model, in combination with standard techniques from graphics, provides a very fertile basis for non-photorealistic image rendering.

Genetic Programming for Non-Photorealistic Rendering

This thesis focuses on developing an evolutionary art system using genetic programming. The main goal is to produce new forms of evolutionary art that filter existing images into new non-photorealistic (NPR) styles, by obtaining images that look like traditional media such as watercolor or pencil, as well as brand new effects. The approach permits GP to generate creative forms of NPR results. The GP language is extended with different techniques and methods inspired from NPR research such as colour mixing expressions, image processing filters and painting algorithm. Colour mixing is a major new contribution, as it enables many familiar and innovative NPR effects to arise. Another major innovation is that many GP functions process the canvas (rendered image), while is dynamically changing. Automatic fitness scoring uses aesthetic evaluation models and statistical analysis, and multi-objective fitness evaluation is used. Results showed a variety of NPR effects, as well as new, creative possibilities.

Non-photorealistic rendering of virtual implant models for computer-assisted fluoroscopy-based surgical procedures

Surgical navigation systems visualize the positions and orientations of surgical instruments and implants as graphical overlays onto a medical image of the operated anatomy on a computer monitor. The orthopaedic surgical navigation systems could be categorized according to the image modalities that are used for the visualization of surgical action. In the so-called CT-based systems or 'surgeon-defined anatomy' based systems, where a 3D volume or surface representation of the operated anatomy could be constructed from the preoperatively acquired tomographic data or through intraoperatively digitized anatomy landmarks, a photorealistic rendering of the surgical action has been identified to greatly improve usability of these navigation systems. However, this may not hold true when the virtual representation of surgical instruments and implants is superimposed onto 2D projection images in a fluoroscopy-based navigation system due to the so-called image occlusion problem. Image occlusion occurs when the field of view of the fluoroscopic image is occupied by the virtual representation of surgical implants or instruments. In these situations, the surgeon may miss part of the image details, even if transparency and/or wire-frame rendering is used. In this paper, we propose to use non-photorealistic rendering to overcome this difficulty. Laboratory testing results on foamed plastic bones during various computer-assisted fluoroscopybased surgical procedures including total hip arthroplasty and long bone fracture reduction and osteosynthesis are shown.

Artistic rendering of the visual cortex

In this paper we explain the processing in the first layers of the visual cortex by simple, complex and endstopped cells, plus grouping cells for line, edge, keypoint and saliency detection. Three visualisations are presented: (a) an integrated scheme that shows activities of simple, complex and end-stopped cells, (b) artistic combinations of selected activity maps that give an impression of global image structure and/or local detail, and (c) NPR on the basis of a 2D brightness model. The cortical image representations offer many possibilities for non-photorealistic rendering.

Perception-based painterly rendering: funcionality and interface design

Painterly rendering (non-photorealistic rendering or NPR) aims at translating photographs into paintings with discrete brush strokes, simulating certain techniques (im- or expressionism) and media (oil or watercolour). Recently, our research into visual perception and models of processes in the visual cortex resulted in a new rendering scheme, in which detected lines and edges at different scales are translated into brush strokes of different sizes. In order to prepare a version which is suitable for many users, including children, the design of the interface in terms of window and menu system is very important. Discussions with artists and non-artists led to three design criteria: (1) the interface must reflect the procedures and possibilities that real painters follow and use, (2) it must be based on only one window, and (3) the menu system must be very simple, avoiding a jungle of menus and sub-menus. This paper explains the interface that has been developed.

非真实感绘制中的线绘制技术及其水墨仿真研究

绘画是人类捕捉、记录及表现不同创意目的的行为，是一种在二维的平面上以手工方式描绘自然的艺术。从远古的石洞壁画到现代的各种绘画艺术作品和插图，随着人类历史的进步，绘画一直扮演着传递信息和艺术表达的角色：作为表达场景信息的手段，在照相技术出现之前，手工绘画一直是最好的方法；而另一方面，作为艺术行为，绘画可以表达作者的精神世界和自我风格。 随着计算机技术的出现和发展，计算机绘图逐渐成熟，并形成一门独立学科：计算机图形学。非真实感绘制(Non-Photorealistic Rendering, NPR)作为计算机图形学的热点研究课题最为接近手工绘画，有别于传统计算机图形学，它是涉及计算机艺术、数字图形处理、认知心理学等多领域的交叉研究领域。非真实感绘制的一个重要作用就是将计算机技术和绘画艺术结合起来，自动或交互地生成具有手绘艺术风格的图像。其目的不在于绘制结果的真实感，而在于表现绘画艺术特质、模拟艺术作品等。非真实感绘制往往依据艺术家的视觉感官或为了突出物体某方面性质，会着重强调某一方面细节而忽略一些其他细节信息。 本文围绕非真实感绘制技术，从模拟线绘画和中国水墨绘画两个方向进行研究。以简单的三维网格模型为输入，力求完全自动化地进行实时绘制。本文工作主要分为两部分：实时线绘制技术和二维水墨扩散仿真。 线条绘画，是表达物理场景既简单又实用的绘画手段，线绘制技术也是非真实感绘制领域最重要的研究课题之一。无论是插图还是艺术绘画，线绘制图像都可以完全表达出三维场景的形状信息而不用借助于其他绘画手段。本文首先总结了前人的线绘制算法，并针对视点相关特征线和视点无关特征线的优缺点，提出了一种融合两类特征线优点的特征线新定义。基于该定义，本文算法以模型曲面的微分几何信息为依据，运用基于图像的绘制技术，利用GPU硬件，实时地绘制出风格可控的线绘制图像。 对于种目繁多的艺术流派，目前NPR技术已经实现了多种绘画艺术风格的模拟，如水彩画、铅笔画、钢笔画、油画等等。水墨画是作为中国传统绘画艺术形式，画风朴素抽象，笔划简单但富表现力，是我国优秀的文化传统。对水墨画的仿真模拟，不仅是传承我国优秀文化的一种新方式，同时也是对NPR技术的拓展。由于本文线绘制算法有较好的线宽控制以及内部着色（灰度渐变），在实现上可以很连贯地由线绘制图像扩展到水墨仿真。在水墨扩散模拟阶段，将线绘制结果图像作为输入，通过在二维水墨扩散仿真实现水墨风格的绘制。本文运用流体力学中常用的LB方法，通过研究水墨扩散的各种现象，修改LB方法的D2Q9模型二维网格间的流体迁移运算以及网格的宏观量更新，模拟水墨在宣纸内的流动。由于LB方法具有良好的并行性，故本文仿真算法可以利用GPU的并行计算能力实时地完成绘制。

一种草图形式的视频摘要生成方法

视频摘要作为一种视频内容的简要表示,能够有效地增强用户浏览和组织视频的效率。提出了一种基于草图的视频摘要生成方法。与以往的静态视频摘要方法不同,该方法结合视频内容分析,利用草图在表达上的简洁性和抽象性,对视频中的主要内容进行表达。首先通过视频分析获取视频中的语义特征并提取关键帧,然后通过交互式的方法从关键帧中生成草图,最后进行摘要布局生成完整的视频摘要。实验结果表明,该方法能够有效突出视频的主要对象和主要事件,并具有较高的用户满意度。

Improved line/edge detection and visual reconstruction

Lines and edges provide important information for object categorization and recognition. In addition, one brightness model is based on a symbolic interpretation of the cortical multi-scale line/edge representation. In this paper we present an improved scheme for line/edge extraction from simple and complex cells and we illustrate the multi-scale representation. This representation can be used for visual reconstruction, but also for nonphotorealistic rendering. Together with keypoints and a new model of disparity estimation, a 3D wireframe representation of e.g. faces can be obtained in the future.

Segmentação fuzzy de imagens e vídeos

Image segmentation is the process of subdiving an image into constituent regions or objects that have similar features. In video segmentation, more than subdividing the frames in object that have similar features, there is a consistency requirement among segmentations of successive frames of the video. Fuzzy segmentation is a region growing technique that assigns to each element in an image (which may have been corrupted by noise and/or shading) a grade of membership between 0 and 1 to an object. In this work we present an application that uses a fuzzy segmentation algorithm to identify and select particles in micrographs and an extension of the algorithm to perform video segmentation. Here, we treat a video shot is treated as a three-dimensional volume with different z slices being occupied by different frames of the video shot. The volume is interactively segmented based on selected seed elements, that will determine the affinity functions based on their motion and color properties. The color information can be extracted from a specific color space or from three channels of a set of color models that are selected based on the correlation of the information from all channels. The motion information is provided into the form of dense optical flows maps. Finally, segmentation of real and synthetic videos and their application in a non-photorealistic rendering (NPR) toll are presented

Renderizações não fotorealísticas para estilização de imagens e vídeos usando areia colorida

Non-Photorealisitc Rendering (NPR) is a class of techniques that aims to reproduce artistic techniques, trying to express feelings and moods on the rendered scenes, giving an aspect of that they had been made "by hand". Another way of defining NPR is that it is the processing of scenes, images or videos into artwork, generating scenes, images or videos that can have the visual appeal of pieces of art, expressing the visual and emotional characteristics of artistic styles. This dissertation presents a new method of NPR for stylization of images and videos, based on a typical artistic expression of the Northeast region of Brazil, that uses colored sand to compose landscape images on the inner surface of glass bottles. This method is comprised by one technique for generating 2D procedural textures of sand, and two techniques that mimic effects created by the artists using their tools. It also presents a method for generating 21 2D animations in sandbox from the stylized video. The temporal coherence within these stylized videos can be enforced on individual objects with the aid of a video segmentation algorithm. The present techniques in this work were used on stylization of synthetic and real videos, something close to impossible to be produced by artist in real life

Photorealistic Surface Rendering with Microfacet Theory

La synthèse d'images dites photoréalistes nécessite d'évaluer numériquement la manière dont la lumière et la matière interagissent physiquement, ce qui, malgré la puissance de calcul impressionnante dont nous bénéficions aujourd'hui et qui ne cesse d'augmenter, est encore bien loin de devenir une tâche triviale pour nos ordinateurs. Ceci est dû en majeure partie à la manière dont nous représentons les objets: afin de reproduire les interactions subtiles qui mènent à la perception du détail, il est nécessaire de modéliser des quantités phénoménales de géométries. Au moment du rendu, cette complexité conduit inexorablement à de lourdes requêtes d'entrées-sorties, qui, couplées à des évaluations d'opérateurs de filtrage complexes, rendent les temps de calcul nécessaires à produire des images sans défaut totalement déraisonnables. Afin de pallier ces limitations sous les contraintes actuelles, il est nécessaire de dériver une représentation multiéchelle de la matière. Dans cette thèse, nous construisons une telle représentation pour la matière dont l'interface correspond à une surface perturbée, une configuration qui se construit généralement via des cartes d'élévations en infographie. Nous dérivons notre représentation dans le contexte de la théorie des microfacettes (conçue à l'origine pour modéliser la réflectance de surfaces rugueuses), que nous présentons d'abord, puis augmentons en deux temps. Dans un premier temps, nous rendons la théorie applicable à travers plusieurs échelles d'observation en la généralisant aux statistiques de microfacettes décentrées. Dans l'autre, nous dérivons une procédure d'inversion capable de reconstruire les statistiques de microfacettes à partir de réponses de réflexion d'un matériau arbitraire dans les configurations de rétroréflexion. Nous montrons comment cette théorie augmentée peut être exploitée afin de dériver un opérateur général et efficace de rééchantillonnage approximatif de cartes d'élévations qui (a) préserve l'anisotropie du transport de la lumière pour n'importe quelle résolution, (b) peut être appliqué en amont du rendu et stocké dans des MIP maps afin de diminuer drastiquement le nombre de requêtes d'entrées-sorties, et (c) simplifie de manière considérable les opérations de filtrage par pixel, le tout conduisant à des temps de rendu plus courts. Afin de valider et démontrer l'efficacité de notre opérateur, nous synthétisons des images photoréalistes anticrenelées et les comparons à des images de référence. De plus, nous fournissons une implantation C++ complète tout au long de la dissertation afin de faciliter la reproduction des résultats obtenus. Nous concluons avec une discussion portant sur les limitations de notre approche, ainsi que sur les verrous restant à lever afin de dériver une représentation multiéchelle de la matière encore plus générale.

Infrared thermography: A non-invasive window into thermal physiology

Infrared thermography is a non-invasive technique that measures mid to long-wave infrared radiation emanating from all objects and converts this to temperature. As an imaging technique, the value of modern infrared thermography is its ability to produce a digitized image or high speed video rendering a thermal map of the scene in false colour. Since temperature is an important environmental parameter influencing animal physiology and metabolic heat production an energetically expensive process, measuring temperature and energy exchange in animals is critical to understanding physiology, especially under field conditions. As a non-contact approach, infrared thermography provides a non-invasive complement to physiological data gathering. One caveat, however, is that only surface temperatures are measured, which guides much research to those thermal events occurring at the skin and insulating regions of the body. As an imaging technique, infrared thermal imaging is also subject to certain uncertainties that require physical modeling, which is typically done via built-in software approaches. Infrared thermal imaging has enabled different insights into the comparative physiology of phenomena ranging from thermogenesis, peripheral blood flow adjustments, evaporative cooling, and to respiratory physiology. In this review, I provide background and guidelines for the use of thermal imaging, primarily aimed at field physiologists and biologists interested in thermal biology. I also discuss some of the better known approaches and discoveries revealed from using thermal imaging with the objective of encouraging more quantitative assessment.

Order and the literary rendering of chaos : children's literature as knowledge, order, and social foundation

Depuis que l'animal humain a conçu un système de technologies pour la pensée abstraite grâce au langage, la guerre contre le monde sauvage est devenu une voie à sens unique vers l'aliénation, la civilisation et la littérature. Le but de ce travail est d'analyser comment les récits civilisationnels donnent une structure à l'expérience par le biais de la ségrégation, de la domestication, de la sélection, et de l'extermination, tandis que les récits sauvages démontrent les possibilités infinies du chaos pour découvrir le monde en toute sa diversité et en lien avec sa communauté de vie. Un des objectifs de cette thèse a été de combler le fossé entre la science et la littérature, et d'examiner l'interdépendance de la fiction et la réalité. Un autre objectif a été de mettre ces récits au cœur d'un dialogue les uns avec les autres, ainsi que de tracer leur expression dans les différentes disciplines et œuvres pour enfants et adultes mais également d’analyser leur manifestations c’est redondant dans la vie réelle. C'est un effort multi-disciplinaires qui se reflète dans la combinaison de méthodes de recherche en anthropologie et en études littéraires. Cette analyse compare et contraste trois livres de fiction pour enfants qui présentent trois différents paradigmes socio-économiques, à savoir, «Winnie-l'Ourson» de Milne qui met en place un monde civilisé monarcho-capitaliste, la trilogie de Nosov sur «les aventures de Neznaika et ses amis» qui présente les défis et les exploits d'une société anarcho-socialiste dans son évolution du primitivisme vers la technologie, et les livres de Moomines de Jansson, qui représentent le chaos, l'anarchie, et l'état sauvage qui contient tout, y compris des épisodes de civilisation. En axant la méthodologie de ma recherche sur la façon dont nous connaissons le monde, j'ai d'abord examiné la construction, la transmission et l'acquisition des connaissances, en particulier à travers la théorie de praxis de Bourdieu et la critique de la civilisation développée dans les études de Zerzan, Ong, et Goody sur les liens entre l'alphabétisation, la dette et l'oppression. Quant à la littérature pour enfants, j'ai choisi trois livres que j’ai connus pendant mon enfance, c'est-à-dire des livres qui sont devenus comme une «langue maternelle» pour moi. En ce sens, ce travail est aussi de «l’anthropologie du champ natif». En outre, j’analyse les prémisses sous-jacentes qui se trouvent non seulement dans les trois livres, mais dans le déroulement des récits de l'état sauvage et de la civilisation dans la vie réelle, des analyses qui paraissent dans cette thèse sous la forme d'extraits d’un journal ethnographique. De même que j’examine la nature de la littérature ainsi que des structures civilisées qui domestiquent le monde au moyen de menaces de mort, je trace aussi la présence de ces récits dans l'expression scientifique (le récit malthusien-darwinien), religieuse, et dans autres expressions culturelles, et réfléchis sur les défis présentés par la théorie anarchiste (Kropotkine) ainsi que par les livres pour enfants écrits du point de vue sauvage, tels que ceux des Moomines.

Parallel Monte Carlo approach for integration of the rendering equation

This paper is addressed to the numerical solving of the rendering equation in realistic image creation. The rendering equation is integral equation describing the light propagation in a scene accordingly to a given illumination model. The used illumination model determines the kernel of the equation under consideration. Nowadays, widely used are the Monte Carlo methods for solving the rendering equation in order to create photorealistic images. In this work we consider the Monte Carlo solving of the rendering equation in the context of the parallel sampling scheme for hemisphere. Our aim is to apply this sampling scheme to stratified Monte Carlo integration method for parallel solving of the rendering equation. The domain for integration of the rendering equation is a hemisphere. We divide the hemispherical domain into a number of equal sub-domains of orthogonal spherical triangles. This domain partitioning allows to solve the rendering equation in parallel. It is known that the Neumann series represent the solution of the integral equation as a infinity sum of integrals. We approximate this sum with a desired truncation error (systematic error) receiving the fixed number of iteration. Then the rendering equation is solved iteratively using Monte Carlo approach. At each iteration we solve multi-dimensional integrals using uniform hemisphere partitioning scheme. An estimate of the rate of convergence is obtained using the stratified Monte Carlo method. This domain partitioning allows easy parallel realization and leads to convergence improvement of the Monte Carlo method. The high performance and Grid computing of the corresponding Monte Carlo scheme are discussed.