High Resolution Image Correspondences for Video Post-Production

We present an algorithm for estimating dense image correspondences. Our versatile approach lends itself to various tasks typical for video post-processing, including image morphing, optical flow estimation, stereo rectification, disparity/depth reconstruction, and baseline adjustment. We incorporate recent advances in feature matching, energy minimization, stereo vision, and data clustering into our approach. At the core of our correspondence estimation we use Efficient Belief Propagation for energy minimization. While state-of-the-art algorithms only work on thumbnail-sized images, our novel feature downsampling scheme in combination with a simple, yet efficient data term compression, can cope with high-resolution data. The incorporation of SIFT (Scale-Invariant Feature Transform) features into data term computation further resolves matching ambiguities, making long-range correspondence estimation possible. We detect occluded areas by evaluating the correspondence symmetry, we further apply Geodesic matting to automatically determine plausible values in these regions.

Every face on Vanity Fair’s Hollywood covers 1995-2008

‘Every face on Vanity Fair’s Hollywood covers 1995-2008’ renders an ethnographic-like study of Hollywood celebrity as a cinematic experience. Viewers are presented with constantly mutating portraits that violently twist and shear into other faces, while an immersive soundscape echoes the turbulent painterly surface. Through technical processes of scaling, looping and image morphing; the work explores a positive affectual response to the seductive power of celebrity imagery. Conceptually, given Vanity Fair magazine’s prestigious stature, the work also performs an ethnographic-mapping of the popularity of Hollywood stars over time, while at the same time creating in-between, ‘mutant’ versions of their visages. The installation explores the potential for fan-based responses to pop culture to lead to artworks that enable a more critical response to the subjective and intersubjective dynamics of celebrity portraiture. Questions are raised about how these cultural forms impact pop culture fans, and their role in the mapping of culture and social experience.

Perceptual Evaluation of Video-Realistic Speech

abstract With many visual speech animation techniques now available, there is a clear need for systematic perceptual evaluation schemes. We describe here our scheme and its application to a new video-realistic (potentially indistinguishable from real recorded video) visual-speech animation system, called Mary 101. Two types of experiments were performed: a) distinguishing visually between real and synthetic image- sequences of the same utterances, ("Turing tests") and b) gauging visual speech recognition by comparing lip-reading performance of the real and synthetic image-sequences of the same utterances ("Intelligibility tests"). Subjects that were presented randomly with either real or synthetic image-sequences could not tell the synthetic from the real sequences above chance level. The same subjects when asked to lip-read the utterances from the same image-sequences recognized speech from real image-sequences significantly better than from synthetic ones. However, performance for both, real and synthetic, were at levels suggested in the literature on lip-reading. We conclude from the two experiments that the animation of Mary 101 is adequate for providing a percept of a talking head. However, additional effort is required to improve the animation for lip-reading purposes like rehabilitation and language learning. In addition, these two tasks could be considered as explicit and implicit perceptual discrimination tasks. In the explicit task (a), each stimulus is classified directly as a synthetic or real image-sequence by detecting a possible difference between the synthetic and the real image-sequences. The implicit perceptual discrimination task (b) consists of a comparison between visual recognition of speech of real and synthetic image-sequences. Our results suggest that implicit perceptual discrimination is a more sensitive method for discrimination between synthetic and real image-sequences than explicit perceptual discrimination.

Image-Based View Synthesis

We present a new method for rendering novel images of flexible 3D objects from a small number of example images in correspondence. The strength of the method is the ability to synthesize images whose viewing position is significantly far away from the viewing cone of the example images ("view extrapolation"), yet without ever modeling the 3D structure of the scene. The method relies on synthesizing a chain of "trilinear tensors" that governs the warping function from the example images to the novel image, together with a multi-dimensional interpolation function that synthesizes the non-rigid motions of the viewed object from the virtual camera position. We show that two closely spaced example images alone are sufficient in practice to synthesize a significant viewing cone, thus demonstrating the ability of representing an object by a relatively small number of model images --- for the purpose of cheap and fast viewers that can run on standard hardware.

Mesh-based vs. Image-based Statistical Appearance Model of the Human Femur: a Preliminary Comparison Study for the Creation of Finite Element Meshes

Statistical models have been recently introduced in computational orthopaedics to investigate the bone mechanical properties across several populations. A fundamental aspect for the construction of statistical models concerns the establishment of accurate anatomical correspondences among the objects of the training dataset. Various methods have been proposed to solve this problem such as mesh morphing or image registration algorithms. The objective of this study is to compare a mesh-based and an image-based statistical appearance model approaches for the creation of nite element(FE) meshes. A computer tomography (CT) dataset of 157 human left femurs was used for the comparison. For each approach, 30 finite element meshes were generated with the models. The quality of the obtained FE meshes was evaluated in terms of volume, size and shape of the elements. Results showed that the quality of the meshes obtained with the image-based approach was higher than the quality of the mesh-based approach. Future studies are required to evaluate the impact of this finding on the final mechanical simulations.

An image-based method to automatically propagate bony landmarks: application to computational spine biomechanics.

In attempts to elucidate the underlying mechanisms of spinal injuries and spinal deformities, several experimental and numerical studies have been conducted to understand the biomechanical behavior of the spine. However, numerical biomechanical studies suffer from uncertainties associated with hard- and soft-tissue anatomies. Currently, these parameters are identified manually on each mesh model prior to simulations. The determination of soft connective tissues on finite element meshes can be a tedious procedure, which limits the number of models used in the numerical studies to a few instances. In order to address these limitations, an image-based method for automatic morphing of soft connective tissues has been proposed. Results showed that the proposed method is capable to accurately determine the spatial locations of predetermined bony landmarks. The present method can be used to automatically generate patient-specific models, which may be helpful in designing studies involving a large number of instances and to understand the mechanical behavior of biomechanical structures across a given population.

Image-based vs. mesh-based statistical appearance models of the human femur: Implications for finite element simulations.

Statistical appearance models have recently been introduced in bone mechanics to investigate bone geometry and mechanical properties in population studies. The establishment of accurate anatomical correspondences is a critical aspect for the construction of reliable models. Depending on the representation of a bone as an image or a mesh, correspondences are detected using image registration or mesh morphing. The objective of this study was to compare image-based and mesh-based statistical appearance models of the femur for finite element (FE) simulations. To this aim, (i) we compared correspondence detection methods on bone surface and in bone volume; (ii) we created an image-based and a mesh-based statistical appearance models from 130 images, which we validated using compactness, representation and generalization, and we analyzed the FE results on 50 recreated bones vs. original bones; (iii) we created 1000 new instances, and we compared the quality of the FE meshes. Results showed that the image-based approach was more accurate in volume correspondence detection and quality of FE meshes, whereas the mesh-based approach was more accurate for surface correspondence detection and model compactness. Based on our results, we recommend the use of image-based statistical appearance models for FE simulations of the femur.

Biomechanical modeling of parotid glands morphing in head & neck radiation therapy treatments

Durante i trattamenti radioterapici dei pazienti oncologici testa-collo, le ghiandole parotidee (PGs) possono essere indebitamente irradiate a seguito di modificazioni volumetriche-spaziali inter/intra-frazione causate da fattori quali il dimagrimento, l’esposizione a radiazioni ionizzanti ed il morphing anatomico degli organi coinvolti nelle aree d’irraggiamento. Il presente lavoro svolto presso la struttura di Fisica Medica e di Radioterapia Oncologica dell’A.O.U di Modena, quale parte del progetto di ricerca del Ministero della Salute (MoH2010, GR-2010-2318757) “ Dose warping methods for IGRT and Adaptive RT: dose accumulation based on organ motion and anatomical variations of the patients during radiation therapy treatments ”, sviluppa un modello biomeccanico in grado di rappresentare il processo di deformazione delle PGs, considerandone la geometria, le proprietà elastiche e l'evoluzione durante il ciclo terapeutico. Il modello di deformazione d’organo è stato realizzato attraverso l’utilizzo di un software agli elementi finiti (FEM). Molteplici superfici mesh, rappresentanti la geometria e l’evoluzione delle parotidi durante le sedute di trattamento, sono state create a partire dai contorni dell’organo definiti dal medico radioterapista sull’immagine tomografica di pianificazione e generati automaticamente sulle immagini di setup e re-positioning giornaliere mediante algoritmi di registrazione rigida/deformabile. I constraints anatomici e il campo di forze del modello sono stati definiti sulla base di ipotesi semplificative considerando l’alterazione strutturale (perdita di cellule acinari) e le barriere anatomiche dovute a strutture circostanti. L’analisi delle mesh ha consentito di studiare la dinamica della deformazione e di individuare le regioni maggiormente soggette a cambiamento. Le previsioni di morphing prodotte dal modello proposto potrebbero essere integrate in un treatment planning system per metodiche di Adaptive Radiation Therapy.