Tecniche per la separazione live di piani di profondita in un flusso video mediante Kinect e una applicazione nell'intrattenimento digitale

Keying e composizione sono da sempre tecniche ampiamente utilizzate in contesti multimediali, quali produzione cinematografica e televisiva; il chroma keying è in particolare la tecnica più popolare, ma presenta una serie di limiti e problematiche. In questo elaborato viene proposta una tecnica alternativa di estrazione, basata sull'uso della profondità, operante in tempo reale e che sfrutta il device Kinect di Microsoft. Sono proposti una serie di algoritmi, basati su tecniche di edge detection, utilizzati per il miglioramento della depth map lungo i bordi di estrazione; viene infine testato il risultato ottenuto dall'implementazione del sistema e proposta una possibile applicazione nell'ambito del teatro multimediale.

Chroma Key Theatre : an examination of similarities and divergences in the use of Green Screen in two experimental theatre works

This paper examines the frame as it contributes to the debate on contemporary intermedial theatre and performance practices in light of increasing astriction between filmic and theatrical discourses. Informed by Auslander (1999), Lehmann (2006), and Giesekam (2007), and through an extrapolation of the tenets Eckersall, Gretchen and Scheer identify in the theory of New Media Dramaturgy, it will analyse two recent works of experimental theatre-making. RUFF (2013), a New York produced solo performance by one of the world’s leading female performers, explores her experiences of having a stroke. Total Dik! (2013), produced in Brisbane, Australia, is an interdisciplinary collaborative performance that examines aspects of dictatorship. They are clearly very different works yet there are a number of significant theatrical similarities in their use of Chroma Key technology and live compositing as material scenic devices. These works overtly and evocatively draw on the cinematic technique and technology of Chroma Key to augment and reveal the tensions and overlaps in their production processes.

Green: Exploring the aestheticized use of chroma-key techniques and technologies in two intermedial productions

Peggy Shaw’s RUFF, (USA 2013) and Queensland Theatre Company’s collaboration with Queensland University of Technology, Total Dik!, (Australia 2013) overtly and evocatively draw on an aestheticized use of the cinematic techniques and technologies of Chroma Key to reveal the tensions in their production and add layers to their performances. In doing so they offer invaluable insight where the filmic and theatrical approaches overlap. This paper draws on Eckersall, Grehan and Scheer’s New Media Dramaturgy (2014) to reposition the frame as a contribution to intermedial theatre and performance practices in light of increasing convergence between seemingly disparate discourses. In RUFF, the scenic environment replicates a chroma-key ‘studio’ which facilitates the reconstruction of memory displaced after a stroke. RUFF uses the screen and projections to recall crooners, lounge singers, movie stars, rock and roll bands, and an eclectic line of eccentric family members living inside Shaw. While the show pays tribute to those who have kept her company across decades of theatrical performance, use of non-composited chroma-key technique as a theatrical device and the work’s taciturn revelation of the production process during performance, play a central role in its exploration of the juxtaposition between its reconstructed form and content. In contrast Total Dik! uses real-time green screen compositing during performance as a scenic device. Actors manipulate scale models, refocus cameras and generate scenes within scenes in the construction of the work’s examination of an isolated Dictator. The ‘studio’ is again replicated as a site for (re)construction, only in this case Total Dik! actively seeks to reveal the process of production as the performance plays out. Building on RUFF, and other works such as By the Way, Meet Vera Stark, (2012) and Hotel Modern’s God’s Beard (2012), this work blends a convergence of mobile technologies, models, and green screen capture to explore aspects of transmedia storytelling in a theatrical environment (Jenkins, 2009, 2013). When a green screen is placed on stage, it reads at once as metaphor and challenge to the language of theatre. It becomes, or rather acts, as a ‘sign’ that alludes to the nature of the reconstructed, recomposited, manipulated and controlled. In RUFF and in Total Dik!, it is also a place where as a mode of production and subsequent reveal, it adds weight to performance. These works are informed by Auslander (1999) and Giesenkam (2007) and speak to and echo Lehmann’s Postdramatic Theatre (2006). This paper’s consideration of the integration of studio technique and live performance as a dynamic approach to multi-layered theatrical production develops our understanding of their combinatory use in a live performance environment.

Efficient spatio-temporal hole filling strategy for Kinect depth maps

In this paper we present an efficient hole filling strategy that improves the quality of the depth maps obtained with the Microsoft Kinect device. The proposed approach is based on a joint-bilateral filtering framework that includes spatial and temporal information. The missing depth values are obtained applying iteratively a joint-bilateral filter to their neighbor pixels. The filter weights are selected considering three different factors: visual data, depth information and a temporal-consistency map. Video and depth data are combined to improve depth map quality in presence of edges and homogeneous regions. Finally, the temporal-consistency map is generated in order to track the reliability of the depth measurements near the hole regions. The obtained depth values are included iteratively in the filtering process of the successive frames and the accuracy of the hole regions depth values increases while new samples are acquired and filtered

A semi-local method for iterative depth-map refinement

We present an iterative hierarchical algorithm for multi-view stereo. The algorithm attempts to utilise as much contextual information as is available to compute highly accurate and robust depth maps. There are three novel aspects to the approach: 1) firstly we incrementally improve the depth fidelity as the algorithm progresses through the image pyramid; 2) secondly we show how to incorporate visual hull information (when available) to constrain depth searches; and 3) we show how to simultaneously enforce the consistency of the depth-map by continual comparison with neighbouring depth-maps. We show that this approach produces highly accurate depth-maps and, since it is essentially a local method, is both extremely fast and simple to implement.

Innovative Stereo Vision-Based Approach to Generate Dense Depth Map of Transportation Infrastructure

Three-dimensional (3-D) spatial data of a transportation infrastructure contain useful information for civil engineering applications, including as-built documentation, on-site safety enhancements, and progress monitoring. Several techniques have been developed for acquiring 3-D point coordinates of infrastructure, such as laser scanning. Although the method yields accurate results, the high device costs and human effort required render the process infeasible for generic applications in the construction industry. A quick and reliable approach, which is based on the principles of stereo vision, is proposed for generating a depth map of an infrastructure. Initially, two images are captured by two similar stereo cameras at the scene of the infrastructure. A Harris feature detector is used to extract feature points from the first view, and an innovative adaptive window-matching technique is used to compute feature point correspondences in the second view. A robust algorithm computes the nonfeature point correspondences. Thus, the correspondences of all the points in the scene are obtained. After all correspondences have been obtained, the geometric principles of stereo vision are used to generate a dense depth map of the scene. The proposed algorithm has been tested on several data sets, and results illustrate its potential for stereo correspondence and depth map generation.

Innovative Stereo Vision-Based Approach to Generate Dense Depth Map of Transportation Infrastructure

Three-dimensional (3-D) spatial data of a transportation infrastructure contain useful information for civil engineering applications, including as-built documentation, on-site safety enhancements, and progress monitoring. Several techniques have been developed for acquiring 3-D point coordinates of infrastructure, such as laser scanning. Although the method yields accurate results, the high device costs and human effort required render the process infeasible for generic applications in the construction industry. A quick and reliable approach, which is based on the principles of stereo vision, is proposed for generating a depth map of an infrastructure. Initially, two images are captured by two similar stereo cameras at the scene of the infrastructure. A Harris feature detector is used to extract feature points from the first view, and an innovative adaptive window-matching technique is used to compute feature point correspondences in the second view. A robust algorithm computes the nonfeature point correspondences. Thus, the correspondences of all the points in the scene are obtained. After all correspondences have been obtained, the geometric principles of stereo vision are used to generate a dense depth map of the scene. The proposed algorithm has been tested on several data sets, and results illustrate its potential for stereo correspondence and depth map generation.

Behavioural plasticity in a large marine herbivore: contrasting patterns of depth utilisation between two green turtle (Chelonia mydas) populations

We used time-depth recorders to measure depth utilisation in gravid green turtles (Chelonia mydas) during the internesting period at northern Cyprus (Mediterranean), a nesting area where individuals feed, and at Ascension Island (mid-Atlantic), a nesting area where individuals fast. There were contrasting patterns of depth utilisation between the two sites, illustrating that the behaviour of this species is shaped by local conditions. For example, the amount of time spent shallower than 4 m was 90% at Cyprus but only 31% at Ascension Island, and there was a clear difference between the mean depth at Cyprus (2.7 m, n=9 internesting intervals) versus Ascension Island (9.5 m, n=6 internesting intervals) (t 5=5.92, P=0.002). At Cyprus, turtles spent the greatest percentage of their time at very shallow depths, where surveys reveated a high abundance of seagrass on which this population feeds. In contrast, the deeper distribution at Ascension Island may reflect the preferred depth for resting on the seabed.

Modeling kinect sensor noise for improved 3D reconstruction and tracking

We contribute an empirically derived noise model for the Kinect sensor. We systematically measure both lateral and axial noise distributions, as a function of both distance and angle of the Kinect to an observed surface. The derived noise model can be used to filter Kinect depth maps for a variety of applications. Our second contribution applies our derived noise model to the KinectFusion system to extend filtering, volumetric fusion, and pose estimation within the pipeline. Qualitative results show our method allows reconstruction of finer details and the ability to reconstruct smaller objects and thinner surfaces. Quantitative results also show our method improves pose estimation accuracy. © 2012 IEEE.

Sorghum stay-green QTL individually reduce post-flowering drought-induced leaf senescence

Sorghum is an important source of food, feed, and biofuel, especially in the semi-arid tropics because this cereal is well adapted to harsh, drought-prone environments. Post-flowering drought adaptation in sorghum is associated with the stay-green phenotype. Alleles that contribute to this complex trait have been mapped to four major QTL, Stg1-Stg4, using a population derived from BTx642 and RTx7000. Near-isogenic RTx7000 lines containing BTx642 DNA spanning one or more of the four stay-green QTL were constructed. The size and location of BTx642 DNA regions in each RTx7000 NIL were analysed using 62 DNA markers spanning the four stay-green QTL. RTx7000 NILs were identified that contained BTx642 DNA completely or partially spanning Stg1, Stg2, Stg3, or Stg4. NILs were also identified that contained sub-portions of each QTL and various combinations of the four major stay-green QTL. Physiological analysis of four RTx7000 NILs containing only Stg1, Stg2, Stg3, or Stg4 showed that BTx642 alleles in each of these loci could contribute to the stay-green phenotype. RTx7000 NILs containing BTx642 DNA corresponding to Stg2 retained more green leaf area at maturity under terminal drought conditions than RTx7000 or the other RTx7000 NILs. Under post-anthesis water deficit, a trend for delayed onset of leaf senescence compared with RTx7000 was also exhibited by the Stg2, Stg3, and Stg4 NILs, while significantly lower rates of leaf senescence in relation to RTx7000 were displayed by all of the Stg NILs to varying degrees, but particularly by the Stg2 NIL. Greener leaves at anthesis relative to RTx7000, indicated by higher SPAD values, were exhibited by the Stg1 and Stg4 NILs. The RTx7000 NILs created in this study provide the starting point for in-depth analysis of stay-green physiology, interaction among stay-green QTL and map-based cloning of the genes that underlie this trait.

Foreground segmentation in depth imagery using depth and spatial dynamic models for video surveillance applications

Low-cost systems that can obtain a high-quality foreground segmentation almostindependently of the existing illumination conditions for indoor environments are verydesirable, especially for security and surveillance applications. In this paper, a novelforeground segmentation algorithm that uses only a Kinect depth sensor is proposedto satisfy the aforementioned system characteristics. This is achieved by combininga mixture of Gaussians-based background subtraction algorithm with a new Bayesiannetwork that robustly predicts the foreground/background regions between consecutivetime steps. The Bayesian network explicitly exploits the intrinsic characteristics ofthe depth data by means of two dynamic models that estimate the spatial and depthevolution of the foreground/background regions. The most remarkable contribution is thedepth-based dynamic model that predicts the changes in the foreground depth distributionbetween consecutive time steps. This is a key difference with regard to visible imagery,where the color/gray distribution of the foreground is typically assumed to be constant.Experiments carried out on two different depth-based databases demonstrate that theproposed combination of algorithms is able to obtain a more accurate segmentation of theforeground/background than other state-of-the art approaches.

Real-time depth camera tracking with CAD models and ICP

In recent years, depth cameras have been widely utilized in camera tracking for augmented and mixed reality. Many of the studies focus on the methods that generate the reference model simultaneously with the tracking and allow operation in unprepared environments. However, methods that rely on predefined CAD models have their advantages. In such methods, the measurement errors are not accumulated to the model, they are tolerant to inaccurate initialization, and the tracking is always performed directly in reference model's coordinate system. In this paper, we present a method for tracking a depth camera with existing CAD models and the Iterative Closest Point (ICP) algorithm. In our approach, we render the CAD model using the latest pose estimate and construct a point cloud from the corresponding depth map. We construct another point cloud from currently captured depth frame, and find the incremental change in the camera pose by aligning the point clouds. We utilize a GPGPU-based implementation of the ICP which efficiently uses all the depth data in the process. The method runs in real-time, it is robust for outliers, and it does not require any preprocessing of the CAD models. We evaluated the approach using the Kinect depth sensor, and compared the results to a 2D edge-based method, to a depth-based SLAM method, and to the ground truth. The results show that the approach is more stable compared to the edge-based method and it suffers less from drift compared to the depth-based SLAM.

A Taxonomy of image matching techniques for stereo vision

Stereo vision is a method of depth perception, in which depth information is inferred from two (or more) images of a scene, taken from different perspectives. Applications of stereo vision include aerial photogrammetry, autonomous vehicle guidance, robotics, industrial automation and stereomicroscopy. A key issue in stereo vision is that of image matching, or identifying corresponding points in a stereo pair. The difference in the positions of corresponding points in image coordinates is termed the parallax or disparity. When the orientation of the two cameras is known, corresponding points may be projected back to find the location of the original object point in world coordinates. Matching techniques are typically categorised according to the nature of the matching primitives they use and the matching strategy they employ. This report provides a detailed taxonomy of image matching techniques, including area based, transform based, feature based, phase based, hybrid, relaxation based, dynamic programming and object space methods. A number of area based matching metrics as well as the rank and census transforms were implemented, in order to investigate their suitability for a real-time stereo sensor for mining automation applications. The requirements of this sensor were speed, robustness, and the ability to produce a dense depth map. The Sum of Absolute Differences matching metric was the least computationally expensive; however, this metric was the most sensitive to radiometric distortion. Metrics such as the Zero Mean Sum of Absolute Differences and Normalised Cross Correlation were the most robust to this type of distortion but introduced additional computational complexity. The rank and census transforms were found to be robust to radiometric distortion, in addition to having low computational complexity. They are therefore prime candidates for a matching algorithm for a stereo sensor for real-time mining applications. A number of issues came to light during this investigation which may merit further work. These include devising a means to evaluate and compare disparity results of different matching algorithms, and finding a method of assigning a level of confidence to a match. Another issue of interest is the possibility of statistically combining the results of different matching algorithms, in order to improve robustness.

Uso de la Realidad Aumentada para facilitar la lectura e interpretación de planos

[ES] Este proyecto genera productos adicionales de trabajos también disponibles en el repositori, en concreto:

高阶广义屏波动方程叠前深度偏移

Attaining sufficient accuracy and efficiency of generalized screen propagator and improving the quality of input gathers are often problems　of wave equation presack depth migration,　in this paper,a high order formula of generalized screen propagator　for one-way wave equation is proposed by using the asymptotic expansion of single-square-root operator. Based on the formula，a new generalized screen propagator is developed ,which is composed of split-step Fourier propagator and high order correction terms，the new generalized screen propagator not only improving calculation precision without sharply increasing the quantity of computation，facilitates the suitability of generalized screen propagator to the media with strong lateral velocity variation.　As wave-equation prestack depth migration is sensitive to the quality of input gathers, which greatly affect the output，and the available seismic data processing system has inability to obtain traveltimes corresponding to the multiple arrivals, to estimate of great residual statics， to merge seismic datum from different projects and to design inverse Q filter, we establish　difference equations with an embodiment of Huygens’s principle for obtaining traveltimes corresponding to the multiple arrivals,bring forward a time variable matching filter for seismic datum merging by using the fast algorithm called Mallat tree for wavelet transformations， put forward a method for estimation of residual statics by applying the optimum model parameters estimated by iterative inversion with three organized algorithm,i.e,the CMP intertrace cross-correlation algorithm,the Laplacian image edge extraction algorithm,and the DFP algorithm, and present phase-shift inverse Q filter based on Futterman’s amplitude and phase-velocity dispersion formula and wave field extrapolation theory. All of their numerical and real data calculating results shows that our theory and method are practical and efficient. Key words: prestack depth migration, generalized screen propagator, residual statics，inverse Q filter ,traveltime，3D seismic datum mergence