Motion capture amb Microsoft Kinect

El proyecto trata de convertirse en una herramienta para animadores 3D, tanto para los que hacen películas como para los que modelan videojuegos, que necesiten de un software para simplificar el trabajo que conlleva animar un modelo 3D. Todo sin necesidad de usar trajes especializados. El proyecto, usando Kinect, convertirá los movimientos captados por la cámara y los agregará al modelo, creando una animación basándose en los movimientos reales de una persona.

Physically based walking simulation

Hem realitzat l’estudi de moviments humans i hem buscat la forma de poder crear aquests moviments en temps real sobre entorns digitals de forma que la feina que han de dur a terme els artistes i animadors sigui reduïda. Hem fet un estudi de les diferents tècniques d’animació de personatges que podem trobar actualment en l’industria de l’entreteniment així com les principals línies de recerca, estudiant detingudament la tècnica més utilitzada, la captura de moviments. La captura de moviments permet enregistrar els moviments d’una persona mitjançant sensors òptics, sensors magnètics i vídeo càmeres. Aquesta informació és emmagatzemada en arxius que després podran ser reproduïts per un personatge en temps real en una aplicació digital. Tot moviment enregistrat ha d’estar associat a un personatge, aquest és el procés de rigging, un dels punts que hem treballat ha estat la creació d’un sistema d’associació de l’esquelet amb la malla del personatge de forma semi-automàtica, reduint la feina de l’animador per a realitzar aquest procés. En les aplicacions en temps real com la realitat virtual, cada cop més s’està simulant l’entorn en el que viuen els personatges mitjançant les lleis de Newton, de forma que tot canvi en el moviment d’un cos ve donat per l’aplicació d’una força sobre aquest. La captura de moviments no escala bé amb aquests entorns degut a que no és capaç de crear noves animacions realistes a partir de l’enregistrada que depenguin de l’interacció amb l’entorn. L’objectiu final del nostre treball ha estat realitzar la creació d’animacions a partir de forces tal i com ho fem en la realitat en temps real. Per a això hem introduït un model muscular i un sistema de balanç sobre el personatge de forma que aquest pugui respondre a les interaccions amb l’entorn simulat mitjançant les lleis de Newton de manera realista.

Drift and ownership toward a distant virtual body.

In body ownership illusions participants feel that a mannequin or virtual body (VB) is their own. Earlier results suggest that body ownership over a body seen from behind in extra personal space is possible when the surrogate body is visually stroked and tapped on its back, while spatially and temporal synchronous tactile stimulation is applied to the participant's back. This result has been disputed with the claim that the results can be explained by self-recognition rather than somatic body ownership. We carried out an experiment with 30 participants in a between-groups design. They all saw the back of a VB 1.2 m in front, that moved in real-time determined by upper body motion capture. All felt tactile stimulation on their back, and for 15 of them this was spatially and temporally synchronous with stimulation that they saw on the back of the VB, but asynchronous for the other 15. After 3 min a revolving fan above the VB descended and stopped at the position of the VB neck. A questionnaire assessed referral of touch to the VB, body ownership, the illusion of drifting forwards toward the VB, and the VB drifting backwards. Heart rate deceleration (HRD) and the amount of head movement during the threat period were used to assess the response to the threat from the fan. Results showed that although referral of touch was significantly greater in the synchronous condition than the asynchronous, there were no other differences between the conditions. However, a further multivariate analysis revealed that in the visuotactile synchronous condition HRD and head movement increased with the illusion of forward drift and decreased with backwards drift. Body ownership contributed positively to these drift sensations. Our conclusion is that the setup results in a contradiction-somatic feelings associated with a distant body-that the brain attempts to resolve by generating drift illusions that would make the two bodies coincide.

Drift and ownership toward a distant virtual body.

In body ownership illusions participants feel that a mannequin or virtual body (VB) is their own. Earlier results suggest that body ownership over a body seen from behind in extra personal space is possible when the surrogate body is visually stroked and tapped on its back, while spatially and temporal synchronous tactile stimulation is applied to the participant's back. This result has been disputed with the claim that the results can be explained by self-recognition rather than somatic body ownership. We carried out an experiment with 30 participants in a between-groups design. They all saw the back of a VB 1.2 m in front, that moved in real-time determined by upper body motion capture. All felt tactile stimulation on their back, and for 15 of them this was spatially and temporally synchronous with stimulation that they saw on the back of the VB, but asynchronous for the other 15. After 3 min a revolving fan above the VB descended and stopped at the position of the VB neck. A questionnaire assessed referral of touch to the VB, body ownership, the illusion of drifting forwards toward the VB, and the VB drifting backwards. Heart rate deceleration (HRD) and the amount of head movement during the threat period were used to assess the response to the threat from the fan. Results showed that although referral of touch was significantly greater in the synchronous condition than the asynchronous, there were no other differences between the conditions. However, a further multivariate analysis revealed that in the visuotactile synchronous condition HRD and head movement increased with the illusion of forward drift and decreased with backwards drift. Body ownership contributed positively to these drift sensations. Our conclusion is that the setup results in a contradiction-somatic feelings associated with a distant body-that the brain attempts to resolve by generating drift illusions that would make the two bodies coincide.

Bowing the violin : a case study for auditory-motor pattern modelling in the context of music performance

This project addresses methodological and technological challenges in the development of multi-modal data acquisition and analysis methods for the representation of instrumental playing technique in music performance through auditory-motor patterning models. The case study is violin playing: a multi-modal database of violin performances has been constructed by recording different musicians while playing short exercises on different violins. The exercise set and recording protocol have been designed to sample the space defined by dynamics (from piano to forte) and tone (from sul tasto to sul ponticello), for each bow stroke type being played on each of the four strings (three different pitches per string) at two different tempi. The data, containing audio, video, and motion capture streams, has been processed and segmented to facilitate upcoming analyses. From the acquired motion data, the positions of the instrument string ends and the bow hair ribbon ends are tracked and processed to obtain a number of bowing descriptors suited for a detailed description and analysis of the bow motion patterns taking place during performance. Likewise, a number of sound perceptual attributes are computed from the audio streams. Besides the methodology and the implementation of a number of data acquisition tools, this project introduces preliminary results from analyzing bowing technique on a multi-modal violin performance database that is unique in its class. A further contribution of this project is the data itself, which will be made available to the scientific community through the repovizz platform.

Motion analysis in terms of wireless video capsule endoscopy

El treball presentat suposa una visió general de l'"Endoscopia amb Càpsula de Vídeo Wireless" i la inspecció de sequències de contraccions intestinals amb les últimes tecnologies de visió per computador. Després de la observació preliminar dels fonaments mèdics requerits, la aplicació de visió per computador es presenta en aquestos termes. En essència, aquest treball proveïx una exhaustiva selecció, descripció i avaluació de cert conjunt de mètodes de processament d'imatges respecte a l'anàlisi de moviment, en el entorn de seqüències d'imatges preses amb una càpsula endoscòpica. Finalment, es presenta una aplicació de software per configurar i emprar de forma ràpida i fàcil un entorn experimental.

HSU-Robbins and Spitzer's theorems for the variations of fractional Brownian motion

Using recent results on the behavior of multiple Wiener-Itô integrals based on Stein's method, we prove Hsu-Robbins and Spitzer's theorems for sequences of correlated random variables related to the increments of the fractional Brownian motion.

A well-posedness theory in measures for some kinetic models of collective motion

We present existence, uniqueness and continuous dependence results for some kinetic equations motivated by models for the collective behavior of large groups of individuals. Models of this kind have been recently proposed to study the behavior of large groups of animals, such as flocks of birds, swarms, or schools of fish. Our aim is to give a well-posedness theory for general models which possibly include a variety of effects: an interaction through a potential, such as a short-range repulsion and long-range attraction; a velocity-averaging effect where individuals try to adapt their own velocity to that of other individuals in their surroundings; and self-propulsion effects, which take into account effects on one individual that are independent of the others. We develop our theory in a space of measures, using mass transportation distances. As consequences of our theory we show also the convergence of particle systems to their corresponding kinetic equations, and the local-in-time convergence to the hydrodynamic limit for one of the models.

Mathematical description of bacterial traveling pulses

The Keller-Segel system has been widely proposed as a model for bacterial waves driven by chemotactic processes. Current experiments on E. coli have shown precise structure of traveling pulses. We present here an alternative mathematical description of traveling pulses at a macroscopic scale. This modeling task is complemented with numerical simulations in accordance with the experimental observations. Our model is derived from an accurate kinetic description of the mesoscopic run-and-tumble process performed by bacteria. This model can account for recent experimental observations with E. coli. Qualitative agreements include the asymmetry of the pulse and transition in the collective behaviour (clustered motion versus dispersion). In addition we can capture quantitatively the main characteristics of the pulse such as the speed and the relative size of tails. This work opens several experimental and theoretical perspectives. Coefficients at the macroscopic level are derived from considerations at the cellular scale. For instance the stiffness of the signal integration process turns out to have a strong effect on collective motion. Furthermore the bottom-up scaling allows to perform preliminary mathematical analysis and write efficient numerical schemes. This model is intended as a predictive tool for the investigation of bacterial collective motion.

Estimating the motion of an underwater robot from a monocular image sequence

When underwater vehicles perform navigation close to the ocean floor, computer vision techniques can be applied to obtain quite accurate motion estimates. The most crucial step in the vision-based estimation of the vehicle motion consists on detecting matchings between image pairs. Here we propose the extensive use of texture analysis as a tool to ameliorate the correspondence problem in underwater images. Once a robust set of correspondences has been found, the three-dimensional motion of the vehicle can be computed with respect to the bed of the sea. Finally, motion estimates allow the construction of a map that could aid to the navigation of the robot

Proposal of a parallel architecture for a motion detection algorithm

This paper proposes a parallel architecture for estimation of the motion of an underwater robot. It is well known that image processing requires a huge amount of computation, mainly at low-level processing where the algorithms are dealing with a great number of data. In a motion estimation algorithm, correspondences between two images have to be solved at the low level. In the underwater imaging, normalised correlation can be a solution in the presence of non-uniform illumination. Due to its regular processing scheme, parallel implementation of the correspondence problem can be an adequate approach to reduce the computation time. Taking into consideration the complexity of the normalised correlation criteria, a new approach using parallel organisation of every processor from the architecture is proposed

Recovering Euclidean deformable models from stereo-motion

In this paper we present a novel structure from motion (SfM) approach able to infer 3D deformable models from uncalibrated stereo images. Using a stereo setup dramatically improves the 3D model estimation when the observed 3D shape is mostly deforming without undergoing strong rigid motion. Our approach first calibrates the stereo system automatically and then computes a single metric rigid structure for each frame. Afterwards, these 3D shapes are aligned to a reference view using a RANSAC method in order to compute the mean shape of the object and to select the subset of points on the object which have remained rigid throughout the sequence without deforming. The selected rigid points are then used to compute frame-wise shape registration and to extract the motion parameters robustly from frame to frame. Finally, all this information is used in a global optimization stage with bundle adjustment which allows to refine the frame-wise initial solution and also to recover the non-rigid 3D model. We show results on synthetic and real data that prove the performance of the proposed method even when there is no rigid motion in the original sequence

Online Robust 3D Mapping Using Structure from Motion Cues

This paper presents a complete solution for creating accurate 3D textured models from monocular video sequences. The methods are developed within the framework of sequential structure from motion, where a 3D model of the environment is maintained and updated as new visual information becomes available. The camera position is recovered by directly associating the 3D scene model with local image observations. Compared to standard structure from motion techniques, this approach decreases the error accumulation while increasing the robustness to scene occlusions and feature association failures. The obtained 3D information is used to generate high quality, composite visual maps of the scene (mosaics). The visual maps are used to create texture-mapped, realistic views of the scene

Predictive motion control of a mirosot mobile robot

This paper discusses predictive motion control of a MiRoSoT robot. The dynamic model of the robot is deduced by taking into account the whole process - robot, vision, control and transmission systems. Based on the obtained dynamic model, an integrated predictive control algorithm is proposed to position precisely with either stationary or moving obstacle avoidance. This objective is achieved automatically by introducing distant constraints into the open-loop optimization of control inputs. Simulation results demonstrate the feasibility of such control strategy for the deduced dynamic model

Comparison of topography- and aperture-dependent motion compensation algorithms for airborne SAR

This letter presents a comparison between threeFourier-based motion compensation (MoCo) algorithms forairborne synthetic aperture radar (SAR) systems. These algorithmscircumvent the limitations of conventional MoCo, namelythe assumption of a reference height and the beam-center approximation.All these approaches rely on the inherent time–frequencyrelation in SAR systems but exploit it differently, with the consequentdifferences in accuracy and computational burden. Aftera brief overview of the three approaches, the performance ofeach algorithm is analyzed with respect to azimuthal topographyaccommodation, angle accommodation, and maximum frequencyof track deviations with which the algorithm can cope. Also, ananalysis on the computational complexity is presented. Quantitativeresults are shown using real data acquired by the ExperimentalSAR system of the German Aerospace Center (DLR).