A user supported object tracking framework for interactive video production

We present a user supported tracking framework that combines automatic tracking with extended user input to create error free tracking results that are suitable for interactive video production. The goal of our approach is to keep the necessary user input as small as possible. In our framework, the user can select between different tracking algorithms - existing ones and new ones that are described in this paper. Furthermore, the user can automatically fuse the results of different tracking algorithms with our robust fusion approach. The tracked object can be marked in more than one frame, which can significantly improve the tracking result. After tracking, the user can validate the results in an easy way, thanks to the support of a powerful interpolation technique. The tracking results are iteratively improved until the complete track has been found. After the iterative editing process the tracking result of each object is stored in an interactive video file that can be loaded by our player for interactive videos.

EVA: Laparoscopic instrument tracking based on endoscopic video analysis for psychomotor skills assessment

INTRODUCTION: The EVA (Endoscopic Video Analysis) tracking system a new tracking system for extracting motions of laparoscopic instruments based on non-obtrusive video tracking was developed. The feasibility of using EVA in laparoscopic settings has been tested in a box trainer setup. METHODS: EVA makes use of an algorithm that employs information of the laparoscopic instrument's shaft edges in the image, the instrument's insertion point, and the camera's optical centre to track the 3D position of the instrument tip. A validation study of EVA comprised a comparison of the measurements achieved with EVA and the TrEndo tracking system. To this end, 42 participants (16 novices, 22 residents, and 4 experts) were asked to perform a peg transfer task in a box trainer. Ten motion-based metrics were used to assess their performance. RESULTS: Construct validation of the EVA has been obtained for seven motion-based metrics. Concurrent validation revealed that there is a strong correlation between the results obtained by EVA and the TrEndo for metrics such as path length (p=0,97), average speed (p=0,94) or economy of volume (p=0,85), proving the viability of EVA. CONCLUSIONS: EVA has been successfully used in the training setup showing potential of endoscopic video analysis to assess laparoscopic psychomotor skills. The results encourage further implementation of video tracking in training setups and in image guided surgery.

A multilevel approach to understanding circadian rhythmicity : social and sleep-dependent effects in ants and mice

Résumé françaisLa majorité des organismes vivants sont soumis à l'alternance du jour et de la nuit, conséquence de la rotation de la terre autour de son axe. Ils ont développé un système interne de mesure du temps, appelé horloge circadienne, leur permettant de s'adapter et de synchroniser leur comportement et leur physiologie aux cycles de lumière. Cette dernière est considérée comme étant le signal majeur entraînant l'horloge interne et. par conséquent, les rythmes journaliers d'éveil et de sommeil. Outre sa régulation circadienne, le sommeil est contrôlé par un processus homéostatique qui détermine son besoin. La contribution de ces deux processus dans le fonctionnement cellulaire du cerveau n'a pas encore été investiguée. La mesure de l'amplitude ainsi que de la prévalence des ondes delta de l'EEG (activité delta) constitue un index très fiable du besoin de sommeil. Il a été démontré que cette activité est génétiquement déterminée et associée à un locus de trait quantitatif situé sur le chromosome 13 de la souris.Grâce à des expériences de privation de sommeil et d'analyses de transcriptome du cerveau dans trois souches de souris présentant diverses réponses à la privation de sommeil, nous avons trouvé que Homerla, localisé dans la région d'intérêt du chromosome 13, est le meilleur marqueur du besoin de sommeil. Homerla est impliqué dans la récupération de l'hyperactivité neuronale induite par le glutamate, grâce à son effet tampon sur le calcium intracellulaire. Une fonction fondamentale du sommeil pourrait donc être de protéger le cerveau et de lui permettre de récupérer après une hyperactivité neuronale imposée par une veille prolongée.De plus, nous avons montré que 2032 transcrits sont exprimés rythmiqueraent dans le cerveau de la souris, parmi lesquels seulement 391 le restent après que les animaux aient été privés de sommeil à différents moments au cours des 24 heures. Cette observation montre clairement que la plupart des changements rythmiques au niveau du transcriptome dépendent du sommeil et non de l'horloge circadienne et souligne ainsi l'importance du sommeil dans la physiologie des mammifères.La plupart des expériences concernant les rythmes circadiens ont été réalisées sur des individus isolés en négligeant l'effet du contexte social sur les comportements circadiens. Les espèces sociales, telles que les fourmis, se caractérisent par une division du travail où une répartition des tâches s'effectue entre ses membres. De plus, certaines d'entre elles doivent être pratiquées en continu comme les soins au couvain tandis que d'autres requièrent une activité rythmique comme le fourragement. Ainsi la fourmi est un excellent modèle pour l'étude de 1 influence du contexte social sur les rythmes circadiens.A ces fins, nous avons décidé d'étudier les rythmes circadiens chez une espèce de fourmi Camponotus fellah et de caractériser au niveau moléculaire son horloge circadienne. Nous avons ainsi développé un système vidéo permettant de suivre l'activité locomotrice de tous les individus d'une colonie. Nos résultats montrent que, bien que la plupart des fourmis soient arythmiques à l'intérieur de la colonie, elles développent d'amples rythmes d'activité en isolation. De plus, ces rythmes disparaissent presque aussitôt que la fourmi est réintroduite dans la colonie. Cette rythmicité observée en isolation semble être générée par l'horloge circadienne car elle persiste en condition constante (obscurité totale). Nous avons ensuite regardé si cette apparente arythmie observée dans la colonie résultait d'un effet masquant des interactions sociales sur les rythmes circadiens d'activité. Nos résultats suggèrent que l'horloge interne est fonctionnelle dans la colonie mais que l'expression de ses rythmes au niveau comportemental est inhibée par les interactions sociales. Les analyses moléculaires du statut de l'horloge dans différents contextes sociaux sont actuellement en cours. Le contexte social semble donc un déterminant majeur du comportement circadien chez la fourmi.AbstractAlmost all living organisms on earth are subjected to the alternance of day and night re-sulting from the rotation of the earth around its axis. They have evolved with an internal timing system, termed the circadian clock, enabling them to adapt and synchronize their behavior and physiology to the daily changes in light and related environmental parame¬ters. Light is thought to be the major cue entraining the circadian clock and consequently the rhythms of rest/activity. In addition to its circadian dependent timing, sleep is reg¬ulated by a homeostatic process that determines its need. The contribution of these two processes in the cellular functioning of the brain has not yet been considered. A highly reliable index of the homeostatic process of sleep is the measure of the amplitude and prevalence of the EEG delta waves (delta activity). It has been shown that sleep need, measured by delta activity, is genetically determined and associated with a Quantitative Trait Locus (QTL) located on the mouse chromosome 13. By using sleep deprivation and brain transcriptome profiling in three inbred mouse strains showing different responses to sleep loss, we found that Homerla, localized within this QTL region is the best transcrip¬tional marker of sleep need. Interestingly Homerla is primarily involved in the recovery from glutamate-induced neuronal hyperactivity by its buffering effect on intracellular cal¬cium. A fundamental function of sleep may therefore reside in the protection and recovery of the brain from a neuronal hyperactivity imposed by prolonged wakefulness.Moreover, time course gene expression experiments showed that 2032 brain tran¬scripts present a rhythmic variation, but only 391 of those remain rhythmic when mice are sleep deprived at four time points around the clock. This finding clearly suggests that most changes in gene transcription over the day are sleep-wake dependent rather than clock dependent and underlines the importance of sleep in mammalian physiology.In the second part of this PhD, I was interested in the social influence on circadian behavior. Most experiments done in the circadian field have been performed on isolated individuals and have therefore ignored the effect of the social context on circadian behav-ior. Eusocial insect species such as ants are characterized by a division of labor: colony tasks are distributed among individuals, some of them requiring continuous activity such as nursing or rhythmic ones such as foraging. Thus ants represent a suitable model to study the influence of the social context on the circadian clock and its output rhythms.The aim of this part was to address the effect of social context on circadian rhythms in the ant species Camponotus fellah and to characterize its circadian clock at the molecu¬lar level. We therefore developed a video tracking system to follow the locomotor activity of all individuals in a colony. Our results show that most ants are arrhythmic within the colony, but develop, when subjected to social isolation, strong rhythms of activity that intriguingly disappear when individuals are reintroduced into the colony. The rhythmicity observed in isolated ants seems to be driven by the circadian clock as it persists under constant conditions (complete darkness). We then tested whether the apparent arrhyth- micity in the colony stemmed from a masking effect of social interactions on circadian rhythms. Indeed, we found that circadian clocks of ants in the colony are functional but their expression at the behavioral level is inhibited by social interactions. The molecular assessment of the circadian clock functional state in the different social context is still under investigation. Our results suggest that social context is a major determinant of circadian behavior in ants.

Holographic optical tweezers combined with back-focal-plane displacement detection

A major problem with holographic optical tweezers (HOTs) is their incompatibility with laser-based position detection methods, such as back-focal-plane interferometry (BFPI). The alternatives generally used with HOTs, like high-speed video tracking, do not offer the same spatial and temporal bandwidths. This has limited the use of this technique in precise quantitative experiments. In this paper, we present an optical trap design that combines digital holography and back-focal-plane displacement detection. We show that, with a particularly simple setup, it is possible to generate a set of multiple holographic traps and an additional static non-holographic trap with orthogonal polarizations and that they can be, therefore, easily separated for measuring positions and forces with the high positional and temporal resolutions of laser-based detection. We prove that measurements from both polarizations contain less than 1% crosstalk and that traps in our setup are harmonic within the typical range. We further tested the instrument in a DNA stretching experiment and we discuss an interesting property of this configuration: the small drift of the differential signal between traps.

Real-time Imaging and Mosaicking of Planar Surfaces

Laajojen pintojen kuvaaminen rajoitetussa työskentelytilassa riittävällä kuvatarkkuudella voi olla vaikeaa. Kuvaaminen on suoritettava osissa ja osat koottava saumattomaksi kokonaisnäkymäksi eli mosaiikkikuvaksi. Kuvauslaitetta käsin siirtelevän käyttäjän on saatava välitöntä palautetta, jotta mosaiikkiin ei jäisi aukkoja ja työ olisi nopeaa. Työn tarkoituksena oli rakentaa pieni, kannettava ja tarkka kuvauslaite paperi- ja painoteollisuuden tarpeisiin sekä kehittää palautteen antamiseen menetelmä, joka koostaaja esittää karkeaa mosaiikkikuvaa tosiajassa. Työssä rakennettiin kaksi kuvauslaitetta: ensimmäinen kuluttajille ja toinen teollisuuteen tarkoitetuista osista. Kuvamateriaali käsiteltiin tavallisella pöytätietokoneella. Videokuvien välinen liike laskettiin yksinkertaisella seurantamenetelmällä ja mosaiikkikuvaa koottiin kameroiden kuvanopeudella. Laskennallista valaistuksenkorjausta tutkittiin ja kehitetty menetelmä otettiin käyttöön. Ensimmäisessä kuvauslaitteessa on ongelmia valaistuksen ja linssivääristymien kanssa tuottaen huonolaatuisia mosaiikkikuvia. Toisessa kuvauslaitteessa nämä ongelmat on korjattu. Seurantamenetelmä toimii hyvin ottaen huomioon sen yksinkertaisuuden ja siihen ehdotetaan monia parannuksia. Työn tulokset osoittavat, että tosiaikainen mosaiikkikuvan koostaminen megapikselin kuvamateriaalista on mahdollista kuluttajille tarkoitetulla tietokonelaitteistolla.

Exploitation de contraintes photométriques et géométriques en vision : application au suivi, au calibrage et à la reconstruction

Cette thése a été réalisée dans le cadre d'une cotutelle avec l'Institut National Polytechnique de Grenoble (France). La recherche a été effectuée au sein des laboratoires de vision 3D (DIRO, UdM) et PERCEPTION-INRIA (Grenoble).

Détection des chutes par calcul homographique

La vidéosurveillance a pour objectif principal de protéger les personnes et les biens en détectant tout comportement anormal. Ceci ne serait possible sans la détection de mouvement dans l’image. Ce processus complexe se base le plus souvent sur une opération de soustraction de l’arrière-plan statique d’une scène sur l’image. Mais il se trouve qu’en vidéosurveillance, des caméras sont souvent en mouvement, engendrant ainsi, un changement significatif de l’arrière-plan; la soustraction de l’arrière-plan devient alors problématique. Nous proposons dans ce travail, une méthode de détection de mouvement et particulièrement de chutes qui s’affranchit de la soustraction de l’arrière-plan et exploite la rotation de la caméra dans la détection du mouvement en utilisant le calcul homographique. Nos résultats sur des données synthétiques et réelles démontrent la faisabilité de cette approche.

Behavior analysis of oithona in hypoxic/anoxic conditions

Senior thesis written for Oceanography 445

Hyperlocomotion associated with transient prenatal vitamin D deficiency is ameliorated by acute restraint

Transient prenatal vitamin D deficiency produces hyperlocomotion in the adult rat. The aim of this study was to examine the effects of acute restraint on the behaviour of DVD and control rats in the open field. Rats were conceived and born to developmentally vitamin D (DVD) deficient or replete (control) dams and, at 8 weeks of age, were monitored for 30 min in an open field using automated video tracking software. Half of the rats were restrained within a towel for 5 min immediately before the open field test. The remainder received minimal handling prior to the open field test. Repeating previous findings, DVD deficient animals had enhanced locomotion during the first 10 min of the open field test compared to control rats. By contrast, there were no differences in locomotor activity after acute restraint stress. The time rats spent in the corners and side of the open field was affected by prenatal diet. DVD rats spent less time in the corners and more time in the side than control rats across the whole 30 min test. This difference was not seen in rats with acute restraint stress. The time spent in the centre was not altered by prenatal diet or acute restraint. Thus, transient prenatal vitamin D deficiency induces a transient spontaneous hyperlocomotion in adulthood that is modulated by acute restraint stress. (c) 2006 Elsevier B.V. All rights reserved.

Human behaviour recognition in data-scarce domains

This paper presents the novel theory for performing multi-agent activity recognition without requiring large training corpora. The reduced need for data means that robust probabilistic recognition can be performed within domains where annotated datasets are traditionally unavailable. Complex human activities are composed from sequences of underlying primitive activities. We do not assume that the exact temporal ordering of primitives is necessary, so can represent complex activity using an unordered bag. Our three-tier architecture comprises low-level video tracking, event analysis and high-level inference. High-level inference is performed using a new, cascading extension of the Rao–Blackwellised Particle Filter. Simulated annealing is used to identify pairs of agents involved in multi-agent activity. We validate our framework using the benchmarked PETS 2006 video surveillance dataset and our own sequences, and achieve a mean recognition F-Score of 0.82. Our approach achieves a mean improvement of 17% over a Hidden Markov Model baseline.

Social dominance modulates eavesdropping in zebrafish

Group living animals may eavesdrop on signalling interactions between conspecifics and integrate it with their own past social experience in order to optimize the use of relevant information from others. However, little is known about this interplay between public (eavesdropped) and private social information. To investigate it, we first manipulated the dominance status of bystander zebrafish. Next, we either allowed or prevented bystanders from observing a fight. Finally, we assessed their behaviour towards the winners and losers of the interaction, using a custom-made video-tracking system and directional analysis. We found that only dominant bystanders who had seen the fight revealed a significant increase in directional focus (a measure of attention) towards the losers of the fights. Furthermore, our results indicate that information about the fighters' acquired status was collected from the signalling interaction itself and not from post-interaction status cues, which implies the existence of individual recognition in zebrafish. Thus, we show for the first time that zebrafish, a highly social model organism, eavesdrop on conspecific agonistic interactions and that this process is modulated by the eavesdroppers' dominance status. We suggest that this type of integration of public and private information may be ubiquitous in social learning processes.

People and object tracking for video annotation

Dissertação para obtenção do Grau de Mestre em Engenharia Informática

Video Object Tracking And Analysis For Computer Assisted Surgery

Pedicle screw insertion technique has made revolution in the surgical treatment of spinal fractures and spinal disorders. Although X- ray fluoroscopy based navigation is popular, there is risk of prolonged exposure to X- ray radiation. Systems that have lower radiation risk are generally quite expensive. The position and orientation of the drill is clinically very important in pedicle screw fixation. In this paper, the position and orientation of the marker on the drill is determined using pattern recognition based methods, using geometric features, obtained from the input video sequence taken from CCD camera. A search is then performed on the video frames after preprocessing, to obtain the exact position and orientation of the drill. An animated graphics, showing the instantaneous position and orientation of the drill is then overlaid on the processed video for real time drill control and navigation