Automatic camera control in virtual environments augmented using multiple sparse videos

Automated virtual camera control has been widely used in animation and interactive virtual environments. We have developed a multiple sparse camera based free view video system prototype that allows users to control the position and orientation of a virtual camera, enabling the observation of a real scene in three dimensions (3D) from any desired viewpoint. Automatic camera control can be activated to follow selected objects by the user. Our method combines a simple geometric model of the scene composed of planes (virtual environment), augmented with visual information from the cameras and pre-computed tracking information of moving targets to generate novel perspective corrected 3D views of the virtual camera and moving objects. To achieve real-time rendering performance, view-dependent textured mapped billboards are used to render the moving objects at their correct locations and foreground masks are used to remove the moving objects from the projected video streams. The current prototype runs on a PC with a common graphics card and can generate virtual 2D views from three cameras of resolution 768 x 576 with several moving objects at about 11 fps. (C)2011 Elsevier Ltd. All rights reserved.

A specific pole placement technique to minimise hardware requirements within a real time terrestrial and cable video echo canceller

The real time hardware architecture of a deterministic video echo canceller (deghoster) system is presented. The deghoster is capable of calculating all the multipath channel distortion characteristics from terrestrial and cable television in one single pass while performing real time video in-line ghost cancellation. The results from the actual system are also presented in this paper.

Real-time Imaging and Mosaicking of Planar Surfaces

Laajojen pintojen kuvaaminen rajoitetussa työskentelytilassa riittävällä kuvatarkkuudella voi olla vaikeaa. Kuvaaminen on suoritettava osissa ja osat koottava saumattomaksi kokonaisnäkymäksi eli mosaiikkikuvaksi. Kuvauslaitetta käsin siirtelevän käyttäjän on saatava välitöntä palautetta, jotta mosaiikkiin ei jäisi aukkoja ja työ olisi nopeaa. Työn tarkoituksena oli rakentaa pieni, kannettava ja tarkka kuvauslaite paperi- ja painoteollisuuden tarpeisiin sekä kehittää palautteen antamiseen menetelmä, joka koostaaja esittää karkeaa mosaiikkikuvaa tosiajassa. Työssä rakennettiin kaksi kuvauslaitetta: ensimmäinen kuluttajille ja toinen teollisuuteen tarkoitetuista osista. Kuvamateriaali käsiteltiin tavallisella pöytätietokoneella. Videokuvien välinen liike laskettiin yksinkertaisella seurantamenetelmällä ja mosaiikkikuvaa koottiin kameroiden kuvanopeudella. Laskennallista valaistuksenkorjausta tutkittiin ja kehitetty menetelmä otettiin käyttöön. Ensimmäisessä kuvauslaitteessa on ongelmia valaistuksen ja linssivääristymien kanssa tuottaen huonolaatuisia mosaiikkikuvia. Toisessa kuvauslaitteessa nämä ongelmat on korjattu. Seurantamenetelmä toimii hyvin ottaen huomioon sen yksinkertaisuuden ja siihen ehdotetaan monia parannuksia. Työn tulokset osoittavat, että tosiaikainen mosaiikkikuvan koostaminen megapikselin kuvamateriaalista on mahdollista kuluttajille tarkoitetulla tietokonelaitteistolla.

Real-time Image-based RGB-D Camera Motion Tracking and Environment Mapping

In this work, image based estimation methods, also known as direct methods, are studied which avoid feature extraction and matching completely. Cost functions use raw pixels as measurements and the goal is to produce precise 3D pose and structure estimates. The cost functions presented minimize the sensor error, because measurements are not transformed or modified. In photometric camera pose estimation, 3D rotation and translation parameters are estimated by minimizing a sequence of image based cost functions, which are non-linear due to perspective projection and lens distortion. In image based structure refinement, on the other hand, 3D structure is refined using a number of additional views and an image based cost metric. Image based estimation methods are particularly useful in conditions where the Lambertian assumption holds, and the 3D points have constant color despite viewing angle. The goal is to improve image based estimation methods, and to produce computationally efficient methods which can be accomodated into real-time applications. The developed image-based 3D pose and structure estimation methods are finally demonstrated in practise in indoor 3D reconstruction use, and in a live augmented reality application.

Evaluation of treatment response after chemoembolisation (TACE) in hepatocellular carcinoma using real time image fusion of contrast-enhanced ultrasound (CEUS) and computed tomography (CT)--preliminary results.

OBJECTIVE To evaluate treatment response of hepatocellular carcinoma (HCC) after transarterial chemoembolization (TACE) with a new real-time imaging fusion technique of contrast-enhanced ultrasound (CEUS) with multi-slice detection computed tomography (CT) in comparison to conventional post-interventional follow-up. MATERIAL AND METHODS 40 patients with HCC (26 male, ages 46-81 years) were evaluated 24 hours after TACE using CEUS with ultrasound volume navigation and image fusion with CT compared to non-enhanced CT and follow-up contrast-enhanced CT after 6-8 weeks. Reduction of tumor vascularization to less than 25% was regarded as "successful" treatment, whereas reduction to levels >25% was considered as "partial" treatment response. Homogenous lipiodol retention was regarded as successful treatment in non-enhanced CT. RESULTS Post-interventional image fusion of CEUS with CT was feasible in all 40 patients. In 24 patients (24/40), post-interventional image fusion with CEUS revealed residual tumor vascularity, that was confirmed by contrast-enhanced CT 6-8 weeks later in 24/24 patients. In 16 patients (16/40), post-interventional image fusion with CEUS demonstrated successful treatment, but follow-up CT detected residual viable tumor (6/16). Non-enhanced CT did not identify any case of treatment failure. Image fusion with CEUS assessed treatment efficacy with a specificity of 100%, sensitivity of 80% and a positive predictive value of 1 (negative predictive value 0.63). CONCLUSIONS Image fusion of CEUS with CT allows a reliable, highly specific post-interventional evaluation of embolization response with good sensitivity without any further radiation exposure. It can detect residual viable tumor at early state, resulting in a close patient monitoring or re-therapy.

Development of a real time Image analysis system for traffic monitoring applications.

Image processing offers unparalleled potential for traffic monitoring and control. For many years engineers have attempted to perfect the art of automatic data abstraction from sequences of video images. This paper outlines a research project undertaken at Napier University by the authors in the field of image processing for automatic traffic analysis. A software based system implementing TRIP algorithms to count cars and measure vehicle speed has been developed by members of the Transport Engineering Research Unit (TERU) at the University. The TRIP algorithm has been ported and evaluated on an IBM PC platform with a view to hardware implementation of the pre-processing routines required for vehicle detection. Results show that a software based traffic counting system is realisable for single window processing. Due to the high volume of data required to be processed for full frames or multiple lanes, system operations in real time are limited. Therefore specific hardware is required to be designed. The paper outlines a hardware design for implementation of inter-frame and background differencing, background updating and shadow removal techniques. Preliminary results showing the processing time and counting accuracy for the routines implemented in software are presented and a real time hardware pre-processing architecture is described.

Efficient Image and Video Representations for Retrieval

Image (Video) retrieval is an interesting problem of retrieving images (videos) similar to the query. Images (Videos) are represented in an input (feature) space and similar images (videos) are obtained by finding nearest neighbors in the input representation space. Numerous input representations both in real valued and binary space have been proposed for conducting faster retrieval. In this thesis, we present techniques that obtain improved input representations for retrieval in both supervised and unsupervised settings for images and videos. Supervised retrieval is a well known problem of retrieving same class images of the query. We address the practical aspects of achieving faster retrieval with binary codes as input representations for the supervised setting in the first part, where binary codes are used as addresses into hash tables. In practice, using binary codes as addresses does not guarantee fast retrieval, as similar images are not mapped to the same binary code (address). We address this problem by presenting an efficient supervised hashing (binary encoding) method that aims to explicitly map all the images of the same class ideally to a unique binary code. We refer to the binary codes of the images as `Semantic Binary Codes' and the unique code for all same class images as `Class Binary Code'. We also propose a new class­ based Hamming metric that dramatically reduces the retrieval times for larger databases, where only hamming distance is computed to the class binary codes. We also propose a Deep semantic binary code model, by replacing the output layer of a popular convolutional Neural Network (AlexNet) with the class binary codes and show that the hashing functions learned in this way outperforms the state­ of ­the art, and at the same time provide fast retrieval times. In the second part, we also address the problem of supervised retrieval by taking into account the relationship between classes. For a given query image, we want to retrieve images that preserve the relative order i.e. we want to retrieve all same class images first and then, the related classes images before different class images. We learn such relationship aware binary codes by minimizing the similarity between inner product of the binary codes and the similarity between the classes. We calculate the similarity between classes using output embedding vectors, which are vector representations of classes. Our method deviates from the other supervised binary encoding schemes as it is the first to use output embeddings for learning hashing functions. We also introduce new performance metrics that take into account the related class retrieval results and show significant gains over the state­ of­ the art. High Dimensional descriptors like Fisher Vectors or Vector of Locally Aggregated Descriptors have shown to improve the performance of many computer vision applications including retrieval. In the third part, we will discuss an unsupervised technique for compressing high dimensional vectors into high dimensional binary codes, to reduce storage complexity. In this approach, we deviate from adopting traditional hyperplane hashing functions and instead learn hyperspherical hashing functions. The proposed method overcomes the computational challenges of directly applying the spherical hashing algorithm that is intractable for compressing high dimensional vectors. A practical hierarchical model that utilizes divide and conquer techniques using the Random Select and Adjust (RSA) procedure to compress such high dimensional vectors is presented. We show that our proposed high dimensional binary codes outperform the binary codes obtained using traditional hyperplane methods for higher compression ratios. In the last part of the thesis, we propose a retrieval based solution to the Zero shot event classification problem - a setting where no training videos are available for the event. To do this, we learn a generic set of concept detectors and represent both videos and query events in the concept space. We then compute similarity between the query event and the video in the concept space and videos similar to the query event are classified as the videos belonging to the event. We show that we significantly boost the performance using concept features from other modalities.

Learning-Based Approach to Real Time Tracking and Analysis of Faces

This paper describes a trainable system capable of tracking faces and facialsfeatures like eyes and nostrils and estimating basic mouth features such as sdegrees of openness and smile in real time. In developing this system, we have addressed the twin issues of image representation and algorithms for learning. We have used the invariance properties of image representations based on Haar wavelets to robustly capture various facial features. Similarly, unlike previous approaches this system is entirely trained using examples and does not rely on a priori (hand-crafted) models of facial features based on optical flow or facial musculature. The system works in several stages that begin with face detection, followed by localization of facial features and estimation of mouth parameters. Each of these stages is formulated as a problem in supervised learning from examples. We apply the new and robust technique of support vector machines (SVM) for classification in the stage of skin segmentation, face detection and eye detection. Estimation of mouth parameters is modeled as a regression from a sparse subset of coefficients (basis functions) of an overcomplete dictionary of Haar wavelets.

Real-time image processing for crop/weed discrimination in maize fields

This paper presents a computer vision system that successfully discriminates between weed patches and crop rows under uncontrolled lighting in real-time. The system consists of two independent subsystems, a fast image processing delivering results in real-time (Fast Image Processing, FIP), and a slower and more accurate processing (Robust Crop Row Detection, RCRD) that is used to correct the first subsystem's mistakes. This combination produces a system that achieves very good results under a wide variety of conditions. Tested on several maize videos taken of different fields and during different years, the system successfully detects an average of 95% of weeds and 80% of crops under different illumination, soil humidity and weed/crop growth conditions. Moreover, the system has been shown to produce acceptable results even under very difficult conditions, such as in the presence of dramatic sowing errors or abrupt camera movements. The computer vision system has been developed for integration into a treatment system because the ideal setup for any weed sprayer system would include a tool that could provide information on the weeds and crops present at each point in real-time, while the tractor mounting the spraying bar is moving

Incorporating global positioning data in real time P2P audio/video streams for mobile devices

We propose an original method to geoposition an audio/video stream with multiple emitters that are at the same time receivers of the mixed signal. The achieved method is suitable for those comes where a list of positions within a designated area is encoded with a degree of precision adjusted to the visualization capabilities; and is also easily extensible to support new requirements. This method extends a previously proposed protocol, without incurring in any performance penalty.

Deep Corals, Deep Learning: Moving the Deep Net Towards Real-Time Image Annotation

The mismatch between human capacity and the acquisition of Big Data such as Earth imagery undermines commitments to Convention on Biological Diversity (CBD) and Aichi targets. Artificial intelligence (AI) solutions to Big Data issues are urgently needed as these could prove to be faster, more accurate, and cheaper. Reducing costs of managing protected areas in remote deep waters and in the High Seas is of great importance, and this is a realm where autonomous technology will be transformative.

Real-time monitoring and kinetic parameter estimation of the affinity interaction of jArtinM and rArtinM with peroxidase glycoprotein by the electrogravimetric technique

ArtinM is a D-mannose binding lectin that has been arousing increasing interest because of its biomedical properties, especially those involving the stimulation of Th1 immune response, which confers protection against intracellular pathogens The potential pharmaceutical applications of ArtinM have motivated the production of its recombinant form (rArtinM) so that it is important to compare the sugar-binding properties of jArtinM and rArtinM in order to take better advantage of the potential applications of the recombinant lectin. In this work, a biosensor framework based on a Quartz Crystal Microbalance was established with the purpose of making a comparative study of the activity of native and recombinant ArtinM protein The QCM transducer was strategically functionalized to use a simple model of protein binding kinetics. This approach allowed for the determination of the binding/dissociation kinetics rate and affinity equilibrium constant of both forms of ArtinM with horseradish peroxidase glycoprotein (HRP), a N-glycosylated protein that contains the trimannoside Man alpha 1-3[Man alpha 1-6]Man, which is a known ligand for jArtinM (Jeyaprakash et al, 2004). Monitoring of the real-time binding of rArtinM shows that it was able to bind HRP, leading to an analytical curve similar to that of jArtinM, with statistically equivalent kinetic rates and affinity equilibrium constants for both forms of ArtinM The lower reactivity of rArtinM with HRP than jArtinM was considered to be due to a difference in the number of Carbohydrate Recognition Domains (CRDs) per molecule of each lectin form rather than to a difference in the energy of binding per CRD of each lectin form. (C) 2010 Elsevier B V. All rights reserved

Genetic associations between carcass traits measured by real-time ultrasound and scrotal circumference and growth traits in Nelore cattle

The aim of the present study was to evaluate the genetic correlations among real-time ultrasound carcass, BW, and scrotal circumference (SC) traits in Nelore cattle. Carcass traits, measured by real-time ultrasound of the live animal, were recorded from 2002 to 2004 on 10 farms across 6 Brazilian states on 2,590 males and females ranging in age from 450 to 599 d. Ultrasound records of LM area (LMA) and backfat thickness (BF) were obtained from cross-sectional images between the 12th and 13th ribs, and rump fat thickness (RF) was measured between the hook and pin bones over the junction between gluteus medius and biceps femoris muscles. Also, BW (n = 22,778) and SC ( n = 5,695) were recorded on animals born between 1998 and 2003. The BW traits were 120, 210, 365, 450, and 550-d standardized BW (W120, W210, W365, W450, and W550), plus BW (WS) and hip height (HH) on the ultrasound scanning date. The SC traits were 365-, 450-, and 550-d standardized SC (SC365, SC450, and SC550). For the BW and SC traits, the database used was from the Nelore Breeding Program-Nelore Brazil. The genetic parameters were estimated with multivariate animal models and REML. Estimated genetic correlations between LMA and other traits were 0.06 (BF), -0.04 ( RF), 0.05 (HH), 0.58 (WS), 0.53 (W120), 0.62 (W210), 0.67 (W365), 0.64 ( W450 and W550), 0.28 (SC365), 0.24 (SC450), and 0.00 ( SC550). Estimated genetic correlations between BF and with other traits were 0.74 ( RF), -0.32 (HH), 0.19 (WS), -0.03 (W120), -0.10 (W210), 0.04 (W365), 0.01 (W450), 0.06 ( W550), 0.17 (SC365 and SC450), and -0.19 (SC550). Estimated genetic correlations between RF and other traits were -0.41 (HH), -0.09 (WS), -0.13 ( W120), -0.09 ( W210), -0.01 ( W365), 0.02 (W450), 0.03 (W550), 0.05 ( SC365), 0.11 ( SC450), and -0.18 (SC550). These estimates indicate that selection for carcass traits measured by real-time ultrasound should not cause antagonism in the genetic improvement of SC and BW traits. Also, selection to increase HH might decrease subcutaneous fat as correlated response. Therefore, to obtain animals suited to specific tropical production systems, carcass, BW, and SC traits should be considered in selection programs.

Real-time tracking and reporting of dynamic events in hierarchical wireless sensor networks

Wireless Sensor Networks (WSNs) are highly distributed systems in which resource allocation (bandwidth, memory) must be performed efficiently to provide a minimum acceptable Quality of Service (QoS) to the regions where critical events occur. In fact, if resources are statically assigned independently from the location and instant of the events, these resources will definitely be misused. In other words, it is more efficient to dynamically grant more resources to sensor nodes affected by critical events, thus providing better network resource management and reducing endto- end delays of event notification and tracking. In this paper, we discuss the use of a WSN management architecture based on the active network management paradigm to provide the real-time tracking and reporting of dynamic events while ensuring efficient resource utilization. The active network management paradigm allows packets to transport not only data, but also program scripts that will be executed in the nodes to dynamically modify the operation of the network. This presumes the use of a runtime execution environment (middleware) in each node to interpret the script. We consider hierarchical (e.g. cluster-tree, two-tiered architecture) WSN topologies since they have been used to improve the timing performance of WSNs as they support deterministic medium access control protocols.

Development of a new chlamydiales-specific real-time PCR and its application to respiratory clinical samples.

Originally composed of the single family Chlamydiaceae, the Chlamydiales order has extended considerably over the last several decades. Chlamydia-related bacteria were added and classified into six different families and family-level lineages: the Criblamydiaceae, Parachlamydiaceae, Piscichlamydiaceae, Rhabdochlamydiaceae, Simkaniaceae, and Waddliaceae. While several members of the Chlamydiaceae family are known pathogens, recent studies showed diverse associations of Chlamydia-related bacteria with human and animal infections. Some of these latter bacteria might be of medical importance since, given their ability to replicate in free-living amoebae, they may also replicate efficiently in other phagocytic cells, including cells of the innate immune system. Thus, a new Chlamydiales-specific real-time PCR targeting the conserved 16S rRNA gene was developed. This new molecular tool can detect at least five DNA copies and show very high specificity without cross-amplification from other bacterial clade DNA. The new PCR was validated with 128 clinical samples positive or negative for Chlamydia trachomatis or C. pneumoniae. Of 65 positive samples, 61 (93.8%) were found to be positive with the new PCR. The four discordant samples, retested with the original test, were determined to be negative or below detection limits. Then, the new PCR was applied to 422 nasopharyngeal swabs taken from children with or without pneumonia; a total of 48 (11.4%) samples were determined to be positive, and 45 of these were successfully sequenced. The majority of the sequences corresponded to Chlamydia-related bacteria and especially to members of the Parachlamydiaceae family.