Improved detection and tracking of objects in surveillance video

Surveillance networks are typically monitored by a few people, viewing several monitors displaying the camera feeds. It is then very difficult for a human operator to effectively detect events as they happen. Recently, computer vision research has begun to address ways to automatically process some of this data, to assist human operators. Object tracking, event recognition, crowd analysis and human identification at a distance are being pursued as a means to aid human operators and improve the security of areas such as transport hubs. The task of object tracking is key to the effective use of more advanced technologies. To recognize an event people and objects must be tracked. Tracking also enhances the performance of tasks such as crowd analysis or human identification. Before an object can be tracked, it must be detected. Motion segmentation techniques, widely employed in tracking systems, produce a binary image in which objects can be located. However, these techniques are prone to errors caused by shadows and lighting changes. Detection routines often fail, either due to erroneous motion caused by noise and lighting effects, or due to the detection routines being unable to split occluded regions into their component objects. Particle filters can be used as a self contained tracking system, and make it unnecessary for the task of detection to be carried out separately except for an initial (often manual) detection to initialise the filter. Particle filters use one or more extracted features to evaluate the likelihood of an object existing at a given point each frame. Such systems however do not easily allow for multiple objects to be tracked robustly, and do not explicitly maintain the identity of tracked objects. This dissertation investigates improvements to the performance of object tracking algorithms through improved motion segmentation and the use of a particle filter. A novel hybrid motion segmentation / optical flow algorithm, capable of simultaneously extracting multiple layers of foreground and optical flow in surveillance video frames is proposed. The algorithm is shown to perform well in the presence of adverse lighting conditions, and the optical flow is capable of extracting a moving object. The proposed algorithm is integrated within a tracking system and evaluated using the ETISEO (Evaluation du Traitement et de lInterpretation de Sequences vidEO - Evaluation for video understanding) database, and significant improvement in detection and tracking performance is demonstrated when compared to a baseline system. A Scalable Condensation Filter (SCF), a particle filter designed to work within an existing tracking system, is also developed. The creation and deletion of modes and maintenance of identity is handled by the underlying tracking system; and the tracking system is able to benefit from the improved performance in uncertain conditions arising from occlusion and noise provided by a particle filter. The system is evaluated using the ETISEO database. The dissertation then investigates fusion schemes for multi-spectral tracking systems. Four fusion schemes for combining a thermal and visual colour modality are evaluated using the OTCBVS (Object Tracking and Classification in and Beyond the Visible Spectrum) database. It is shown that a middle fusion scheme yields the best results and demonstrates a significant improvement in performance when compared to a system using either mode individually. Findings from the thesis contribute to improve the performance of semi-automated video processing and therefore improve security in areas under surveillance.

Vision-based detection and tracking of aerial targets for UAV collision avoidance

Machine vision represents a particularly attractive solution for sensing and detecting potential collision-course targets due to the relatively low cost, size, weight, and power requirements of the sensors involved (as opposed to radar). This paper describes the development and evaluation of a vision-based collision detection algorithm suitable for fixed-wing aerial robotics. The system was evaluated using highly realistic vision data of the moments leading up to a collision. Based on the collected data, our detection approaches were able to detect targets at distances ranging from 400m to about 900m. These distances (with some assumptions about closing speeds and aircraft trajectories) translate to an advanced warning of between 8-10 seconds ahead of impact, which approaches the 12.5 second response time recommended for human pilots. We make use of the enormous potential of graphic processing units to achieve processing rates of 30Hz (for images of size 1024-by- 768). Currently, integration in the final platform is under way.

Description and evaluation of a Newton-based electronic appetite rating system for temporal tracking of appetite in human subjects

This study assessed the reliability and validity of a palm-top-based electronic appetite rating system (EARS) in relation to the traditional paper and pen method. Twenty healthy subjects [10 male (M) and 10 female (F)] — mean age M=31 years (S.D.=8), F=27 years (S.D.=5); mean BMI M=24 (S.D.=2), F=21 (S.D.=5) — participated in a 4-day protocol. Measurements were made on days 1 and 4. Subjects were given paper and an EARS to log hourly subjective motivation to eat during waking hours. Food intake and meal times were fixed. Subjects were given a maintenance diet (comprising 40% fat, 47% carbohydrate and 13% protein by energy) calculated at 1.6×Resting Metabolic Rate (RMR), as three isoenergetic meals. Bland and Altman's test for bias between two measurement techniques found significant differences between EARS and paper and pen for two of eight responses (hunger and fullness). Regression analysis confirmed that there were no day, sex or order effects between ratings obtained using either technique. For 15 subjects, there was no significant difference between results, with a linear relationship between the two methods that explained most of the variance (r2 ranged from 62.6 to 98.6). The slope for all subjects was less than 1, which was partly explained by a tendency for bias at the extreme end of results on the EARS technique. These data suggest that the EARS is a useful and reliable technique for real-time data collection in appetite research but that it should not be used interchangeably with paper and pen techniques.

The use of visual analogue scales to assess motivation to eat in human subjects: a review of their reliability and validity with an evaluation of new hand-held computerized stsytems for temporal tracking of appetite ratings.

This present paper reviews the reliability and validity of visual analogue scales (VAS) in terms of (1) their ability to predict feeding behaviour, (2) their sensitivity to experimental manipulations, and (3) their reproducibility. VAS correlate with, but do not reliably predict, energy intake to the extent that they could be used as a proxy of energy intake. They do predict meal initiation in subjects eating their normal diets in their normal environment. Under laboratory conditions, subjectively rated motivation to eat using VAS is sensitive to experimental manipulations and has been found to be reproducible in relation to those experimental regimens. Other work has found them not to be reproducible in relation to repeated protocols. On balance, it would appear, in as much as it is possible to quantify, that VAS exhibit a good degree of within-subject reliability and validity in that they predict with reasonable certainty, meal initiation and amount eaten, and are sensitive to experimental manipulations. This reliability and validity appears more pronounced under the controlled (but more arti®cial) conditions of the laboratory where the signal : noise ratio in experiments appears to be elevated relative to real life. It appears that VAS are best used in within-subject, repeated-measures designs where the effect of different treatments can be compared under similar circumstances. They are best used in conjunction with other measures (e.g. feeding behaviour, changes in plasma metabolites) rather than as proxies for these variables. New hand-held electronic appetite rating systems (EARS) have been developed to increase reliability of data capture and decrease investigator workload. Recent studies have compared these with traditional pen and paper (P&P) VAS. The EARS have been found to be sensitive to experimental manipulations and reproducible relative to P&P. However, subjects appear to exhibit a signi®cantly more constrained use of the scale when using the EARS relative to the P&P. For this reason it is recommended that the two techniques are not used interchangeably

Fixed-wing attitude estimation using temporal tracking of the horizon and optical flow

In this paper, a method has been developed for estimating pitch angle, roll angle and aircraft body rates based on horizon detection and temporal tracking using a forward-looking camera, without assistance from other sensors. Using an image processing front-end, we select several lines in an image that may or may not correspond to the true horizon. The optical flow at each candidate line is calculated, which may be used to measure the body rates of the aircraft. Using an Extended Kalman Filter (EKF), the aircraft state is propagated using a motion model and a candidate horizon line is associated using a statistical test based on the optical flow measurements and the location of the horizon. Once associated, the selected horizon line, along with the associated optical flow, is used as a measurement to the EKF. To test the accuracy of the algorithm, two flights were conducted, one using a highly dynamic Uninhabited Airborne Vehicle (UAV) in clear flight conditions and the other in a human-piloted Cessna 172 in conditions where the horizon was partially obscured by terrain, haze and smoke. The UAV flight resulted in pitch and roll error standard deviations of 0.42◦ and 0.71◦ respectively when compared with a truth attitude source. The Cessna flight resulted in pitch and roll error standard deviations of 1.79◦ and 1.75◦ respectively. The benefits of selecting and tracking the horizon using a motion model and optical flow rather than naively relying on the image processing front-end is also demonstrated.

Cooperative multi-AUV tracking of phytoplankton blooms based on ocean model predictions

In recent years, ocean scientists have started to employ many new forms of technology as integral pieces in oceanographic data collection for the study and prediction of complex and dynamic ocean phenomena. One area of technological advancement in ocean sampling if the use of Autonomous Underwater Vehicles (AUVs) as mobile sensor plat- forms. Currently, most AUV deployments execute a lawnmower- type pattern or repeated transects for surveys and sampling missions. An advantage of these missions is that the regularity of the trajectory design generally makes it easier to extract the exact path of the vehicle via post-processing. However, if the deployment region for the pattern is poorly selected, the AUV can entirely miss collecting data during an event of specific interest. Here, we consider an innovative technology toolchain to assist in determining the deployment location and executed paths for AUVs to maximize scientific information gain about dynamically evolving ocean phenomena. In particular, we provide an assessment of computed paths based on ocean model predictions designed to put AUVs in the right place at the right time to gather data related to the understanding of algal and phytoplankton blooms.

Learning object dynamics for smooth tracking of moving lip contours

A new technique is proposed for learning the dynamic characteristics of a deformable object, applied in particular to the problem of lip-tracking. Experimental results are given which demonstrate that the use of dynamic models allows the system to track more robustly under adverse conditions and to correct spurious, poorly tracked frames

Temporal tracking of mineralization and transcriptional developments of shell formation during the early life history of pearl oyster Pinctada maxima

Molluscan larval ontogeny is a highly conserved process comprising three principal developmental stages. A characteristic unique to each of these stages is shell design, termed prodissoconch I, prodissoconch II and dissoconch. These shells vary in morphology, mineralogy and microstructure. The discrete temporal transitions in shell biomineralization between these larval stages are utilized in this study to investigate transcriptional involvement in several distinct biomineralization events. Scanning electron microscopy and X-ray diffraction analysis of P. maxima larvae and juveniles collected throughout post-embryonic ontogenesis, document the mineralogy and microstructure of each shelled stage as well as establishing a timeline for transitions in biomineralization. P. maxima larval samples most representative of these biomineralization distinctions and transitions were analyzed for differential gene expression on the microarray platform PmaxArray 1.0. A number of transcripts are reported as differentially expressed in correlation to the mineralization events of P. maxima larval ontogeny. Some of those isolated are known shell matrix genes while others are novel; these are discussed in relation to potential shell formation roles. This interdisciplinary investigation has linked the shell developments of P. maxima larval ontogeny with corresponding gene expression profiles, furthering the elucidation of shell biomineralization.

Autonomous aerial navigation and tracking of marine animals

In this paper, we describe the development of an independent and on-board visual servoing system which allows a computationally impoverished aerial vehicle to autonomously identify and track a moving surface target. Our image segmentation and target identification algorithms were developed with the specific task of monitoring whales at sea but could be adapted for other targets. Observing whales is important for many marine biology tasks and is currently performed manually from the shore or from boats. We also present hardware experiments which demonstrate the capabilities of our algorithms for object identification and tracking that enable a flying vehicle to track a moving target.

Comparative eye tracking of experts and novices in web single sign-on

Security indicators in web browsers alert users to the presence of a secure connection between their computer and a web server; many studies have shown that such indicators are largely ignored by users in general. In other areas of computer security, research has shown that technical expertise can decrease user susceptibility to attacks. In this work, we examine whether computer or security expertise affects use of web browser security indicators. Our study takes place in the context of web-based single sign-on, in which a user can use credentials from a single identity provider to login to many relying websites; single sign-on is a more complex, and hence more difficult, security task for users. In our study, we used eye trackers and surveyed participants to examine the cues individuals use and those they report using, respectively. Our results show that users with security expertise are more likely to self-report looking at security indicators, and eye-tracking data shows they have longer gaze duration at security indicators than those without security expertise. However, computer expertise alone is not correlated with recorded use of security indicators. In survey questions, neither experts nor novices demonstrate a good understanding of the security consequences of web-based single sign-on.

SERS substrate for detection of explosives

A novel gold coated femtosecond laser nanostructured sapphire surface – an “optical nose” - based on surface-enhanced Raman spectroscopy (SERS) for detecting vapours of explosive substances was investigated. Four different nitroaromatic vapours at room temperature were tested. Sensor responses were unambiguous and showed response in the range of 0.05 – 15 uM at 25 °C. The laser fabricated substrate nanostructures produced up to an eight-fold increase in Raman signal over that observed on the unstructured portions of the substrate. This work demonstrates a simple sensing system that is compatible with commercial manufacturing practices to detect taggants in explosives which can undertake as part of an integrated security or investigative mission.

A multi-genome analysis approach enables tracking of the invasion of a single Russian wheat aphid (Diuraphis noxia) clone throughout the New World

This study investigated the population genetics, demographic history and pathway of invasion of the Russian wheat aphid (RWA) from its native range in Central Asia, the Middle East and Europe to South Africa and the Americas. We screened microsatellite markers, mitochondrial DNA and endosymbiont genes in 504 RWA clones from nineteen populations worldwide. Following pathway analyses of microsatellite and endosymbiont data, we postulate that Turkey and Syria were the most likely sources of invasion to Kenya and South Africa, respectively. Furthermore, we found that one clone transferred between South Africa and the Americas was most likely responsible for the New World invasion. Finally, endosymbiont DNA was found to be a high resolution population genetic marker, extremely useful for studies of invasion over a relatively short evolutionary history time frame. This study has provided valuable insights into the factors that may have facilitated the recent global invasion by this damaging pest.

Tracking of physical activity in young children

The purpose of this study was to determine whether physical activity behavior tracks during early childhood. Forty-seven children (22 males, 25 females) aged 3-4 yr at the beginning of the study were followed over a 3-yr period. Heart rates were measured at least 2 and up to 4 d . yr(-1) with a Quantum XL Telemetry heart rate monitor. Physical activity was quantified as the percentage of observed minutes between 3:00 and 6:00 p.m. during which heart rate was 50% or more above individual resting heart rate (PAHR-50 Index). Tracking of physical activity was analyzed using Pearson and Spearman correlations. Yearly PAHR-50 index tertiles were created and examined for percent agreement and Cohen's kappa. Repeated measures ANOVA was used to calculate the intraclass correlation coefficient across the 3 yr of the study. Spearman rank order correlations ranged from 0.57 to 0.66 (P < 0.0001). Percent agreement ranged from 49% to 62%. The intraclass R for the 3 yr was 0.81. It was concluded that physical activity behavior tends to track during early childhood.

Tracking of physical activity, physical inactivity, and health-related physical fitness in rural youth

This study examined the tracking of selected measures of physical activity, inactivity, and fitness in a cohort of rural youth. Students (N = 181, 54.7% female, 63.5% African American) completed test batteries during their fifth-(age = 10.7 +/- 0.7 years), sixth-, and seventh-grade years. The Previous Day Physical Activity Recall (PDPAR) was used to assess 30-min blocks of vigorous physical activity (VPA), moderate-to-vigorous physical activity (MVPA), TV watching and other sedentary activities, and estimated energy expenditure (EE). Fitness measures included the PWC 170 cycle ergometer test, strength tests, triceps skinfold thickness, and BMI. Intraclass correlation coefficients (ICCs) for VPA, MVPA, and after-school EE ranged from 0.63 to 0.78. ICCs ranged from 0.49 to 0.71 for measures of inactivity and from 0.78 to 0.82 for the fitness measures. These results indicate that measures of physical activity, inactivity, and physical fitness tend to track during the transition from elementary to middle school.