Visual surveillance using deformable models of vehicles

This paper presents recent developments to a vision-based traffic surveillance system which relies extensively on the use of geometrical and scene context. Firstly, a highly parametrised 3-D model is reported, able to adopt the shape of a wide variety of different classes of vehicle (e.g. cars, vans, buses etc.), and its subsequent specialisation to a generic car class which accounts for commonly encountered types of car (including saloon, batchback and estate cars). Sample data collected from video images, by means of an interactive tool, have been subjected to principal component analysis (PCA) to define a deformable model having 6 degrees of freedom. Secondly, a new pose refinement technique using “active” models is described, able to recover both the pose of a rigid object, and the structure of a deformable model; an assessment of its performance is examined in comparison with previously reported “passive” model-based techniques in the context of traffic surveillance. The new method is more stable, and requires fewer iterations, especially when the number of free parameters increases, but shows somewhat poorer convergence. Typical applications for this work include robot surveillance and navigation tasks.

Vehicle subtype, make and model classification from side profile video

This paper addresses the challenging domain of vehicle classification from pole-mounted roadway cameras, specifically from side-profile views. A new public vehicle dataset is made available consisting of over 10000 side profile images (86 make/model and 9 sub-type classes). 5 state-of-the-art classifiers are applied to the dataset, with the best achieving high classification rates of 98.7% for sub-type and 99.7- 99.9% for make and model recognition, confirming the assertion made that single vehicle side profile images can be used for robust classification.

Meeting detection in video through semantic analysis

In this paper we present a novel approach to detect people meeting. The proposed approach works by translating people behaviour from trajectory information into semantic terms. Having available a semantic model of the meeting behaviour, the event detection is performed in the semantic domain. The model is learnt employing a soft-computing clustering algorithm that combines trajectory information and motion semantic terms. A stable representation can be obtained from a series of examples. Results obtained on a series of videos with different types of meeting situations show that the proposed approach can learn a generic model that can effectively be applied on the behaviour recognition of meeting situations.

Vision-based target tracking and surveillance with robust set-valued state estimation

Tracking a target from a video stream (or a sequence of image frames) involves nonlinear measurements in Cartesian coordinates. However, the target dynamics, modeled in Cartesian coordinates, result in a linear system. We present a robust linear filter based on an analytical nonlinear to linear measurement conversion algorithm. Using ideas from robust control theory, a rigorous theoretical analysis is given which guarantees that the state estimation error for the filter is bounded, i.e., a measure against filter divergence is obtained. In fact, an ellipsoidal set-valued estimate is obtained which is guaranteed to contain the true target location with an arbitrarily high probability. The algorithm is particularly suited to visual surveillance and tracking applications involving targets moving on a plane.

Ozbot and haptics : remote surveillance to physical presence

This paper reports on robotic and haptic technologies and capabilities developed for the law enforcement and defence community within Australia by the Centre for Intelligent Systems Research (CISR). The OzBot series of small and medium surveillance robots have been designed in Australia and evaluated by law enforcement and defence personnel to determine suitability and ruggedness in a variety of environments. Using custom developed digital electronics and featuring expandable data busses including RS485, I2C, RS232, video and Ethernet, the robots can be directly connected to many off the shelf payloads such as gas sensors, x-ray sources and camera systems including thermal and night vision. Differentiating the OzBot platform from its peers is its ability to be integrated directly with haptic technology or the 'haptic bubble' developed by CISR. Haptic interfaces allow an operator to physically 'feel' remote environments through position-force control and experience realistic force feedback. By adding the capability to remotely grasp an object, feel its weight, texture and other physical properties in real-time from the remote ground control unit, an operator's situational awareness is greatly improved through Haptic augmentation in an environment where remote-system feedback is often limited.

Pedestrian detection for mobile bus surveillance

In this paper, we present a system for pedestrian detection involving scenes captured by mobile bus surveillance cameras in busy city streets. Our approach integrates scene localization, foreground and background separation, and pedestrian detection modules into a unified detection framework. The scene localization module performs a two stage clustering of the video data. In the first stage, SIFT Homography is applied to cluster frames in terms of their structural similarities and second stage further clusters these aligned frames in terms of lighting. This produces clusters of images which are differential in viewpoint and lighting. A kernel density estimation (KDE) method for colour and gradient foreground-background separation are then used to construct background model for each image cluster which is subsequently used to detect all foreground pixels. Finally, using a hierarchical template matching approach, pedestrians can be identified. We have tested our system on a set of real bus video datasets and the experimental results verify that our system works well in practice.

Calibration of audio-video sensors for multi-modal event indexing

This paper addresses the coordinated use of video and audio cues to capture and index surveillance events with multimodal labels. The focus of this paper is the development of a joint-sensor calibration technique that uses audio-visual observations to improve the calibration process. One significant feature of this approach is the ability to continuously check and update the calibration status of the sensor suite, making it resilient to independent drift in the individual sensors. We present scenarios in which this system is used to enhance surveillance.

Distributed query processing for mobile surveillance

Addressing core issues in mobile surveillance, we present an architecture for querying and retrieving distributed, semi-permanent multi-modal data through challenged networks with limited connectivity. The system provides a rich set of queries for spatio-temporal querying in a surveillance context, and uses the network availability to provide best quality of service. It incrementally and adaptively refines the query, using data already retrieved that exists on static platforms and on-demand data that it requests from mobile platforms. We demonstrate the system using a real surveillance system on a mobile 20 bus transport network coupled with static bus depot infrastructure. In addition, we show the robustness of the system in handling different conditions in the underlying infrastructure by running simulations on a real, but historic dataset collected in an offline manner.

Multi-modal abnormality detection in video with unknown data segmentation

This paper examines a new problem in large scale stream data: abnormality detection which is localized to a data segmentation process. Unlike traditional abnormality detection methods which typically build one unified model across data stream, we propose that building multiple detection models focused on different coherent sections of the video stream would result in better detection performance. One key challenge is to segment the data into coherent sections as the number of segments is not known in advance and can vary greatly across cameras; and a principled way approach is required. To this end, we first employ the recently proposed infinite HMM and collapsed Gibbs inference to automatically infer data segmentation followed by constructing abnormality detection models which are localized to each segmentation. We demonstrate the superior performance of the proposed framework in a real-world surveillance camera data over 14 days.

Vehicular traffic surveillance and road lane detection using radar interferometry

Speed enforcement on public roadways is an important issue in order to guarantee road security and to reduce the number and seriousness of traffic accidents. Traditionally, this task has been partially solved using radar and/or laser technologies and, more recently, using video-camera based systems. All these systems have significant shortcomings that have yet to be overcome. The main drawback of classical Doppler radar technology is that the velocity measurement fails when several vehicles are in the radars beam. Modern radar systems are able to measure speed and range between vehicle and radar. However, this is not enough to discriminate the lane where the vehicle is driving on. The limitation of several vehicles in the beam is overcome using laser technology. However, laser systems have another important limitation: They cannot measure the speed of several vehicles simultaneously. Novel video-camera systems, based on license plate identification, solve the previous drawbacks, but they have the problem that they can only measure average speed but never top-speed. This paper studies the feasibility of using an interferometric linear frequency modulated continuous wave radar to improve top-speed enforcement on roadways. Two different systems based on down-the-road and across-the-road radar configurations are presented. The main advantage of the proposed solutions is they can simultaneously measure speed, range, and lane of several vehicles, allowing the univocal identification of the offenders. A detailed analysis about the operation and accuracy of these solutions is reported. In addition, the feasibility of the proposed techniques has been demonstrated with simulations and real experiments using a Ka-band interferometric radar developed by our research group.

Cognitive Surveillance Architecture for Scenario Understanding

Uno de los mayores retos para la comunidad científica es conseguir que las máquinas posean en un futuro la capacidad del sistema visual y cognitivo humanos, de forma que, por ejemplo, en entornos de video vigilancia, puedan llegar a proporcionar de manera automática una descripción fiable de lo que está ocurriendo en la escena. En la presente tesis, mediante la propuesta de un marco de trabajo de referencia, se discuten y plantean los pasos necesarios para el desarrollo de sistemas más inteligentes capaces de extraer y analizar, a diferentes niveles de abstracción y mediante distintos módulos de procesamiento independientes, la información necesaria para comprender qué está sucediendo en un conjunto amplio de escenarios de distinta naturaleza. Se parte de un análisis de requisitos y se identifican los retos para este tipo de sistemas en la actualidad, lo que constituye en sí mismo los objetivos de esta tesis, contribuyendo así a un modelo de datos basado en el conocimiento que permitirá analizar distintas situaciones en las que personas y vehículos son los actores principales, dejando no obstante la puerta abierta a la adaptación a otros dominios. Así mismo, se estudian los distintos procesos que se pueden lanzar a nivel interno así como la necesidad de integrar mecanismos de realimentación a distintos niveles que permitan al sistema adaptarse mejor a cambios en el entorno. Como resultado, se propone un marco de referencia jerárquico que integra las capacidades de percepción, interpretación y aprendizaje para superar los retos identificados en este ámbito; y así poder desarrollar sistemas de vigilancia más robustos, flexibles e inteligentes, capaces de operar en una variedad de entornos. Resultados experimentales ejecutados sobre distintas muestras de datos (secuencias de vídeo principalmente) demuestran la efectividad del marco de trabajo propuesto respecto a otros propuestos en el pasado. Un primer caso de estudio, permite demostrar la creación de un sistema de monitorización de entornos de parking en exteriores para la detección de vehículos y el análisis de plazas libres de aparcamiento. Un segundo caso de estudio, permite demostrar la flexibilidad del marco de referencia propuesto para adaptarse a los requisitos de un entorno de vigilancia completamente distinto, como es un hogar inteligente donde el análisis automático de actividades de la vida cotidiana centra la atención del estudio. ABSTRACT One of the most ambitious objectives for the Computer Vision and Pattern Recognition research community is that machines can achieve similar capacities to the human's visual and cognitive system, and thus provide a trustworthy description of what is happening in the scene under surveillance. Thus, a number of well-established scenario understanding architectural frameworks to develop applications working on a variety of environments can be found in the literature. In this Thesis, a highly descriptive methodology for the development of scene understanding applications is presented. It consists of a set of formal guidelines to let machines extract and analyse, at different levels of abstraction and by means of independent processing modules that interact with each other, the necessary information to understand a broad set of different real World surveillance scenarios. Taking into account the challenges that working at both low and high levels offer, we contribute with a highly descriptive knowledge-based data model for the analysis of different situations in which people and vehicles are the main actors, leaving the door open for the development of interesting applications in diverse smart domains. Recommendations to let systems achieve high-level behaviour understanding will be also provided. Furthermore, feedback mechanisms are proposed to be integrated in order to let any system to understand better the environment and the logical context around, reducing thus the uncertainty and noise, and increasing its robustness and precision in front of low-level or high-level errors. As a result, a hierarchical cognitive architecture of reference which integrates the necessary perception, interpretation, attention and learning capabilities to overcome main challenges identified in this area of research is proposed; thus allowing to develop more robust, flexible and smart surveillance systems to cope with the different requirements of a variety of environments. Once crucial issues that should be treated explicitly in the design of this kind of systems have been formulated and discussed, experimental results shows the effectiveness of the proposed framework compared with other proposed in the past. Two case studies were implemented to test the capabilities of the framework. The first case study presents how the proposed framework can be used to create intelligent parking monitoring systems. The second case study demonstrates the flexibility of the system to cope with the requirements of a completely different environment, a smart home where activities of daily living are performed. Finally, general conclusions and future work lines to further enhancing the capabilities of the proposed framework are presented.

Inconsistencies in Guidelines for Visual Health Surveillance of VDT Workers

Objectives: In Europe, 25% of workers use video display terminals (VDTs). Occupational health surveillance has been considered a key element in the protection of these workers. Nevertheless, it is unclear if guidelines available for this purpose, based on EU standards and available evidence, meet currently accepted quality criteria. The aim of this study was to appraise three sets of European VDT guidelines (UK, France, Spain) in which regulatory and evidence-based approaches for visual health have been formulated and recommendations for practice made. Methods: Three independent appraisers used an adapted AGREE instrument with seven domains to appraise the guidelines. A modified nominal group technique approach was used in two consecutive phases: first, individual evaluation of the three guidelines simultaneously, and second, a face-to-face meeting of appraisers to discuss scoring. Analysis of ratings obtained in each domain and variability among appraisers was undertaken (correlation and kappa coefficients). Results: All guidelines had low domain scores. The domain evaluated most highly was Scope and purpose, while Applicability was scored minimally. The UK guidelines had the highest overall score, and the Spanish ones had the lowest. The analysis of reliability and differences between scores in each domain showed a high level of agreement. Conclusions: These results suggest current guidelines used in these countries need an update. The formulation of evidence-base European guidelines on VDT could help to reduce the significant variation of national guidelines, which may have an impact on practical application.

Parallel Computational Intelligence-Based Multi-Camera Surveillance System

In this work, we present a multi-camera surveillance system based on the use of self-organizing neural networks to represent events on video. The system processes several tasks in parallel using GPUs (graphic processor units). It addresses multiple vision tasks at various levels, such as segmentation, representation or characterization, analysis and monitoring of the movement. These features allow the construction of a robust representation of the environment and interpret the behavior of mobile agents in the scene. It is also necessary to integrate the vision module into a global system that operates in a complex environment by receiving images from multiple acquisition devices at video frequency. Offering relevant information to higher level systems, monitoring and making decisions in real time, it must accomplish a set of requirements, such as: time constraints, high availability, robustness, high processing speed and re-configurability. We have built a system able to represent and analyze the motion in video acquired by a multi-camera network and to process multi-source data in parallel on a multi-GPU architecture.