Hierarchical monitoring of people's behaviors in complex environments using multiple cameras

We present a distributed, surveillance system that works in large and complex indoor environments. To track and recognize behaviors of people, we propose the use of the Abstract Hidden Markov Model (AHMM), which can be considered as an extension of the Hidden Markov Model (HMM), where the single Markov chain in the HMM is replaced by a hierarchy of Markov policies. In this policy hierarchy, each behavior can be represented as a policy at the corresponding level of abstraction. The noisy observations are handled in the same way as an HMM and an efficient Rao-Blackwellised particle filter method is used to compute the probabilities of the current policy at different levels of the hierarchy The novelty of the paper lies in the implementation of a scalable framework in the context of both the scale of behaviors and the size of the environment, making it ideal for distributed surveillance. The results of the system demonstrate the ability to answer queries about people's behaviors at different levels of details using multiple cameras in a large and complex indoor environment.

Learning people movement model from multiple cameras for behaviour recognition

In surveillance systems for monitoring people behaviours, it is important to build systems that can adapt to the signatures of people's tasks and movements in the environment. At the same time, it is important to cope with noisy observations produced by a set of cameras with possibly different characteristics. In previous work, we have implemented a distributed surveillance system designed for complex indoor environments [1]. The system uses the Abstract Hidden Markov mEmory Model (AHMEM) for modelling and specifying complex human behaviours that can take place in the environment. Given a sequence of observations from a set of cameras, the system employs approximate probabilistic inference to compute the likelihood of different possible behaviours in real-time. This paper describes the techniques that can be used to learn the different camera noise models and the human movement models to be used in this system. The system is able to monitor and classify people behaviours as data is being gathered, and we provide classification results showing the system is able to identify behaviours of people from their movement signatures.

Coordination of multiple cameras to track multiple people

In this paper, we present a distributed surveillance system that uses multiple cheap static cameras to track multiple people in indoor environments. The system has a set of Camera Processing Modules and a Central Module to coordinate the tracking tasks among the cameras. Since each object in the scene can be tracked by a number of cameras, the problem is how to choose the most appropriate camera for each object. We propose a novel algorithm to allocate objects to cameras using the object-to-camera distance while taking into account occlusion. The algorithm attempts to assign objects in the overlapping fields of view to the nearest camera which can see the object without occlusion. Experimental results show that the system can coordinate cameras to track people properly and can deal well with occlusion.

Measuring depth accuracy in RGBD cameras

This paper presents the comparison between the Microsoft Kinect depth sensor and the Asus Xtion for computer vision applications. Depth sensors, known as RGBD cameras,
project an infrared pattern and calculate the depth from the reflected light using an infrared sensitive camera. In this research, we compare the depth sensing capabilities of the two sensors under various conditions. The purpose is to give the reader a background to whether use the Microsoft Kinect or Asus Xtion sensor to solve a specific computer vision problem. The properties of the two depth sensors were investigated by conducting a series of experiments evaluating the accuracy of the sensors under various conditions, which shows the advantages and disadvantages of both Microsoft Kinect and Asus Xtion sensor.

Identification of prey captures in Australian Fur Seals (Arctocephalus pusillus doriferus) using head-mounted accelerometers: field validation with animal-borne video cameras

This study investigated prey captures in free-ranging adult female Australian fur seals (Arctocephalus pusillus doriferus) using head-mounted 3-axis accelerometers and animal-borne video cameras. Acceleration data was used to identify individual attempted prey captures (APC), and video data were used to independently verify APC and prey types. Results demonstrated that head-mounted accelerometers could detect individual APC but were unable to distinguish among prey types (fish, cephalopod, stingray) or between successful captures and unsuccessful capture attempts. Mean detection rate (true positive rate) on individual animals in the testing subset ranged from 67-100%, and mean detection on the testing subset averaged across 4 animals ranged from 82-97%. Mean False positive (FP) rate ranged from 15-67% individually in the testing subset, and 26-59% averaged across 4 animals. Surge and sway had significantly greater detection rates, but also conversely greater FP rates compared to heave. Video data also indicated that some head movements recorded by the accelerometers were unrelated to APC and that a peak in acceleration variance did not always equate to an individual prey item. The results of the present study indicate that head-mounted accelerometers provide a complementary tool for investigating foraging behaviour in pinnipeds, but that detection and FP correction factors need to be applied for reliable field application.

Cameras and body memory create new blends of performance with visual art practice

I propose that a learnt somatic experience of dance can translate into another discipline such as visual art. In my visual art practice I combine both photography, which is traditionally seen as a still medium, and performance in order to create a new form of embodiment. I have developed two series of art works of prints and video made in response to the Australian landscape. By analyzing my method of movement and photography I will describe how an embodied dance language can result in a material outcome – a series of drawings of light and movement, a body signature made possible through old and new technology. I have activated a performative state while capturing images discovering new ways of using technology reliant upon my knowledge of dance, performance and photography. Making a human size camera to make analogue prints I gained an intuitive knowledge of light – a skill that has become foundational in performance and photography. In response to space and light in the Australian landscape I then built a custom made camera that allows for the longest possible time to capture an image. I move while taking the image and use the camera as if an eye at the end of my arm. In this way I activate dance skills and embodied knowledge of space, timing and light, opening up a radical space for new thinking, making and performing.Furthermore this process engages memory and sentiment embodied through age. Many artists have responded to the unique qualities of the Australian landscape and by using a performative/photographic approach I have engaged with my own body memory. Being brought up in the landscape and training in ballet my body has acquired memories at a cellular level. My method has given memory a voice. In doing these works I have become conscious of how unconscious memories of the space and light in the landscape is a movement vocabulary activated in a way that ‘feels’ like dancing. As an ageing person this experience is profound and the resultant materialisation of the photographs and videos leave a material record of the event. The sentiments evoked through my process bridge the past with the present, the body with the mind, memory with body and space connecting disciplines in a new way.The materialisation of artworks itself continues cross-disciplinary processes using a technique that is a continuum of the performative. Through using technology I release memory of the landscape and pixel by pixel build imagery that relies on and is a part of the performative process. It is a photographic performance dance manifesting as pigment on paper exhibited a gallery context. The exhibition allows a space for the viewer to respond - re-membering the universal the act of moving. The works titled ‘body signatures’ and ‘Fly Rhythm’ become a communicative device in the gallery context.My paper through an analysis of process and methods used in making the two series will talk to several of the subjects listed and reveal a new way of connecting performance and visual art and old and new technologies.

Wearable cameras can reduce dietary under-reporting: doubly labelled water validation of a camera-assisted 24 h recall

Preliminary research has suggested that wearable cameras may reduce under-reporting of energy intake (EI) in self-reported dietary assessment. The aim of the present study was to test the validity of a wearable camera-assisted 24 h dietary recall against the doubly labelled water (DLW) technique. Total energy expenditure (TEE) was assessed over 15 d using the DLW protocol among forty adults (n 20 males, age 35 (sd 17) years, BMI 27 (sd 4) kg/m2 and n 20 females, age 28 (sd 7) years, BMI 22 (sd 2) kg/m2). EI was assessed using three multiple-pass 24 h dietary recalls (MP24) on days 2-4, 8-10 and 13-15. On the days before each nutrition assessment, participants wore an automated wearable camera (SenseCam (SC)) in free-living conditions. The wearable camera images were viewed by the participants following the completion of the dietary recall, and their changes in self-reported intakes were recorded (MP24+SC). TEE and EI assessed by the MP24 and MP24+SC methods were compared. Among men, the MP24 and MP24+SC measures underestimated TEE by 17 and 9%, respectively (P< 0.001 and P= 0.02). Among women, these measures underestimated TEE by 13 and 7%, respectively (P< 0.001 and P= 0.004). The assistance of the wearable camera (MP24+SC) reduced the magnitude of under-reporting by 8% for men and 6% for women compared with the MP24 alone (P< 0.001 and P< 0.001). The increase in EI was predominantly from the addition of 265 unreported foods (often snacks) as revealed by the participants during the image review. Wearable cameras enhance the accuracy of self-report by providing passive and objective information regarding dietary intake. High-definition image sensors and increased imaging frequency may improve the accuracy further.

The mythical properties of police body-worn cameras: A solution in the search of a problem

Over the past few years, Australian police agencies have begun to enthusiastically introduce body-worn cameras on police personnel. These devices are now either implemented or under trial across the country. There is also an emergent ‘surveillance consensus’ (Hempel and Töpfer 2009) concerning their use amongst Australian police. While more detailed empirical examination of information flows that shape this surveillance consensus is warranted, this contribution to the debate seeks to draw from policing scholarship to critically explore the intersections between the rationalizations for body-worn cameras and the broader policing scholarship. More directly, body-worn cameras cannot be understood in narrow instrumental terms, but must be located within the broader literature on governing police and the law and order politics that surrounds many contemporary police and criminal justice reforms (Cox 2015; Gregg and Wilson 2015). I begin with a summary of the introduction of body-worn cameras in Australia. The article then identifies five problems body-worn cameras purportedly address and provides a brief case summary indicating how current ‘privacy protections’ fail to establish real limits to the collection, use, and dissemination of images from body-worn cameras.

New insights: animal-borne cameras and accelerometers reveal the secret lives of cryptic species

Logging cameras and accelerometers have opened our eyes to the secret lives of many enigmatic species. Here some of the new opportunities provided by this technology are reviewed. Recent discoveries are highlighted including the observation of selective feeding on energy-rich parts of prey. As such, biologging cameras provide new opportunities for consideration of selective feeding within the same sort of theoretical framework (marginal value theory/optimal foraging) that exploitation of prey patches has been examined. A recent study with the world's largest bony fish, the ocean sunfish (Mola mola), is highlighted where animal-borne cameras allowed the ground-truthing of data sets collected with depth recorders and accelerometers. This synergistic use of a range of biologging approaches will help drive an holistic understanding of the free-living behaviour of a range of species.

Image matching using TI multi-wavelet transform

A multi-resolution image matching technique based on multiwavelets followed by a coarse to fine strategy is presented. The technique addresses the estimation of optimal corresponding points and the corresponding disparity maps in the presence of occlusion, ambiguity and illuminative variations in the two perspective views taken by two different cameras or at different lighting conditions. The problem of occlusion and ambiguity is addressed by a geometric topological refining approach along with the uniqueness constraint whereas the illuminative variation is dealt by using windowed normalized correlation.

Disparity estimation using TI multi-wavelet transform

A multi-resolution image matching technique based on translation invariant discrete multi-wavelet transform followed by a coarse to fine matching strategy is presented. The technique addresses the estimation of optimal corresponding points and the corresponding disparity maps in the presence of occlusion, ambiguity and illuminative variations in the two perspective views taken by two different cameras or at different lighting conditions. The problem of occlusion and ambiguity is addressed explicitly by a geometric optimization approach along with the uniqueness constraint whereas the illuminative variation is dealt with by using windowed normalized correlation on the discrete multi-wavelet coefficients.

Robust video/ultrasonic fusion based estimation for automotive applications

We describe how object estimation by a stationary or a non-stationary camera can be improved using recently-developed robust estimation ideas. The robustness of vision-based systems can be improved significantly by employing a Robust Extended Kalman Filter (REKF). The system performance is also enhanced by increasing the spatial diveristy in measurements via employing additional cameras for video capture. We describe a normal-flow based image segmentation technique to identify the object for the application of our proposed state estimation technique. Our simulations demonstrate that dynamic system modelling coupled with the application of a REKF significantly improves the estimation system performance, especially when large uncertainties are present.

Robust video/ultrasonic fusion-based estimation for automotive applications

In this paper, we use recently developed robust estimation ideas to improve object tracking by a stationary or nonstationary camera. Large uncertainties are always present in vision-based systems, particularly, in relation to the estimation of the initial state as well as the measurement of object motion. The robustness of these systems can be significantly improved by employing a robust extended Kalman filter (REKF). The system performance can also be enhanced by increasing the spatial diversity in measurements via employing additional cameras for video capture. We compare the performances of various image segmentation techniques in moving-object localization and show that normal-flow-based segmentation yields comparable results to, but requires significantly less time than, optical-flow-based segmentation. We also demonstrate with simulations that dynamic system modeling coupled with the application of an REKF significantly improves the estimation system performance, particularly, when subjected to large uncertainties.