Improved fade and dissolve detection for reliable video segmentation

We present improved algorithms for automatic fade and dissolve detection in digital video analysis. We devise new two-step algorithms for fade and dissolve detection and introduce a method for eliminating false positives from a list of detected candidate transitions. In our detailed study of these gradual shot transitions, our objective has been to accurately classify the type of transitions (fade-in, fade-out, and dissolve) and to precisely locate the boundary of the transitions. This distinguishes our work from early work in scene change detection which focuses on identifying the existence of a transition rather than its precise temporal extent. We evaluate our algorithms against two other commonly used methods on a comprehensive data set, and demonstrate the improved performance due to our enhancements.

New enhancements to cut, fade, and dissolve detection processes in video segmentation

We present improved algorithms for cut, fade, and dissolve detection which are fundamental steps in digital video analysis. In particular, we propose a new adaptive threshold determination method that is shown to reduce artifacts created by noise and motion in scene cut detection. We also describe new two-step algorithms for fade and dissolve detection, and introduce a method for eliminating false positives from a list of detected candidate transitions. In our detailed study of these gradual shot transitions, our objective has been to accurately classify the type of transitions (fade-in, fade-out, and dissolve) and to precisely locate the boundary of the transitions. This distinguishes our work from other early work in scene change detection which tends to focus primarily on identifying the existence of a transition rather than its precise temporal extent. We evaluate our improved algorithms against two other commonly used shot detection techniques on a comprehensive data set, and demonstrate the improved performance due to our enhancements.

Bayesian nonparametric approaches to abnormality detection in video surveillance

In data science, anomaly detection is the process of identifying the items, events or observations which do not conform to expected patterns in a dataset. As widely acknowledged in the computer vision community and security management, discovering suspicious events is the key issue for abnormal detection in video surveil-lance. The important steps in identifying such events include stream data segmentation and hidden patterns discovery. However, the crucial challenge in stream data segmenta-tion and hidden patterns discovery are the number of coherent segments in surveillance stream and the number of traffic patterns are unknown and hard to specify. Therefore, in this paper we revisit the abnormality detection problem through the lens of Bayesian nonparametric (BNP) and develop a novel usage of BNP methods for this problem. In particular, we employ the Infinite Hidden Markov Model and Bayesian Nonparamet-ric Factor Analysis for stream data segmentation and pattern discovery. In addition, we introduce an interactive system allowing users to inspect and browse suspicious events.

Scene extraction in motion pictures

This paper addresses the challenge of bridging the semantic gap between the rich meaning users desire when they query to locate and browse media and the shallowness of media descriptions that can be computed in today's content management systems. To facilitate high-level semantics-based content annotation and interpretation, we tackle the problem of automatic decomposition of motion pictures into meaningful story units, namely scenes. Since a scene is a complicated and subjective concept, we first propose guidelines from fill production to determine when a scene change occurs. We then investigate different rules and conventions followed as part of Fill Grammar that would guide and shape an algorithmic solution for determining a scene. Two different techniques using intershot analysis are proposed as solutions in this paper. In addition, we present different refinement mechanisms, such as film-punctuation detection founded on Film Grammar, to further improve the results. These refinement techniques demonstrate significant improvements in overall performance. Furthermore, we analyze errors in the context of film-production techniques, which offer useful insights into the limitations of our method.

Determining dramatic intensification via flashing lights in movies

Movie directors and producers worldwide, in their quest to narrate a good story that warrants repeated audience viewing, use many cinematic elements to intensify and clarify the viewing experience. One such element that directors manipulate is lighting. In this paper we examine one aspect of lighting, namely flashing lights, and its role as an intensifier of dramatic effects in film. We present an algorithm for robust extraction of flashing lights and a simple mechanism to group detected flashing lights into flashing light scenes and analyze the role of these segments in story narration. In addition, we demonstrate how flashing lights detection can improve the performance of shot-based video segmentation. Experiments on a number of video sequences extracted from real movies yields good results. Our technique detects 90.4% of flashing lights. The detected flashing lights correctly eliminates 92.7% of false cuts in these sequences. In addition, data support is compiled to demonstrate the association between flashing light scenes and certain dramatic intensification events such as supernatural power, crisis or excitement.

Segmentation of intentional human gestures for sports video annotation

We present results on the recognition of intentional human gestures for video annotation and retrieval. We define a gesture as a particular, repeatable, human movement having a predefined meaning. An obvious application of the work is in sports video annotation where umpire gestures indicate specific events. Our approach is to augment video with data obtained from accelerometers worn as wrist bands by one or more officials. We present the recognition performance using a Hidden Markov Model approach for gesture modeling with both isolated gestures and gestures segmented from a stream.

Multi-modal abnormality detection in video with unknown data segmentation

This paper examines a new problem in large scale stream data: abnormality detection which is localized to a data segmentation process. Unlike traditional abnormality detection methods which typically build one unified model across data stream, we propose that building multiple detection models focused on different coherent sections of the video stream would result in better detection performance. One key challenge is to segment the data into coherent sections as the number of segments is not known in advance and can vary greatly across cameras; and a principled way approach is required. To this end, we first employ the recently proposed infinite HMM and collapsed Gibbs inference to automatically infer data segmentation followed by constructing abnormality detection models which are localized to each segmentation. We demonstrate the superior performance of the proposed framework in a real-world surveillance camera data over 14 days.

Robust video/ultrasonic fusion based estimation for automotive applications

We describe how object estimation by a stationary or a non-stationary camera can be improved using recently-developed robust estimation ideas. The robustness of vision-based systems can be improved significantly by employing a Robust Extended Kalman Filter (REKF). The system performance is also enhanced by increasing the spatial diveristy in measurements via employing additional cameras for video capture. We describe a normal-flow based image segmentation technique to identify the object for the application of our proposed state estimation technique. Our simulations demonstrate that dynamic system modelling coupled with the application of a REKF significantly improves the estimation system performance, especially when large uncertainties are present.

Robust video/ultrasonic fusion-based estimation for automotive applications

In this paper, we use recently developed robust estimation ideas to improve object tracking by a stationary or nonstationary camera. Large uncertainties are always present in vision-based systems, particularly, in relation to the estimation of the initial state as well as the measurement of object motion. The robustness of these systems can be significantly improved by employing a robust extended Kalman filter (REKF). The system performance can also be enhanced by increasing the spatial diversity in measurements via employing additional cameras for video capture. We compare the performances of various image segmentation techniques in moving-object localization and show that normal-flow-based segmentation yields comparable results to, but requires significantly less time than, optical-flow-based segmentation. We also demonstrate with simulations that dynamic system modeling coupled with the application of an REKF significantly improves the estimation system performance, particularly, when subjected to large uncertainties.

Hierarchical topical segmentation in instructional films based on cinematic expressive functions

In this paper, we propose a novel solution for segmenting an instructional video into hierarchical topical sections. Incorporating the knowledge of education-oriented film theory with our previous study of expressive functions namely the content density and the thematic functions, we develop an algorithm to effectively structuralize an instructional video into a two-tiered hierarchy of topical sections at the main and sub-topic levels. Our experimental results on a set of ten industrial instructional videos demonstrate the validity of the detection scheme.

Automatic labeling of sports video using umpire gesture recognition

We present results on an extension to our approach for automatic sports video annotation. Sports video is augmented with accelerometer data from wrist bands worn by umpires in the game. We solve the problem of automatic segmentation and robust gesture classification using a hierarchical hidden Markov model in conjunction with a filler model. The hierarchical model allows us to consider gestures at different levels of abstraction and the filler model allows us to handle extraneous umpire movements. Results are presented for labeling video for a game of Cricket.

Human activity learning and segmentation using partially hidden discriminative models

Learning and understanding the typical patterns in the daily activities and routines of people from low-level sensory data is an important problem in many application domains such as building smart environments, or providing intelligent assistance. Traditional approaches to this problem typically rely on supervised learning and generative models such as the hidden Markov models and its extensions. While activity data can be readily acquired from pervasive sensors, e.g. in smart environments, providing manual labels to support supervised training is often extremely expensive. In this paper, we propose a new approach based on semi-supervised training of partially hidden discriminative models such as the conditional random field (CRF) and the maximum entropy Markov model (MEMM). We show that these models allow us to incorporate both labeled and unlabeled data for learning, and at the same time, provide us with the flexibility and accuracy of the discriminative framework. Our experimental results in the video surveillance domain illustrate that these models can perform better than their generative counterpart, the partially hidden Markov model, even when a substantial amount of labels are unavailable.

Enabling electronic medicine at Kiwicare : the case of video conferencing adoption for psychiatry in New Zealand

Telemedicine emerges as a viable solution to New Zealand health providers in reaching out to rural patients, in offering medical services and conducting administrative meetings and training. No research exists about adoption of telemedicine in New Zealand. The purpose of this case study was to explain factors influencing adoption of telemedicine utilizing video conferencing technology (TMVC) within a New Zealand hospital known as KiwiCare. Since TMVC is part of IT, tackling it from within technological innovation literature may assist in providing an insight into its adoption within KiwiCare and into the literature. Findings indicate weak presence of critical assessment into technological innovation factors prior to the adoption decision, thereby leading to its weak utilization. Factors like complexity, compatibility and trialability were not assessed extensively by KiwiCare and would have hindered TMVC adoption. TMVC was mainly assessed according to its relative advantage and to its cost effectiveness along with other facilitating and accelerating factors. This is essential but should be alongside technological and other influencing factors highlighted in the literature.

Strategies to develop divided attention skills through video game training

Research .over the past two decades has confirmed the learning opportunities that video games can offer. The objectives of the current study were to investigate the ability of video games to enhance divided attention skills and compare these skills based on individuals' level of expertise in video game playing. Female participants aged between 17-25 years categorised as experts or novices, were divided into experimental and control groups. All participants completed the pre and post-test of divided attention between which only the experimental group received video game training. Results indicate that participants who received video game practice achieved an increase in their dual-attention skills compared to those who did not receive any training, with novices displaying a greater
enhancement in perfonnance. Implications include the provision of video game training to enhance divided attention skills in air pilot training, driving, and heavyequipment operation apart from other tasks necessitating dual-task efficiency.