Using decision-tree to automatically construct learned-heuristics for events classification in sports video

Automatic events classification is an essential requirement for constructing an effective sports video summary. It has become a well-known theory that the high-level semantics in sport video can be “computationally interpreted” based on the occurrences of specific audio and visual features which can be extracted automatically. State-of-the-art solutions for features-based event classification have only relied on either manual-knowledge based heuristics or machine learning. To bridge the gaps, we have successfully combined the two approaches by using learning-based heuristics. The heuristics are constructed automatically using decision tree while manual supervision is only required to check the features and highlight contained in each training segment. Thus, fully automated construction of classification system for sports video events has been achieved. A comprehensive experiment on 10 hours video dataset, with five full-match soccer and five full-match basketball videos, has demonstrated the effectiveness/robustness of our algorithms.

Analysis of environmental sounds as indexical signs in film

In this paper, we investigate the problem of classifying a subset of environmental sounds in movie audio tracks that indicate specific indexical semiotic use. These environmental sounds are used to signify and enhance events occurring in film scenes. We propose a classification system for detecting the presence of violence and car chase scenes in film by classifying ten various environmental sounds that form the constituent audio events of these scenes using a number of old and new audio features. Experiments with our classification system on pure test sounds resulted in a correct event classification rate of 88.9%. We also present the results of the classifier on the mixed audio tracks of several scenes taken from The Mummy and Lethal Weapon 2. The classification of sound events is the first step towards determining the presence of the complex sound scenes within film audio and describing the thematic content of the scenes.

Detecting indexical signs in film audio for scene interpretation

In this paper, we study the sound tracks in films and their indexical semiotic usage by developing a classification system that detects complex sound scenes and their constituent sound events in cinema. We investigate two main issues in this paper: Determination of what constitutes the presence of a high level sound scene and inferences about the thematic content of the scene that can be drawn from this presence, and classification of environmental sounds in the audio track of the scene, to assist in the automatic detection of the high level scene. Experiments with our classification system on pure sounds resulted in a correct event classification rate of 88.9%. When the audio content of a number of film scenes was examined, though a lower accuracy resulted with sound event detection due to the presence of mixed sounds, the film audio samples were generally classified with the correct high-level sound scene label, enabling correct inferences about the story content of the scenes.

Quantitative experimental comparison of single-beam, sidescan, and multibeam benthic habitat maps

Map comparison is a relatively uncommon practice in acoustic seabed classification to date, contrary to the field of land remote sensing, where it has been developed extensively over recent decades. The aim here is to illustrate the benefits of map comparison in the underwater realm with a case study of three maps independently describing the seabed habitats of the Te Matuku Marine Reserve (Hauraki Gulf, New Zealand). The maps are obtained from a QTC View classification of a single-beam echosounder (SBES) dataset, manual segmentation of a sidescan sonar (SSS) mosaic, and automatic classification of a backscatter dataset from a multibeam echosounder (MBES). The maps are compared using pixel-to-pixel similarity measures derived from the literature in land remote sensing. All measures agree in presenting the MBES and SSS maps as the most similar, and the SBES and SSS maps as the least similar. The results are discussed with reference to the potential of MBES backscatter as an alternative to SSS mosaic for imagery segmentation and to the potential of joint SBES–SSS survey for improved habitat mapping. Other applications of map-similarity measures in acoustic classification of the seabed are suggested.

Hydro-acoustic remote sensing of benthic biological communities on the shallow South East Australian continental shelf

Information regarding the composition and extent of benthic habitats on the South East Australian continental shelf is limited. In this habitat mapping study, multibeam echosounder (MBES) data are integrated with precisely geo-referenced video ground-truth data to quantify benthic biotic communities at Cape Nelson, Victoria, Australia. Using an automated decision tree classification approach, 5 representative biotic groups defined from video analysis were related to hydro-acoustically derived variables in the Cape Nelson survey area. Using a combination of multibeam bathymetry, backscatter and derivative products produced highest overall accuracy (87%) and kappa statistic (0.83). This study demonstrates that decision tree classifiers are capable of integrating variable data types for mapping distributions of benthic biological assemblages, which are important in maintaining biodiversity and other system services in the marine environment.

Semantic trajectory based event detection and event pattern mining

Video event detection is an effective way to automatically understand the semantic content of the video. However, due to the mismatch between low-level visual features and high-level semantics, the research of video event detection encounters a number of challenges, such as how to extract the suitable information from video, how to represent the event, how to build up reasoning mechanism to infer the event according to video information. In this paper, we propose a novel event detection method. The method detects the video event based on the semantic trajectory, which is a high-level semantic description of the moving object’s trajectory in the video. The proposed method consists of three phases to transform low-level visual features to middle-level raw trajectory information and then to high-level semantic trajectory information. Event reasoning is then carried out with the assistance of semantic trajectory information and background knowledge. Additionally, to release the users’ burden in manual event definition, a method is further proposed to automatically discover the event-related semantic trajectory pattern from the sample semantic trajectories. Furthermore, in order to effectively use the discovered semantic trajectory patterns, the associative classification-based event detection framework is adopted to discover the possibly occurred event. Empirical studies show our methods can effectively and efficiently detect video events.