Multi-level semantic analysis for sports video

There has been a huge increase in the utilization of video as one of the most preferred type of media due to its content richness for many significant applications including sports. To sustain an ongoing rapid growth of sports video, there is an emerging demand for a sophisticated content-based indexing system. Users recall video contents in a high-level abstraction while video is generally stored as an arbitrary sequence of audio-visual tracks. To bridge this gap, this paper will demonstrate the use of domain knowledge and characteristics to design the extraction of high-level concepts directly from audio-visual features. In particular, we propose a multi-level semantic analysis framework to optimize the sharing of domain characteristics.

Multichannel biomedical time series clustering via hierarchical probabilistic latent semantic analysis

Biomedical time series clustering that automatically groups a collection of time series according to their internal similarity is of importance for medical record management and inspection such as bio-signals archiving and retrieval. In this paper, a novel framework that automatically groups a set of unlabelled multichannel biomedical time series according to their internal structural similarity is proposed. Specifically, we treat a multichannel biomedical time series as a document and extract local segments from the time series as words. We extend a topic model, i.e., the Hierarchical probabilistic Latent Semantic Analysis (H-pLSA), which was originally developed for visual motion analysis to cluster a set of unlabelled multichannel time series. The H-pLSA models each channel of the multichannel time series using a local pLSA in the first layer. The topics learned in the local pLSA are then fed to a global pLSA in the second layer to discover the categories of multichannel time series. Experiments on a dataset extracted from multichannel Electrocardiography (ECG) signals demonstrate that the proposed method performs better than previous state-of-the-art approaches and is relatively robust to the variations of parameters including length of local segments and dictionary size. Although the experimental evaluation used the multichannel ECG signals in a biometric scenario, the proposed algorithm is a universal framework for multichannel biomedical time series clustering according to their structural similarity, which has many applications in biomedical time series management.

Semantic analysis in location privacy preserving

With the increasing use of location-based services, location privacy has recently started raising serious concerns. Location perturbation and obfuscation are most widely used for location privacy preserving. To protect a user from being identified, a cloaked spatial region that contains other k - 1 nearest neighbors of the user is submitted to the location-based service provider, instead of the accurate position. In this paper, we consider the location-aware applications that services are different among regions. In such scenarios, the semantic distance between users should be considered besides the Euclidean distance for searching the neighbors of a user. We define a novel distance measurement that combines the semantic and the Euclidean distance to address the privacy-preserving issue in the aforementioned applications. We also present an algorithm kNNH to implement our proposed method. Moreover, we conduct performance study experiments on the proposed algorithm. The experimental results further suggest that the proposed distance metric and the algorithm can successfully retain the utility of the location services while preserving users' privacy.

Finding the beat : an analysis of the rhythmic elements of motion pictures

This paper presents a new study on the application of the framework of Computational Media Aesthetics to the problem of automated understanding of film. Leveraging Film Grammar as the means to closing the "semantic gap" in media analysis, we examine film rhythm, a powerful narrative concept used to endow structure and form to the film compositionally and enhance its lyrical quality experientially. The novelty of this paper lies in the specification and investigation of the rhythmic elements that are present in two cinematic devices; namely motion and editing patterns, and their potential usefulness to automated content annotation and management systems. In our rhythm model, motion behavior is classified as being either nonexistent, fluid or staccato for a given shot. Shot neighborhoods in movies are then grouped by proportional makeup of these motion behavioral classes to yield seven high-level rhythmic arrangements that prove to be adept at indicating likely scene content (e.g. dialogue or chase sequence) in our experiments. The second part of our investigation presents a computational model to detect editing patterns as either metric, accelerated, decelerated or free. Details of the algorithm for the extraction of these classes are presented, along with experimental results on real movie data. We show with an investigation of combined rhythmic patterns that, while detailed content identification via rhythm types alone is not possible by virtue of the fact that film is not codified to this level in terms of rhythmic elements, analysis of the combined motion/editing rhythms can allow us to determine that the content has changed and hypothesize as to why this is so. We present three such categories of change and demonstrate their efficacy for capturing useful film elements (e.g. scene change precipitated by plot event), by providing data support from five motion pictures.

Ricoeur on responsibility

There are at least two themes in Paul Ricoeur’s recent essay, ‘The Concept of Responsibility: An Essay in Semantic Analysis’ (Ricoeur 2000). The first of these is in the foreground of the essay. It concerns how the concept of responsibility has evolved in recent times from a delimited juridical notion to a much broader moral concept. The second theme remains in the background of the essay and alludes to theses that Ricoeur has developed in his book, Oneself as Another (Ricoeur 1992). This theme concerns how responsibility relates to personal and moral identity and how it emerges dialectically from social formation and from an eliminable subjectivity. In this paper I will explicate Ricoeur’s first theme and also explore how the second theme might solve the problem of the unassumable scope and range of our responsibilities which the first theme might suggest.

The language of subjective well-being

The Language of Depression project is a linguistic study of the language of Acute Care Hospital patients suffering depression with the ultimate aim of enabling medical and nursing staff to become more aware of their patients’ depression and immediately refer them for psychological or psychiatric help. As part of that larger project, and following recent developments in positive psychology (e.g. Seligman 2002) this paper will focus exclusively on the control group, that is, the language of those Acute Care Hospital patients deemed non-depressed. The data comprise 30 minute interviews between the patients and a Consultation-liaison psychiatrist. Prior to interview, the patients were screened using the Brief Case-find for Depression (Clarke et al. 1994). From the screening, patients were then deemed likely to be depressed and likely to be non-depressed. This paper reports on the analysis of 10 patients deemed as non-depressed. Using the linguistic theory of Systemic Functional Linguistics, the data were analysed for their Appraisal features (e.g. Martin and Rose 2003). Appraisal analysis provides a lexico-semantic analysis that is concerned with how speakers use language to evaluate as well as negotiate relationships. The Appraisal analysis has been used to identify in the language of non-depressed patients the types of attitudes that facilitate psychological well-being. This paper will present some analysed extracts from the interviews to show how key features of subjective well-being are realised in the language of non-depressed Acute Care Hospital patients.

A statistical-driven approach for automatic classification of events in AFL video highlights

Due to the repetitive and lengthy nature, automatic content-based summarization is essential to extract a more compact and interesting representation of sport video. State-of-the art approaches have confirmed that high-level semantic in sport video can be detected based on the occurrences of specific audio and visual features (also known as cinematic). However, most of them still rely heavily on manual investigation to construct the algorithms for highlight detection. Thus, the primary aim of this paper is to demonstrate how the statistics of cinematic features within play-break sequences can be used to less-subjectively construct highlight classification rules. To verify the effectiveness of our algorithms, we will present some experimental results using six AFL (Australian Football League) matches from different broadcasters. At this stage, we have successfully classified each play-break sequence into: goal, behind, mark, tackle, and non-highlight. These events are chosen since they are commonly used for broadcasted AFL highlights. The proposed algorithms have also been tested successfully with soccer video.

A probabilistic framework for extracting narrative act boundaries and semantics in motion pictures

This work constitutes the first attempt to extract the important narrative structure, the 3-Act storytelling paradigm in film. Widely prevalent in the domain of film, it forms the foundation and framework in which a film can be made to function as an effective tool for story telling, and its extraction is a vital step in automatic content management for film data. The identification of act boundaries allows for structuralizing film at a level far higher than existing segmentation frameworks, which include shot detection and scene identification, and provides a basis for inferences about the semantic content of dramatic events in film. A novel act boundary likelihood function for Act 1 and 2 is derived using a Bayesian formulation under guidance from film grammar, tested under many configurations and the results are reported for experiments involving 25 full-length movies. The result proves to be a useful tool in both the automatic and semi-interactive setting for semantic analysis of film, with potential application to analogues occuring in many other domains, including news, training video, sitcoms.

Narrative structure detection through audio pace

We use the concept of film pace, expressed through the audio, to analyse the broad level narrative structure of film. The narrative structure is divided into visual narration, action sections, and audio narration, plot development sections. We hypothesise, that changes in the narrative structure signal a change in audio content, which is reflected by a change in audio pace. We test this hypothesis using a number of audio feature functions, that reflect the audio pace, to detect changes in narrative structure for 8 films of varying genres. The properties of the energy were then used to determine the. audio pace feature corresponding to the narrative, structure for each film analysed. The method was successful in determining the narrative structure for 1 of the films, achieving an overall precision of 76.4% and recall of 80.3%, We map the properties of the speech and energy of film audio to the higher level semantic concept of audio pace. The audio pace was in turn applied to a higher level semantic analysis of the structure of film.

Indexing narrative structure and semantics in motion pictures with a probabilistic framework

This work constitutes the first attempt to extract an important narrative structure, the 3-Act story telling paradigm, in film. This narrative structure is prevalent in the domain of film as it forms the foundation and framework in which the film can be made to function as an effective tool for story telling, and its extraction is a vital step in automatic content management for film data. A novel act boundary likelihood function for Act 1 is derived using a Bayesian formulation under guidance from film grammar, tested under many configurations and the results are reported for experiments involving 25 full length movies. The formulation is shown to be a useful tool in both the automatic and semi-interactive setting for semantic analysis of film.

Wire - a formal intermediate language for binary analysis

Wire is a intermediate language to enable static program analysis on low level objects such as native executables. It has practical benefit in analysing the structure and semantics of malware, or for identifying software defects in closed source software. In this paper we describe how an executable program is disassembled and translated to the Wire intermediate language. We define the formal syntax and operational semantics of Wire and discuss our justifications for its language features. We use Wire in our previous work Malwise, a malware variant detection system. We also examine applications for when a formally defined intermediate language is given. Our results include showing the semantic equivalence between obfuscated and non obfuscated code samples. These examples stem from the obfuscations commonly used by malware.

Modelling multilevel data in multimedia : A hierarchical factor analysis approach

Multimedia content understanding research requires rigorous approach to deal with the complexity of the data. At the crux of this problem is the method to deal with multilevel data whose structure exists at multiple scales and across data sources. A common example is modeling tags jointly with images to improve retrieval, classification and tag recommendation. Associated contextual observation, such as metadata, is rich that can be exploited for content analysis. A major challenge is the need for a principal approach to systematically incorporate associated media with the primary data source of interest. Taking a factor modeling approach, we propose a framework that can discover low-dimensional structures for a primary data source together with other associated information. We cast this task as a subspace learning problem under the framework of Bayesian nonparametrics and thus the subspace dimensionality and the number of clusters are automatically learnt from data instead of setting these parameters a priori. Using Beta processes as the building block, we construct random measures in a hierarchical structure to generate multiple data sources and capture their shared statistical at the same time. The model parameters are inferred efficiently using a novel combination of Gibbs and slice sampling. We demonstrate the applicability of the proposed model in three applications: image retrieval, automatic tag recommendation and image classification. Experiments using two real-world datasets show that our approach outperforms various state-of-the-art related methods.