Content structure discovery in educational videos using shared structures in the hierarchical hidden Markov models

In this paper, we present an application of the hierarchical HMM for structure discovery in educational videos. The HHMM has recently been extended to accommodate the concept of shared structure, ie: a state might multiply inherit from more than one parents. Utilising the expressiveness of this model, we concentrate on a specific class of video -educational videos - in which the hierarchy of semantic units is simpler and clearly defined in terms of topics and its subunits. We model the hierarchy of topical structures by an HHMM and demonstrate the usefulness of the model in detecting topic transitions.

Post processing of multimedia information - concepts, problems, and techniques

Currently, most research work on multimedia information processing is focused on multimedia information storage and retrieval, especially indexing and content-based access of multimedia information. We consider multimedia information processing should include one more level-post-processing. Here "post-processing" means further processing of retrieved multimedia information, which includes fusion of multimedia information and reasoning with multimedia information to reach new conclusions. In this paper, the three levels of multimedia information processing storage, retrieval, and post-processing- are discussed. The concepts and problems of multimedia information post-processing are identified. Potential techniques that can be used in post-processing are suggested, By highlighting the problems in multimedia information post-processing, hopefully this paper will stimulate further research on this important but ignored topic.

Image featuring for retrieval of multimedia documents

The utilization of massive multimedia documents collections, such as multimedia documents in the global Internet, needs search engines which can rank using both text and image evidence. Massive size and (dynamic) nature of collection can make manual indexing prohibitively expensive in such situations. Traditional search engines utilize only text components of multimedia documents. But there are information needs, which require the utilization of image evidence. In this paper, we investigate image-feature for large and heterogeneous collections. Both the nature and complexities of information needs are key elements for an effective retrieval. Retrieval needs that depend on perceptual similarities (as found in art galleries, building architecture) require the utilization of visual cues. In such situations, the retrieval of multimedia document based on image ranking can provide higher effectiveness. Experimental results show that effectiveness of ranking based on image feature can be higher where perceptual similarities are key elements for retrieval than the retrieval effectiveness of algorithms based on text ranking algorithms

Content-based video indexing for sports applications using integrated multi-modal approach

This thesis presents a research work based on an integrated multi-modal approach for sports video indexing and retrieval. By combining specific features extractable from multiple (audio-visual) modalities, generic structure and specific events can be detected and classified. During browsing and retrieval, users will benefit from the integration of high-level semantic and some descriptive mid-level features such as whistle and close-up view of player(s). The main objective is to contribute to the three major components of sports video indexing systems. The first component is a set of powerful techniques to extract audio-visual features and semantic contents automatically. The main purposes are to reduce manual annotations and to summarize the lengthy contents into a compact, meaningful and more enjoyable presentation. The second component is an expressive and flexible indexing technique that supports gradual index construction. Indexing scheme is essential to determine the methods by which users can access a video database. The third and last component is a query language that can generate dynamic video summaries for smart browsing and support user-oriented retrievals.

Calibration of audio-video sensors for multi-modal event indexing

This paper addresses the coordinated use of video and audio cues to capture and index surveillance events with multimodal labels. The focus of this paper is the development of a joint-sensor calibration technique that uses audio-visual observations to improve the calibration process. One significant feature of this approach is the ability to continuously check and update the calibration status of the sensor suite, making it resilient to independent drift in the individual sensors. We present scenarios in which this system is used to enhance surveillance.

Media independent knowledge indexing and retrieval

Composing a multimedia presentation may require creation or generation of suitable images and video segments, as well as animation, sound, or special effects. Obtaining images or video sequences can be prohibitively expensive when costs of travel to location, equipment, staff, etc, are considered. Those problems can be alleviated with the use of pictorial and video digital libraries, such libraries require methods for comprehensive indexing and annotation of stored items and efficient retrieval tools.

We propose a system based on user oriented perceptions as they influence query formation in image and video retrieval. We present a method based on user dependent conceptual structures for creating and maintaining indexes to images and video sequences.

Convergent journalism : the fundamentals of multimedia reporting

This book explains the phenomenon of media convergence, defines what has been until recently a confusing topic, describes the main business models, provides case studies of successful converged newsrooms around the world, and explains how to introduce convergence into the newsroom. Stephen Quinn provides a practical introduction to the changing landscape of news reporting, and has written a useful book for students and professionals alike.

Multimedia journalism: putting it all together

A combination of sound, image, text, and interactivity-hereafter referred to as multimedia journalism-gives media practitioners a new way to tell stories, using the strengths of each medium to produce a more compelling package. From the outset, a multimedia journalist must appreciate the potential and power of each medium and capitalize on those strengths. In doing so the multimedia reporter produces journalism that is well beyond what a single medium can do. The multimedia reporter needs to know how to use a variety of digital tools, but the essential requirement is a multimedia mind-set. Loosely articulated, this mind-set requires the ability to conceive of stories that go beyond a single medium when it is appropriate to use more than one medium. Sometimes one medium is sufficient to provide what audiences need. Implicit in this statement is an understanding of audiences (note the plural form) and an acceptance that the fundamental role of journalism remains constant, which is to inform, educate, and entertain.

Can multimedia meet tertiary educational needs better than the conventional lecture? A case study

Educational researchers have long derided the university lecture as an effective mode of delivery of educational materials, but currently there are many reports on the advantages offered by computer. In this study a multimedia solution was sought to replace existing face to face lectures because it appeared to offer a close 'media versus need' match. Consequently, a decision was made to develop a design template for an interactive computer based program that would be suitable for a range of subject content. In order to personalise the instruction, a large video insert was incorporated as the main screen's most prominent design feature. From here the learner could navigate to support material including interactive simulations. The program was piloted with a small group of students and, in particular, the student tracking data that the program automatically generates yield some interesting learning style information.

A touch of sugar : a multimedia case study to facilitate student dietitians learning about the clinical management of diabetes

This paper describes the development, implementation and evaluation of a multimedia case study on dietetic case management of diabetes. A multimedia web-based program was developed to use the interactive capacity of the web-based environment to enable student dietitians to develop the skills of clinical reasoning and to trigger their learning about the nutritional management of diabetes. A case study of a person with diabetes was developed using a simulated medical history, video clips and sound clips. The students were asked to manage the patient online by responding to the patient's questions, attending a team meeting and outpatient clinic, attending to food service tasks and responding to visual cues. Tutors were able to access the student's responses to submissions online. Evaluation of the program was by questionnaire, which gathered quantitative and qualitative data on the student's perceptions of their experiences in using the web-based case. The students rated the content and the interactive parts of the case highly but experienced technical difficulties and found the case took too much time to complete.

A new indexing method for high dimensional dataset

Indexing high dimensional datasets has attracted extensive attention from many researchers in the last decade. Since R-tree type of index structures are known as suffering curse of dimensionality problems, Pyramid-tree type of index structures, which are based on the B-tree, have been proposed to break the curse of dimensionality. However, for high dimensional data, the number of pyramids is often insufficient to discriminate data points when the number of dimensions is high. Its effectiveness degrades dramatically with the increase of dimensionality. In this paper, we focus on one particular issue of curse of dimensionality; that is, the surface of a hypercube in a high dimensional space approaches 100% of the total hypercube volume when the number of dimensions approaches infinite. We propose a new indexing method based on the surface of dimensionality. We prove that the Pyramid tree technology is a special case of our method. The results of our experiments demonstrate clear priority of our novel method.

Reducing the bandwidth requirements of P2P keyword indexing

This paper describes the design and evaluation of a federated, peer-to-peer indexing system, which can be used to integrate the resources of local systems into a globally addressable index using a distributed hash table. The salient feature of the indexing systems design is the efficient dissemination of term-document indices using a combination of duplicate elimination, leaf set forwarding and conventional techniques such as aggressive index pruning, index compression, and batching. Together these indexing strategies help to reduce the number of RPC operations required to locate the nodes responsible for a section of the index, as well as the bandwidth utilization and the latency of the indexing service. Using empirical observation we evaluate the performance benefits of these cumulative optimizations and show that these design trade-offs can significantly improve indexing performance when using a distributed hash table.

Using object and trajectory analysis to facilitate indexing and retrieval of video

This paper aims to show that by using low level feature extraction, motion and object identifying and tracking methods, features can be extracted and indexed for efficient and effective retrieval for video; such as an awards ceremony video. Video scene/shot analysis and key frame extraction are used as a foundation to identify objects in video and be able to find spatial relationships within the video. The compounding of low level features such as colour, texture and abstract object identification lead into higher level real object identification and tracking and scene detection. The main focus is on using a video style that is different to the heavily used sports and news genres. Using different video styles can open the door to creating methods that could encompass all video types instead of specialized methods for each specific style of video.

Reasoning with multimedia information using symbolic projection

In many multimedia application systems, it is not the final goal to retrieve the relevant multimedia information from different multimedia information sources. Rather, post-processing of the retrieved multimedia information is needed. For example, the retrieved information is used as “known facts”. The systems will do some reasoning to obtain further conclusions based on these multimedia form “known facts”. We call this reasoning with multimedia information. Most current research work in multimedia information processing is focused on multimedia information retrieval, but post-processing the retrieved information is more or less ignored. This paper explores the way to tackle this problem by using symbolic projection. A case study of reasoning with still image information is presented. Some extensions to symbolic projection- introducing auxiliary pictorial objects in symbolic pictures that need to be processed-are discussed. We expect this paper will stimulate further research on this important but ignored topic.