Semi-automatic knowledge extraction, representation and context-sensitive intelligent retrieval of video content using collateral context modelling with scalable ontological networks

Automatic indexing and retrieval of digital data poses major challenges. The main problem arises from the ever increasing mass of digital media and the lack of efficient methods for indexing and retrieval of such data based on the semantic content rather than keywords. To enable intelligent web interactions, or even web filtering, we need to be capable of interpreting the information base in an intelligent manner. For a number of years research has been ongoing in the field of ontological engineering with the aim of using ontologies to add such (meta) knowledge to information. In this paper, we describe the architecture of a system (Dynamic REtrieval Analysis and semantic metadata Management (DREAM)) designed to automatically and intelligently index huge repositories of special effects video clips, based on their semantic content, using a network of scalable ontologies to enable intelligent retrieval. The DREAM Demonstrator has been evaluated as deployed in the film post-production phase to support the process of storage, indexing and retrieval of large data sets of special effects video clips as an exemplar application domain. This paper provides its performance and usability results and highlights the scope for future enhancements of the DREAM architecture which has proven successful in its first and possibly most challenging proving ground, namely film production, where it is already in routine use within our test bed Partners' creative processes. (C) 2009 Published by Elsevier B.V.

Video activity extraction and reporting with incremental unsupervised learning

The present work presents a new method for activity extraction and reporting from video based on the aggregation of fuzzy relations. Trajectory clustering is first employed mainly to discover the points of entry and exit of mobiles appearing in the scene. In a second step, proximity relations between resulting clusters of detected mobiles and contextual elements from the scene are modeled employing fuzzy relations. These can then be aggregated employing typical soft-computing algebra. A clustering algorithm based on the transitive closure calculation of the fuzzy relations allows building the structure of the scene and characterises the ongoing different activities of the scene. Discovered activity zones can be reported as activity maps with different granularities thanks to the analysis of the transitive closure matrix. Taking advantage of the soft relation properties, activity zones and related activities can be labeled in a more human-like language. We present results obtained on real videos corresponding to apron monitoring in the Toulouse airport in France.

Modeling camera orientation and 3D structure from a sequence of images taken by a perambulating commercial video camera

In this paper we report the degree of reliability of image sequences taken by off-the-shelf TV cameras for modeling camera rotation and reconstructing 3D structure using computer vision techniques. This is done in spite of the fact that computer vision systems usually use imaging devices that are specifically designed for the human vision. Our scenario consists of a static scene and a mobile camera moving through the scene. The scene is any long axial building dominated by features along the three principal orientations and with at least one wall containing prominent repetitive planar features such as doors, windows bricks etc. The camera is an ordinary commercial camcorder moving along the axial axis of the scene and is allowed to rotate freely within the range +/- 10 degrees in all directions. This makes it possible that the camera be held by a walking unprofessional cameraman with normal gait, or to be mounted on a mobile robot. The system has been tested successfully on sequence of images of a variety of structured, but fairly cluttered scenes taken by different walking cameramen. The potential application areas of the system include medicine, robotics and photogrammetry.

Deterministic IIR video deghoster for all ghost carrier phases

It is known that certain video deghoster systems cannot fully process the induced signal derived from the quadrature carrier forming nature of the VSB filter under a multipath condition. A new deterministic IIR deghoster filter structure is given which is capable of deghosting terrestrial video for any relative ghost carrier phase.

Cancellation of residual ghost VSB components from video IIR deghoster systems

Certain deghosters suffer from the presence of distortion caused by the quadrature forming nature of the IF VSB filter operating on a ghosted IF signal. By analysing this a priori effect, a specific deghoster solution is found by using the phase quadrature detected IF signal to cancel the VSB induced ghost quadrature distortions from the detected inphase signal before deghosting takes place.

The concept of a new video detector for ghosted IF signals

The effect of terrestrial television multipath signals on the intermediate frequency (IF) vestigial sideband filter and the video detector are discussed. A new detector is proposed which, by processing the detected phase quadrature information, derives the correct phase for synchronous detection in the presence of multipath effects. This minimizes dispersion and produces a detected video signal with the linear addition of any ghosts.

Vidibus: a low-cost, modular bus system for real-time video processing

The advantages of standard bus systems have been appreciated for many years. The ability to connect only those modules required to perform a given task has both technical and commercial advantages over a system with a fixed architecture which cannot be easily expanded or updated. Although such bus standards have proliferated in the microprocessor field, a general purpose low-cost standard for digital video processing has yet to gain acceptance. The paper describes the likely requirements of such a system, and discusses three currently available commercial systems. A new bus specification known as Vidibus, developed to fulfil these requirements, is presented. Results from applications already implemented using this real-time bus system are also given.

A specific pole placement technique to minimise hardware requirements within a real time terrestrial and cable video echo canceller

The real time hardware architecture of a deterministic video echo canceller (deghoster) system is presented. The deghoster is capable of calculating all the multipath channel distortion characteristics from terrestrial and cable television in one single pass while performing real time video in-line ghost cancellation. The results from the actual system are also presented in this paper.

Local oscillator phase lock under multipath for video deghoster systems

Under multipath conditions, standard Video Intermediate Frequency (VIF) detectors generate a local oscillator phase error and consequently produce a dispersed non-ideal detected video signal due to the presence of additional IF carriers. The dispersed video causes problems when attempting to identify and remove the multipath interference, or ghosts, by the use of Digital Signal Processing and digital filtering. A digital phase lock system is presented which derives the correct phase for synchronous detection in the presence of multipath by using correlation information that has already been calculated as part of the deghosting process. As a result, the video deghoster system is made simpler, faster and more economical.

The findings from a single pass deghoster prototype for terrestrial transmitted video

A deterministic prototype video deghoster is presented which is capable of calculating all the multipath channel distortion characteristics in one single pass and subsequently removing the multipath distortions, commonly termed ghosts. Within the system, a channel identification algorithm finds in isolation all the ghost components while a dedicated DSP filter subsystem is capable of removing ghosts in real time. The results from the system are presented.

A new method for multipath channel characterisation for terrestrial transmitted video

Under the multipath conditions of terrestrial television transmission, ghost carriers cause additive information to be generated by the VSB filter within the television receiver. By analysis of this a priori effect of the VSB filter under a ghosted condition the inphase and phase quadrature detected video signals are defined. Derived from these results, a new algorithm based upon correlation techniques is presented which finds the characteristics of the amplitude and delay of individual ghosts. These characteristics are then passed to a deterministic deghoster to minimise ghost effects.