Semi-automatic knowledge extraction, representation and context-sensitive intelligent retrieval of video content using collateral context modelling with scalable ontological networks

Automatic indexing and retrieval of digital data poses major challenges. The main problem arises from the ever increasing mass of digital media and the lack of efficient methods for indexing and retrieval of such data based on the semantic content rather than keywords. To enable intelligent web interactions, or even web filtering, we need to be capable of interpreting the information base in an intelligent manner. For a number of years research has been ongoing in the field of ontological engineering with the aim of using ontologies to add such (meta) knowledge to information. In this paper, we describe the architecture of a system (Dynamic REtrieval Analysis and semantic metadata Management (DREAM)) designed to automatically and intelligently index huge repositories of special effects video clips, based on their semantic content, using a network of scalable ontologies to enable intelligent retrieval. The DREAM Demonstrator has been evaluated as deployed in the film post-production phase to support the process of storage, indexing and retrieval of large data sets of special effects video clips as an exemplar application domain. This paper provides its performance and usability results and highlights the scope for future enhancements of the DREAM architecture which has proven successful in its first and possibly most challenging proving ground, namely film production, where it is already in routine use within our test bed Partners' creative processes. (C) 2009 Published by Elsevier B.V.

Semantic-associative visual content labelling and retrieval: A multimodal approach

A novel framework referred to as collaterally confirmed labelling (CCL) is proposed, aiming at localising the visual semantics to regions of interest in images with textual keywords. Both the primary image and collateral textual modalities are exploited in a mutually co-referencing and complementary fashion. The collateral content and context-based knowledge is used to bias the mapping from the low-level region-based visual primitives to the high-level visual concepts defined in a visual vocabulary. We introduce the notion of collateral context, which is represented as a co-occurrence matrix of the visual keywords. A collaborative mapping scheme is devised using statistical methods like Gaussian distribution or Euclidean distance together with collateral content and context-driven inference mechanism. We introduce a novel high-level visual content descriptor that is devised for performing semantic-based image classification and retrieval. The proposed image feature vector model is fundamentally underpinned by the CCL framework. Two different high-level image feature vector models are developed based on the CCL labelling of results for the purposes of image data clustering and retrieval, respectively. A subset of the Corel image collection has been used for evaluating our proposed method. The experimental results to-date already indicate that the proposed semantic-based visual content descriptors outperform both traditional visual and textual image feature models. (C) 2007 Elsevier B.V. All rights reserved.

Collaterally cued labelling framework underpinning semantic-level visual content descriptor

In this paper, we introduce a novel high-level visual content descriptor devised for performing semantic-based image classification and retrieval. The work can be treated as an attempt for bridging the so called "semantic gap". The proposed image feature vector model is fundamentally underpinned by an automatic image labelling framework, called Collaterally Cued Labelling (CCL), which incorporates the collateral knowledge extracted from the collateral texts accompanying the images with the state-of-the-art low-level visual feature extraction techniques for automatically assigning textual keywords to image regions. A subset of the Corel image collection was used for evaluating the proposed method. The experimental results indicate that our semantic-level visual content descriptors outperform both conventional visual and textual image feature models.

The relation between content and structure in language production: an analysis of speech errors in semantic dementia

In order to explore the impact of a degraded semantic system on the structure of language production, we analysed transcripts from autobiographical memory interviews to identify naturally-occurring speech errors by eight patients with semantic dementia (SD) and eight age-matched normal speakers. Relative to controls, patients were significantly more likely to (a) substitute and omit open class words, (b) substitute (but not omit) closed class words, (c) substitute incorrect complex morphological forms and (d) produce semantically and/or syntactically anomalous sentences. Phonological errors were scarce in both groups. The study confirms previous evidence of SD patients’ problems with open class content words which are replaced by higher frequency, less specific terms. It presents the first evidence that SD patients have problems with closed class items and make syntactic as well as semantic speech errors, although these grammatical abnormalities are mostly subtle rather than gross. The results can be explained by the semantic deficit which disrupts the representation of a pre-verbal message, lexical retrieval and the early stages of grammatical encoding.

Capturing temporality and intentionality in information systems

Information modelling is a topic that has been researched a great deal, but still many questions around it have not been solved. An information model is essential in the design of a database which is the core of an information system. Currently most of databases only deal with information that represents facts, or asserted information. The ability of capturing semantic aspect has to be improved, and yet other types, such as temporal and intentional information, should be considered. Semantic Analysis, a method of information modelling, has offered a way to handle various aspects of information. It employs the domain knowledge and communication acts as sources of information modelling. It lends itself to a uniform structure whereby semantic, temporal and intentional information can be captured, which builds a sound foundation for building a semantic temporal database.

Development of a semantic graph-based scene description

Software representations of scenes, i.e. the modelling of objects in space, are used in many application domains. Current modelling and scene description standards focus on visualisation dimensions, and are intrinsically limited by their dependence upon their semantic interpretation and contextual application by humans. In this paper we propose the need for an open, extensible and semantically rich modelling language, which facilitates a machine-readable semantic structure. We critically review existing standards and techniques, and highlight a need for a semantically focussed scene description language. Based on this defined need we propose a preliminary solution, based on hypergraph theory, and reflect on application domains.

HR-semantics roadmap: the semantic challenges and opportunities in the Human Resources domain

Purpose: This document identifies the challenges and opportunities in applying the ontology technology in the Human Resources domain. Target users: A reference for both the HR and the ontology communities. Also, to be used as a roadmap for the OOA itself, within the HR domain. Background: During the discussion panel at the OOA kick-off workshop, which was attended by more than 50 HR and ontology experts, the need for this roadmap was realized. It was obvious that the current understanding of the problem of semantics in HR is fragmented and only partial solutions exist. People from both the HR and the ontology communities speak different languages, have different understandings, and are not aware of existing solutions.

Investigating orthographic and semantic aspects of word learning in poor comprehenders

This study compared orthographic and semantic aspects of word learning in children who differed in reading comprehension skill. Poor comprehenders and controls matched for age (9-10 years), nonverbal ability and decoding skill were trained to pronounce 20 visually presented nonwords, 10 in a consistent way and 10 in an inconsistent way. They then had an opportunity to infer the meanings of the new words from story context. Orthographic learning was measured in three ways: the number of trials taken to learn to pronounce nonwords correctly, orthographic choice and spelling. Across all measures, consistent items were easier than inconsistent items and poor comprehenders did not differ from control children. Semantic learning was assessed on three occasions, using a nonword-picture matching task. While poor comprehenders showed equivalent semantic learning to controls immediately after exposure to nonword meaning, this knowledge was not well retained over time. Results are discussed in terms of the language and reading skills of poor comprehenders and in relation to current models of reading development.

The role of self-teaching in learning orthographic and semantic aspects of new words

This study explores how children learn the meaning (semantics) and spelling patterns (orthography) of novel words encountered in story context. English-speaking children (N = 88) aged 7 to 8 years read 8 stories and each story contained 1 novel word repeated 4 times. Semantic cues were provided by the story context such that children could infer the meaning of the word (specific context) or the category that the word belonged to (general context). Following story reading, posttests indicated that children showed reliable semantic and orthographic learning. Decoding was the strongest predictor of orthographic learning, indicating that self-teaching via phonological recoding was important for this aspect of word learning. In contrast, oral vocabulary emerged as the strongest predictor of semantic learning.

Is prospective memory enhanced by cue-action semantic relatedness and enactment at encoding?

Benefits and costs on prospective memory performance, of enactment at encoding and a semantic association between a cue-action word pair, were investigated in two experiments. Findings revealed superior performance for both younger and older adults following enactment, in contrast to verbal encoding, and when cue-action semantic relatedness was high. Although younger adults outperformed older adults, age did not moderate benefits of cue-action relatedness or enactment. Findings from a second experiment revealed that the inclusion of an instruction to perform a prospective memory task led to increments in response latency to items from the ongoing activity in which that task was embedded, relative to latencies when the ongoing task only was performed. However, this task interference ‘cost’ did not differ as a function of either cue-action relatedness or enactment. We argue that the high number of cue-action pairs employed here influenced meta-cognitive consciousness, hence determining attention allocation, in all experimental conditions.