Semi-automatic annotation and retrieval of visual content using the topic map technology

There are still major challenges in the area of automatic indexing and retrieval of multimedia content data for very large multimedia content corpora. Current indexing and retrieval applications still use keywords to index multimedia content and those keywords usually do not provide any knowledge about the semantic content of the data. With the increasing amount of multimedia content, it is inefficient to continue with this approach. In this paper, we describe the project DREAM, which addresses such challenges by proposing a new framework for semi-automatic annotation and retrieval of multimedia based on the semantic content. The framework uses the Topic Map Technology, as a tool to model the knowledge automatically extracted from the multimedia content using an Automatic Labelling Engine. We describe how we acquire knowledge from the content and represent this knowledge using the support of NLP to automatically generate Topic Maps. The framework is described in the context of film post-production.

Semi-automatic knowledge extraction, representation and context-sensitive intelligent retrieval of video content using collateral context modelling with scalable ontological networks

Automatic indexing and retrieval of digital data poses major challenges. The main problem arises from the ever increasing mass of digital media and the lack of efficient methods for indexing and retrieval of such data based on the semantic content rather than keywords. To enable intelligent web interactions, or even web filtering, we need to be capable of interpreting the information base in an intelligent manner. For a number of years research has been ongoing in the field of ontological engineering with the aim of using ontologies to add such (meta) knowledge to information. In this paper, we describe the architecture of a system (Dynamic REtrieval Analysis and semantic metadata Management (DREAM)) designed to automatically and intelligently index huge repositories of special effects video clips, based on their semantic content, using a network of scalable ontologies to enable intelligent retrieval. The DREAM Demonstrator has been evaluated as deployed in the film post-production phase to support the process of storage, indexing and retrieval of large data sets of special effects video clips as an exemplar application domain. This paper provides its performance and usability results and highlights the scope for future enhancements of the DREAM architecture which has proven successful in its first and possibly most challenging proving ground, namely film production, where it is already in routine use within our test bed Partners' creative processes. (C) 2009 Published by Elsevier B.V.

Semantic-associative visual content labelling and retrieval: A multimodal approach

A novel framework referred to as collaterally confirmed labelling (CCL) is proposed, aiming at localising the visual semantics to regions of interest in images with textual keywords. Both the primary image and collateral textual modalities are exploited in a mutually co-referencing and complementary fashion. The collateral content and context-based knowledge is used to bias the mapping from the low-level region-based visual primitives to the high-level visual concepts defined in a visual vocabulary. We introduce the notion of collateral context, which is represented as a co-occurrence matrix of the visual keywords. A collaborative mapping scheme is devised using statistical methods like Gaussian distribution or Euclidean distance together with collateral content and context-driven inference mechanism. We introduce a novel high-level visual content descriptor that is devised for performing semantic-based image classification and retrieval. The proposed image feature vector model is fundamentally underpinned by the CCL framework. Two different high-level image feature vector models are developed based on the CCL labelling of results for the purposes of image data clustering and retrieval, respectively. A subset of the Corel image collection has been used for evaluating our proposed method. The experimental results to-date already indicate that the proposed semantic-based visual content descriptors outperform both traditional visual and textual image feature models. (C) 2007 Elsevier B.V. All rights reserved.

Visual motion interferes with lexical decision on motion words

Embodied theories of cognition propose that neural substrates used in experiencing the referent of a word, for example perceiving upward motion, should be engaged in weaker form when that word, for example ‘rise’, is comprehended. Motivated by the finding that the perception of irrelevant background motion at near-threshold, but not supra-threshold, levels interferes with task execution, we assessed whether interference from near-threshold background motion was modulated by its congruence with the meaning of words (semantic content) when participants completed a lexical decision task (deciding if a string of letters is a real word or not). Reaction times for motion words, such as ‘rise’ or ‘fall’, were slower when the direction of visual motion and the ‘motion’ of the word were incongruent — but only when the visual motion was at nearthreshold levels. When motion was supra-threshold, the distribution of error rates, not reaction times, implicated low-level motion processing in the semantic processing of motion words. As the perception of near-threshold signals is not likely to be influenced by strategies, our results support a close contact between semantic information and perceptual systems.

Analysing lexical richness in French learner language: what frequency lists and teacher judgements can tell us about basic and advanced words

We show how teacher judgements can be used to assess the quality of vocabulary used by L2 learners of French.

Language dominance and lexical diversity: how bilinguals and L2 learners differ in their knowledge and use of French lexical and functional items

This chapter compares lexical diversity of French words used by Dutch-French bilinguals, English-French bilinguals and Flemish L2 learners of French.

Processing empty categories in a second language: When naturalistic exposure fills the (intermediate) gap

An ongoing debate on second language (L2) processing revolves around whether or not L2 learners process syntactic information similarly to monolinguals (L1), and what factors lead to a native-like processing. According to the Shallow Structure Hypothesis (Clahsen & Felser, 2006a), L2 learners’ processing does not include abstract syntactic features, such as intermediate gaps of wh-movement, but relies more on lexical/semantic information. Other researchers have suggested that naturalistic L2 exposure can lead to native-like processing (Dussias, 2003). This study investigates the effect of naturalistic exposure in processing wh-dependencies. Twenty-six advanced Greek learners of L2 English with an average nine years of naturalistic exposure, 30 with classroom exposure, and 30 native speakers of English completed a self-paced reading task with sentences involving intermediate gaps. L2 learners with naturalistic exposure showed evidence of native-like processing of the intermediate gaps, suggesting that linguistic immersion can lead to native-like abstract syntactic processing in the L2.

Production and comprehension of English and Hindi in multilingual transcortical aphasia

This study investigates the two later-acquired but proficient languages, English and Hindi, of two multilingual individuals with transcortical aphasia (right basal ganglia lesion in GN and brain stem lesion in GS). Dissociation between lexical and syntactic profiles in both the languages with a uniform performance across the languages at the lexical level and an uneven performance across the languages at the syntactic level was observed. Their performances are discussed in relation to the implicit/explicit language processes (Paradis, 1994 and Paradis, 2004) and the declarative/procedural model (Ullman, 2001b and Ullman, 2005) of bilingual language processing. Additionally, their syntactic performance is interpreted in relation to the salient grammatical contrasts between English and Hindi.

Measuring lexical diversity among L2 learners of French: an exploration of the validity of D, MTLD and HD-D as measures of language ability

In this study two new measures of lexical diversity are tested for the first time on French. The usefulness of these measures, MTLD (McCarthy and Jarvis (2010 and this volume) ) and HD-D (McCarthy and Jarvis 2007), in predicting different aspects of language proficiency is assessed and compared with D (Malvern and Richards 1997; Malvern, Richards, Chipere and Durán 2004) and Maas (1972) in analyses of stories told by two groups of learners (n=41) of two different proficiency levels and one group of native speakers of French (n=23). The importance of careful lemmatization in studies of lexical diversity which involve highly inflected languages is also demonstrated. The paper shows that the measures of lexical diversity under study are valid proxies for language ability in that they explain up to 62 percent of the variance in French C-test scores, and up to 33 percent of the variance in a measure of complexity. The paper also provides evidence that dependence on segment size continues to be a problem for the measures of lexical diversity discussed in this paper. The paper concludes that limiting the range of text lengths or even keeping text length constant is the safest option in analysing lexical diversity.

Lexical access, story re-telling and sequencing skills in adults who clutter and those who do not

Cluttering is a rate-based disorder of fluency, the scope of whose diagnostic criteria currently remains unclear. This paper reports preliminary findings from a larger study which aims to determine whether cluttering can be associated with language disturbances as well as motor and rate based ones. Subtests from the Mt Wilga High Level Language Test (MWHLLT) were used to determine whether people who clutter (PWC) have word finding difficulties, and use significantly more maze behaviours compared to controls, during story re-telling and simple sequencing tasks. Independent t tests showed that PWC were significantly slower than control participants in lexical access and sentence completion tasks, but returned mixed findings when PWCs were required to name items within a semantic category. PWC produced significantly more maze behaviour than controls in a task where participants were required to explain how to undertake commonly performed actions, but no difference in use of maze behaviour was found between the two groups when retelling a story from memory. The implications of these findings are discussed

Sometimes they use it, sometimes they don’t: an epistemological discussion of L2 morphological production and its use as a competence measurement

The main goal of all approaches to adult second language acquisition (SLA) is to accurately describe and explain the overall acquisition process. To accomplish this, SLA researchers must come to agree on some key issues. In this commentary, I defend the necessity of the competence/performance distinction and how this relates to why an examination of morphological production presents challenges for SLA research. I suggest that such a methodology is meaningful only when it is dovetailed with procedures that test for related syntactic/semantic knowledge.

Broadly speaking: vocabulary in semantic dementia shifts towards general, semantically diverse words

One of the cardinal features of semantic dementia (SD) is a steady reduction in expressive vocabulary. We investigated the nature of this breakdown by assessing the psycholinguistic characteristics of words produced spontaneously by SD patients during an autobiographical memory interview. Speech was analysed with respect to frequency and imageability, and a recently-developed measure called semantic diversity. This measure quantifies the degree to which a word can be used in a broad range of different linguistic contexts. We used this measure in a formal exploration of the tendency for SD patients to replace specific terms with more vague and general words, on the assumption that more specific words are used in a more constrained set of contexts. Relative to healthy controls, patients were less likely to produce low-frequency, high-imageability words, and more likely to produce highly frequent, abstract words. These changes in the lexical-semantic landscape were related to semantic diversity: the highly frequent and abstract words most prevalent in the patients' speech were also the most semantically diverse. In fact, when the speech samples of healthy controls were artificially engineered such that low semantic diversity words (e.g., garage, spanner) were replaced with broader terms (e.g., place, thing), the characteristics of their speech production came to closely resemble that of SD patients. A similar simulation in which low-frequency words were replaced was less successful in replicating the patient data. These findings indicate systematic biases in the deterioration of lexical-semantic space in SD. As conceptual knowledge degrades, speech increasingly consists of general terms that can be applied in a broad range of linguistic contexts and convey less specific information.