Multimedia-Datenbanken : Retrieval, Suchalgorithmen und Anfragebearbeitung

Multimedia, Datenbanken, inhaltsbasiertes Retrieval, Multimedia-Retrieval

Electrophysiological and behavioral correlates of stable automatic semantic retrieval in aging

Previous studies have shown both declining and stable semantic-memory abilities during healthy aging. There is consistent evidence that semantic processes involving controlled mechanisms weaken with age. In contrast, results of aging studies on automatic semantic retrieval are often inconsistent, probably due to methodological limitations and differences. The present study therefore examines age-related alterations in automatic semantic retrieval and memory structure with a novel combination of critical methodological factors, i.e., the selection of subjects, a well-designed paradigm, and electrophysiological methods that result in unambiguous signal markers. Healthy young and elderly participants performed lexical decisions on visually presented word/non-word pairs with a stimulus onset asynchrony (SOA) of 150 ms. Behavioral and electrophysiological data were measured, and the N400-LPC complex, an event-related potential component sensitive to lexical-semantic retrieval, was analyzed by power and topographic distribution of electrical brain activity. Both age groups exhibited semantic priming (SP) and concreteness effects in behavioral reaction time and the electrophysiological N400-LPC complex. Importantly, elderly subjects did not differ significantly from the young in their lexical decision and SP performances as well as in the N400-LPC SP effect. The only difference was an age-related delay measured in the N400-LPC microstate. This could be attributed to existing age effects in controlled functions, as further supported by the replicated age difference in word fluency. The present results add new behavioral and neurophysiological evidence to earlier findings, by showing that automatic semantic retrieval remains stable in global signal strength and topographic distribution during healthy aging.

Exploring Hidden Coherent Feature Groups and Temporal Semantics for Multimedia Big Data Analysis

Thanks to the advanced technologies and social networks that allow the data to be widely shared among the Internet, there is an explosion of pervasive multimedia data, generating high demands of multimedia services and applications in various areas for people to easily access and manage multimedia data. Towards such demands, multimedia big data analysis has become an emerging hot topic in both industry and academia, which ranges from basic infrastructure, management, search, and mining to security, privacy, and applications. Within the scope of this dissertation, a multimedia big data analysis framework is proposed for semantic information management and retrieval with a focus on rare event detection in videos. The proposed framework is able to explore hidden semantic feature groups in multimedia data and incorporate temporal semantics, especially for video event detection. First, a hierarchical semantic data representation is presented to alleviate the semantic gap issue, and the Hidden Coherent Feature Group (HCFG) analysis method is proposed to capture the correlation between features and separate the original feature set into semantic groups, seamlessly integrating multimedia data in multiple modalities. Next, an Importance Factor based Temporal Multiple Correspondence Analysis (i.e., IF-TMCA) approach is presented for effective event detection. Specifically, the HCFG algorithm is integrated with the Hierarchical Information Gain Analysis (HIGA) method to generate the Importance Factor (IF) for producing the initial detection results. Then, the TMCA algorithm is proposed to efficiently incorporate temporal semantics for re-ranking and improving the final performance. At last, a sampling-based ensemble learning mechanism is applied to further accommodate the imbalanced datasets. In addition to the multimedia semantic representation and class imbalance problems, lack of organization is another critical issue for multimedia big data analysis. In this framework, an affinity propagation-based summarization method is also proposed to transform the unorganized data into a better structure with clean and well-organized information. The whole framework has been thoroughly evaluated across multiple domains, such as soccer goal event detection and disaster information management.

A multimedia indexing and retrieval framework for multimedia database systems

The main challenges of multimedia data retrieval lie in the effective mapping between low-level features and high-level concepts, and in the individual users' subjective perceptions of multimedia content. ^ The objectives of this dissertation are to develop an integrated multimedia indexing and retrieval framework with the aim to bridge the gap between semantic concepts and low-level features. To achieve this goal, a set of core techniques have been developed, including image segmentation, content-based image retrieval, object tracking, video indexing, and video event detection. These core techniques are integrated in a systematic way to enable the semantic search for images/videos, and can be tailored to solve the problems in other multimedia related domains. In image retrieval, two new methods of bridging the semantic gap are proposed: (1) for general content-based image retrieval, a stochastic mechanism is utilized to enable the long-term learning of high-level concepts from a set of training data, such as user access frequencies and access patterns of images. (2) In addition to whole-image retrieval, a novel multiple instance learning framework is proposed for object-based image retrieval, by which a user is allowed to more effectively search for images that contain multiple objects of interest. An enhanced image segmentation algorithm is developed to extract the object information from images. This segmentation algorithm is further used in video indexing and retrieval, by which a robust video shot/scene segmentation method is developed based on low-level visual feature comparison, object tracking, and audio analysis. Based on shot boundaries, a novel data mining framework is further proposed to detect events in soccer videos, while fully utilizing the multi-modality features and object information obtained through video shot/scene detection. ^ Another contribution of this dissertation is the potential of the above techniques to be tailored and applied to other multimedia applications. This is demonstrated by their utilization in traffic video surveillance applications. The enhanced image segmentation algorithm, coupled with an adaptive background learning algorithm, improves the performance of vehicle identification. A sophisticated object tracking algorithm is proposed to track individual vehicles, while the spatial and temporal relationships of vehicle objects are modeled by an abstract semantic model. ^

Formulierung von Nutzerpräferenzen in Multimedia-Retrieval-Systemen

Multimedia, retrieval, multimedia-retrieval-system, multimedia query languages, weighting, preferences

Role of left inferior prefrontal cortex in retrieval of semantic knowledge: A reevaluation

A number of neuroimaging findings have been interpreted as evidence that the left inferior frontal gyrus (IFG) subserves retrieval of semantic knowledge. We provide a fundamentally different interpretation, that it is not retrieval of semantic knowledge per se that is associated with left IFG activity but rather selection of information among competing alternatives from semantic memory. Selection demands were varied across three semantic tasks in a single group of subjects. Functional magnetic resonance imaging signal in overlapping regions of left IFG was dependent on selection demands in all three tasks. In addition, the degree of semantic processing was varied independently of selection demands in one of the tasks. The absence of left IFG activity for this comparison counters the argument that the effects of selection can be attributed solely to variations in degree of semantic retrieval. Our findings suggest that it is selection, not retrieval, of semantic knowledge that drives activity in the left IFG.

A generalized multidimensional index structure for multimedia data to support content-based similarity searches in a collaborative search environment

Since multimedia data, such as images and videos, are way more expressive and informative than ordinary text-based data, people find it more attractive to communicate and express with them. Additionally, with the rising popularity of social networking tools such as Facebook and Twitter, multimedia information retrieval can no longer be considered a solitary task. Rather, people constantly collaborate with one another while searching and retrieving information. But the very cause of the popularity of multimedia data, the huge and different types of information a single data object can carry, makes their management a challenging task. Multimedia data is commonly represented as multidimensional feature vectors and carry high-level semantic information. These two characteristics make them very different from traditional alpha-numeric data. Thus, to try to manage them with frameworks and rationales designed for primitive alpha-numeric data, will be inefficient. An index structure is the backbone of any database management system. It has been seen that index structures present in existing relational database management frameworks cannot handle multimedia data effectively. Thus, in this dissertation, a generalized multidimensional index structure is proposed which accommodates the atypical multidimensional representation and the semantic information carried by different multimedia data seamlessly from within one single framework. Additionally, the dissertation investigates the evolving relationships among multimedia data in a collaborative environment and how such information can help to customize the design of the proposed index structure, when it is used to manage multimedia data in a shared environment. Extensive experiments were conducted to present the usability and better performance of the proposed framework over current state-of-art approaches.

FTMOntology: An ontology to fill the semantic gap between music, mood, personality, and human physiology

This paper presents a domain ontology, the FeelingTheMusic Ontology - FTMOntology. FTMOntology is designed to represent the complex domain of music and how it relates to other domains like mood, personality and physiology. This includes representing the main concepts and relations of music domain with each of the above-mentioned domains. The concepts and relations between music, mood, personality and physiology. The main contribution of this work is to model and relate these different domains in a consistent ontology. © 2011 Springer-Verlag.

Does Age Matter? Semantic Memory in Healthy Young and Elderly Adults

Background: Semantic memory processes have been well described in literature. However, the available findings are mostly based on relatively young subjects and concrete word material (e.g. tree). Comparatively little information exists about semantic memory for abstract words (e.g. mind) and possible age related changes in semantic retrieval. In this respect, we developed a paradigm that is useful to investigate the implicit (i.e. attentionindependent) access to concrete and abstract semantic memory. These processes were then compared between young and elderly healthy subjects. Methods: A well established tool for investigating semantic memory processes is the semantic priming paradigm, which consists both of semantically unrelated and related word pairs. In our behavioral task these noun-noun word pairs were further divided into concrete, abstract and matched pronounceable non-word conditions. With this premise, the young and elderly participants performed a lexical decision task: they were asked to press a choice of two buttons as an indication for whether the word pair contained a non-word or not. In order to minimize controlled (i.e. attention-dependent) retrieval strategies, a short stimulus onset asynchrony (SOA) of 150ms was set. Reaction time (RT) changes and accuracy to related and unrelated words (priming effect) in the abstract vs. concrete condition (concreteness effect) were the dependent variables of interest. Results and Discussion: Statistical analysis confirmed both a significant priming effect (i.e. shorter RTs in semantically related compared to unrelated words) and a concreteness effect (i.e. RT decrease for concrete compared to abstract words) in the young and elderly subjects. There was no age difference in accuracy. The only age effect was a commonly known general slowing in RT over all conditions. In conclusion, age is not a critical factor in the implicit access to abstract and concrete semantic memory.

Altered Electrophysiology of Semantic Word Processing in Alzheimer's Disease and Semantic Dementia

With the progressing course of Alzheimer's disease (AD), deficits in declarative memory increasingly restrict the patients' daily activities. Besides the more apparent episodic (biographical) memory impairments, the semantic (factual) memory is also affected by this neurodegenerative disorder. The episodic pathology is well explored; instead the underlying neurophysiological mechanisms of the semantic deficits remain unclear. For a profound understanding of semantic memory processes in general and in AD patients, the present study compares AD patients with healthy controls and Semantic Dementia (SD) patients, a dementia subgroup that shows isolated semantic memory impairments. We investigate the semantic memory retrieval during the recording of an electroencephalogram, while subjects perform a semantic priming task. Precisely, the task demands lexical (word/nonword) decisions on sequentially presented word pairs, consisting of semantically related or unrelated prime-target combinations. Our analysis focuses on group-dependent differences in the amplitude and topography of the event related potentials (ERP) evoked by related vs. unrelated target words. AD patients are expected to differ from healthy controls in semantic retrieval functions. The semantic storage system itself, however, is thought to remain preserved in AD, while SD patients presumably suffer from the actual loss of semantic representations.

The neural basis for writing from dictation in the temporoparietal cortex.

Cortical electrical stimulation mapping was used to study neural substrates of the function of writing in the temporoparietal cortex. We identified the sites involved in oral language (sentence reading and naming) and writing from dictation, in order to spare these areas during removal of brain tumours in 30 patients (23 in the left, and 7 in the right hemisphere). Electrostimulation of the cortex impaired writing ability in 62 restricted cortical areas (.25 cm2). These were found in left temporoparietal lobes and were mostly located along the superior temporal gyrus (Brodmann's areas 22 and 42). Stimulation of right temporoparietal lobes in right-handed patients produced no writing impairments. However there was a high variability of location between individuals. Stimulation resulted in combined symptoms (affecting oral language and writing) in fourteen patients, whereas in eight other patients, stimulation-induced pure agraphia symptoms with no oral language disturbance in twelve of the identified areas. Each detected area affected writing in a different way. We detected the various different stages of the auditory-to-motor pathway of writing from dictation: either through comprehension of the dictated sentences (word deafness areas), lexico-semantic retrieval, or phonologic processing. In group analysis, barycentres of all different types of writing interferences reveal a hierarchical functional organization along the superior temporal gyrus from initial word recognition to lexico-semantic and phonologic processes along the ventral and the dorsal comprehension pathways, supporting the previously described auditory-to-motor process. The left posterior Sylvian region supports different aspects of writing function that are extremely specialized and localized, sometimes being segregated in a way that could account for the occurrence of pure agraphia that has long-been described in cases of damage to this region.

Interface issues in the ESCHER visual database editor

Presentation at the 1997 Dagstuhl Seminar "Evaluation of Multimedia Information Retrieval", Norbert Fuhr, Keith van Rijsbergen, Alan F. Smeaton (eds.), Dagstuhl Seminar Report 175, 14.04. - 18.04.97 (9716). - Abstract: This presentation will introduce ESCHER, a database editor which supports visualization in non-standard applications in engineering, science, tourism and the entertainment industry. It was originally based on the extended nested relational data model and is currently extended to include object-relational properties like inheritance, object types, integrity constraints and methods. It serves as a research platform into areas such as multimedia and visual information systems, QBE-like queries, computer-supported concurrent work (CSCW) and novel storage techniques. In its role as a Visual Information System, a database editor must support browsing and navigation. ESCHER provides this access to data by means of so called fingers. They generalize the cursor paradigm in graphical and text editors. On the graphical display, a finger is reflected by a colored area which corresponds to the object a finger is currently pointing at. In a table more than one finger may point to objects, one of which is the active finger and is used for navigating through the table. The talk will mostly concentrate on giving examples for this type of navigation and will discuss some of the architectural needs for fast object traversal and display. ESCHER is available as public domain software from our ftp site in Kassel. The portable C source can be easily compiled for any machine running UNIX and OSF/Motif, in particular our working environments IBM RS/6000 and Intel-based LINUX systems. A porting to Tcl/Tk is under way.

Representação iterativa: um modelo para repositórios digitais

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Modelos de recuperação de informação e web semântica: a questão da relevância

Introduction: In the Web environment, there is a need for greater care with regard to the processing of descriptive and thematic information. The concern with the recovery of information in computer systems precedes the development of the first personal computers. Models of information retrieval have been and are today widely used in databases specific to a field whose scope is known. Objectives: Verify how the issue of relevance is treated in the main computer models of information retrieval and, especially, as the issue is addressed in the future of the Web, the called Semantic Web. Methodology: Bibliographical research. Results: In the classical models studied here, it was realized that the main concern is retrieving documents whose description is closest to the search expression used by the user, which does not necessarily imply that this really needs. In semantic retrieval is the use of ontologies, feature that extends the user's search for a wider range of possible relevant options. Conclusions: The relevance is a subjective judgment and inherent to the user, it will depend on the interaction with the system and especially the fact that he expects to recover in your search. Systems that are based on a model of relevance are not popular, because it requires greater interaction and depend on the user's disposal. The Semantic Web is so far the initiative more efficient in the case of information retrieval in the digital environment.

Representação iterativa e folksonomia assistida para repositórios digitais

Information retrieval has been much discussed within Information Science lately. The search for quality information compatible with the users needs became the object of constant research.Using the Internet as a source of dissemination of knowledge has suggested new models of information storage, such as digital repositories, which have been used in academic research as the main form of autoarchiving and disseminating information, but with an information structure that suggests better descriptions of resources and hence better retrieval.Thus the objective is to improve the process of information retrieval, presenting a proposal for a structural model in the context of the semantic web, addressing the use of web 2.0 and web 3.0 in digital repositories, enabling semantic retrieval of information through building a data layer called Iterative Representation.The present study is characterized as descriptive and analytical, based on document analysis, divided into two parts: the first, characterized by direct observation of non-participatory tools that implement digital repositories, as well as digital repositories already instantiated, and the second with scanning feature, which suggests an innovative model for repositories, with the use of structures of knowledge representation and user participation in building a vocabulary domain. The model suggested and proposed ─ Iterative Representation ─ will allow to tailor the digital repositories using Folksonomy and also controlled vocabulary of the field in order to generate a data layer iterative, which allows feedback information, and semantic retrieval of information, through the structural model designed for repositories. The suggested model resulted in the formulation of the thesis that through Iterative Representation it is possible to establish a process of semantic retrieval of information in digital repositories.