A multimedia indexing and retrieval framework for multimedia database systems

The main challenges of multimedia data retrieval lie in the effective mapping between low-level features and high-level concepts, and in the individual users' subjective perceptions of multimedia content. ^ The objectives of this dissertation are to develop an integrated multimedia indexing and retrieval framework with the aim to bridge the gap between semantic concepts and low-level features. To achieve this goal, a set of core techniques have been developed, including image segmentation, content-based image retrieval, object tracking, video indexing, and video event detection. These core techniques are integrated in a systematic way to enable the semantic search for images/videos, and can be tailored to solve the problems in other multimedia related domains. In image retrieval, two new methods of bridging the semantic gap are proposed: (1) for general content-based image retrieval, a stochastic mechanism is utilized to enable the long-term learning of high-level concepts from a set of training data, such as user access frequencies and access patterns of images. (2) In addition to whole-image retrieval, a novel multiple instance learning framework is proposed for object-based image retrieval, by which a user is allowed to more effectively search for images that contain multiple objects of interest. An enhanced image segmentation algorithm is developed to extract the object information from images. This segmentation algorithm is further used in video indexing and retrieval, by which a robust video shot/scene segmentation method is developed based on low-level visual feature comparison, object tracking, and audio analysis. Based on shot boundaries, a novel data mining framework is further proposed to detect events in soccer videos, while fully utilizing the multi-modality features and object information obtained through video shot/scene detection. ^ Another contribution of this dissertation is the potential of the above techniques to be tailored and applied to other multimedia applications. This is demonstrated by their utilization in traffic video surveillance applications. The enhanced image segmentation algorithm, coupled with an adaptive background learning algorithm, improves the performance of vehicle identification. A sophisticated object tracking algorithm is proposed to track individual vehicles, while the spatial and temporal relationships of vehicle objects are modeled by an abstract semantic model. ^

Image synthesis based on a model of human vision

Modern computer graphics systems are able to construct renderings of such high quality that viewers are deceived into regarding the images as coming from a photographic source. Large amounts of computing resources are expended in this rendering process, using complex mathematical models of lighting and shading. However, psychophysical experiments have revealed that viewers only regard certain informative regions within a presented image. Furthermore, it has been shown that these visually important regions contain low-level visual feature differences that attract the attention of the viewer. This thesis will present a new approach to image synthesis that exploits these experimental findings by modulating the spatial quality of image regions by their visual importance. Efficiency gains are therefore reaped, without sacrificing much of the perceived quality of the image. Two tasks must be undertaken to achieve this goal. Firstly, the design of an appropriate region-based model of visual importance, and secondly, the modification of progressive rendering techniques to effect an importance-based rendering approach. A rule-based fuzzy logic model is presented that computes, using spatial feature differences, the relative visual importance of regions in an image. This model improves upon previous work by incorporating threshold effects induced by global feature difference distributions and by using texture concentration measures. A modified approach to progressive ray-tracing is also presented. This new approach uses the visual importance model to guide the progressive refinement of an image. In addition, this concept of visual importance has been incorporated into supersampling, texture mapping and computer animation techniques. Experimental results are presented, illustrating the efficiency gains reaped from using this method of progressive rendering. This visual importance-based rendering approach is expected to have applications in the entertainment industry, where image fidelity may be sacrificed for efficiency purposes, as long as the overall visual impression of the scene is maintained. Different aspects of the approach should find many other applications in image compression, image retrieval, progressive data transmission and active robotic vision.

A database for person re-identification in multi-camera surveillance networks

Person re-identification involves recognising individuals in different locations across a network of cameras and is a challenging task due to a large number of varying factors such as pose (both subject and camera) and ambient lighting conditions. Existing databases do not adequately capture these variations, making evaluations of proposed techniques difficult. In this paper, we present a new challenging multi-camera surveillance database designed for the task of person re-identification. This database consists of 150 unscripted sequences of subjects travelling in a building environment though up to eight camera views, appearing from various angles and in varying illumination conditions. A flexible XML-based evaluation protocol is provided to allow a highly configurable evaluation setup, enabling a variety of scenarios relating to pose and lighting conditions to be evaluated. A baseline person re-identification system consisting of colour, height and texture models is demonstrated on this database.

数字图书馆中图像信息库的检索技术

简要介绍了基于内容的检索技术在数字图书馆中图像信息库检索方面的应用 ,着重对基于对象特征的检索技术做了一定的探讨。并给出了检索效果评价准则

Fast Contour Matching Using Approximate Earth Mover's Distance

Weighted graph matching is a good way to align a pair of shapes represented by a set of descriptive local features; the set of correspondences produced by the minimum cost of matching features from one shape to the features of the other often reveals how similar the two shapes are. However, due to the complexity of computing the exact minimum cost matching, previous algorithms could only run efficiently when using a limited number of features per shape, and could not scale to perform retrievals from large databases. We present a contour matching algorithm that quickly computes the minimum weight matching between sets of descriptive local features using a recently introduced low-distortion embedding of the Earth Mover's Distance (EMD) into a normed space. Given a novel embedded contour, the nearest neighbors in a database of embedded contours are retrieved in sublinear time via approximate nearest neighbors search. We demonstrate our shape matching method on databases of 10,000 images of human figures and 60,000 images of handwritten digits.

Image indexing and retrieval using an ART-2A neural network architecture

Traditional content-based image retrieval (CBIR) systems use low-level features such as colors, shapes, and textures of images. Although, users make queries based on semantics, which are not easily related to such low-level characteristics. Recent works on CBIR confirm that researchers have been trying to map visual low-level characteristics and high-level semantics. The relation between low-level characteristics and image textual information has motivated this article which proposes a model for automatic classification and categorization of words associated to images. This proposal considers a self-organizing neural network architecture, which classifies textual information without previous learning. Experimental results compare the performance results of the text-based approach to an image retrieval system based on low-level features. (c) 2008 Wiley Periodicals, Inc.

Convex Hulls in Image Processing : A Scoping Review

The demands of image processing related systems are robustness, high recognition rates, capability to handle incomplete digital information, and magnanimous flexibility in capturing shape of an object in an image. It is exactly here that, the role of convex hulls comes to play. The objective of this paper is twofold. First, we summarize the state of the art in computational convex hull development for researchers interested in using convex hull image processing to build their intuition, or generate nontrivial models. Secondly, we present several applications involving convex hulls in image processing related tasks. By this, we have striven to show researchers the rich and varied set of applications they can contribute to. This paper also makes a humble effort to enthuse prospective researchers in this area. We hope that the resulting awareness will result in new advances for specific image recognition applications.

Seleção seqüencial de descritores por análise da semântica para recuperação de imagens baseada no conteúdo

With the rapid growth of databases of various types (text, multimedia, etc..), There exist a need to propose methods for ordering, access and retrieve data in a simple and fast way. The images databases, in addition to these needs, require a representation of the images so that the semantic content characteristics are considered. Accordingly, several proposals such as the textual annotations based retrieval has been made. In the annotations approach, the recovery is based on the comparison between the textual description that a user can make of images and descriptions of the images stored in database. Among its drawbacks, it is noted that the textual description is very dependent on the observer, in addition to the computational effort required to describe all the images in database. Another approach is the content based image retrieval - CBIR, where each image is represented by low-level features such as: color, shape, texture, etc. In this sense, the results in the area of CBIR has been very promising. However, the representation of the images semantic by low-level features is an open problem. New algorithms for the extraction of features as well as new methods of indexing have been proposed in the literature. However, these algorithms become increasingly complex. So, doing an analysis, it is natural to ask whether there is a relationship between semantics and low-level features extracted in an image? and if there is a relationship, which descriptors better represent the semantic? which leads us to a new question: how to use descriptors to represent the content of the images?. The work presented in this thesis, proposes a method to analyze the relationship between low-level descriptors and semantics in an attempt to answer the questions before. Still, it was observed that there are three possibilities of indexing images: Using composed characteristic vectors, using parallel and independent index structures (for each descriptor or set of them) and using characteristic vectors sorted in sequential order. Thus, the first two forms have been widely studied and applied in literature, but there were no records of the third way has even been explored. So this thesis also proposes to index using a sequential structure of descriptors and also the order of these descriptors should be based on the relationship that exists between each descriptor and semantics of the users. Finally, the proposed index in this thesis revealed better than the traditional approachs and yet, was showed experimentally that the order in this sequence is important and there is a direct relationship between this order and the relationship of low-level descriptors with the semantics of the users

Monooxygenases involved in the n-alkanes metabolism by Rhodococcus sp. BCP1: molecular characterization and expression of alkB gene

Bioremediation implies the use of living organisms, primarily microorganisms, to convert environmental contaminants into less toxic forms. The impact of the consequences of hydrocarbon release in the environment maintain a high research interest in the study of microbial metabolisms associated with the biodegradation of aromatic and aliphatic hydrocarbons but also in the analysis of microbial enzymes that can convert petroleum substrates to value-added products. The studies described in this Thesis fall within the research field that directs the efforts into identifying gene/proteins involved in the catabolism of n-alkanes and into studying the regulatory mechanisms leading to their oxidation. In particular the studies were aimed at investigating the molecular aspects of the ability of Rhodococcus sp. BCP1 to grow on aliphatic hydrocarbons as sole carbon and energy sources. We studied the ability of Rhodococcus sp. BCP1 to grow on gaseous (C2-C4), liquid (C5-C16) and solid (C17-C28) n-alkanes that resulted to be biochemically correlated with the activity of one or more monooxygenases. In order to identify the alkane monooxygenase that is involved in the n-alkanes degradation pathway in Rhodococcus sp. BCP1, PCR-based methodology was applied by using degenerate primers targeting AlkB monooxygenase family members. As result, a chromosomal region, including the alkB gene cluster, was cloned from Rhodococcus sp. BCP1 genome. We characterized the products of this alkB gene cluster and the products of the orfs included in the flanking regions by comparative analysis with the homologues in the database. alkB gene expression studies were carried out by RT-PCR and by the construction of a promoter probe vector containing the lacZ gene downstream of the alkB promoter. B-galactosidase assays revealed the alkB promoter activity induced by n-alkanes and by n-alkanes metabolic products. Furthermore, the transcriptional start of alkB gene was determined by primer extension procedure. A proteomic approach was subsequently applied to compare the protein patterns expressed by BCP1 growing on n-butane, n-hexane, n-hexadecane or n-eicosane with the protein pattern expressed by BCP1 growing on succinate. The accumulation of enzymes specifically induced on n-alkanes was determined. These enzymes were identified by tandem mass spectrometry (LC/MS/MS). Finally, a prm gene, homologue to the gene family coding for soluble di-iron monooxygenases (SDIMOs), has been isolated from Rhodococcus sp. BCP1 genome. This gene product could be involved in the degradation of gaseous n-alkanes in this Rhodococcus strain. The versatility in utilizing hydrocarbons and the discovery of new remarkable metabolic activities outline the potential applications of this microorganism in environmental and industrial biotechnologies.

Epidemiological and functional characterization of Streptococcus pneumoniae pili

Streptococcus pneumoniae is an important life threatening human pathogen causing agent of invasive diseases such as otitis media, pneumonia, sepsis and meningitis, but is also a common inhabitant of the respiratory tract of children and healthy adults. Likewise most streptococci, S. pneumoniae decorates its surface with adhesive pili, composed of covalently linked subunits and involved in the attachment to epithelial cells and virulence. The pneumococcal pili are encoded by two genomic regions, pilus islet 1 (PI-1), and pilus islet-2 (PI-2), which are present in about 30% and 16% of the pneumococcal strains, respectively. PI-1 exists in three clonally related variants, whereas PI-2 is highly conserved. The presence of the islets does not correlate with the serotype of the strains, but with the genotype (as determined by Multi Locus Sequence Typing). The prevalence of PI-1 and PI-2 positive strains is similar in isolates from invasive disease and carriage. To better dissect a possible association between PIs presence and disease we evaluated the distribution of the two PIs in a panel of 113 acute otitis media (AOM) clinical isolates from Israel. PI-1 was present in 30.1% (N=34) of the isolates tested, and PI-2 in 7% (N=8). We found that 50% of the PI-1 positive isolates belonged to the international clones Spain9V-3 (ST156) and Taiwan19F-14 (ST236), and that PI-2 was not present in the absence of Pl-1. In conclusion, there was no correlation between PIs presence and AOM, and, in general, the observed differences in PIs prevalence are strictly dependent upon regional differences in the distribution of the clones. Finally, in the AOM collection the prevalence of PI-1 was higher among antibiotic resistant isolates, confirming previous indications obtained by the in silico analysis of the MLST database collection. Since the pilus-1 subunits were shown to confer protection in mouse models of infection both in active and passive immunization studies, and were regarded as potential candidates for a new generation of protein-based vaccines, the functional characterization was mainly focused on S. pneumoniae pilus -1 components. The pneumococcal pilus-1 is composed of three subunits, RrgA, RrgB and RrgC, each stabilized by intra-molecular isopeptide bonds and covalently polymerized by means of inter-molecular isopeptide bonds to form an extended fibre. The pilus shaft is a multimeric structure mainly composed by the RrgB backbone subunit. The minor ancillary proteins are located at the tip and at the base of the pilus, where they have been proposed to act as the major adhesin (RrgA) and as the pilus anchor (RrgC), respectively. RrgA is protective in in vivo mouse models, and exists in two variants (clades I and II). Mapping of the sequence variability onto the RrgA structure predicted from X-ray data showed that the diversity was restricted to the “head” of the protein, which contains the putative binding domains, whereas the elongated “stalk” was mostly conserved. To investigate whether this variability could influence the adhesive capacity of RrgA and to map the regions important for binding, two full-length protein variants and three recombinant RrgA portions were tested for adhesion to lung epithelial cells and to purified extracellular matrix (ECM) components. The two RrgA variants displayed similar binding abilities, whereas none of the recombinant fragments adhered at levels comparable to those of the full-length protein, suggesting that proper folding and structural arrangement are crucial to retain protein functionality. Furthermore, the two RrgA variants were shown to be cross-reactive in vitro and cross-protective in vivo in a murine model of passive immunization. Taken together, these data indicate that the region implicated in adhesion and the functional epitopes responsible for the protective ability of RrgA may be conserved and that the considerable level of variation found within the “head” domain of RrgA may have been generated by immunologic pressure without impairing the functional integrity of the pilus.

Sviluppo di un'applicazione bioinformatica per la gestione dei dati di antibiotico sensibilità di isolati clinici

Il problema dell'antibiotico-resistenza è un problema di sanità pubblica per affrontare il quale è necessario un sistema di sorveglianza basato sulla raccolta e l'analisi dei dati epidemiologici di laboratorio. Il progetto di dottorato è consistito nello sviluppo di una applicazione web per la gestione di tali dati di antibiotico sensibilità di isolati clinici utilizzabile a livello di ospedale. Si è creata una piattaforma web associata a un database relazionale per avere un’applicazione dinamica che potesse essere aggiornata facilmente inserendo nuovi dati senza dover manualmente modificare le pagine HTML che compongono l’applicazione stessa. E’ stato utilizzato il database open-source MySQL in quanto presenta numerosi vantaggi: estremamente stabile, elevate prestazioni, supportato da una grande comunità online ed inoltre gratuito. Il contenuto dinamico dell’applicazione web deve essere generato da un linguaggio di programmazione tipo “scripting” che automatizzi operazioni di inserimento, modifica, cancellazione, visualizzazione di larghe quantità di dati. E’ stato scelto il PHP, linguaggio open-source sviluppato appositamente per la realizzazione di pagine web dinamiche, perfettamente utilizzabile con il database MySQL. E’ stata definita l’architettura del database creando le tabelle contenenti i dati e le relazioni tra di esse: le anagrafiche, i dati relativi ai campioni, microrganismi isolati e agli antibiogrammi con le categorie interpretative relative al dato antibiotico. Definite tabelle e relazioni del database è stato scritto il codice associato alle funzioni principali: inserimento manuale di antibiogrammi, importazione di antibiogrammi multipli provenienti da file esportati da strumenti automatizzati, modifica/eliminazione degli antibiogrammi precedenti inseriti nel sistema, analisi dei dati presenti nel database con tendenze e andamenti relativi alla prevalenza di specie microbiche e alla chemioresistenza degli stessi, corredate da grafici. Lo sviluppo ha incluso continui test delle funzioni via via implementate usando reali dati clinici e sono stati introdotti appositi controlli e l’introduzione di una semplice e pulita veste grafica.

Valutazione di un sistema di recupero di immagini dermatologiche

Questa tesi si inserisce in un progetto di ricerca fra il gruppo di Matematica della Visione del Prof. Ferri e CA-MI S.r.l. volto a progettare un sistema di recupero di immagini mediante il quale un dermatologo potrà acquisire l’immagine di una lesione e recuperare da un database classificato le immagini più somiglianti. Il concetto stesso di “somiglianza” è formalmente realizzato da una parte dell’omologia persistente (funzioni di taglia). Questa tesi utilizza tali metodi al fine di ottenere una combinazione ottimale dei diversi classificatori che si ottengono utilizzando la modularità intrinseca nella teoria. A questo scopo vengono impiegati due modelli e diversi metodi numerici.

Cloud Computing Service for Managing Large Medical Image Data-Sets Using Balanced Collaborative Agents

Managing large medical image collections is an increasingly demanding important issue in many hospitals and other medical settings. A huge amount of this information is daily generated, which requires robust and agile systems. In this paper we present a distributed multi-agent system capable of managing very large medical image datasets. In this approach, agents extract low-level information from images and store them in a data structure implemented in a relational database. The data structure can also store semantic information related to images and particular regions. A distinctive aspect of our work is that a single image can be divided so that the resultant sub-images can be stored and managed separately by different agents to improve performance in data accessing and processing. The system also offers the possibility of applying some region-based operations and filters on images, facilitating image classification. These operations can be performed directly on data structures in the database.

Image analysis by moment invariants using a set of step-like basis functions

Moment invariants have been thoroughly studied and repeatedly proposed as one of the most powerful tools for 2D shape identification. In this paper a set of such descriptors is proposed, being the basis functions discontinuous in a finite number of points. The goal of using discontinuous functions is to avoid the Gibbs phenomenon, and therefore to yield a better approximation capability for discontinuous signals, as images. Moreover, the proposed set of moments allows the definition of rotation invariants, being this the other main design concern. Translation and scale invariance are achieved by means of standard image normalization. Tests are conducted to evaluate the behavior of these descriptors in noisy environments, where images are corrupted with Gaussian noise up to different SNR values. Results are compared to those obtained using Zernike moments, showing that the proposed descriptor has the same performance in image retrieval tasks in noisy environments, but demanding much less computational power for every stage in the query chain.