Efficient dimensionality reduction and one-class classification for content-based image retrieval

The thesis investigates various machine learning approaches to reducing data dimensionality, and studies the impact of asymmetric data on learning in image retrieval. Efficient algorithms are proposed to reduce the data dimensionality. Integration strategies for one-class classification are designed to address asymmetric data issue and improve retrieval effectiveness.

Series feature aggregation for content-based image retrieval

Feature aggregation is a critical technique in content-based image retrieval systems that employ multiple visual features to characterize image content. One problem in feature aggregation is that image similarity in different feature spaces can not be directly comparable with each other. To address this problem, a new feature aggregation approach, series feature aggregation (SFA), is proposed in this paper. In contrast to merging incomparable feature distances in different feature spaces to get aggregated image similarity in the conventional feature aggregation approach, the series feature aggregation directly deal with images in each feature space to avoid comparing different feature distances. SFA is effectively filtering out irrelevant images using individual features in each stage and the remaining images are images that collectively described by all features. Experiments, conducted with IAPR TC-12 benchmark image collection (ImageCLEF2006) that contains over 20,000 photographic images and defined queries, have shown that SFA can outperform the parallel feature aggregation and linear distance combination schemes. Furthermore, SFA is able to retrieve more relevant images in top ranked outputs that brings better user experience in finding more relevant images quickly.

An unified framework based on p-norm for feature aggregation in content- based image retrieval

Feature aggregation is a critical technique in content- based image retrieval systems that employ multiple visual features to characterize image content. In this paper, the p-norm is introduced to feature aggregation that provides a framework to unify various previous feature aggregation schemes such as linear combination, Euclidean distance, Boolean logic and decision fusion schemes in which previous schemes are instances. Some insights of the mechanism of how various aggregation schemes work are discussed through the effects of model parameters in the unified framework. Experiments show that performances vary over feature aggregation schemes that necessitates an unified framework in order to optimize the retrieval performance according to individual queries and user query concept. Revealing experimental results conducted with IAPR TC-12 ImageCLEF2006 benchmark collection that contains over 20,000 photographic images are presented and discussed.

Watermarking protocol for protecting user's right in content based image retrieval

Content based image retrieval (CBIR) is a technique to search for images relevant to the user’s query from an image collection.In last decade, most attention has been paid to improve the retrieval performance. However, there is no significant effort to investigate the security concerning in CBIR. Under the query by example (QBE) paradigm, the user supplies an image as a query and the system returns a set of retrieved results. If the query image includes user’s private information, an untrusted server provider of CBIR may distribute it illegally, which leads to the user’s right problem. In this paper, we propose an interactive watermarking protocol to address this problem. A watermark is inserted into the query image by the user in encrypted domain without knowing the exact content. The server provider of CBIR will get the watermarked query image and uses it to perform image retrieval. In case where the user finds an unauthorized copy, a watermark in the unauthorized copy will be used as evidence to prove that the user’s legal right is infringed by the server provider.

Ranking method for optimizing precision/recall of content-based image retrieval

The ranking method is a key element of Content-based Image Retrieval (CBIR) system, which can affect the final retrieval performance. In the literature, previous ranking methods based on either distance or probability do not explicitly relate to precision and recall, which are normally used to evaluate the performance of CBIR systems. In this paper, a novel ranking method based on relative density is proposed to improve the probability based approach by ranking images in the class. The proposed method can achieve optimal precision and recall. The experiments conducted on a large photographic collection show significant improvements of retrieval performance.

Content based image retrieval using unclean positive examples

Conventional content-based image retrieval (CBIR) schemes employing relevance feedback may suffer from some problems in the practical applications. First, most ordinary users would like to complete their search in a single interaction especially on the web. Second, it is time consuming and difficult to label a lot of negative examples with sufficient variety. Third, ordinary users may introduce some noisy examples into the query. This correspondence explores solutions to a new issue that image retrieval using unclean positive examples. In the proposed scheme, multiple feature distances are combined to obtain image similarity using classification technology. To handle the noisy positive examples, a new two-step strategy is proposed by incorporating the methods of data cleaning and noise tolerant classifier. The extensive experiments carried out on two different real image collections validate the effectiveness of the proposed scheme.

A fuzzy clustering approach to content-based image retrieval

In this study, an interactive Content-Based Image Retrieval (CBIR) system that allows searching and retrieving images from databases is designed and developed. Based on the fuzzy c-means clustering algorithm, the CBIR system fuses color and texture features in image segmentation. A technique to form compound queries based on the combined features of different images is devised. This technique allows users to have a better control on the search criteria, thus a higher retrieval performance can be achieved. A database consisting of skin cancer imagery is used to demonstrate the applicability of the CBIR system.

A scalable re-ranking method for content-based image retrieval

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

A scalable approach for content based image retrieval in cloud datacenter

The emergence of cloud datacenters enhances the capability of online data storage. Since massive data is stored in datacenters, it is necessary to effectively locate and access interest data in such a distributed system. However, traditional search techniques only allow users to search images over exact-match keywords through a centralized index. These techniques cannot satisfy the requirements of content based image retrieval (CBIR). In this paper, we propose a scalable image retrieval framework which can efficiently support content similarity search and semantic search in the distributed environment. Its key idea is to integrate image feature vectors into distributed hash tables (DHTs) by exploiting the property of locality sensitive hashing (LSH). Thus, images with similar content are most likely gathered into the same node without the knowledge of any global information. For searching semantically close images, the relevance feedback is adopted in our system to overcome the gap between low-level features and high-level features. We show that our approach yields high recall rate with good load balance and only requires a few number of hops.

An information foraging theory based user study of an adaptive user interaction framework for content-based image retrieval

This paper presents the design and results of a task-based user study, based on Information Foraging Theory, on a novel user interaction framework - uInteract - for content-based image retrieval (CBIR). The framework includes a four-factor user interaction model and an interactive interface. The user study involves three focused evaluations, 12 simulated real life search tasks with different complexity levels, 12 comparative systems and 50 subjects. Information Foraging Theory is applied to the user study design and the quantitative data analysis. The systematic findings have not only shown how effective and easy to use the uInteract framework is, but also illustrate the value of Information Foraging Theory for interpreting user interaction with CBIR. © 2011 Springer-Verlag Berlin Heidelberg.

Applying Information Foraging Theory to understand user interaction with content-based image retrieval

The paper proposes an ISE (Information goal, Search strategy, Evaluation threshold) user classification model based on Information Foraging Theory for understanding user interaction with content-based image retrieval (CBIR). The proposed model is verified by a multiple linear regression analysis based on 50 users' interaction features collected from a task-based user study of interactive CBIR systems. To our best knowledge, this is the first principled user classification model in CBIR verified by a formal and systematic qualitative analysis of extensive user interaction data. Copyright 2010 ACM.

A four-factor user interaction model for content-based image retrieval

In order to bridge the “Semantic gap”, a number of relevance feedback (RF) mechanisms have been applied to content-based image retrieval (CBIR). However current RF techniques in most existing CBIR systems still lack satisfactory user interaction although some work has been done to improve the interaction as well as the search accuracy. In this paper, we propose a four-factor user interaction model and investigate its effects on CBIR by an empirical evaluation. Whilst the model was developed for our research purposes, we believe the model could be adapted to any content-based search system.

Enabling effective user interactions in content-based image retrieval

This paper presents an interactive content-based image retrieval framework—uInteract, for delivering a novel four-factor user interaction model visually. The four-factor user interaction model is an interactive relevance feedback mechanism that we proposed, aiming to improve the interaction between users and the CBIR system and in turn users overall search experience. In this paper, we present how the framework is developed to deliver the four-factor user interaction model, and how the visual interface is designed to support user interaction activities. From our preliminary user evaluation result on the ease of use and usefulness of the proposed framework, we have learnt what the users like about the framework and the aspects we could improve in future studies. Whilst the framework is developed for our research purposes, we believe the functionalities could be adapted to any content-based image search framework.

Comparing dissimilarity measures for content-based image retrieval

Dissimilarity measurement plays a crucial role in content-based image retrieval, where data objects and queries are represented as vectors in high-dimensional content feature spaces. Given the large number of dissimilarity measures that exist in many fields, a crucial research question arises: Is there a dependency, if yes, what is the dependency, of a dissimilarity measure’s retrieval performance, on different feature spaces? In this paper, we summarize fourteen core dissimilarity measures and classify them into three categories. A systematic performance comparison is carried out to test the effectiveness of these dissimilarity measures with six different feature spaces and some of their combinations on the Corel image collection. From our experimental results, we have drawn a number of observations and insights on dissimilarity measurement in content-based image retrieval, which will lay a foundation for developing more effective image search technologies.