Multimedia search capabilities of Chinese language search engines

This paper reports results from a study exploring the multimedia search functionality of Chinese language search engines. Web searching in Chinese (Mandarin) is a growing research area and a technical challenge for popular commercial Web search engines. Few studies have been conducted on Chinese language search engines. We investigate two research questions: which Chinese language search engines provide multimedia searching, and what multimedia search functionalities are available in Chinese language Web search engines. Specifically, we examine each Web search engine's (1) features permitting Chinese language multimedia searches, (2) extent of search personalization and user control of multimedia search variables, and (3) the relationships between Web search engines and their features in the Chinese context. Key findings show that Chinese language Web search engines offer limited multimedia search functionality, and general search engines provide a wider range of features than specialized multimedia search engines. Study results have implications for Chinese Web users, Website designers and Web search engine developers. © 2009 Elsevier Ltd. All rights reserved.

Analyzing web multimedia query reformulation behavior

Current multimedia Web search engines still use keywords as the primary means to search. Due to the richness in multimedia contents, general users constantly experience some difficulties in formulating textual queries that are representative enough for their needs. As a result, query reformulation becomes part of an inevitable process in most multimedia searches. Previous Web query formulation studies did not investigate the modification sequences and thus can only report limited findings on the reformulation behavior. In this study, we propose an automatic approach to examine multimedia query reformulation using large-scale transaction logs. The key findings show that search term replacement is the most dominant type of modifications in visual searches but less important in audio searches. Image search users prefer the specified search strategy more than video and audio users. There is also a clear tendency to replace terms with synonyms or associated terms in visual queries. The analysis of the search strategies in different types of multimedia searching provides some insights into user’s searching behavior, which can contribute to the design of future query formulation assistance for keyword-based Web multimedia retrieval systems.

A study and comparison of multimedia Web searching : 1997-2006

Searching for multimedia is an important activity for users of Web search engines. Studying user's interactions with Web search engine multimedia buttons, including image, audio, and video, is important for the development of multimedia Web search systems. This article provides results from a Weblog analysis study of multimedia Web searching by Dogpile users in 2006. The study analyzes the (a) duration, size, and structure of Web search queries and sessions; (b) user demographics; (c) most popular multimedia Web searching terms; and (d) use of advanced Web search techniques including Boolean and natural language. The current study findings are compared with results from previous multimedia Web searching studies. The key findings are: (a) Since 1997, image search consistently is the dominant media type searched followed by audio and video; (b) multimedia search duration is still short (>50% of searching episodes are <1 min), using few search terms; (c) many multimedia searches are for information about people, especially in audio search; and (d) multimedia search has begun to shift from entertainment to other categories such as medical, sports, and technology (based on the most repeated terms). Implications for design of Web multimedia search engines are discussed.

Review of Apple iPad news applications : new media service

The launch of the Apple iPad on January 2010 has seen considerable interest from the newspaper and publishing industry in developing content and business models for the tablet PC device that can address the limits of both the print and online news and information media products. It is early days in the iPad’s evolution, and we wait to see what competitor devices will emerge in the near future. It is apparent, however, that it has become a significant “niche” product, with considerable potential for mass market expansion over the next few years, possibly at the expense of netbook sales. The scope for the iPad and tablet PCs to become a “fourth screen” for users, alongside the TV, PC and mobile phone, is in early stages of evolution. The study used five criteria to assess iPad apps: • Content: timeliness; archive; personalisation; content depth; advertisements; the use of multimedia; and the extent to which the content was in sync with the provider brand. • Useability: degree of static content; ability to control multimedia; file size; page clutter; resolution; signposts; and customisation. • Interactivity: hyperlinks; ability to contribute content or provide feedback to news items; depth of multimedia; search function; ability to use plug-ins and linking; ability to highlight, rate and/or save items; functions that may facilitate a community of users. • Transactions capabilities: ecommerce functionality; purchase and download process; user privacy and transaction security. • Openness: degree of linking to outside sources; reader contribution processes; anonymity measures; and application code ownership.

TEXT-MESS: Intelligent, Interactive and Multilingual Text Mining based on Human Language Technologies, TIN2006-15265-C06

The goal of the project is to analyze, experiment, and develop intelligent, interactive and multilingual Text Mining technologies, as a key element of the next generation of search engines, systems with the capacity to find "the need behind the query". This new generation will provide specialized services and interfaces according to the search domain and type of information needed. Moreover, it will integrate textual search (websites) and multimedia search (images, audio, video), it will be able to find and organize information, rather than generating ranked lists of websites.

A generalized multidimensional index structure for multimedia data to support content-based similarity searches in a collaborative search environment

Since multimedia data, such as images and videos, are way more expressive and informative than ordinary text-based data, people find it more attractive to communicate and express with them. Additionally, with the rising popularity of social networking tools such as Facebook and Twitter, multimedia information retrieval can no longer be considered a solitary task. Rather, people constantly collaborate with one another while searching and retrieving information. But the very cause of the popularity of multimedia data, the huge and different types of information a single data object can carry, makes their management a challenging task. Multimedia data is commonly represented as multidimensional feature vectors and carry high-level semantic information. These two characteristics make them very different from traditional alpha-numeric data. Thus, to try to manage them with frameworks and rationales designed for primitive alpha-numeric data, will be inefficient. An index structure is the backbone of any database management system. It has been seen that index structures present in existing relational database management frameworks cannot handle multimedia data effectively. Thus, in this dissertation, a generalized multidimensional index structure is proposed which accommodates the atypical multidimensional representation and the semantic information carried by different multimedia data seamlessly from within one single framework. Additionally, the dissertation investigates the evolving relationships among multimedia data in a collaborative environment and how such information can help to customize the design of the proposed index structure, when it is used to manage multimedia data in a shared environment. Extensive experiments were conducted to present the usability and better performance of the proposed framework over current state-of-art approaches.

A semantic middleware to enhance current multimedia retrieval systems with content-based functionalities

210 p. : graf.

Video Search: New Challenges in the Pervasive Digital Video Era

The explosion of multimedia digital content and the development of technologies that go beyond traditional broadcast and TV have rendered access to such content important for all end-users of these technologies. While originally developed for providing access to multimedia digital libraries, video search technologies assume now a more demanding role. In this paper, we attempt to shed light onto this new role of video search technologies, looking at the rapid developments in the related market, the lessons learned from state of art video search prototypes developed mainly in the digital libraries context and the new technological challenges that have risen. We focus on one of the latter, i.e., the development of cross-media decision mechanisms, drawing examples from REVEAL THIS, an FP6 project on the retrieval of video and language for the home user. We argue, that efficient video search holds a key to the usability of the new ”pervasive digital video” technologies and that it should involve cross-media decision mechanisms.

Multipedia: Enriching DBpedia with Multimedia Information

Enriching knowledge bases with multimedia information makes it possible to complement textual descriptions with visual and audio information. Such complementary information can help users to understand the meaning of assertions, and in general improve the user experience with the knowledge base. In this paper we address the problem of how to enrich ontology instances with candidate images retrieved from existing Web search engines. DBpedia has evolved into a major hub in the Linked Data cloud, interconnecting millions of entities organized under a consistent ontology. Our approach taps into the Wikipedia corpus to gather context information for DBpedia instances and takes advantage of image tagging information when this is available to calculate semantic relatedness between instances and candidate images. We performed experiments with focus on the particularly challenging problem of highly ambiguous names. Both methods presented in this work outperformed the baseline. Our best method leveraged context words from Wikipedia, tags from Flickr and type information from DBpedia to achieve an average precision of 80%.

Desarrollo de una metodología de vigilancia tecnológica para la toma de decisiones : aplicación a las tecnologías multimedia

La aplicación de la vigilancia tecnología se ha mostrado como una de las herramientas más importantes para ganar competitividad y mejorar las actividades de innovación de las empresas. La vigilancia se basa en captar las informaciones, normalmente patentes y publicaciones científicas, más relevantes para un determinado campo tecnológico y valorarlas para influir en la toma de decisiones. Las fases clásicas son: búsqueda, análisis y comunicación de la información. El trabajo se soportará tanto en herramientas comerciales como en otras que deberá desarrollar el estudiante, y tendrá como objetivo fundamental el desarrollar una metodología, basada en la vigilancia, para tomar decisiones sobre tecnologías y en este caso aplicadas a las tecnologías multimedia. El objetivo principal en la propuesta de una metodología genérica de vigilancia tecnológica (VT/IC) para la toma de decisiones con un ejemplo de aplicación en las tecnologías multimedia y que más adelante se explicitó en TV 3D. La necesidad de que el proceso de VT/IC se soporte en una metodología es imprescindible si queremos darle la importancia que debe tener en el ciclo productivo de cualquier tipo de organización y muy especialmente en una organización involucrada en investigación y desarrollo (I+D+i). Esta metodología posibilitará, entre otras cosas, que estos procesos que conforman la VT/IC puedan integrarse en una organización compartiendo los procesos productivos, de administración y de dirección de la organización. Permitirá una medición de su funcionamiento y las posibles modificaciones para obtener un mejor funcionamiento. Proveerá a los posibles elementos involucrados en la VT/IC de la documentación, procesos, herramientas, elementos de medición de un sistema definido, publicado, medido y analizado de trabajo. Finalmente a modo de ejemplo de un proceso de consulta VT/IC utilizaremos el criterio de búsqueda genérico 3D TV propuesto. Estructura del PFC: Para lograr estos objetivos el trabajo ha sido dividido en 6 etapas: 1.- Descripción del PFC: Una presentación del PFC y su desarrollo. 2.- Vigilancia tecnológica: Desarrollo del concepto de VT/IC, los efectos esperados, beneficios y riesgos de la VT/IC, concepto de inteligencia competitiva (IC), concepto aplicado de vigilancia tecnológica e inteligencia competitiva (VT/IC). 3.- Técnicas de análisis empresarial donde la VT/IC es útil: para empezar a entender como debe ser la VT/IC analizamos como una organización utiliza las distintas técnicas de análisis y que información aportan cada una de ellas, finalmente analizamos como la VT/IC ayuda a esas técnicas de análisis. 4.- Gestión de las fuentes de información: análisis de los tipos de fuentes de información y sus herramientas de búsqueda asociadas. 5.- Metodología propuesta de la VT/IC: desarrollo de la metodología de implementación y de funcionamiento de una unidad de VT/IC. 6.- Observatorio: a modo de ejemplo, “3d TV”. ABSTRACT. The application of surveillance technology has proven to be one of the most important to increase competitiveness and improve the innovation activities of enterprises tools. Surveillance is based on capturing the information, usually patents and scientific publications most relevant to a given technological field and assess them to influence decision making. The classical phases are: search, analysis and communication of information. The work will support both commercial and other tools to be developed by the student, and will have as main objective to develop a methodology, based on monitoring to make decisions about technologies and in this case applied to multimedia technologies. The main objective in the proposed generic methodology for technological awareness (VT / IC) for decision making with an example application in multimedia technologies and later made explicit 3D TV. The need for the process of VT / CI support methodology is essential if we give it the importance it should have in the production cycle of any organization and especially in an organization involved in research and development (R + D + i). This methodology will allow, among other things, that these processes that make up the VT / IC can be integrated into an organization sharing production processes, management and direction of the organization. It will allow a measurement of its performance and possible changes for better performance. It will provide the possible elements involved in the VT / IC documentation, processes, tools, measuring elements of a defined system, published, measured and analyzed work. Finally an example of a consultation process VT / IC use generic search criteria proposed 3D TV. Structure of the PFC: To achieve these objectives the work has been divided into 6 stages: 1. PFC Description: A presentation of the PFC and its development. 2. Technology Watch: Concept Development of VT / IC, expected effects, benefits and risks of VT / IC concept of competitive intelligence (CI) concept applied technology watch and competitive intelligence (VT / IC). 3. Business analysis techniques where VT / IC is useful: to begin to understand how it should be the VT / IC analyze how an organization uses different analysis techniques and information provide each finally analyze how the VT / IC helps these analysis techniques. 4. Management information sources: analysis of the types of information sources and their associated search tools. 5. proposed methodology VT / IC: methodology development and operational deployment of a unit of VT / IC. 6. Observatory: by way of example, "3D TV".

SaveRF: Towards efficient relevance feedback search

In multimedia retrieval, a query is typically interactively refined towards the ‘optimal’ answers by exploiting user feedback. However, in existing work, in each iteration, the refined query is re-evaluated. This is not only inefficient but fails to exploit the answers that may be common between iterations. In this paper, we introduce a new approach called SaveRF (Save random accesses in Relevance Feedback) for iterative relevance feedback search. SaveRF predicts the potential candidates for the next iteration and maintains this small set for efficient sequential scan. By doing so, repeated candidate accesses can be saved, hence reducing the number of random accesses. In addition, efficient scan on the overlap before the search starts also tightens the search space with smaller pruning radius. We implemented SaveRF and our experimental study on real life data sets show that it can reduce the I/O cost significantly.

Communicating Cuneiform:The Evolution of a Multimedia Cuneiform Database

Our paper presents the work of the Cuneiform Digital Forensic Project (CDFP), an interdisciplinary project at The University of Birmingham, concerned with the development of a multimedia database to support scholarly research into cuneiform, wedge-shaped writing imprinted onto clay tablets and indeed the earliest real form of writing. We describe the evolutionary design process and dynamic research and developmental cycles associated with the database. Unlike traditional publications, the electronic publication of resources offers the possibility of almost continuous revisions with the integration and support of new media and interfaces. However, if on-line resources are to win the favor and confidence of their respective communities there must be a clear distinction between published and maintainable resources, and, developmental content. Published material should, ideally, be supported via standard web-browser interfaces with fully integrated tools so that users receive a reliable, homogenous and intuitive flow of information and media relevant to their needs. We discuss the inherent dynamics of the design and publication of our on-line resource, starting with the basic design and maintenance aspects of the electronic database, which includes photographic instances of cuneiform signs, and shows how the continuous review process identifies areas for further research and development, for example, the “sign processor” graphical search tool and three-dimensional content, the results of which then feedback into the maintained resource.

A multimedia indexing and retrieval framework for multimedia database systems

The main challenges of multimedia data retrieval lie in the effective mapping between low-level features and high-level concepts, and in the individual users' subjective perceptions of multimedia content. ^ The objectives of this dissertation are to develop an integrated multimedia indexing and retrieval framework with the aim to bridge the gap between semantic concepts and low-level features. To achieve this goal, a set of core techniques have been developed, including image segmentation, content-based image retrieval, object tracking, video indexing, and video event detection. These core techniques are integrated in a systematic way to enable the semantic search for images/videos, and can be tailored to solve the problems in other multimedia related domains. In image retrieval, two new methods of bridging the semantic gap are proposed: (1) for general content-based image retrieval, a stochastic mechanism is utilized to enable the long-term learning of high-level concepts from a set of training data, such as user access frequencies and access patterns of images. (2) In addition to whole-image retrieval, a novel multiple instance learning framework is proposed for object-based image retrieval, by which a user is allowed to more effectively search for images that contain multiple objects of interest. An enhanced image segmentation algorithm is developed to extract the object information from images. This segmentation algorithm is further used in video indexing and retrieval, by which a robust video shot/scene segmentation method is developed based on low-level visual feature comparison, object tracking, and audio analysis. Based on shot boundaries, a novel data mining framework is further proposed to detect events in soccer videos, while fully utilizing the multi-modality features and object information obtained through video shot/scene detection. ^ Another contribution of this dissertation is the potential of the above techniques to be tailored and applied to other multimedia applications. This is demonstrated by their utilization in traffic video surveillance applications. The enhanced image segmentation algorithm, coupled with an adaptive background learning algorithm, improves the performance of vehicle identification. A sophisticated object tracking algorithm is proposed to track individual vehicles, while the spatial and temporal relationships of vehicle objects are modeled by an abstract semantic model. ^

A scalable multimedia content processing framework with application to TV shopping

The advent of smart TVs has reshaped the TV-consumer interaction by combining TVs with mobile-like applications and access to the Internet. However, consumers are still unable to seamlessly interact with the contents being streamed. An example of such limitation is TV shopping, in which a consumer makes a purchase of a product or item displayed in the current TV show. Currently, consumers can only stop the current show and attempt to find a similar item in the Web or an actual store. It would be more convenient if the consumer could interact with the TV to purchase interesting items. ^ Towards the realization of TV shopping, this dissertation proposes a scalable multimedia content processing framework. Two main challenges in TV shopping are addressed: the efficient detection of products in the content stream, and the retrieval of similar products given a consumer-selected product. The proposed framework consists of three components. The first component performs computational and temporal aware multimedia abstraction to select a reduced number of frames that summarize the important information in the video stream. By both reducing the number of frames and taking into account the computational cost of the subsequent detection phase, this component component allows the efficient detection of products in the stream. The second component realizes the detection phase. It executes scalable product detection using multi-cue optimization. Additional information cues are formulated into an optimization problem that allows the detection of complex products, i.e., those that do not have a rigid form and can appear in various poses. After the second component identifies products in the video stream, the consumer can select an interesting one for which similar ones must be located in a product database. To this end, the third component of the framework consists of an efficient, multi-dimensional, tree-based indexing method for multimedia databases. The proposed index mechanism serves as the backbone of the search. Moreover, it is able to efficiently bridge the semantic gap and perception subjectivity issues during the retrieval process to provide more relevant results.^