A multimedia indexing and retrieval framework for multimedia database systems

The main challenges of multimedia data retrieval lie in the effective mapping between low-level features and high-level concepts, and in the individual users' subjective perceptions of multimedia content. ^ The objectives of this dissertation are to develop an integrated multimedia indexing and retrieval framework with the aim to bridge the gap between semantic concepts and low-level features. To achieve this goal, a set of core techniques have been developed, including image segmentation, content-based image retrieval, object tracking, video indexing, and video event detection. These core techniques are integrated in a systematic way to enable the semantic search for images/videos, and can be tailored to solve the problems in other multimedia related domains. In image retrieval, two new methods of bridging the semantic gap are proposed: (1) for general content-based image retrieval, a stochastic mechanism is utilized to enable the long-term learning of high-level concepts from a set of training data, such as user access frequencies and access patterns of images. (2) In addition to whole-image retrieval, a novel multiple instance learning framework is proposed for object-based image retrieval, by which a user is allowed to more effectively search for images that contain multiple objects of interest. An enhanced image segmentation algorithm is developed to extract the object information from images. This segmentation algorithm is further used in video indexing and retrieval, by which a robust video shot/scene segmentation method is developed based on low-level visual feature comparison, object tracking, and audio analysis. Based on shot boundaries, a novel data mining framework is further proposed to detect events in soccer videos, while fully utilizing the multi-modality features and object information obtained through video shot/scene detection. ^ Another contribution of this dissertation is the potential of the above techniques to be tailored and applied to other multimedia applications. This is demonstrated by their utilization in traffic video surveillance applications. The enhanced image segmentation algorithm, coupled with an adaptive background learning algorithm, improves the performance of vehicle identification. A sophisticated object tracking algorithm is proposed to track individual vehicles, while the spatial and temporal relationships of vehicle objects are modeled by an abstract semantic model. ^

The role of teleost grazers in a relatively pristine seagrass ecosystem

Trophic downgrading of ecosystems necessitates a functional understanding of trophic cascades. Identifying the presence of cascades, and the mechanisms through which they occur, is particularly important for seagrass meadows, which are among the most threatened ecosystems on Earth. Shark Bay, Western Australia provides a model system to investigate the potential importance of top-down effects in a relatively pristine seagrass ecosystem. The role of megagrazers in the Shark Bay system has been previously investigated, but the role of macrograzers (i.e., teleosts), and their importance relative to megagrazers, remains unknown. The objective of my dissertation was to elucidate the importance of teleost macrograzers in transmitting top-down effects in seagrass ecosystems. Seagrasses and macroalgae were the main food of the abundant teleost Pelates octolineatus, but stable isotopic values suggested that algae may contribute a larger portion of assimilated food than suggested by gut contents. Pelates octolineatus is at risk from numerous predators, with pied cormorants (Phalacrocorax varius) taking the majority of tethered P. octolineatus. Using a combination of fish trapping and unbaited underwater video surveillance, I found that the relative abundance of P. octolineatus was greater in interior areas of seagrass banks during the cold season, and that the mean length of P. octolineatus was greater in these areas compared to along edges of banks. Finally, I used seagrass transplants and exclosure experiments to determine the relative effect of megagrazers and macrograzers on the establishment and persistence of three species of seagrasses in interior microhabitats. Teleost grazing had the largest impact on seagrass species with the highest nutrient content, and these impacts were primarily observed during the warm season. My findings are consistent with predictions of a behaviorally-mediated trophic cascade initiated by tiger sharks (Galeocerdo cuvier) and transmitted through herbivorous fishes and their predators.

The Role of Teleost Grazers in a Relatively Pristine Seagrass Ecosystem

Trophic downgrading of ecosystems necessitates a functional understanding of trophic cascades. Identifying the presence of cascades, and the mechanisms through which they occur, is particularly important for seagrass meadows, which are among the most threatened ecosystems on Earth. Shark Bay, Western Australia provides a model system to investigate the potential importance of top-down effects in a relatively pristine seagrass ecosystem. The role of megagrazers in the Shark Bay system has been previously investigated, but the role of macrograzers (i.e., teleosts), and their importance relative to megagrazers, remains unknown. The objective of my dissertation was to elucidate the importance of teleost macrograzers in transmitting top-down effects in seagrass ecosystems. Seagrasses and macroalgae were the main food of the abundant teleost Pelates octolineatus, but stable isotopic values suggested that algae may contribute a larger portion of assimilated food than suggested by gut contents. Pelates octolineatus is at risk from numerous predators, with pied cormorants (Phalacrocorax varius) taking the majority of tethered P. octolineatus. Using a combination of fish trapping and unbaited underwater video surveillance, I found that the relative abundance of P. octolineatus was greater in interior areas of seagrass banks during the cold season, and that the mean length of P. octolineatus was greater in these areas compared to along edges of banks. Finally, I used seagrass transplants and exclosure experiments to determine the relative effect of megagrazers and macrograzers on the establishment and persistence of three species of seagrasses in interior microhabitats. Teleost grazing had the largest impact on seagrass species with the highest nutrient content, and these impacts were primarily observed during the warm season. My findings are consistent with predictions of a behaviorally-mediated trophic cascade initiated by tiger sharks (Galeocerdo cuvier) and transmitted through herbivorous fishes and their predators.

Digital surveillance based on video CODEC System-on-a-Chip (SoC) platforms

Today, most conventional surveillance networks are based on analog system, which has a lot of constraints like manpower and high-bandwidth requirements. It becomes the barrier for today's surveillance network development. This dissertation describes a digital surveillance network architecture based on the H.264 coding/decoding (CODEC) System-on-a-Chip (SoC) platform. The proposed digital surveillance network architecture includes three major layers: software layer, hardware layer, and the network layer. The following outlines the contributions to the proposed digital surveillance network architecture. (1) We implement an object recognition system and an object categorization system on the software layer by applying several Digital Image Processing (DIP) algorithms. (2) For better compression ratio and higher video quality transfer, we implement two new modules on the hardware layer of the H.264 CODEC core, i.e., the background elimination module and the Directional Discrete Cosine Transform (DDCT) module. (3) Furthermore, we introduce a Digital Signal Processor (DSP) sub-system on the main bus of H.264 SoC platforms as the major hardware support system for our software architecture. Thus we combine the software and hardware platforms to be an intelligent surveillance node. Lab results show that the proposed surveillance node can dramatically save the network resources like bandwidth and storage capacity.