Exploring Hidden Coherent Feature Groups and Temporal Semantics for Multimedia Big Data Analysis

Thanks to the advanced technologies and social networks that allow the data to be widely shared among the Internet, there is an explosion of pervasive multimedia data, generating high demands of multimedia services and applications in various areas for people to easily access and manage multimedia data. Towards such demands, multimedia big data analysis has become an emerging hot topic in both industry and academia, which ranges from basic infrastructure, management, search, and mining to security, privacy, and applications. Within the scope of this dissertation, a multimedia big data analysis framework is proposed for semantic information management and retrieval with a focus on rare event detection in videos. The proposed framework is able to explore hidden semantic feature groups in multimedia data and incorporate temporal semantics, especially for video event detection. First, a hierarchical semantic data representation is presented to alleviate the semantic gap issue, and the Hidden Coherent Feature Group (HCFG) analysis method is proposed to capture the correlation between features and separate the original feature set into semantic groups, seamlessly integrating multimedia data in multiple modalities. Next, an Importance Factor based Temporal Multiple Correspondence Analysis (i.e., IF-TMCA) approach is presented for effective event detection. Specifically, the HCFG algorithm is integrated with the Hierarchical Information Gain Analysis (HIGA) method to generate the Importance Factor (IF) for producing the initial detection results. Then, the TMCA algorithm is proposed to efficiently incorporate temporal semantics for re-ranking and improving the final performance. At last, a sampling-based ensemble learning mechanism is applied to further accommodate the imbalanced datasets. In addition to the multimedia semantic representation and class imbalance problems, lack of organization is another critical issue for multimedia big data analysis. In this framework, an affinity propagation-based summarization method is also proposed to transform the unorganized data into a better structure with clean and well-organized information. The whole framework has been thoroughly evaluated across multiple domains, such as soccer goal event detection and disaster information management.

The effect of multimedia instruction on achievement of college history students

The purpose of this study was to investigate the effect of multimedia instruction on achievement of college students in AMR 2010 from exploration and discovery to 1865. A non-equivalent control group design was used. The dependent variable was achievement. The independent variables were learning styles, method of instruction, and visual clarifiers (notes). The study was conducted using two history sections from Palm Beach Community College, in Boca Raton, Florida, between August and December, 1998. Data were obtained by means of placement scores, posttests, the Productivity Environmental Preference Survey (PEPS), and a researcher-developed student survey. Statistical analysis of the data was done using SPSS statistical software. Demographic variables were compared using Chi square. T tests were run on the posttests to determine the equality of variances. The posttest scores of the groups were compared using the analysis of covariance (ANCOVA) at the .05 level of significance. The first hypothesis there is a significant difference in students' learning of U.S. History when students receive multimedia instruction was supported, F (1, 52) = 16.88, p < .0005, and F = (1, 53) = 8.52, p < .005 for Tests 2 and 3, respectively. The second hypothesis there is a significant difference on the effectiveness of multimedia instruction based on students' various learning preferences was not supported. The last hypotheses there is a significant difference on students' learning of U.S. History when students whose first language is other than English and students who need remediation receive visual clarifiers were not supported. Analysis of covariance (ANCOVA) indicated no difference between the groups on Test 1, Test 2, or Test 3: F (1, 45) = .01, p < .940, F (1, 52) = .77, p < .385, and F (1, 53) =.17, p < .678, respectively, for language. Analysis of covariance (ANCOVA) indicated no significant difference on Test 1, Test 2, or Test 3, between the groups on the variable remediation: F (1, 45) = .31, p < .580, F (1, 52) = 1.44, p < .236, and F (1, 53) = .21, p < .645, respectively. ^

A multimedia indexing and retrieval framework for multimedia database systems

The main challenges of multimedia data retrieval lie in the effective mapping between low-level features and high-level concepts, and in the individual users' subjective perceptions of multimedia content. ^ The objectives of this dissertation are to develop an integrated multimedia indexing and retrieval framework with the aim to bridge the gap between semantic concepts and low-level features. To achieve this goal, a set of core techniques have been developed, including image segmentation, content-based image retrieval, object tracking, video indexing, and video event detection. These core techniques are integrated in a systematic way to enable the semantic search for images/videos, and can be tailored to solve the problems in other multimedia related domains. In image retrieval, two new methods of bridging the semantic gap are proposed: (1) for general content-based image retrieval, a stochastic mechanism is utilized to enable the long-term learning of high-level concepts from a set of training data, such as user access frequencies and access patterns of images. (2) In addition to whole-image retrieval, a novel multiple instance learning framework is proposed for object-based image retrieval, by which a user is allowed to more effectively search for images that contain multiple objects of interest. An enhanced image segmentation algorithm is developed to extract the object information from images. This segmentation algorithm is further used in video indexing and retrieval, by which a robust video shot/scene segmentation method is developed based on low-level visual feature comparison, object tracking, and audio analysis. Based on shot boundaries, a novel data mining framework is further proposed to detect events in soccer videos, while fully utilizing the multi-modality features and object information obtained through video shot/scene detection. ^ Another contribution of this dissertation is the potential of the above techniques to be tailored and applied to other multimedia applications. This is demonstrated by their utilization in traffic video surveillance applications. The enhanced image segmentation algorithm, coupled with an adaptive background learning algorithm, improves the performance of vehicle identification. A sophisticated object tracking algorithm is proposed to track individual vehicles, while the spatial and temporal relationships of vehicle objects are modeled by an abstract semantic model. ^

An approach for design and analysis of User-centric Communication Middleware

Today, the development of domain-specific communication applications is both time-consuming and error-prone because the low-level communication services provided by the existing systems and networks are primitive and often heterogeneous. Multimedia communication applications are typically built on top of low-level network abstractions such as TCP/UDP socket, SIP (Session Initiation Protocol) and RTP (Real-time Transport Protocol) APIs. The User-centric Communication Middleware (UCM) is proposed to encapsulate the networking complexity and heterogeneity of basic multimedia and multi-party communication for upper-layer communication applications. And UCM provides a unified user-centric communication service to diverse communication applications ranging from a simple phone call and video conferencing to specialized communication applications like disaster management and telemedicine. It makes it easier to the development of domain-specific communication applications. The UCM abstraction and API is proposed to achieve these goals. The dissertation also tries to integrate the formal method into UCM development process. The formal model is created for UCM using SAM methodology. Some design errors are found during model creation because the formal method forces to give the precise description of UCM. By using the SAM tool, formal UCM model is translated to Promela formula model. In the dissertation, some system properties are defined as temporal logic formulas. These temporal logic formulas are manually translated to promela formulas which are individually integrated with promela formula model of UCM and verified using SPIN tool. Formal analysis used here helps verify the system properties (for example multiparty multimedia protocol) and dig out the bugs of systems.

DIMUSE: An integrated framework for distributed multimedia system with database management and security support

With the recent explosion in the complexity and amount of digital multimedia data, there has been a huge impact on the operations of various organizations in distinct areas, such as government services, education, medical care, business, entertainment, etc. To satisfy the growing demand of multimedia data management systems, an integrated framework called DIMUSE is proposed and deployed for distributed multimedia applications to offer a full scope of multimedia related tools and provide appealing experiences for the users. This research mainly focuses on video database modeling and retrieval by addressing a set of core challenges. First, a comprehensive multimedia database modeling mechanism called Hierarchical Markov Model Mediator (HMMM) is proposed to model high dimensional media data including video objects, low-level visual/audio features, as well as historical access patterns and frequencies. The associated retrieval and ranking algorithms are designed to support not only the general queries, but also the complicated temporal event pattern queries. Second, system training and learning methodologies are incorporated such that user interests are mined efficiently to improve the retrieval performance. Third, video clustering techniques are proposed to continuously increase the searching speed and accuracy by architecting a more efficient multimedia database structure. A distributed video management and retrieval system is designed and implemented to demonstrate the overall performance. The proposed approach is further customized for a mobile-based video retrieval system to solve the perception subjectivity issue by considering individual user's profile. Moreover, to deal with security and privacy issues and concerns in distributed multimedia applications, DIMUSE also incorporates a practical framework called SMARXO, which supports multilevel multimedia security control. SMARXO efficiently combines role-based access control (RBAC), XML and object-relational database management system (ORDBMS) to achieve the target of proficient security control. A distributed multimedia management system named DMMManager (Distributed MultiMedia Manager) is developed with the proposed framework DEMUR; to support multimedia capturing, analysis, retrieval, authoring and presentation in one single framework.

The effect of multimedia instruction on achievement of college history students

The purpose of this study was to investigate the effect of multimedia instruction on achievement of college students in AMH 2010 from exploration and discovery to1865. A non-equivalent control group design was used. The dependent variable was achievement. The independent variables were learning styles method of instruction, and visual clarifiers (notes). The study was conducted using two history sections from Palm Beach Community College, in Boca Raton, Florida, between August and December, 1998. Data were obtained by means of placement scores, posttests, the Productivity Environmental Preference Survey (PEPS), and a researcher-developed student survey. Statistical analysis of the data was done using SPSS statistical software. Demographic variables were compared using Chi square. T tests were run on the posttests to determine the equality of variances. The posttest scores of the groups were compared using the analysis of covariance (ANCOVA) at the .05 level of significance. The first hypothesis there is a significant difference in students' learning of U.S. History when students receive multimedia instruction was supported, F = (1, 52)= 688, p < .0005, and F = (1, 53) = 8.52, p < .005for Tests 2 and 3, respectively. The second hypothesis there is a significant difference on the effectiveness of multimedia instruction based on students' various learning preferences was not supported. The last hypotheses there is a significant difference on students' learning of U.S. History when students whose first language is other than English and students who need remediation receive visual clarifiers were not supported. Analysis of covariance (ANCOVA) indicated no difference between the groups on Test 1, Test 2, or Test 3: F (1, 4 5)= .01, p < .940, F (l, 52) = .77, p < .385, and F (1,53) =.17, p > .678, respectively, for language. Analysis of covariance (ANCOVA) indicated no significant difference on Test 1, Test 2, or Test 3, between the groups on the variable remediation: F (1, 45) = .31, p < .580, F (1, 52) = 1.44, p < .236, and F (1, 53) = .21, p < .645, respectively.