Reasoning with multimedia information using symbolic projection

In many multimedia application systems, it is not the final goal to retrieve the relevant multimedia information from different multimedia information sources. Rather, post-processing of the retrieved multimedia information is needed. For example, the retrieved information is used as “known facts”. The systems will do some reasoning to obtain further conclusions based on these multimedia form “known facts”. We call this reasoning with multimedia information. Most current research work in multimedia information processing is focused on multimedia information retrieval, but post-processing the retrieved information is more or less ignored. This paper explores the way to tackle this problem by using symbolic projection. A case study of reasoning with still image information is presented. Some extensions to symbolic projection- introducing auxiliary pictorial objects in symbolic pictures that need to be processed-are discussed. We expect this paper will stimulate further research on this important but ignored topic.

Post processing of multimedia information - concepts, problems, and techniques

Currently, most research work on multimedia information processing is focused on multimedia information storage and retrieval, especially indexing and content-based access of multimedia information. We consider multimedia information processing should include one more level-post-processing. Here "post-processing" means further processing of retrieved multimedia information, which includes fusion of multimedia information and reasoning with multimedia information to reach new conclusions. In this paper, the three levels of multimedia information processing storage, retrieval, and post-processing- are discussed. The concepts and problems of multimedia information post-processing are identified. Potential techniques that can be used in post-processing are suggested, By highlighting the problems in multimedia information post-processing, hopefully this paper will stimulate further research on this important but ignored topic.

Conceptual representation for multimedia information

Multimedia information is now routinely available in the forms of text, pictures, animation and sound. Although text objects are relatively easy to deal with (in terms of information search and retrieval), other information bearing objects (such as sound, images, animation) are more difficult to index. Our research is aimed at developing better ways of representing multimedia objects by using a conceptual representation based on Schank's conceptual dependencies. Moreover, the representation allows for users' individual interpretations to be embedded in the system. This will alleviate the problems associated with traditional semantic networks by allowing for coexistence of multiple views of the same information. The viability of the approach is tested, and the preliminary results reported.

MMM 2005 : proceedings of the 11th International Multimedia Modelling Conference : 12-14 January, 2005, Melbourne, Australia

The International Multimedia Modeling conference series is an annual forum to discuss the efficient representation, processing, interaction, integration, communication, and retrieval of multimedia information.
In particular, the 11th International Multimedia Modeling Conference (MMM2005) concentrates on common modeling frameworks for integrating the diverse fields of visual, audio, video, and virtual world information.
MMM2005 deals with emerging Multimedia Modeling topics that include:
• Audio Analysis and Modeling
• Video Manipulation and Modeling
• Video Mining and MPEG
• Image Modeling and Editing
• Image Retrieval
• Multimedia Presentation and Knowledge Sharing
• AI and Image Recognition
• Mobile and Virtual Multimedia Environments

Proceedings : 10th International Multimedia Modeling Conference : MMM 2004 : Brisbane, Australia, 5-7 January 2004

The International Multimedia Modelling conference series is an annual forum to discuss the efficient representation, processing, interaction, integration, communication, and retrieval of multimedia information.
In particular, the 10th International Multimedia Modelling Conference (MMM2004) concentrates on common modelling frameworks for integrating the diverse fields of visual, audio, video, and
virtual world information.
MMM2004 deals with emerging Multimedia Modelling topics including:
• Multimedia Databases
• Audio Processing, Coding and Encryption
• Network Games and Animation
• Video Applications
• Multimedia Frameworks and QoS
• Topological and 3D Geometric Modelling
• Image Applications
• Image Retrieval
• Modelling / Editing / Virtual Environment
• Video Retrieval and Browsing

Networking of Multimedia Women Event : beyond epsilon science - "where to look and how to realize new opportunities"

"Networking of Multimedia Women" event is a continuation of an on-going conversation in the multimedia research community and efforts by the ACM SIGMM to engage and promote female researchers in multimedia community, enable networking of junior and senior female researchers, and give insights towards successful professional careers based on examples. This year, the event will have a theme, called "Beyond Epsilon Science", where preeminent senior female researchers from academia, industry and government, Svetha Venkatesh, Nalini Venkatasubramanian, Dulce Ponceleon, Susanne Boll, and Maria Zemankova will present and discuss how to go beyond epsilon science, where to look for big ideas with high social impact, as well as how to obtain funding to realize these ideas, innovations and opportunities. Their current research projects and funding efforts, and their personal experiences will drive the event's discussions, awareness of major research and funding initiatives, answers to open questions and insights into successful professional careers.

Between context-aware media capture and multimedia content analysis - where do we find the promised land?

Various issues related to the multimedia information retrieval and media access are discussed. The feasible solutions for automatic signal-based analysis of media content are analyzed. The extent of user involvement in the content creation process is emphasized. The applications driving the creation and usage of context and metadata are also elaborated.

"You Tube and I Find" - personalizing multimedia content access

Recent growth in broadband access and proliferation of small personal devices that capture images and videos has led to explosive growth of multimedia content available everywhereVfrom personal disks to the Web. While digital media capture and upload has become nearly universal with newer device technology, there is still a need for better tools and technologies to search large collections of multimedia data and to find and deliver the right content to a user according to her current needs and preferences. A renewed focus on the subjective dimension in the multimedia lifecycle, fromcreation, distribution, to delivery and consumption, is required to address this need beyond what is feasible today. Integration of the subjective aspects of the media itselfVits affective, perceptual, and physiological potential (both intended and achieved), together with those of the users themselves will allow for personalizing the content access, beyond today’s facility. This integration, transforming the traditional multimedia information retrieval (MIR) indexes to more effectively answer specific user needs, will allow a richer degree of personalization predicated on user intention and mode of interaction, relationship to the producer, content of the media, and their history and lifestyle. In this paper, we identify the challenges in achieving this integration, current approaches to interpreting content creation processes, to user modelling and profiling, and to personalized content selection, and we detail future directions. The structure of the paper is as follows: In Section I, we introduce the problem and present some definitions. In Section II, we present a review of the aspects of personalized content and current approaches for the same. Section III discusses the problem of obtaining metadata that is required for personalized media creation and present eMediate as a case study of an integrated media capture environment. Section IV presents the MAGIC system as a case study of capturing effective descriptive data and putting users first in distributed learning delivery. The aspects of modelling the user are presented as a case study in using user’s personality as a way to personalize summaries in Section V. Finally, Section VI concludes the paper with a discussion on the emerging challenges and the open problems.

Agent-based hybrid framework for decision making on complex problems

Electronic commerce and the Internet have created demand for automated systems that can make complex decisions utilizing information from multiple sources. Because the information is uncertain, dynamic, distributed, and heterogeneous in nature, these systems require a great diversity of intelligent techniques including expert systems, fuzzy logic, neural networks, and genetic algorithms. However, in complex decision making, many different components or sub-tasks are involved, each of which requires different types of processing. Thus multiple such techniques are required resulting in systems called hybrid intelligent systems. That is, hybrid solutions are crucial for complex problem solving and decision making. There is a growing demand for these systems in many areas including financial investment planning, engineering design, medical diagnosis, and cognitive simulation. However, the design and development of these systems is difficult because they have a large number of parts or components that have many interactions. From a multi-agent perspective, agents in multi-agent systems (MAS) are autonomous and can engage in flexible, high-level interactions. MASs are good at complex, dynamic interactions. Thus a multi-agent perspective is suitable for modeling, design, and construction of hybrid intelligent systems. The aim of this thesis is to develop an agent-based framework for constructing hybrid intelligent systems which are mainly used for complex problem solving and decision making. Existing software development techniques (typically, object-oriented) are inadequate for modeling agent-based hybrid intelligent systems. There is a fundamental mismatch between the concepts used by object-oriented developers and the agent-oriented view. Although there are some agent-oriented methodologies such as the Gaia methodology, there is still no specifically tailored methodology available for analyzing and designing agent-based hybrid intelligent systems. To this end, a methodology is proposed, which is specifically tailored to the analysis and design of agent-based hybrid intelligent systems. The methodology consists of six models - role model, interaction model, agent model, skill model, knowledge model, and organizational model. This methodology differs from other agent-oriented methodologies in its skill and knowledge models. As good decisions and problem solutions are mainly based on adequate information, rich knowledge, and appropriate skills to use knowledge and information, these two models are of paramount importance in modeling complex problem solving and decision making. Follow the methodology, an agent-based framework for hybrid intelligent system construction used in complex problem solving and decision making was developed. The framework has several crucial characteristics that differentiate this research from others. Four important issues relating to the framework are also investigated. These cover the building of an ontology for financial investment, matchmaking in middle agents, reasoning in problem solving and decision making, and decision aggregation in MASs. The thesis demonstrates how to build a domain-specific ontology and how to access it in a MAS by building a financial ontology. It is argued that the practical performance of service provider agents has a significant impact on the matchmaking outcomes of middle agents. It is proposed to consider service provider agents' track records in matchmaking. A way to provide initial values for the track records of service provider agents is also suggested. The concept of ‘reasoning with multimedia information’ is introduced, and reasoning with still image information using symbolic projection theory is proposed. How to choose suitable aggregation operations is demonstrated through financial investment application and three approaches are proposed - the stationary agent approach, the token-passing approach, and the mobile agent approach to implementing decision aggregation in MASs. Based on the framework, a prototype was built and applied to financial investment planning. This prototype consists of one serving agent, one interface agent, one decision aggregation agent, one planning agent, four decision making agents, and five service provider agents. Experiments were conducted on the prototype. The experimental results show the framework is flexible, robust, and fully workable. All agents derived from the methodology exhibit their behaviors correctly as specified.

Media-independent knowledge representation via UMART : unified mental annotation and retrieval tool

Multimedia information is now routinely available in the forms of text, pictures, animation and sound. Although text objects are relatively easy to deal with (in terms of information search and retrieval), other information bearing objects (such as sound, images, animation) are more difficult to index. Our research is aimed at developing better ways of representing multimedia objects by using a conceptual representation based on Schank's conceptual dependencies. Moreover, the representation allows for users' individual interpretations to be embedded in the system. This will alleviate the problems associated with traditional semantic networks by allowing for coexistence of multiple views of the same information. The viability of the approach is tested, and the preliminary results reported.

Can multimedia meet tertiary educational needs better than the conventional lecture? A case study

Educational researchers have long derided the university lecture as an effective mode of delivery of educational materials, but currently there are many reports on the advantages offered by computer. In this study a multimedia solution was sought to replace existing face to face lectures because it appeared to offer a close 'media versus need' match. Consequently, a decision was made to develop a design template for an interactive computer based program that would be suitable for a range of subject content. In order to personalise the instruction, a large video insert was incorporated as the main screen's most prominent design feature. From here the learner could navigate to support material including interactive simulations. The program was piloted with a small group of students and, in particular, the student tracking data that the program automatically generates yield some interesting learning style information.

Computer-aided design decision support: interfacing knowledge and information

Computer-aided design decision support has proved to be an elusive and intangible project for many researchers as they seek to encapsulate information and knowledge-based systems as useful multifunctional data structures. Definitions of ‘knowledge', ‘information', ‘facts', and ‘data' become semantic footballs in the struggle to identify what designers actually do, and what level of support would suit them best, and how that support might be offered. The Construction Primer is a database-drivable interactive multimedia environment that provides readily updated access to many levels of information aimed to suit students and practitioners alike. This is hardly a novelty in itself. The innovative interface and metadata structures, however, combine with the willingness of national building control legislators, standards authorities, materials producers, building research organisations, and specification services to make the Construction Primer a versatile design decision support vehicle. It is both compatible with most working methodologies while remaining reasonably future-proof. This paper describes the structure of the project and highlights the importance of sound planning and strict adhesion to library-standard metadata protocols as a means to avoid the support becoming too specific or too paradigmatic.

Thinking multimedia

Children in our society have access to many information resources and communication options. As we witness the convergence of art, literacy and publishing, individuals need to learn how to make sense of information presented in many different forms, and how to construct their own communications in multiple media.
Thinking Multimedia is a program that has developed out of many projects that I have run in several school and some tertiary institutions over the past 12 years. It is an attempt to integrate skills and knowledge from different academic disciplines and to encourage students to understand learning processes and their own learning preferences. The course, offered at this stage at Year 10 level at St Catherine’s School in Melbourne, aims to provide background and basic skills in how to construct and deconstruct information in multiple media and to provide students with the opportunity to explore a ‘real need’ project of their own in a project-based team environment. The course is supported by an online resource and discussion component.
In this presentation I will explain the background to the Thinking Multimedia program and explore some of the work by the students involved.