A framework towards a multi-modal fingerprinting scheme for multimedia assets

Fingerprinting is a well known approach for identifying multimedia data without having the original data present but instead what amounts to its essence or 'DNA'. Current approaches show insufficient deployment of various types of knowledge that could be brought to bear in providing a fingerprinting framework that remains effective, efficient and can accommodate both the whole as well as elemental protection at appropriate levels of abstraction to suit various Zones of Interest (ZoI) in an image or cross media artefact. The proposed framework aims to deliver selective composite fingerprinting that is powerfully aided by leveraging both multi-modal information as well as a rich spectrum of collateral context knowledge including both image-level collaterals and also the inevitably needed market intelligence knowledge such as customers' social networks interests profiling which we can deploy as a crucial component of our fingerprinting collateral knowledge.

Situation assessment through multi-modal sensing of dynamic environments to support cognitive robot control

Awareness of emerging situations in a dynamic operational environment of a robotic assistive device is an essential capability of such a cognitive system, based on its effective and efficient assessment of the prevailing situation. This allows the system to interact with the environment in a sensible (semi)autonomous / pro-active manner without the need for frequent interventions from a supervisor. In this paper, we report a novel generic Situation Assessment Architecture for robotic systems directly assisting humans as developed in the CORBYS project. This paper presents the overall architecture for situation assessment and its application in proof-of-concept Demonstrators as developed and validated within the CORBYS project. These include a robotic human follower and a mobile gait rehabilitation robotic system. We present an overview of the structure and functionality of the Situation Assessment Architecture for robotic systems with results and observations as collected from initial validation on the two CORBYS Demonstrators.

Content-based video indexing for sports applications using integrated multi-modal approach

To sustain an ongoing rapid growth of video information, there is an emerging demand for a sophisticated content-based video indexing system. However, current video indexing solutions are still immature and lack of any standard. This doctoral consists of a research work based on an integrated multi-modal approach for sports video indexing and retrieval. By combining specific features extractable from multiple audio-visual modalities, generic structure and specific events can be detected and classified. During browsing and retrieval, users will benefit from the integration of high-level semantic and some descriptive mid-level features such as whistle and close-up view of player(s).

Issues arising from the implementation of a comprehensive assessment approach in a large, multi-modal Bachelor of Commerce Unit

Recent literature in higher education argues university assessment has been too narrow and hasn’t adequately reflected the quality, breadth and depth of students’ learning. Research shows students often prioritise and learn what they need to know for formal, graded assessment and disregard other academic content seen as less relevant to those requirements. The predominance of essays and examinations has therefore tended to constrain learning. The case for a more comprehensive approach has been clearly articulated. So what happens when staff take up the unique challenge of designing fair and uniform assessment for a large, core, multi-modal, multi-campus unit offered nationally and internationally?
When developing an undergraduate Bachelor of Commerce unit at Deakin University, staff considered the most appropriate ways to assess a range of conceptual understandings and communication skills. This resulted in the mapping and adoption of a comprehensive approach incorporating teacher, peer, and self-assessment aspects, individual and group work, oral and written presentations, and the use of portfolios and journals. Particular practices were adopted to control workloads, ensure fairness in marking, and overcome some problems generally associated with group work. When implementing the approach, practical issues arose that demanded adjustments. This paper details the approach taken, outlines research activities, and discusses the practical implications of issues that arose.

Content-based video indexing for sports applications using integrated multi-modal approach

This thesis presents a research work based on an integrated multi-modal approach for sports video indexing and retrieval. By combining specific features extractable from multiple (audio-visual) modalities, generic structure and specific events can be detected and classified. During browsing and retrieval, users will benefit from the integration of high-level semantic and some descriptive mid-level features such as whistle and close-up view of player(s). The main objective is to contribute to the three major components of sports video indexing systems. The first component is a set of powerful techniques to extract audio-visual features and semantic contents automatically. The main purposes are to reduce manual annotations and to summarize the lengthy contents into a compact, meaningful and more enjoyable presentation. The second component is an expressive and flexible indexing technique that supports gradual index construction. Indexing scheme is essential to determine the methods by which users can access a video database. The third and last component is a query language that can generate dynamic video summaries for smart browsing and support user-oriented retrievals.

Issues arising from the implementation of a comprehensive assessment approach in a large, multi-modal Bachelor of Commerce unit

Recent literature in higher education argues university assessment has been too narrow and hasn’t adequately reflected the quality, breadth and depth of students’ learning. Research shows students often prioritise and learn what they need to know for formal, graded assessment and disregard other academic content seen as less relevant to those requirements. The predominance of essays and examinations has therefore tended to constrain learning. The case for a more comprehensive approach has been clearly articulated. So what happens when staff take up the unique challenge of designing fair and uniform assessment for a large, core, multi-modal, multi-campus unit offered nationally and internationally?

When developing an undergraduate Bachelor of Commerce unit at Deakin University, staff considered the most appropriate ways to assess a range of conceptual understandings and communication skills. This resulted in the mapping and adoption of a comprehensive approach incorporating teacher, peer, and self-assessment aspects, individual and group work, oral and written presentations, and the use of portfolios and journals. Particular practices were adopted to control workloads, ensure fairness in marking, and overcome some problems generally associated with group work. When implementing the approach, practical issues arose that demanded adjustments. This paper details the approach taken, outlines research activities, and discusses the practical implications of issues that arose.

Mining multi-modal crime patterns at different levels of granularity using hierarchical clustering

The appearance of patterns could be found in different modalities of a domain, where the different modalities refer to the data sources that constitute different aspects of a domain. Particularly, the domain of our discussion refers to crime and the different modalities refer to the different data sources such as offender data, weapon data, etc. in crime domain. In addition, patterns also exist in different levels of granularity for each modality. In order to have a thorough understanding a domain, it is important to reveal the hidden patterns through the data explorations at different levels of granularity and for each modality. Therefore, this paper presents a new model for identifying patterns that exist in different levels of granularity for different modes of crime data. A hierarchical clustering approach - growing self organising maps (GSOM) has been deployed. Furthermore, the model is enhanced with experiments that exhibit the significance of exploring data at different granularities.

Calibration of audio-video sensors for multi-modal event indexing

This paper addresses the coordinated use of video and audio cues to capture and index surveillance events with multimodal labels. The focus of this paper is the development of a joint-sensor calibration technique that uses audio-visual observations to improve the calibration process. One significant feature of this approach is the ability to continuously check and update the calibration status of the sensor suite, making it resilient to independent drift in the individual sensors. We present scenarios in which this system is used to enhance surveillance.

A coarse-grained parallel genetic algorithm employing cluster analysis for multi-modal numerical optimisation

This paper describes a technique for improving the performance of parallel genetic algorithms on multi-modal numerical optimisation problems. It employs a cluster analysis algorithm to identify regions of the search space in which more than one sub-population is sampling. Overlapping clusters are merged in one sub-population whilst a simple derating function is applied to samples in all other sub-populations to discourage them from further sampling in that region. This approach leads to a better distribution of the search effort across multiple subpopulations and helps to prevent premature convergence. On the test problems used, significant performance improvements over the traditional island model implementation are realised.

Osteo-cise : Strong bones for life': protocol for a community-based randomised controlled trial of a multi-modal exercise and osteoporosis education program for older adults at risk of falls and fractures

Background : Osteoporosis affects over 220 million people worldwide, and currently there is no 'cure' for the disease. Thus, there is a need to develop evidence-based, safe and acceptable prevention strategies at the population level that target multiple risk factors for fragility fractures to reduce the health and economic burden of the condition.

Methods : The 'Osteo-cise: Strong Bones for Life' study will investigate the effectiveness and feasibility of a multi-component targeted exercise, osteoporosis education/awareness and behavioural change program for improving bone health and muscle function, and reducing falls risk in community-dwelling older adults at an increased risk of fracture. Men and women aged 60 years or above will participate in an 18-month randomised controlled trial comprising a 12-month structured and supervised community-based program and a 6-month 'research to practise' translational phase. Participants will be randomly assigned to either the 'Osteo-cise' intervention or a self-management control group. The intervention will comprise a multi-modal exercise program incorporating high velocity progressive resistance training, moderate impact weight-bearing exercise and high challenging balance exercises performed three times weekly at local community-based fitness centres. A behavioural change program will be used to enhance exercise adoption and adherence to the program. Community-based osteoporosis education seminars will be conducted to improve participant knowledge and understanding of the risk factors and preventative measures for osteoporosis, falls and fractures. The primary outcomes measures, to be collected at baseline, 6, 12, and 18 months, will include DXA-derived hip and spine bone mineral density measurements and functional muscle power (timed stair-climb test). Secondary outcomes measures include: MRI-assessed distal femur and proximal tibia trabecular bone micro-architecture, lower limb and back maximal muscle strength, balance and function (four square step test, functional reach test, timed up-and-go test and 30-second sit-to-stand), falls incidence and health-related quality of life. Cost-effectiveness will also be assessed.

Discussion : The findings from the Osteo-cise: Strong Bones for Life study will provide new information on the efficacy of a targeted multi-modal community-based exercise program incorporating high velocity resistance training, together with an osteoporosis education and behavioural change program for improving multiple risk factors for falls and fracture in older adults at risk of fragility fracture. Trial Registration: Australian New Zealand Clinical Trials Registry reference ACTRN12609000100291

Multi-modal abnormality detection in video with unknown data segmentation

This paper examines a new problem in large scale stream data: abnormality detection which is localized to a data segmentation process. Unlike traditional abnormality detection methods which typically build one unified model across data stream, we propose that building multiple detection models focused on different coherent sections of the video stream would result in better detection performance. One key challenge is to segment the data into coherent sections as the number of segments is not known in advance and can vary greatly across cameras; and a principled way approach is required. To this end, we first employ the recently proposed infinite HMM and collapsed Gibbs inference to automatically infer data segmentation followed by constructing abnormality detection models which are localized to each segmentation. We demonstrate the superior performance of the proposed framework in a real-world surveillance camera data over 14 days.