Incorporating domain knowledge with video and voice data analysis in news broadcasts

This paper addresses the area of video annotation, indexing and retrieval, and shows how a set of tools can be employed, along with domain knowledge, to detect narrative structure in broadcast news. The initial structure is detected using low-level audio visual processing in conjunction with domain knowledge. Higher level processing may then utilize the initial structure detected to direct processing to improve and extend the initial classification.

The structure detected breaks a news broadcast into segments, each of which contains a single topic of discussion. Further the segments are labeled as a) anchor person or reporter, b) footage with a voice over or c) sound bite. This labeling may be used to provide a summary, for example by presenting a thumbnail for each reporter present in a section of the video. The inclusion of domain knowledge in computation allows more directed application of high level processing, giving much greater efficiency of effort expended. This allows valid deductions to be made about structure and semantics of the contents of a news video stream, as demonstrated by our experiments on CNN news broadcasts.

Research on seismic signals for vehicle targets and recognition by data fusion

This paper researches seismic signals of typical vehicle targets in order to extract features and to recognize vehicle targets. As a data fusion method, the technique of artificial neural networks combined with genetic algorithm(ANNCGA) is applied for recognition of seismic signals that belong to different kinds of vehicle targets. The technique of ANNCGA and its architecture have been presented. The algorithm had been used for classification and recognition of seismic signals of vehicle targets in the outdoor environment. Through experiments, it can be proven that seismic properties of target acquired are correct, ANNCGA data fusion method is effective to solve the problem of target recognition.

Sensor fusion based localization of a mobile user in a wireless network

This paper applies sensor fusion to the localization problem of a mobile user. We propose that the use of direction of arrival (DOA) estimations along with received signal strength measurements can increase the accuracy and robustness of location estimations. The DOA estimations are incapable of providing multi-dimensional positioning alone, while signal strength methods are prone to high uncertainties. A Robust Extended Kalman Filter (REKF) is used to derive the state estimate of the mobile user's position, and successfully track the mobile users with less system complexity, as it requires measurements from only one base station. Therefore, localization of mobile users can be performed at the single base station. Furthermore, the technique is robust against system uncertainties caused by the inherent deterministic nature of the mobility model. Through simulation, we show the accuracy of our prediction algorithm and the simplicity of its implementation.

Multi-sensory face biometric fusion (for personal identification)

The objective of this work is to recognize faces using sets of images in visual and thermal spectra. This is challenging because the former is greatly affected by illumination changes, while the latter frequently contains occlusions due to eye-wear and is inherently less discriminative. Our method is based on a fusion of the two modalities. Specifically: we examine (i) the effects of preprocessing of data in each domain, (ii) the fusion of holistic and local facial appearance, and (iii) propose an algorithm for combining the similarity scores in visual and thermal spectra in the presence of prescription glasses and significant pose variations, using a small number of training images (5-7). Our system achieved a high correct identification rate of 97% on a freely available test set of 29 individuals and extreme illumination changes.

Structural damage identification based on self-fitting ARMAX model and multi-sensor data fusion

Statistical time series methods have proven to be a promising technique in structural health monitoring, since it provides a direct form of data analysis and eliminates the requirement for domain transformation. Latest research in structural health monitoring presents a number of statistical models that have been successfully used to construct quantified models of vibration response signals. Although a majority of these studies present viable results, the aspects of practical implementation, statistical model construction and decision-making procedures are often vaguely defined or omitted from presented work. In this article, a comprehensive methodology is developed, which essentially utilizes an auto-regressive moving average with exogenous input model to create quantified model estimates of experimentally acquired response signals. An iterative self-fitting algorithm is proposed to construct and fit the auto-regressive moving average with exogenous input model, which is capable of integrally finding an optimum set of auto-regressive moving average with exogenous input model parameters. After creating a dataset of quantified response signals, an unlabelled response signal can be identified according to a 'closest-fit' available in the dataset. A unique averaging method is proposed and implemented for multi-sensor data fusion to decrease the margin of error with sensors, thus increasing the reliability of global damage identification. To demonstrate the effectiveness of the developed methodology, a steel frame structure subjected to various bolt-connection damage scenarios is tested. Damage identification results from the experimental study suggest that the proposed methodology can be employed as an efficient and functional damage identification tool. © The Author(s) 2014.

An adaptive complementary filter for inertial sensor based data fusion to track upper body motion

  Remote human activity monitoring is critical and essential in physiotherapy with respect to the skyrocketing healthcare expenditure and the fast aging population. One of frequently used method to monitor human activity is wearing inertial sensors since it is low-cost and accurate. However, the measurements of those sensors are able only to estimate the orientation and rotation angles with respect to actual movement angles, because of differences in the body’s co-ordination system and the sensor’s co-ordination system. There were numerous studies being conducted to improve the accuracy of estimation, though there is potential for further discussions on improving accuracy by replacing heavy algorithms to less complexity. This research is an attempt to propose an adaptive complementary filter for identifying human upper arm movements. Further, this article discusses a feasibility of upper arm rehabilitation using the proposed adaptive complementary filter and inertial measurement sensors. The proposed algorithm is tested with four healthy subjects wearing an inertial sensor against gold standard, which is the VICON system. It demonstrated root mean squared error of 8.77◦ for upper body limb orientation estimation when compared to gold standard VICON optical motion capture system.