Assessment of Speech Dialog Systems using Multi-Modal Cognitive Load Analysis and Driving Performance Metrics

In this paper, cognitive load analysis via acoustic- and CAN-Bus-based driver performance metrics is employed to assess two different commercial speech dialog systems (SDS) during in-vehicle use. Several metrics are proposed to measure increases in stress, distraction and cognitive load and we compare these measures with statistical analysis of the speech recognition component of each SDS. It is found that care must be taken when designing an SDS as it may increase cognitive load which can be observed through increased speech response delay (SRD), changes in speech production due to negative emotion towards the SDS, and decreased driving performance on lateral control tasks. From this study, guidelines are presented for designing systems which are to be used in vehicular environments.

Multi-edge decimation in multi-modal 3D collaborative applications

Multi-resolution modelling has become essential as modern 3D applications demand 3D objects with higher LODs (LOD). Multi-modal devices such as PDAs and UMPCs do not have sufficient resources to handle the original 3D objects. The increased usage of collaborative applications has created many challenges for remote manipulation working with 3D objects of different quality. This paper studies how we can improve multi-resolution techniques by performing multiedge decimation and using annotative commands. It also investigates how devices with poorer quality 3D object can participate in collaborative actions.

New modal feed network for a compact monopole array with isolated ports

An element spacing of less than half a wavelength introduces strong mutual coupling between the ports of compact antenna arrays. The strong coupling causes significant system performance degradation. A decoupling network may compensate for the mutual coupling. Alternatively, port decoupling can be achieved using a modal feed network. In response to an input signal at one of the input ports, this feed network excites the antenna elements in accordance with one of the eigenvectors of the array scattering parameter matrix. In this paper, a novel 4-element monopole array is described. The feed network of the array is implemented as a planar ring-type circuit in stripline with four coupled line sections. The new configuration offers a significant reduction in size, resulting in a very compact array.

Improved correlation-based modal strain energy method for global damage detection of truss bridge structures

The previous investigations have shown that the modal strain energy correlation method, MSEC, could successfully identify the damage of truss bridge structures. However, it has to incorporate the sensitivity matrix to estimate damage and is not reliable in certain damage detection cases. This paper presents an improved MSEC method where the prediction of modal strain energy change vector is differently obtained by running the eigensolutions on-line in optimisation iterations. The particular trail damage treatment group maximising the fitness function close to unity is identified as the detected damage location. This improvement is then compared with the original MSEC method along with other typical correlation-based methods on the finite element model of a simple truss bridge. The contributions to damage detection accuracy of each considered mode is also weighed and discussed. The iterative searching process is operated by using genetic algorithm. The results demonstrate that the improved MSEC method suffices the demand in detecting the damage of truss bridge structures, even when noised measurement is considered.

Damage detection for truss bridge structures using correlation-based structural modal strain energy

This paper presents the feasibility of using structural modal strain energy as a parameter employed in correlation- based damage detection method for truss bridge structures. It is an extension of the damage detection method adopting multiple damage location assurance criterion. In this paper, the sensitivity of modal strain energy to damage obtained from the analytical model is incorporated into the correlation objective function. Firstly, the sensitivity matrix of modal strain energy to damage is conducted offline, and for an arbitrary damage case, the correlation coefficient (objective function) is calculated by multiplying the sensitivity matrix and damage vector. Then, a genetic algorithm is used to iteratively search the damage vector maximising the correlation between the corresponding modal strain energy change (hypothesised) and its counterpart in measurement. The proposed method is simulated and compared with the conventional methods, e.g. frequency-error method, coordinate modal assurance criterion and multiple damage location assurance criterion using mode shapes on a numerical truss bridge structure. The result demonstrates the modal strain energy correlation method is able to yield acceptable damage detection outcomes with less computing efforts, even in a noise contaminated condition.

Influence of axial deformation on modal strain energy

Axial loads of load bearing elements impact on the vibration characteristics. Several methods have been developed to quantify axial loads and hence axial deformations of individual structural members using their natural frequencies. Nevertheless, these methods cannot be applied to individual members in structural framing systems as the natural frequency is a global parameter for the entire framing system. This paper proposes an innovative method which uses modal strain energy phenomenon to quantify axial deformations of load bearing elements of structural framing systems. The procedure is illustrated through examples and results confirm that the proposed method has an ability to quantify the axial deformations of individual elements of structural framing systems

Multi-modal object tracking using dynamic performance metrics

Intelligent surveillance systems typically use a single visual spectrum modality for their input. These systems work well in controlled conditions, but often fail when lighting is poor, or environmental effects such as shadows, dust or smoke are present. Thermal spectrum imagery is not as susceptible to environmental effects, however thermal imaging sensors are more sensitive to noise and they are only gray scale, making distinguishing between objects difficult. Several approaches to combining the visual and thermal modalities have been proposed, however they are limited by assuming that both modalities are perfuming equally well. When one modality fails, existing approaches are unable to detect the drop in performance and disregard the under performing modality. In this paper, a novel middle fusion approach for combining visual and thermal spectrum images for object tracking is proposed. Motion and object detection is performed on each modality and the object detection results for each modality are fused base on the current performance of each modality. Modality performance is determined by comparing the number of objects tracked by the system with the number detected by each mode, with a small allowance made for objects entering and exiting the scene. The tracking performance of the proposed fusion scheme is compared with performance of the visual and thermal modes individually, and a baseline middle fusion scheme. Improvement in tracking performance using the proposed fusion approach is demonstrated. The proposed approach is also shown to be able to detect the failure of an individual modality and disregard its results, ensuring performance is not degraded in such situations.

An improved modal strain energy method for damage assessment

The modal strain energy method, which depends on the vibration characteristics of the structure, has been reasonably successful in identifying and localising damage in the structure. However, existing strain energy methods require the first few modes to be measured to provide meaningful damage detection. Use of individual modes with existing strain energy methods may indicate false alarms or may not detect the damage at or near the nodal points. This paper proposes a new modal strain energy based damage index which can detect and localize the damage using any one of the modes measured and illustrates its application for beam structures. It becomes evident that the proposed strain energy based damage index also has potential for damage quantification.

Modal parameter estimation in the presence of non-linearities

This paper investigates the use of time-frequency techniques to assist in the estimation of power system modes which are resolvable by a Digital Fourier Transform (DFT). The limitations of linear estimation techniques in the presence of large disturbances which excite system non-linearities, particularly the swing equation non-linearity are shown. Where a nonlinearity manifests itself as time varying modal frequencies the Wigner-Ville Distribution (WVD) is used to describe the variation in modal frequencies and construct a window over which standard linear estimation techniques can be used. The error obtained even in the presence of multiple resolvable modes is better than 2%.

Quantifying axial deformations of core shear walls using modal parameters

Differential axial deformation between column elements and shear wall elements of cores increase with building height and geometric complexity. Adverse effects due to the differential axial deformation reduce building performance and life time serviceability. Quantifying axial deformations using ambient measurements from vibrating wire, external mechanical and electronic strain gauges in order to acquire adequate provisions to mitigate the adverse effects is well established method. However, these gauges require installing in or on elements to acquire continuous measurements and hence use of these gauges is uneconomical and inconvenient. This motivates to develop a method to quantify the axial deformations. This paper proposes an innovative method based on modal parameters to quantify axial deformations of shear wall elements in cores of buildings. Capabilities of the method are presented though an illustrative example.

Multi-modal summarization of key events and top players in sports tournament videos

To detect and annotate the key events of live sports videos, we need to tackle the semantic gaps of audio-visual information. Previous work has successfully extracted semantic from the time-stamped web match reports, which are synchronized with the video contents. However, web and social media articles with no time-stamps have not been fully leveraged, despite they are increasingly used to complement the coverage of major sporting tournaments. This paper aims to address this limitation using a novel multimodal summarization framework that is based on sentiment analysis and players' popularity. It uses audiovisual contents, web articles, blogs, and commentators' speech to automatically annotate and visualize the key events and key players in a sports tournament coverage. The experimental results demonstrate that the automatically generated video summaries are aligned with the events identified from the official website match reports.