Comparison of Image Transform-Based Features for Visual Speech Recognition in Clean and Corrupted Videos

We present results of a study into the performance of a variety of different image transform-based feature types for speaker-independent visual speech recognition of isolated digits. This includes the first reported use of features extracted using a discrete curvelet transform. The study will show a comparison of some methods for selecting features of each feature type and show the relative benefits of both static and dynamic visual features. The performance of the features will be tested on both clean video data and also video data corrupted in a variety of ways to assess each feature type's robustness to potential real-world conditions. One of the test conditions involves a novel form of video corruption we call jitter which simulates camera and/or head movement during recording.

Robust Audio-Visual Speech Recognition under Noisy Audio-Video Conditions

This paper presents the maximum weighted stream posterior (MWSP) model as a robust and efficient stream integration method for audio-visual speech recognition in environments, where the audio or video streams may be subjected to unknown and time-varying corruption. A significant advantage of MWSP is that it does not require any specific measurements of the signal in either stream to calculate appropriate stream weights during recognition, and as such it is modality-independent. This also means that MWSP complements and can be used alongside many of the other approaches that have been proposed in the literature for this problem. For evaluation we used the large XM2VTS database for speaker-independent audio-visual speech recognition. The extensive tests include both clean and corrupted utterances with corruption added in either/both the video and audio streams using a variety of types (e.g., MPEG-4 video compression) and levels of noise. The experiments show that this approach gives excellent performance in comparison to another well-known dynamic stream weighting approach and also compared to any fixed-weighted integration approach in both clean conditions or when noise is added to either stream. Furthermore, our experiments show that the MWSP approach dynamically selects suitable integration weights on a frame-by-frame basis according to the level of noise in the streams and also according to the naturally fluctuating relative reliability of the modalities even in clean conditions. The MWSP approach is shown to maintain robust recognition performance in all tested conditions, while requiring no prior knowledge about the type or level of noise.

Test stimulus characteristics determine the perceived speed of the dynamic motion aftereffect

Using a speed-matching task, we measured the speed tuning of the dynamic motion aftereVect (MAE). The results of our Wrst experiment, in which we co-varied dot speed in the adaptation and test stimuli, revealed a speed tuning function. We sought to tease apart what contribution, if any, the test stimulus makes towards the observed speed tuning. This was examined by independently manipulating dot speed in the adaptation and test stimuli, and measuring the eVect this had on the perceived speed of the dynamic MAE. The results revealed that the speed tuning of the dynamic MAE is determined, not by the speed of the adaptation stimulus, but by the local motion characteristics of the dynamic test stimulus. The role of the test stimulus in determining the perceived speed of the dynamic MAE was conWrmed by showing that, if one uses a test stimulus containing two sources of local speed information, observers report seeing a transparent MAE; this is despite the fact that adaptation is induced using a single-speed stimulus. Thus while the adaptation stimulus necessarily determines perceived direction of the dynamic MAE, its perceived speed is determined by the test stimulus. This dissociation of speed and direction supports the notion that the processing of these two visual attributes may be partially independent.

The visual processing of motion-defined transparency

Our understanding of how the visual system processes motion transparency, the phenomenon by which multiple directions of motion are perceived to co-exist in the same spatial region, has grown considerably in the past decade. There is compelling evidence that the process is driven by global-motion mechanisms. Consequently, although transparently moving surfaces are readily segmented over an extended space, the visual system cannot separate two motion signals that co-exist in the same local region. A related issue is whether the visual system can detect transparently moving surfaces simultaneously, or whether the component signals encounter a serial â??bottleneckâ?? during their processing? Our initial results show that, at sufficiently short stimulus durations, observers cannot accurately detect two superimposed directions; yet they have no difficulty in detecting one pattern direction in noise, supporting the serial-bottleneck scenario. However, in a second experiment, the difference in performance between the two tasks disappears when the component patterns are segregated. This discrepancy between the processing of transparent and non-overlapping patterns may be a consequence of suppressed activity of global-motion mechanisms when the transparent surfaces are presented in the same depth plane. To test this explanation, we repeated our initial experiment while separating the motion components in depth. The marked improvement in performance leads us to conclude that transparent motion signals are represented simultaneously.

Neuromuscular adaptation during skill acquisition on a two degree-of-freedom target-acquisition task: Dynamic movement

In this experiment, we examined the extent to which the spatiotemporal reorganization of muscle synergies mediates skill acquisition on a two degree-of-freedom (df) target-acquisition task. Eight participants completed five practice sessions on consecutive days. During each session they practiced movements to eight target positions presented by a visual display. The movements required combinations of flexion/extension and pronation/supination of the elbow joint complex. During practice sessions, eight targets displaced 5.4 cm from the start position ( representing joint excursions of 54) were presented 16 times. During pre- and posttests, participants acquired the targets at two distances (3.6 cm [36 degrees] and 7.2 cm [72 degrees]). EMG data were recorded from eight muscles contributing to the movements during the pre- and posttests. Most targets were acquired more rapidly after the practice period. Performance improvements were, in most target directions, accompanied by increases in the smoothness of the movement trajectories. When target acquisition required movement in both dfs, there were also practice-related decreases in the extent to which the trajectories deviated from a direct path to the target. The contribution of monofunctional muscles ( those producing torque in a single df) increased with practice during movements in which they acted as agonists. The activity in bifunctional muscles ( those contributing torque in both dfs) remained at pretest levels in most movements. The results suggest that performance gains were mediated primarily by changes in the spatial organization of muscles synergies. These changes were expressed most prominently in terms of the magnitude of activation of the monofunctional muscles.

Dynamic Linear Precoding for the Exploitation of Known Interference in MIMO Broadcast Systems

This paper introduces a novel channel inversion (CI) precoding scheme for the downlink of phase shift keying (PSK)-based multiple input multiple output (MIMO) systems. In contrast to common practice where knowledge of the interference is used to eliminate it, the main idea proposed here is to use this knowledge to glean benefit from the interference. It will be shown that the system performance can be enhanced by exploiting some of the existent inter-channel interference (ICI). This is achieved by applying partial channel inversion such that the constructive part of ICI is preserved and exploited while the destructive part is eliminated by means of CI precoding. By doing so, the effective signal to interference-plus-noise ratio (SINR) delivered to the mobile unit (MU) receivers is enhanced without the need to invest additional transmitted signal power at the MIMO base station (BS). It is shown that the trade-off to this benefit is a minor increase in the complexity of the BS processing. The presented theoretical analysis and simulations demonstrate that due to the SINR enhancement, significant performance and throughput gains are offered by the proposed MIMO precoding technique compared to its conventional counterparts.

Adaptations of lateral hand movements to early and late visual occlusion in catching

Recent studies suggested that the control of hand movements in catching involves continuous vision-based adjustments. More insight into these adjustments may be gained by examining the effects of occluding different parts of the ball trajectory. Here, we examined the effects of such occlusion on lateral hand movements when catching balls approaching from different directions, with the occlusion conditions presented in blocks or in randomized order. The analyses showed that late occlusion only had an effect during the blocked presentation, and early occlusion only during the randomized presentation. During the randomized presentation movement biases were more leftward if the preceding trial was an early occlusion trial. The effect of early occlusion during the randomized presentation suggests that the observed leftward movement bias relates to the rightward visual acceleration inherent to the ball trajectories used, while its absence during the blocked presentation seems to reflect trial-by-trial adaptations in the visuomotor gain, reminiscent of dynamic gain control in the smooth pursuit system. The movement biases during the late occlusion block were interpreted in terms of an incomplete motion extrapolation--a reduction of the velocity gain--caused by the fact that participants never saw the to-be-extrapolated part of the ball trajectory. These results underscore that continuous movement adjustments for catching do not only depend on visual information, but also on visuomotor adaptations based on non-visual information.

The use of a dynamic truck-trailer drive-by system to monitor bridge damping

Bridge structures are continuously subject to degradation due to the environment, ageing and excess loading. Periodic monitoring of bridges is therefore a key part of any maintenance strategy as it can give early warning if a bridge becomes unsafe. This article investigates an alternative method for the monitoring of bridge dynamic behaviour: a truck-trailer vehicle system, with accelerometers fitted to the axles of the trailer. The method aims to detect changes in the damping of a bridge, which may indicate the existence of damage. A simplified vehicle-bridge interaction model is used in theoretical simulations to assess the effectiveness of the method in detecting those changes. The influence of road profile roughness on the vehicle vibration is overcome by recording accelerations from both axles of a trailer and then analysing the spectra of the difference in the accelerations between the two axles. The effectiveness of the approach in detecting damage simulated as a loss in stiffness is also investigated. In addition, the sensitivity of the approach to the vehicle speed, road roughness class, bridge span length, changes in the equal axle properties and noise is investigated.

Adaptive NormalHedge for robust visual tracking

In this paper, we propose a novel visual tracking framework, based on a decision-theoretic online learning algorithm namely NormalHedge. To make NormalHedge more robust against noise, we propose an adaptive NormalHedge algorithm, which exploits the historic information of each expert to perform more accurate prediction than the standard NormalHedge. Technically, we use a set of weighted experts to predict the state of the target to be tracked over time. The weight of each expert is online learned by pushing the cumulative regret of the learner towards that of the expert. Our simulation experiments demonstrate the effectiveness of the proposed adaptive NormalHedge, compared to the standard NormalHedge method. Furthermore, the experimental results of several challenging video sequences show that the proposed tracking method outperforms several state-of-the-art methods.

A Correlation between the Dynamic Viscosity and Crystallisation of Felodipine from its Amorphous Polymer Solid Dispersion

Purpose Previously, it has been reported that molecular mobility determines the rate of molecular approach to crystal surfaces, while entropy relates to the probability of that approaching molecule having the desirable configuration for further growth of the existing crystal; and the free energy dictates the probability of that molecule not returning to the liquid phase1. If we plot the crystal growth rate and viscosity of a supercooled liquid in a log-log format, the relationship between the two is linear, indicating the influence viscosity has upon crystal growth rate. However, such approximation has been derived from pure drug compounds and it is apparent that further understanding of crystallization from drug-polymer solid dispersion is required in order to stabilise drugs embedded within amorphous polymeric solid dispersions. Methods Mixtures of felodipine and polymer (HPMCAS-HF, PVPK15 and Soluplus®) at specified compositions were prepared using a Restch MM200 ball mill. To examine crystal growth within amorphous solid dispersions, samples were prepared by melting 5-10 mg of ball milled mixture at 150°C for 3-5 minutes on a glass slip pre-cleaned with methanol and acetone. All prepared samples were confirmed to be crystal free by visual observation using a polarised light microscope (Olympus BX50). Prepared samples were stored at 0% RH (P2O5), inside desiccators, maintained in ovens at 80°C. For the dynamic viscosity measurement, approximately 100-200mg ball milled mixture was heated on the base plate of a rotational rheometer at 150°C for 5 minutes and the top plate was lowered to a defined gap to form a good contact with the material. The sandwiched amorphous material was heated to 80°C and the viscosity was measured. Results The equation was used to probe the correlation of viscosity to crystal growth rate. In comparison to the value of xi in log-log equation reported from pure drug compound, a value of 1.63 was obtained for FD-polymer solid dispersions irrespective of the polymer involved. &#8733 Conclusion The high xi value suggests stronger viscosity dependence may exist for amorphous FD once incorporated with amorphous polymer.

Monitoring Bridge Dynamic Behaviour using an Instrumented Two Axle Vehicle

Highway structures such as bridges are subject to continuous degradation primarily due to ageing, loading and environmental factors. A rational transport policy must monitor and provide adequate maintenance to this infrastructure to guarantee the required levels of transport service and safety. Increasingly in recent years, bridges are being instrumented and monitored on an ongoing basis due to the implementation of Bridge Management Systems. This is very effective and provides a high level of protection to the public and early warning if the bridge becomes unsafe. However, the process can be expensive and time consuming, requiring the installation of sensors and data acquisition electronics on the bridge. This paper investigates the use of an instrumented 2-axle vehicle fitted with accelerometers to monitor the dynamic behaviour of a bridge network in a simple and cost-effective manner. A simplified half car-beam interaction model is used to simulate the passage of a vehicle over a bridge. This investigation involves the frequency domain analysis of the axle accelerations as the vehicle crosses the bridge. The spectrum of the acceleration record contains noise, vehicle, bridge and road frequency components. Therefore, the bridge dynamic behaviour is monitored in simulations for both smooth and rough road surfaces. The vehicle mass and axle spacing are varied in simulations along with bridge structural damping in order to analyse the sensitivity of the vehicle accelerations to a change in bridge properties. These vehicle accelerations can be obtained for different periods of time and serve as a useful tool to monitor the variation of bridge frequency and damping with time.

Investigation into DCT Feature Selection for Visual Lip-Based Biometric Authentication

This paper investigated using lip movements as a behavioural biometric for person authentication. The system was trained, evaluated and tested using the XM2VTS dataset, following the Lausanne Protocol configuration II. Features were selected from the DCT coefficients of the greyscale lip image. This paper investigated the number of DCT coefficients selected, the selection process, and static and dynamic feature combinations. Using a Gaussian Mixture Model - Universal Background Model framework an Equal Error Rate of 2.20% was achieved during evaluation and on an unseen test set a False Acceptance Rate of 1.7% and False Rejection Rate of 3.0% was achieved. This compares favourably with face authentication results on the same dataset whilst not being susceptible to spoofing attacks.

Noise resilient discrete-time cross-relation sensor characterisation

In-situ characterisation of thermocouple sensors is a challenging problem. Recently the authors presented a blind characterisation technique based on the cross-relation method of blind identification. The method allows in-situ identification of two thermocouple probes, each with a different dynamic response, using only sampled sensor measurement data. While the technique offers certain advantages over alternative methods, including low estimation variance and the ability to compensate for noise induced bias, the robustness of the method is limited by the multimodal nature of the cost function. In this paper, a normalisation term is proposed which improves the convexity of
the cost function. Further, a normalisation and bias compensation hybrid approach is presented that exploits the advantages of both normalisation and bias compensation. It is found that the optimum of the hybrid cost function is less biased and more stable than when only normalisation is applied. All results were verified by simulation.