Lip detection for audio-visual speech recognition in-car environment

Acoustically, car cabins are extremely noisy and as a consequence audio-only, in-car voice recognition systems perform poorly. As the visual modality is immune to acoustic noise, using the visual lip information from the driver is seen as a viable strategy in circumventing this problem by using audio visual automatic speech recognition (AVASR). However, implementing AVASR requires a system being able to accurately locate and track the drivers face and lip area in real-time. In this paper we present such an approach using the Viola-Jones algorithm. Using the AVICAR [1] in-car database, we show that the Viola- Jones approach is a suitable method of locating and tracking the driver’s lips despite the visual variability of illumination and head pose for audio-visual speech recognition system.

Visual localisation in outdoor industrial building environments

This paper presents a vision-based method of vehicle localisation that has been developed and tested on a large forklift type robotic vehicle which operates in a mainly outdoor industrial setting. The localiser uses a sparse 3D edgemap of the environment and a particle filter to estimate the pose of the vehicle. The vehicle operates in dynamic and non-uniform outdoor lighting conditions, an issue that is addressed by using knowledge of the scene to intelligently adjust the camera exposure and hence improve the quality of the information in the image. Results from the industrial vehicle are shown and compared to another laser-based localiser which acts as a ground truth. An improved likelihood metric, using peredge calculation, is presented and has shown to be 40% more accurate in estimating rotation. Visual localization results from the vehicle driving an arbitrary 1.5km path during a bright sunny period show an average position error of 0.44m and rotation error of 0.62deg.

The public sector’s perspective on procuring public works projects : comparing the views of practitioners in Hong Kong and Australia

Hong Kong has been one of the early jurisdictions to adopt Public Private Partnership (PPP) model for delivering large public infrastructure projects. The development of this procurement approach in Hong Kong has followed an intricate path. As such, it is believed that there are a number of areas which are interesting to unveil. As part of a comprehensive research study looking at implementing PPPs, interviews with experienced local industrial practitioners from the public sector were conducted to realize their perspective on the topic of procuring public works projects. Amongst these interviews, fourteen were launched government officials and advisers. The interview findings show that the majority of the Hong Kong and Australian interviewees had previously conducted some kind of research in the field of PPP. Both groups of interviewees agreed that “PPPs gain private sector’s added efficiency/expertise/management skills” when compared to projects procured traditionally. Also, both groups of interviewees felt that projects best suited to use PPP are those that have an “Economic business case”. The interviewees believed that “Contractor’s performance” could be used as key performance indicators for PPP projects. A large number of critical success factors were identified by the interviewees for PPP projects; two of these were similar for both groups of interviewees. These included “Project objectives well defined” and “Partnership spirit/commitment/trust”. Finally it was found that in-house guidance materials were more common in the organizations of the Australian interviewees compared to the Hong Kong ones. This paper studies the views of the public sector towards the topic of PPPs in Hong Kong and Australia, which helps to answer some of the queries that both academics and the private sector in these jurisdictions are keen to know. As a result the private sector can be more prepared when negotiating with the public sector and realise their needs better, academics on the other hand are provided a wider perspective of this topic benefiting the research industry at large.

The researcher’s perspective on procuring public works projects

Purpose: As part of a comprehensive research study looking at implementing PPPs, interviews with experienced researchers were conducted to realize their views on private sector involvement in public works projects. Design / methodology / approach: Amongst these interviews, five were launched with academics from Hong Kong and Australia, and two were conducted with Legislative Councillors of the Hong Kong Special Administration Region (HKSAR) government. Findings: The interview findings show that both Hong Kong and Australian interviewees had previously conducted some kind of research in the field of PPP. The interviewees highlighted that “Different risk profiles” and “Private sector more innovative / efficient” were the main differences between projects that were procured by PPP and traditionally. Other differences include risk transfer. In a PPP arrangement the public sector passes on a substantial amount of the project risks to the private sector, whereas in a traditional case the public sector would take the largest responsibility in bearing these risks. Another common feature of the private sector is that they tend to be more efficient and innovative when compared to the public sector hence their expertise is often reflected in PPP projects. The interviewees agreed that the key performance indicators for PPP projects were unique depending on the individual project. The critical success factors mentioned by both groups of interviewees included “Transparent process”, “Project dependent” and “Market need”. Due to the fact that PPP projects tend to be large scaled costly projects, adequate transparency in the process is necessary in order to demonstrate that a fair selection and tendering process is conducted. A market need for the project is also important to ensure that the project will be financially secure and that the private sector can make a reasonable profit to cover their project expenditure. Originality / value: The findings from this study have enabled a comparative analysis between the views of researchers in two completely different jurisdictions. With the growing popularity to implement PPP projects, it is believed that the results presented in this paper would be of interest to the industry at large.

Biologically inspired visual landmark processing for simultaneous localization and mapping

This paper illustrates a method for finding useful visual landmarks for performing simultaneous localization and mapping (SLAM). The method is based loosely on biological principles, using layers of filtering and pooling to create learned templates that correspond to different views of the environment. Rather than using a set of landmarks and reporting range and bearing to the landmark, this system maps views to poses. The challenge is to produce a system that produces the same view for small changes in robot pose, but provides different views for larger changes in pose. The method has been developed to interface with the RatSLAM system, a biologically inspired method of SLAM. The paper describes the method of learning and recalling visual landmarks in detail, and shows the performance of the visual system in real robot tests.

Building Indigenous research capacity : a personal perspective

The International Network of Indigenous Health Knowledge and Development (INIHKD) Conference was held from Monday 24 May to Friday 28 May 2010 at Kiana Lodge, Port Madison Indian Reservation, Suquamish Nation, Washington State, United States of America. The overall theme for the 4th Biennial Conference was ‘Knowing Our Roots: Indigenous Medicines, Health Knowledges and Best Practices’.

Audio visual automatic speech recognition in vehicles

Acoustically, car cabins are extremely noisy and as a consequence, existing audio-only speech recognition systems, for voice-based control of vehicle functions such as the GPS based navigator, perform poorly. Audio-only speech recognition systems fail to make use of the visual modality of speech (eg: lip movements). As the visual modality is immune to acoustic noise, utilising this visual information in conjunction with an audio only speech recognition system has the potential to improve the accuracy of the system. The field of recognising speech using both auditory and visual inputs is known as Audio Visual Speech Recognition (AVSR). Continuous research in AVASR field has been ongoing for the past twenty-five years with notable progress being made. However, the practical deployment of AVASR systems for use in a variety of real-world applications has not yet emerged. The main reason is due to most research to date neglecting to address variabilities in the visual domain such as illumination and viewpoint in the design of the visual front-end of the AVSR system. In this paper we present an AVASR system in a real-world car environment using the AVICAR database [1], which is publicly available in-car database and we show that the use of visual speech conjunction with the audio modality is a better approach to improve the robustness and effectiveness of voice-only recognition systems in car cabin environments.

Exploring visual features through Gabor representations for facial expression detection

Gabor representations have been widely used in facial analysis (face recognition, face detection and facial expression detection) due to their biological relevance and computational properties. Two popular Gabor representations used in literature are: 1) Log-Gabor and 2) Gabor energy filters. Even though these representations are somewhat similar, they also have distinct differences as the Log-Gabor filters mimic the simple cells in the visual cortex while the Gabor energy filters emulate the complex cells, which causes subtle differences in the responses. In this paper, we analyze the difference between these two Gabor representations and quantify these differences on the task of facial action unit (AU) detection. In our experiments conducted on the Cohn-Kanade dataset, we report an average area underneath the ROC curve (A`) of 92.60% across 17 AUs for the Gabor energy filters, while the Log-Gabor representation achieved an average A` of 96.11%. This result suggests that small spatial differences that the Log-Gabor filters pick up on are more useful for AU detection than the differences in contours and edges that the Gabor energy filters extract.

Cascading appearance-based features for visual voice activity detection

The detection of voice activity is a challenging problem, especially when the level of acoustic noise is high. Most current approaches only utilise the audio signal, making them susceptible to acoustic noise. An obvious approach to overcome this is to use the visual modality. The current state-of-the-art visual feature extraction technique is one that uses a cascade of visual features (i.e. 2D-DCT, feature mean normalisation, interstep LDA). In this paper, we investigate the effectiveness of this technique for the task of visual voice activity detection (VAD), and analyse each stage of the cascade and quantify the relative improvement in performance gained by each successive stage. The experiments were conducted on the CUAVE database and our results highlight that the dynamics of the visual modality can be used to good effect to improve visual voice activity detection performance.

Part 2. Chemical and physical restraints in the management of mechanically ventilated patients in the ICU: A patient perspective

An important goal of the care for the mechanically ventilated patient is to minimize patient discomfort and anxiety. This is partly achieved by frequent use of chemical and physical restraints. The majority of patients in intensive care will receive some form of sedation. The goal and use of sedation has changed considerably over the past few decades with literature evidencing trends toward overall lighter sedation levels and daily interruption of sedation. Conversely, the use of physical restraint for the ventilated patient in ICU differs considerably between nations and continents. A large portion of the literature on the use of physical restraint is from general hospital wards and residential homes, and not from the ICU environment. Recent literature suggests minimal use of physical restraint in the ICU, and that reduction programmes have been initiated. However, very few papers illuminate the patient's experience of physical and chemical restraints as a treatment strategy. In Part 1 of this two-part review, the evidence on chemical and physical restraints was explored with specific focus on definitions of terms, unplanned extubation, agitation, delirium as well as the impact of nurse–patient ratios in the ICU on these issues. This paper, Part 2, examines the evidence related to chemical and physical restraints from the mechanically ventilated patient's perspective.

Wide-angle visual feature matching for outdoor localization

Wide-angle images exhibit significant distortion for which existing scale-space detectors such as the scale-invariant feature transform (SIFT) are inappropriate. The required scale-space images for feature detection are correctly obtained through the convolution of the image, mapped to the sphere, with the spherical Gaussian. A new visual key-point detector, based on this principle, is developed and several computational approaches to the convolution are investigated in both the spatial and frequency domain. In particular, a close approximation is developed that has comparable computation time to conventional SIFT but with improved matching performance. Results are presented for monocular wide-angle outdoor image sequences obtained using fisheye and equiangular catadioptric cameras. We evaluate the overall matching performance (recall versus 1-precision) of these methods compared to conventional SIFT. We also demonstrate the use of the technique for variable frame-rate visual odometry and its application to place recognition.

Mobile robot navigation as a planar visual servoing problem

This paper demonstrates some interesting connections between the hitherto disparate fields of mobile robot navigation and image-based visual servoing. A planar formulation of the well-known image-based visual servoing method leads to a bearing-only navigation system that requires no explicit localization and directly yields desired velocity. The well known benefits of image-based visual servoing such as robustness apply also to the planar case. Simulation results are presented.

Dynamic Image-based visual servo control using centroid and optic flow features

This paper considers the question of designing a fully image-based visual servo control for a class of dynamic systems. The work is motivated by the ongoing development of image-based visual servo control of small aerial robotic vehicles. The kinematics and dynamics of a rigid-body dynamical system (such as a vehicle airframe) maneuvering over a flat target plane with observable features are expressed in terms of an unnormalized spherical centroid and an optic flow measurement. The image-plane dynamics with respect to force input are dependent on the height of the camera above the target plane. This dependence is compensated by introducing virtual height dynamics and adaptive estimation in the proposed control. A fully nonlinear adaptive control design is provided that ensures asymptotic stability of the closed-loop system for all feasible initial conditions. The choice of control gains is based on an analysis of the asymptotic dynamics of the system. Results from a realistic simulation are presented that demonstrate the performance of the closed-loop system. To the author's knowledge, this paper documents the first time that an image-based visual servo control has been proposed for a dynamic system using vision measurement for both position and velocity.

Visual motion estimation for an autonomous underwater reef monitoring robot

Performing reliable localisation and navigation within highly unstructured underwater coral reef environments is a difficult task at the best of times. Typical research and commercial underwater vehicles use expensive acoustic positioning and sonar systems which require significant external infrastructure to operate effectively. This paper is focused on the development of a robust vision-based motion estimation technique using low-cost sensors for performing real-time autonomous and untethered environmental monitoring tasks in the Great Barrier Reef without the use of acoustic positioning. The technique is experimentally shown to provide accurate odometry and terrain profile information suitable for input into the vehicle controller to perform a range of environmental monitoring tasks.

Visual motion estimation for an autonomous underwater reef monitoring robot

Performing reliable localisation and navigation within highly unstructured underwater coral reef environments is a difficult task at the best of times. Typical research and commercial underwater vehicles use expensive acoustic positioning and sonar systems which require significant external infrastructure to operate effectively. This paper is focused on the development of a robust vision-based motion estimation technique using low-cost sensors for performing real-time autonomous and untethered environmental monitoring tasks in the Great Barrier Reef without the use of acoustic positioning. The technique is experimentally shown to provide accurate odometry and terrain profile information suitable for input into the vehicle controller to perform a range of environmental monitoring tasks.