Improving visual noise insensitivity in small vocabulary audio visual speech recognition applications

Visual noise insensitivity is important to audio visual speech recognition (AVSR). Visual noise can take on a number of forms such as varying frame rate, occlusion, lighting or speaker variabilities. The use of a high dimensional secondary classifier on the word likelihood scores from both the audio and video modalities is investigated for the purposes of adaptive fusion. Preliminary results are presented demonstrating performance above the catastrophic fusion boundary for our confidence measure irrespective of the type of visual noise presented to it. Our experiments were restricted to small vocabulary applications.

Can audio-visual speech recognition outperform acoustically enhanced speech recognition in automotive environment?

The use of visual features in the form of lip movements to improve the performance of acoustic speech recognition has been shown to work well, particularly in noisy acoustic conditions. However, whether this technique can outperform speech recognition incorporating well-known acoustic enhancement techniques, such as spectral subtraction, or multi-channel beamforming is not known. This is an important question to be answered especially in an automotive environment, for the design of an efficient human-vehicle computer interface. We perform a variety of speech recognition experiments on a challenging automotive speech dataset and results show that synchronous HMM-based audio-visual fusion can outperform traditional single as well as multi-channel acoustic speech enhancement techniques. We also show that further improvement in recognition performance can be obtained by fusing speech-enhanced audio with the visual modality, demonstrating the complementary nature of the two robust speech recognition approaches.

Safety enhancement of operator protection systems on self-propelled mining equipment

In the long term, with development of skill, knowledge, exposure and confidence within the engineering profession, rigorous analysis techniques have the potential to become a reliable and far more comprehensive method for design and verification of the structural adequacy of OPS, write Nimal J Perera, David P Thambiratnam and Brian Clark. This paper explores the potential to enhance operator safety of self-propelled mechanical plant subjected to roll over and impact of falling objects using the non-linear and dynamic response simulation capabilities of analytical processes to supplement quasi-static testing methods prescribed in International and Australian Codes of Practice for bolt on Operator Protection Systems (OPS) that are post fitted. The paper is based on research work carried out by the authors at the Queensland University of Technology (QUT) over a period of three years by instrumentation of prototype tests, scale model tests in the laboratory and rigorous analysis using validated Finite Element (FE) Models. The FE codes used were ABAQUS for implicit analysis and LSDYNA for explicit analysis. The rigorous analysis and dynamic simulation technique described in the paper can be used to investigate the structural response due to accident scenarios such as multiple roll over, impact of multiple objects and combinations of such events and thereby enhance the safety and performance of Roll Over and Falling Object Protection Systems (ROPS and FOPS). The analytical techniques are based on sound engineering principles and well established practice for investigation of dynamic impact on all self propelled vehicles. They are used for many other similar applications where experimental techniques are not feasible.

Patterns and transitions of query reformulation during web searching

Purpose – To investigate and identify the patterns of interaction between searchers and search engine during web searching. Design/methodology/approach – The authors examined 2,465,145 interactions from 534,507 users of Dogpile.com submitted on May 6, 2005, and compared query reformulation patterns. They investigated the type of query modifications and query modification transitions within sessions. Findings – The paper identifies three strong query reformulation transition patterns: between specialization and generalization; between video and audio, and between content change and system assistance. In addition, the findings show that web and images content were the most popular media collections. Originality/value – This research sheds light on the more complex aspects of web searching involving query modifications.

The Profiling of Property, Plant & Equipment (PPE) Contributions in Australia and Malaysia Public Listed Construction Companies

The purpose of this paper is to study the profiling of property, plant and equipment (PPE) contributions in Australia and Malaysia construction companies. A company’s worth is usually based on the listed share price on the stock exchange. In arriving at the net profit, the contribution of PPE in the company’s assets is somehow being neglected. This paper will investigate the followings; firstly the level of PPE contribution in the construction firms by comparing the PPE contributions to the company’s asset as a whole which includes fixed (non-current) assets and current assets. This will determine the true strength of the companies, rather than relying on the share prices alone. Secondly, the paper will determine the trend of company’s asset ownership to show the company’s performance of the PPE ownership during the period of study. The data is based on the selected construction companies listed on the Australian Stock Exchange (ASX) and Malaysian Stock Exchange, known as Bursa Malaysia. The profiling will help to determine the strength of the construction firms based on the PPE holding, and the level of PPE ownerships in the two countries construction firms during the period of study.

Learnability and discriminability of melodic medical equipment alarms

Melodic alarms proposed in the IEC 60601-1-8 standard for medical electrical equipment were tested for learnability and discriminability. Thirty-three non-anaesthetist participants learned the alarms over two sessions of practice, with or without mnemonics suggested in the standard. Fewer than 30% of participants could identify the alarms with 100% accuracy at the end of practice. Confusions persisted between pairs of alarms, especially if mnemonics were used during learning (p = 0.011). Participants responded faster (p < 0.00001) and more accurately (p = 0.002) to medium priority alarms than to high priority alarms, even though they rated the high priority alarms as sounding more urgent (p < 0.00001). Participants with at least 1 year of formal musical training identified the alarms more accurately (p = 0.0002) than musically untrained participants, and found the task easier overall (p < 0.00001). More intensive studies of the IEC 60601-1-8 alarms are needed for their effectiveness to be determined.

An audio wiki supporting mobile collaboration

Wikis have proved to be very effective collaboration and knowledge management tools in large variety of fields thanks to their simplicity and flexible nature. Another important development for the internet is the emergence of powerful mobile devices supported by fast and reliable wireless networks. The combination of these developments begs the question of how to extend wikis on mobile devices and how to leverage mobile devices' rich modalities to supplement current wikis. Realizing that composing and consuming through auditory channel is the most natural and efficient way for mobile device user, this paper explores the use of audio as the medium of wiki. Our work, as the first step towards this direction, creates a framework called Mobile Audio Wiki which facilitates asynchronous audio-mediated collaboration on the move. In this paper, we present the design of Mobile Audio Wiki. As a part of such design, we propose an innovative approach for a light-weight audio content annotation system for enabling group editing, versioning and cross-linking among audio clips. To elucidate the novel collaboration model introduced by Mobile Audio Wiki, its four usage modes are identified and presented in storyboard format. Finally, we describe the initial design for presentation and navigation of Mobile Audio Wiki.

Multiple cameras for audio-visual speech recognition in an automotive environment

Audio-visualspeechrecognition, or the combination of visual lip-reading with traditional acoustic speechrecognition, has been previously shown to provide a considerable improvement over acoustic-only approaches in noisy environments, such as that present in an automotive cabin. The research presented in this paper will extend upon the established audio-visualspeechrecognition literature to show that further improvements in speechrecognition accuracy can be obtained when multiple frontal or near-frontal views of a speaker's face are available. A series of visualspeechrecognition experiments using a four-stream visual synchronous hidden Markov model (SHMM) are conducted on the four-camera AVICAR automotiveaudio-visualspeech database. We study the relative contribution between the side and central orientated cameras in improving visualspeechrecognition accuracy. Finally combination of the four visual streams with a single audio stream in a five-stream SHMM demonstrates a relative improvement of over 56% in word recognition accuracy when compared to the acoustic-only approach in the noisiest conditions of the AVICAR database.

Automatic Audio Segmentation Using the Generalized Likelihood Ratio

This paper presents a novel technique for segmenting an audio stream into homogeneous regions according to speaker identities, background noise, music, environmental and channel conditions. Audio segmentation is useful in audio diarization systems, which aim to annotate an input audio stream with information that attributes temporal regions of the audio into their specific sources. The segmentation method introduced in this paper is performed using the Generalized Likelihood Ratio (GLR), computed between two adjacent sliding windows over preprocessed speech. This approach is inspired by the popular segmentation method proposed by the pioneering work of Chen and Gopalakrishnan, using the Bayesian Information Criterion (BIC) with an expanding search window. This paper will aim to identify and address the shortcomings associated with such an approach. The result obtained by the proposed segmentation strategy is evaluated on the 2002 Rich Transcription (RT-02) Evaluation dataset, and a miss rate of 19.47% and a false alarm rate of 16.94% is achieved at the optimal threshold.

Disaster strikes, then what? Using evaluation in narrative driven (oral history & digital storytelling) community-based projects

In 2011 Queensland suffered both floods and cyclones, leaving residents without homes and their communities in ruins (2011). This paper presents how researchers from QUT, who are also members of the Oral History Association of Australia (OHAA) Queensland’s chapter, are using oral history, photographs, videography and digital storytelling to help heal and empower rural communities around the state and how evaluation has become a key element of our research. QUT researchers ran storytelling workshops in the capital city of Brisbane i early 2011, after the city suffered sever flooding. Cyclone Yasi then struck the town of Cardwell (in February 2011) destroying their historical museum and recording equipment. We delivered an 'emergency workshop', offering participants hands on use of the equipment, ethical and interviewing theory, so that the community could start to build a new collection. We included oral history workshops as well as sessions on how best to use a video camera, digital camera and creative writing sessions, so the community would also know how to make 'products' or exhibition pieces out of the interviews they were recording. We returned six months later to conduct follow-up workshops and the material produced by and with the community had been amazing. More funding has now been secured to replicate audio/visual/writing workshops in other remote rural Queensland communities including Townsville, Mackay and Cunnamulla and Toowoomba in 2012, highlighting the need for a multi media approach, to leverage the most out of OH interviews as a mechanism to restore and promote community resilience and pride.

Downtime model development for construction equipment management

Downtime (DT) caused by non-availability of equipment and equipment breakdown has non-trivial impact on the performance of construction projects. Earlier research has often addressed this fact, but it has rarely explained the causes and consequences of DT – especially in the context of developing countries. This paper presents a DT model to address this issue. Using this model, the generic factors and processes related to DT are identified, and the impact of DT is quantified. By applying the model framework to nine road projects in Nepal, the impact of DT is explored in terms of its duration and cost. The research findings highlight how various factors and processes interact with each other to create DT, and mitigate or exacerbate its impact on project performance. It is suggested that construction companies need to adopt proactive equipment management and maintenance programs to minimize the impact of DT.

Factors associated with suboptimal adherence to antiretroviral therapy in Viet Nam: A cross-sectional study using audio computer-assisted self-interview (ACASI)

Background: Optimal adherence to antiretroviral therapy (ART) is necessary for people living with HIV/AIDS (PLHIV). There have been relatively few systematic analyses of factors that promote or inhibit adherence to antiretroviral therapy among PLHIV in Asia. This study assessed ART adherence and examined factors associated with suboptimal adherence in northern Viet Nam. Methods: Data from 615 PLHIV on ART in two urban and three rural outpatient clinics were collected by medical record extraction and from patient interviews using audio computer-assisted self-interview (ACASI). Results: The prevalence of suboptimal adherence was estimated to be 24.9% via a visual analogue scale (VAS) of past-month dose-missing and 29.1% using a modified Adult AIDS Clinical Trial Group scale for on-time dose-taking in the past 4 days. Factors significantly associated with the more conservative VAS score were: depression (p < 0.001), side-effect experiences (p < 0.001), heavy alcohol use (p = 0.001), chance health locus of control (p = 0.003), low perceived quality of information from care providers (p = 0.04) and low social connectedness (p = 0.03). Illicit drug use alone was not significantly associated with suboptimal adherence, but interacted with heavy alcohol use to reduce adherence (p < 0.001). Conclusions: This is the largest survey of ART adherence yet reported from Asia and the first in a developing country to use the ACASI method in this context. The evidence strongly indicates that ART services in Viet Nam should include screening and treatment for depression, linkage with alcohol and/or drug dependence treatment, and counselling to address the belief that chance or luck determines health outcomes.

A novel representation of bioacoustic events for content-based search in field audio data

Bioacoustic data can provide an important base for environmental monitoring. To explore a large amount of field recordings collected, an automated similarity search algorithm is presented in this paper. A region of an audio defined by frequency and time bounds is provided by a user; the content of the region is used to construct a query. In the retrieving process, our algorithm will automatically scan through recordings to search for similar regions. In detail, we present a feature extraction approach based on the visual content of vocalisations – in this case ridges, and develop a generic regional representation of vocalisations for indexing. Our feature extraction method works best for bird vocalisations showing ridge characteristics. The regional representation method allows the content of an arbitrary region of a continuous recording to be described in a compressed format.