Speech recognition in adverse environments using lip information

The performance of automatic speech recognition systems deteriorates in the presence of noise. One known solution is to incorporate video information with an existing acoustic speech recognition system. We investigate the performance of the individual acoustic and visual sub-systems and then examine different ways in which the integration of the two systems may be performed. The system is to be implemented in real time on a Texas Instruments' TMS320C80 DSP.

Robust speaker verification via fusion of speech and lip modalities

This paper investigates the use of lip information, in conjunction with speech information, for robust speaker verification in the presence of background noise. It has been previously shown in our own work, and in the work of others, that features extracted from a speaker's moving lips hold speaker dependencies which are complementary with speech features. We demonstrate that the fusion of lip and speech information allows for a highly robust speaker verification system which outperforms the performance of either sub-system. We present a new technique for determining the weighting to be applied to each modality so as to optimize the performance of the fused system. Given a correct weighting, lip information is shown to be highly effective for reducing the false acceptance and false rejection error rates in the presence of background noise

The use of temporal speech and lip information for multi-modal speaker identification via multi-stream HMMs

Investigates the use of temporal lip information, in conjunction with speech information, for robust, text-dependent speaker identification. We propose that significant speaker-dependent information can be obtained from moving lips, enabling speaker recognition systems to be highly robust in the presence of noise. The fusion structure for the audio and visual information is based around the use of multi-stream hidden Markov models (MSHMM), with audio and visual features forming two independent data streams. Recent work with multi-modal MSHMMs has been performed successfully for the task of speech recognition. The use of temporal lip information for speaker identification has been performed previously (T.J. Wark et al., 1998), however this has been restricted to output fusion via single-stream HMMs. We present an extension to this previous work, and show that a MSHMM is a valid structure for multi-modal speaker identification

The use of speech and lip modalities for robust speaker verification under adverse conditions

Investigates the use of lip information, in conjunction with speech information, for robust speaker verification in the presence of background noise. We have previously shown (Int. Conf. on Acoustics, Speech and Signal Proc., vol. 6, pp. 3693-3696, May 1998) that features extracted from a speaker's moving lips hold speaker dependencies which are complementary with speech features. We demonstrate that the fusion of lip and speech information allows for a highly robust speaker verification system which outperforms either subsystem individually. We present a new technique for determining the weighting to be applied to each modality so as to optimize the performance of the fused system. Given a correct weighting, lip information is shown to be highly effective for reducing the false acceptance and false rejection error rates in the presence of background noise

Microscopic and spectroscopic investigation of poly(3-hexylthiophene) interaction with carbon nanotubes

The inclusion of carbon nanotubes in polymer matrix has been proposed to enhance the polymer’s physical and electrical properties. In this study, microscopic and spectroscopic techniques are used to investigate the interaction between poly(3-hexylthiophene) (P3HT) and nanotubes and the reciprocal modification of physical properties. The presence of P3HT-covered nanotubes dispersed in the polymer matrix has been observed by atomic force microscopy and transmission electron microscopy. Then, the modification of P3HT optical properties due to nanotube inclusion has been evidenced with spectroscopic techniques like absorption and Raman spectroscopy. The study is completed with detailed nanoscale analysis by scanning probe techniques. The ordered self assembly of polymer adhering on the nanotube is unveiled by showing an example of helical wrapping of P3HT. Scanning tunneling spectroscopy study provides information on the electronic structure of nanotube-polymer assembly, revealing the charge transfer from P3HT to the nanotube.

Can audio-visual speech recognition outperform acoustically enhanced speech recognition in automotive environment?

The use of visual features in the form of lip movements to improve the performance of acoustic speech recognition has been shown to work well, particularly in noisy acoustic conditions. However, whether this technique can outperform speech recognition incorporating well-known acoustic enhancement techniques, such as spectral subtraction, or multi-channel beamforming is not known. This is an important question to be answered especially in an automotive environment, for the design of an efficient human-vehicle computer interface. We perform a variety of speech recognition experiments on a challenging automotive speech dataset and results show that synchronous HMM-based audio-visual fusion can outperform traditional single as well as multi-channel acoustic speech enhancement techniques. We also show that further improvement in recognition performance can be obtained by fusing speech-enhanced audio with the visual modality, demonstrating the complementary nature of the two robust speech recognition approaches.

A review of locative media, mobile and embodied spatial interaction

Highlights ► Provides a review of the history and development of locative media. ► Outlines different human-computer interaction techniques applied in locative media. ► Discusses how locative media applications have changed interaction affordances in and of physical spaces. ► Discusses practices of people in urban settings that evolved through these new affordances. ► Provides an overview on methods to investigate and elaborate design principles for future locative media.

Bending and bundling of metal-free vertically aligned ZnO nanowires due to electrostatic interaction

Bending and bundling was observed from vertically aligned arrays of ZnO nanowires with flat (0001) top surfaces, which were synthesized using a vapor-phase method without metal catalysts. Sufficient evidence was found to exclude electron-beam bombardment during scanning electron microscopy as a cause for bending and bundling. We attribute the bending and bundling to electrostatic interactions due to charged (0001) polar surfaces, and also discussed the threshold surface charge densities for the bending and bundling based on a simple cantilever-bending model. Some growth features were indicative of the operation of electrostatic interactions during the growth.

Aggravation of ADAMTS and MMP production and ERK1/2 pathway in the interaction of osteoarthritic subchondral bone osteoblasts and articular cartilage chondrocytes : a possible pathogenic role in osteoarthritis

Introduction: Degradative enzymes, such as A disintegrin and metalloproteinase with thrombospondin motifs (ADAMTS) and matrix metalloproteinases (MMPs), play key roles in osteoarthritis (OA) development. The aim of the present study was to investigate if cross-talk between subchondral bone osteoblasts (SBOs) and articular cartilage chondrocytes (ACCs) in OA alters the expression and regulation of ADAMTS5, ADAMTS4, MMP-1, MMP-2, MMP-3, MMP-8, MMP-9 and MMP-13, and also to test the possible involvement of mitogen activated protein kinase (MAPK) signaling pathway during this process. Methods: ACCs and SBOs were isolated from normal and OA patients. An in vitro co-culture model was developed to study the regulation of ADAMTS and MMPs under normal and OA joint cross-talk conditions. MAPK-ERK inhibitor, PD98059 was applied to delineate the involvement of specific pathway during this interaction process. Results: Indirect co-culture of OA SBOs with normal ACCs resulted in significantly increased expression of ADAMTS5, ADAMTS4, MMP-2, MMP-3 and MMP-9 in ACCs, whereas co-culture of OA ACCs led to increased MMP-1 and MMP-2 expression in normal SBOs. The upregulation of ADAMTS and MMPs under these conditions was correlated with activation of the MAPK-ERK1/2 signaling pathway and the addition of the MAPK-ERK inhibitor, PD98059, reversed the overexpression of ADAMTS and MMPs in co-cultures. Conclusion: In summary, we believe, these results add to the evidence that in human OA, altered bi-directional signals transmitted between SBOs and ACCs significantly impacts the critical features of both cartilage and bone by producing abnormal levels of ADAMTS and MMPs. Furthermore, we have demonstrated for the first time that this altered cross-talk was mediated by the phosphorylation of MAPK-ERK1/2 signaling pathway.

Mechanism of interaction of hydrocalumites (Ca/Al-LDH) with methyl orange and acidic scarlet GR

The development of new materials for water purification is of universal importance. Among these types of materials are layered double hydroxides (LDHs). Non-ionic materials pose a significant problem as pollutants. The interaction of methyl orange (MO) and acidic scarlet GR (GR) adsorption on hydrocalumite (Ca/Al-LDH-Cl) were studied by X-ray diffraction (XRD), infrared spectroscopy (MIR), scanning electron microscope (SEM) and near-infrared spectroscopy (NIR). The XRD results revealed that the basal spacing of Ca/Al-LDH-MO was expanded to 2.45 nm, and the MO molecules were intercalated with a inter-penetrating bilayer model in the gallery of LDH, with 49o tilting angle. Yet Ca/Al-LDH-GR was kept the same d-value as Ca/Al-LDH-Cl. The NIR spectrum for Ca/Al-LDH-MO showed a prominent band around 5994 cm-1, assigned to the combination result of the N-H stretching vibrations, which was considered as a mark to assess MO- ion intercalation into Ca/Al-LDH-Cl interlayers. From SEM images, the particle morphology of Ca/Al-LDH-MO mainly changed to irregular platelets, with a “honey-comb” like structure. Yet the Ca/Al-LDH-GR maintained regular hexagons platelets, which was similar to that of Ca/Al-LDH-Cl. All results indicated that MO- ion was intercalated into Ca/Al-LDH-Cl interlayers, and acidic scarlet GR was only adsorped upon Ca/Al-LDH-Cl surfaces.

Detecting gesture force peaks for intuitive interaction

With the release of the Nintendo Wii in 2006, the use of haptic force gestures has become a very popular form of input for interactive entertainment. However, current gesture recognition techniques utilised in Nintendo Wii games fall prey to a lack of control when it comes to recognising simple gestures. This paper presents a simple gesture recognition technique called Peak Testing which gives greater control over gesture interaction. This recognition technique locates force peaks in continuous force data (provided by a gesture device such as the Wiimote) and then cancels any peaks which are not meant for input. Peak Testing is therefore technically able to identify movements in any direction. This paper applies this recognition technique to control virtual instruments and investigates how users respond to this interaction. The technique is then explored as the basis for a robust way to navigate menus with a simple flick of the wrist. We propose that this flick-form of interaction could be a very intuitive way to navigate Nintendo Wii menus instead of the current pointer techniques implemented.

Bridging the representation and interaction challenges of mobile context-aware computing: designing agile ridesharing

The increasing capability of mobile devices and social networks to gather contextual and social data has led to increased interest in context-aware computing for mobile applications. This paper explores ways of reconciling two different viewpoints of context, representational and interactional, that have arisen respectively from technical and social science perspectives on context-aware computing. Through a case study in agile ridesharing, the importance of dynamic context control, historical context and broader context is discussed. We build upon earlier work that has sought to address the divide by further explicating the problem in the mobile context and expanding on the design approaches.

That's what governments do : exploring fundamental barriers to public–philanthropic interaction : the example of indigenous well-being

Decades of intervention have made variable impact on the inequality between indigenous and non-indigenous well-being across the world. Unacceptable differences in life expectancy alone mark indigenous need as an area where greater understanding of public and private funding approaches and their interaction may deliver real benefits. Both the public and the third sector have been active in trying to address the disadvantage experienced by Australia’s indigenous people. The interaction between the indigenous cause philanthropy system and the wider geopolitical landscape in Australia is revealing barriers and insights that may apply in other challenging policy terrain. The research reported here draws upon two empirical studies aimed at understanding the issues facing philanthropy in Australia,including the impact of government agency both independently and as it contrasts with philanthropy. The two different cultures are evident and two levers (greater system flexibility and closer engagement) are suggested as important in moving forward the philanthropy/government relationship in this area.

Physical characteristics of small ions and their interaction with ultrafine particles in the urban environment

In recent years, the effect of ions and ultrafine particles on ambient air quality and human health has been well documented, however, knowledge about their sources, concentrations and interactions within different types of urban environments remains limited. This thesis presents the results of numerous field studies aimed at quantifying variations in ion concentration with distance from the source, as well as identifying the dynamics of the particle ionisation processes which lead to the formation of charged particles in the air. In order to select the most appropriate measurement instruments and locations for the studies, a literature review was also conducted on studies that reported ion and ultrafine particle emissions from different sources in a typical urban environment. The initial study involved laboratory experiments on the attachment of ions to aerosols, so as to gain a better understanding of the interaction between ions and particles. This study determined the efficiency of corona ions at charging and removing particles from the air, as a function of different particle number and ion concentrations. The results showed that particle number loss was directly proportional to particle charge concentration, and that higher small ion concentrations led to higher particle deposition rates in all size ranges investigated. Nanoparticles were also observed to decrease with increasing particle charge concentration, due to their higher Brownian mobility and subsequent attachment to charged particles. Given that corona discharge from high voltage powerlines is considered one of the major ion sources in urban areas, a detailed study was then conducted under three parallel overhead powerlines, with a steady wind blowing in a perpendicular direction to the lines. The results showed that large sections of the lines did not produce any corona at all, while strong positive emissions were observed from discrete components such as a particular set of spacers on one of the lines. Measurements were also conducted at eight upwind and downwind points perpendicular to the powerlines, spanning a total distance of about 160m. The maximum positive small and large ion concentrations, and DC electric field were observed at a point 20 m downwind from the lines, with median values of 4.4×103 cm-3, 1.3×103 cm-3 and 530 V m-1, respectively. It was estimated that, at this point, less than 7% of the total number of particles was charged. The electrical parameters decreased steadily with increasing downwind distance from the lines but remained significantly higher than background levels at the limit of the measurements. Moreover, vehicles are one of the most prevalent ion and particle emitting sources in urban environments, and therefore, experiments were also conducted behind a motor vehicle exhaust pipe and near busy motorways, with the aim of quantifying small ion and particle charge concentration, as well as their distribution as a function of distance from the source. The study found that approximately equal numbers of positive and negative ions were observed in the vehicle exhaust plume, as well as near motorways, of which heavy duty vehicles were believed to be the main contributor. In addition, cluster ion concentration was observed to decrease rapidly within the first 10-15 m from the road and ion-ion recombination and ion-aerosol attachment were the most likely cause of ion depletion, rather than dilution and turbulence related processes. In addition to the above-mentioned dominant ion sources, other sources also exist within urban environments where intensive human activities take place. In this part of the study, airborne concentrations of small ions, particles and net particle charge were measured at 32 different outdoor sites in and around Brisbane, Australia, which were classified into seven different groups as follows: park, woodland, city centre, residential, freeway, powerlines and power substation. Whilst the study confirmed that powerlines, power substations and freeways were the main ion sources in an urban environment, it also suggested that not all powerlines emitted ions, only those with discrete corona discharge points. In addition to the main ion sources, higher ion concentrations were also observed environments affected by vehicle traffic and human activities, such as the city centre and residential areas. A considerable number of ions were also observed in a woodland area and it is still unclear if they were emitted directly from the trees, or if they originated from some other local source. Overall, it was found that different types of environments had different types of ion sources, which could be classified as unipolar or bipolar particle sources, as well as ion sources that co-exist with particle sources. In general, fewer small ions were observed at sites with co-existing sources, however particle charge was often higher due to the effect of ion-particle attachment. In summary, this study quantified ion concentrations in typical urban environments, identified major charge sources in urban areas, and determined the spatial dispersion of ions as a function of distance from the source, as well as their controlling factors. The study also presented ion-aerosol attachment efficiencies under high ion concentration conditions, both in the laboratory and in real outdoor environments. The outcomes of these studies addressed the aims of this work and advanced understanding of the charge status of aerosols in the urban environment.

An actor-network research frame for analysing complex socio-technical situations

This paper presents a “research frame” which we have found useful in analyzing complex socio- technical situations. The research frame is based on aspects of actor-network theory: “interressment”, “enrollment”, “points of passage” and the “trial of strength”. Each of these aspects are described in turn, making clear their purpose in the overall research frame. Having established the research frame it is used to analyse two examples. First, the use of speech recognition technology is examined in two different contexts, showing how to apply the frame to compare and contrast current situations. Next, a current medical consultation context is described and the research frame is used to consider how it could change with innovative technology. In both examples, the research frame shows that the use of an artefact or technology must be considered together with the context in which it is used.