Improving visual noise insensitivity in small vocabulary audio visual speech recognition applications

Visual noise insensitivity is important to audio visual speech recognition (AVSR). Visual noise can take on a number of forms such as varying frame rate, occlusion, lighting or speaker variabilities. The use of a high dimensional secondary classifier on the word likelihood scores from both the audio and video modalities is investigated for the purposes of adaptive fusion. Preliminary results are presented demonstrating performance above the catastrophic fusion boundary for our confidence measure irrespective of the type of visual noise presented to it. Our experiments were restricted to small vocabulary applications.

Speech recognition in adverse environments using lip information

The performance of automatic speech recognition systems deteriorates in the presence of noise. One known solution is to incorporate video information with an existing acoustic speech recognition system. We investigate the performance of the individual acoustic and visual sub-systems and then examine different ways in which the integration of the two systems may be performed. The system is to be implemented in real time on a Texas Instruments' TMS320C80 DSP.

Robust speaker verification via fusion of speech and lip modalities

This paper investigates the use of lip information, in conjunction with speech information, for robust speaker verification in the presence of background noise. It has been previously shown in our own work, and in the work of others, that features extracted from a speaker's moving lips hold speaker dependencies which are complementary with speech features. We demonstrate that the fusion of lip and speech information allows for a highly robust speaker verification system which outperforms the performance of either sub-system. We present a new technique for determining the weighting to be applied to each modality so as to optimize the performance of the fused system. Given a correct weighting, lip information is shown to be highly effective for reducing the false acceptance and false rejection error rates in the presence of background noise

The use of temporal speech and lip information for multi-modal speaker identification via multi-stream HMMs

Investigates the use of temporal lip information, in conjunction with speech information, for robust, text-dependent speaker identification. We propose that significant speaker-dependent information can be obtained from moving lips, enabling speaker recognition systems to be highly robust in the presence of noise. The fusion structure for the audio and visual information is based around the use of multi-stream hidden Markov models (MSHMM), with audio and visual features forming two independent data streams. Recent work with multi-modal MSHMMs has been performed successfully for the task of speech recognition. The use of temporal lip information for speaker identification has been performed previously (T.J. Wark et al., 1998), however this has been restricted to output fusion via single-stream HMMs. We present an extension to this previous work, and show that a MSHMM is a valid structure for multi-modal speaker identification

The use of speech and lip modalities for robust speaker verification under adverse conditions

Investigates the use of lip information, in conjunction with speech information, for robust speaker verification in the presence of background noise. We have previously shown (Int. Conf. on Acoustics, Speech and Signal Proc., vol. 6, pp. 3693-3696, May 1998) that features extracted from a speaker's moving lips hold speaker dependencies which are complementary with speech features. We demonstrate that the fusion of lip and speech information allows for a highly robust speaker verification system which outperforms either subsystem individually. We present a new technique for determining the weighting to be applied to each modality so as to optimize the performance of the fused system. Given a correct weighting, lip information is shown to be highly effective for reducing the false acceptance and false rejection error rates in the presence of background noise

Can audio-visual speech recognition outperform acoustically enhanced speech recognition in automotive environment?

The use of visual features in the form of lip movements to improve the performance of acoustic speech recognition has been shown to work well, particularly in noisy acoustic conditions. However, whether this technique can outperform speech recognition incorporating well-known acoustic enhancement techniques, such as spectral subtraction, or multi-channel beamforming is not known. This is an important question to be answered especially in an automotive environment, for the design of an efficient human-vehicle computer interface. We perform a variety of speech recognition experiments on a challenging automotive speech dataset and results show that synchronous HMM-based audio-visual fusion can outperform traditional single as well as multi-channel acoustic speech enhancement techniques. We also show that further improvement in recognition performance can be obtained by fusing speech-enhanced audio with the visual modality, demonstrating the complementary nature of the two robust speech recognition approaches.

Psychoanalytic perspectives on substance use and antisocial personality disorder

Psychoanalysis and related psychodynamic psychotherapies have historically had a limited engagement with substance use and antisocial personality disorders. This in part reflects an early preoccupation with ‘transference neuroses’ and in part reflects later de-emphasis of diagnosis and focus on therapeutic process. Nonetheless, psychoanalytic perspectives can usefully inform thinking about approaches to treatment of such disorders and there are psychoanalytic constructs that have specific relevance to their treatment. This paper reviews some prominent strands of psychoanalytic thinking as they pertain to the treatment of substance abuse and antisocial personality disorders. It is argued that, while Freudian formulations lead to a primarily pessimistic view of the prospect of treatment of such disorders, both the British object relations and the North American self psychology traditions suggest potentially productive approaches. Finally the limited empirical evidence from brief psychodynamically informed treatments of substance use disorders is reviewed. It is concluded that such treatments are not demonstrably effective but that, since no form of psychotherapy has established high efficacy with substance use disorders, brief psychdynamic therapies are not necessarily of lesser value than other treatments and may have specific value for particular individuals and in particular treatment contexts.

Work-related subjective experiences among community residents with schizophrenia or schizoaffective disorder

Objective: To develop a self-report scale of subjective experiences of illness perceived to impact on employment functioning, as an alternative to a diagnostic perspective, for anticipating the vocational assistance needs of people with schizophrenia or schizoaffective disorders. Method: A repeated measures pilot study (n1 = 26, n2 = 21) of community residents with schizophrenia identified a set of work-related subjective experiences perceived to impact on employment functioning. Items with the best psychometric properties were applied in a 12 month longitudinal survey of urban residents with schizophrenia or schizoaffective disorder (n1 = 104; n2 = 94; n3 = 94). Results: Construct validity, factor structure, responsiveness, internal consistency, stability, and criterion validity investigations produced favourable results. Work-related subjective experiences provide information about the intersection of the person, the disorder, and expectations of employment functioning, which suggest new opportunities for vocational professionals to explore and discuss individual assistance needs. Conclusion: Further psychometric investigations of test-retest reliability, discriminant and predictive validity, and research applications in supported employment and vocational rehabilitation, are recommended. Subject to adequate psychometric properties, the new measure promises to facilitate exploring: individuals' specific subjective experiences; how each is perceived to contribute to employment restrictions; and the corresponding implications for specialized treatment, vocational interventions and workplace accommodations.

An Investigation of Organic Factors in the Neuropsychological Functioning of Patients with Borderline Personality Disorder

The hypothesis to be tested in this study was that the cognitive deficits that have been documented in patients with Borderline Personality Disorder (BPD) are largely the consequence of organic insult, either developmental or acquired. Using a cross–sectional design, 80 subjects (males and females) who met the criteria for BPD participated in the study. They completed a battery of neuropsychological tests and a comprehensive interview assessing organic status as well as measures of the potentially confounding factors of current levels of depression and anxiety. It was expected that BPD-patients with a probable history of organic insult would perform significantly worse than would BPD patients without such a history. Analyses of the results provided partial support for the hypothesis. Subjects with both BPD and a history of organic insult were significantly more impaired on several measures including measures of attention than were BPD only subjects. The results suggested that the impaired cognitive performance of persons diagnosed with BPD may, in part, be attributed to organic factors.