Hidden Conditional Random Fields for Visual Speech Recognition

In this paper we present the application of Hidden Conditional Random Fields (HCRFs) to modelling speech for visual speech recognition. HCRFs may be easily adapted to model long range dependencies across an observation sequence. As a result visual word recognition performance can be improved as the model is able to take more of a contextual approach to generating state sequences. Results are presented from a speaker-dependent, isolated digit, visual speech recognition task using comparisons with a baseline HMM system. We firstly illustrate that word recognition rates on clean video using HCRFs can be improved by increasing the number of past and future observations being taken into account by each state. Secondly we compare model performances using various levels of video compression on the test set. As far as we are aware this is the first attempted use of HCRFs for visual speech recognition.

Imitation and 'theory of mind' competencies in discrimination of autism from other neurodevelopmental disorders

Several studies have reported imitative deficits in autism spectrum disorder (ASD). However, it is still debated if imitative deficits are specific to ASD or shared with clinical groups with similar mental impairment and motor difficulties. We investigated whether imitative tasks can be used to discriminate ASD children from typically developing children (TD) and children with general developmental delay (GDD). We applied discriminant function analyses to the performance of these groups on three imitation tasks and tests of dexterity, motor planning, verbal skills, theory of mind (ToM). Analyses revealed two significant dimensions. The first represented impairment of dexterity and verbal ability, and discriminated TD from GDD children. Once these differences were accounted for, differences in ToM and the three imitation tasks accounted for a significant proportion of the remaining intergroup variance and discriminated the ASD group from other groups. Further analyses revealed that inclusion of imitative tasks increased the specificity and sensitivity of ASD classification and that imitative tasks considered alone were able to reliably discriminate ASD, TD and GDD. The results suggest that imitation and theory of mind impairment in autism may stem from a common domain of origin separate from general cognitive and motor skill.

Missing the targets and being misunderstood: Child phonology and interactive problems

Effects of vowel variation on interaction are considered, with particular relevance to their role in conversational breakdown. The effect of speaker knowledge and experience is noted as a variable in developmental progress which must inform profiling decisions, and the need for appropriate taxonomies of speech varieties is emphasized as a precursor to clinical and educational assessments. It is noted, too, that a shared sociolinguistic background between speaker and listener does not always resolve difficulties arising from non-target realizations, casting some doubt on ideas that assessors always possess a guaranteed sense of phonological variability and its effects. Hence, an informed understanding of phonological variation, rather than merely awareness that such variation exists, is advocated.

Inter-Frame Contextual Modelling For Visual Speech Recognition

In this paper, we present a new approach to visual speech recognition which improves contextual modelling by combining Inter-Frame Dependent and Hidden Markov Models. This approach captures contextual information in visual speech that may be lost using a Hidden Markov Model alone. We apply contextual modelling to a large speaker independent isolated digit recognition task, and compare our approach to two commonly adopted feature based techniques for incorporating speech dynamics. Results are presented from baseline feature based systems and the combined modelling technique. We illustrate that both of these techniques achieve similar levels of performance when used independently. However significant improvements in performance can be achieved through a combination of the two. In particular we report an improvement in excess of 17% relative Word Error Rate in comparison to our best baseline system.