Visual Speech Segmentation: Using Facial Cues to Locate Word Boundaries in Continuous Speech

Speech is typically a multimodal phenomenon, yet few studies have focused on the exclusive contributions of visual cues to language acquisition. To address this gap, we investigated whether visual prosodic information can facilitate speech segmentation. Previous research has demonstrated that language learners can use lexical stress and pitch cues to segment speech and that learners can extract this information from talking faces. Thus, we created an artificial speech stream that contained minimal segmentation cues and paired it with two synchronous facial displays in which visual prosody was either informative or uninformative for identifying word boundaries. Across three familiarisation conditions (audio stream alone, facial streams alone, and paired audiovisual), learning occurred only when the facial displays were informative to word boundaries, suggesting that facial cues can help learners solve the early challenges of language acquisition.

Seeing music performance: Visual influences on perception and experience

Drawing from ethnographic, empirical, and historical/cultural perspectives, we examine the extent to which visual aspects of music contribute to the communication that takes place between performers and their listeners. First, we introduce a framework for understanding how media and genres shape aural and visual experiences of music. Second, we present case studies of two performances, and describe the relation between visual and aural aspects of performance. Third, we report empirical evidence that visual aspects of performance reliably influence perceptions of musical structure (pitch related features) and affective interpretations of music. Finally, we trace new and old media trajectories of aural and visual dimensions of music, and highlight how our conceptions, perceptions and appreciation of music are intertwined with technological innovation and media deployment strategies.

Infants' selective attention to faces and prosody of speech: The roles of intersensory redundancy and exploratory time

One of the overarching questions in the field of infant perceptual and cognitive development concerns how selective attention is organized during early development to facilitate learning. The following study examined how infants' selective attention to properties of social events (i.e., prosody of speech and facial identity) changes in real time as a function of intersensory redundancy (redundant audiovisual, nonredundant unimodal visual) and exploratory time. Intersensory redundancy refers to the spatially coordinated and temporally synchronous occurrence of information across multiple senses. Real time macro- and micro-structural change in infants' scanning patterns of dynamic faces was also examined. ^ According to the Intersensory Redundancy Hypothesis, information presented redundantly and in temporal synchrony across two or more senses recruits infants' selective attention and facilitates perceptual learning of highly salient amodal properties (properties that can be perceived across several sensory modalities such as the prosody of speech) at the expense of less salient modality specific properties. Conversely, information presented to only one sense facilitates infants' learning of modality specific properties (properties that are specific to a particular sensory modality such as facial features) at the expense of amodal properties (Bahrick & Lickliter, 2000, 2002). ^ Infants' selective attention and discrimination of prosody of speech and facial configuration was assessed in a modified visual paired comparison paradigm. In redundant audiovisual stimulation, it was predicted infants would show discrimination of prosody of speech in the early phases of exploration and facial configuration in the later phases of exploration. Conversely, in nonredundant unimodal visual stimulation, it was predicted infants would show discrimination of facial identity in the early phases of exploration and prosody of speech in the later phases of exploration. Results provided support for the first prediction and indicated that following redundant audiovisual exposure, infants showed discrimination of prosody of speech earlier in processing time than discrimination of facial identity. Data from the nonredundant unimodal visual condition provided partial support for the second prediction and indicated that infants showed discrimination of facial identity, but not prosody of speech. The dissertation study contributes to the understanding of the nature of infants' selective attention and processing of social events across exploratory time.^

Advancing Bag-of-visual-words Representations For Lesion Classification In Retinal Images.

Diabetic Retinopathy (DR) is a complication of diabetes that can lead to blindness if not readily discovered. Automated screening algorithms have the potential to improve identification of patients who need further medical attention. However, the identification of lesions must be accurate to be useful for clinical application. The bag-of-visual-words (BoVW) algorithm employs a maximum-margin classifier in a flexible framework that is able to detect the most common DR-related lesions such as microaneurysms, cotton-wool spots and hard exudates. BoVW allows to bypass the need for pre- and post-processing of the retinographic images, as well as the need of specific ad hoc techniques for identification of each type of lesion. An extensive evaluation of the BoVW model, using three large retinograph datasets (DR1, DR2 and Messidor) with different resolution and collected by different healthcare personnel, was performed. The results demonstrate that the BoVW classification approach can identify different lesions within an image without having to utilize different algorithms for each lesion reducing processing time and providing a more flexible diagnostic system. Our BoVW scheme is based on sparse low-level feature detection with a Speeded-Up Robust Features (SURF) local descriptor, and mid-level features based on semi-soft coding with max pooling. The best BoVW representation for retinal image classification was an area under the receiver operating characteristic curve (AUC-ROC) of 97.8% (exudates) and 93.5% (red lesions), applying a cross-dataset validation protocol. To assess the accuracy for detecting cases that require referral within one year, the sparse extraction technique associated with semi-soft coding and max pooling obtained an AUC of 94.2 ± 2.0%, outperforming current methods. Those results indicate that, for retinal image classification tasks in clinical practice, BoVW is equal and, in some instances, surpasses results obtained using dense detection (widely believed to be the best choice in many vision problems) for the low-level descriptors.

Visual Navigation In The Neotropical Ant Odontomachus Hastatus (formicidae, Ponerinae), A Predominantly Nocturnal, Canopy-dwelling Predator Of The Atlantic Rainforest.

The arboreal ant Odontomachus hastatus nests among roots of epiphytic bromeliads in the sandy forest at Cardoso Island (Brazil). Crepuscular and nocturnal foragers travel up to 8m to search for arthropod prey in the canopy, where silhouettes of leaves and branches potentially provide directional information. We investigated the relevance of visual cues (canopy, horizon patterns) during navigation in O. hastatus. Laboratory experiments using a captive ant colony and a round foraging arena revealed that an artificial canopy pattern above the ants and horizon visual marks are effective orientation cues for homing O. hastatus. On the other hand, foragers that were only given a tridimensional landmark (cylinder) or chemical marks were unable to home correctly. Navigation by visual cues in O. hastatus is in accordance with other diurnal arboreal ants. Nocturnal luminosity (moon, stars) is apparently sufficient to produce contrasting silhouettes from the canopy and surrounding vegetation, thus providing orientation cues. Contrary to the plain floor of the round arena, chemical cues may be important for marking bifurcated arboreal routes. This experimental demonstration of the use of visual cues by a predominantly nocturnal arboreal ant provides important information for comparative studies on the evolution of spatial orientation behavior in ants. This article is part of a Special Issue entitled: Neotropical Behaviour.

Applications Of Visual Evoked Potentials And Fourier-domain Optical Coherence Tomography In Parkinson's Disease: A Controlled Study.

The goal of this cross-sectional observational study was to quantify the pattern-shift visual evoked potentials (VEP) and the thickness as well as the volume of retinal layers using optical coherence tomography (OCT) across a cohort of Parkinson's disease (PD) patients and age-matched controls. Forty-three PD patients and 38 controls were enrolled. All participants underwent a detailed neurological and ophthalmologic evaluation. Idiopathic PD cases were included. Cases with glaucoma or increased intra-ocular pressure were excluded. Patients were assessed by VEP and high-resolution Fourier-domain OCT, which quantified the inner and outer thicknesses of the retinal layers. VEP latencies and the thicknesses of the retinal layers were the main outcome measures. The mean age, with standard deviation (SD), of the PD patients and controls were 63.1 (7.5) and 62.4 (7.2) years, respectively. The patients were predominantly in the initial Hoehn-Yahr (HY) disease stages (34.8% in stage 1 or 1.5, and 55.8 % in stage 2). The VEP latencies and the thicknesses as well as the volumes of the retinal inner and outer layers of the groups were similar. A negative correlation between the retinal thickness and the age was noted in both groups. The thickness of the retinal nerve fibre layer (RNFL) was 102.7 μm in PD patients vs. 104.2 μm in controls. The thicknesses of retinal layers, VEP, and RNFL of PD patients were similar to those of the controls. Despite the use of a representative cohort of PD patients and high-resolution OCT in this study, further studies are required to establish the validity of using OCT and VEP measurements as the anatomic and functional biomarkers for the evaluation of retinal and visual pathways in PD patients.

Physical Activity Interventions For Children And Youth With Visual Impairments.

The authors conducted a systematic literature review on physical activity interventions for children and youth with visual impairment (VI). Five databases were searched to identify studies involving the population of interest and physical activity practices. After evaluating 2,495 records, the authors found 18 original full-text studies published in English they considered eligible. They identified 8 structured exercise-training studies that yielded overall positive effect on physical-fitness and motor-skill outcomes. Five leisure-time-physical-activity and 5 instructional-strategy interventions were also found with promising proposals to engage and instruct children and youth with VI to lead an active lifestyle. However, the current research on physical activity interventions for children and youth with VI is still limited by an absence of high-quality research designs, low sample sizes, use of nonvalidated outcome measures, and lack of generalizability, which need to be addressed in future studies.

Método para avaliação da conduta visual de lactentes

The purpose of this study is to introduce a method to evaluate visual functions in infants in the first three months of life. An adaptation of the Guide for the Assessment of Visual Ability in Infants (Gagliardo, 1997) was used. The instrument was a ring with string. It was implemented a pilot study with 33 infants, selected according to the following criteria: neonates well enough to go home within two days of birth; 1 to 3 months of chronological age; monthly evaluation with no absence; subjects living in Campinas/SP metropolitan area. In the first month we observed: visual fixation (93,9%); eye contact (90,9%); horizontal tracking (72,7%); inspects surroundings (97,0%). In the third month, we observed: inspects own hands (42,4%) and increased movements of arms (36,4%). This method allowed the evaluation of visual functions in infants, according to the chronological age. Alterations in this function will facilitate immediate referral to medical services for diagnoses.

Visual demonstration of the ionic strength effect in the classroom: the Debye-Hückel limiting law

The effects of ionic strength on ions in aqueous solutions are quite relevant, especially for biochemical systems, in which proteins and amino acids are involved. The teaching of this topic and more specifically, the Debye-Hückel limiting law, is central in chemistry undergraduate courses. In this work, we present a description of an experimental procedure based on the color change of aqueous solutions of bromocresol green (BCG), driven by addition of electrolyte. The contribution of charge product (z+|z-|) to the Debye-Hückel limiting law is demonstrated when the effects of NaCl and Na2SO4 on the color of BCG solutions are compared.

Prosody, focus, and word order

155

Estudo da circulação retrobulbar e do campo visual após dose única oral de citrato de sildenafil (Viagra®)

Purpose: To analyze the effects of 100 mg of sildenafil citrate (Viagra®) on the retrobulbar circulation and visual field. Methods: A double masked, placebo controlled study was conducted in 10 males with a mean age of 27.7 + 5.68 years. The right eye of each volunteer underwent orbital color Doppler imaging and automated perimetry (Humphrey, program 30-2, Full-Threshold Strategy) at 3 occasions: baseline, 1 hour after placebo and 1 hour after 100 mg of sildenafil. The foveal threshold and the mean deviation (MD) were analyzed by automated perimetry on the three occasions. Color Doppler imaging allowed the measurement of the peak systolic velocity (PSV), end diastolic velocity (EDV) and Pourcelot index (PI) in the central retinal artery and ophthalmic artery. Results: The foveal threshold and the mean deviation did not show a significant change following the administration of sildenafil. The ophthalmic artery peak systolic velocity and end diastolic velocity significantly increased after the administration of sildenafil (p<0.001). The hemodynamic parameters in the central retinal artery and the ophthalmic artery PI did not significantly change. Conclusions: Sildefanil citrate increased the blood flow velocities in the ophthalmic artery in normal subjects, with no significant changes in the foveal threshold and mean deviation in the automated perimetry.

Utilização de recursos ópticos e equipamentos por escolares com deficiência visual

PURPOSE: To verify perceptions and conduct of students with visual impairment regarding devices and equipment utilized in schooling process. METHODS: A transversal descriptive study on a population of 12-year-old or older students in schooling process, affected by congenital or acquired visual impairment, inserted in the government teaching system of Campinas during the year 2000. An interview quiz, created based on an exploratory study was applied. RESULTS: A group of 26 students, 46% of them with low vision and 53.8% affected by blindness was obtained. Most of the students were from fundamental teaching courses (65.4%), studying in schools with classrooms provided with devices (73.1%). Among the resources used in reading and writing activities, 94.1% of the students reported they used the Braille system and 81.8% reported that the reading subject was dictated by a colleague. Most of the students with low vision wore glasses (91.7%), and 33.3% utilized a magnifying glass as optical devices. Among the non-optical devices, the most common were the environmental ones, getting closer to the blackboard (75.0%) and to the window (66.7%) for better lighting. CONCLUSIONS: It became evident that students with low vision eye-sight made use of devices meant for bearers of blindness, such as applying the Braille system. A reduced number of low vision students making use of optical and non-optical devices applicable to their problems were observed, indicating a probable unawareness of their visual potential and the appropriate devices to improve efficiency.

Comportamento visual e perfil socioeconômico e demográfico de recém-nascidos prematuros da Maternidade do Hospital das Clínicas de Pernambuco - UFPE

OBJECTIVE: To characterize the behavior of premature newborns in the first year of chronological age. METHODS: This is a cross-sectional descriptive study, bound to a longitudinal study titled: Comparison of visual behavior on the first quarter of year of life of premature nursling born at two maternities of Recife/PE. The sample was composed by 52 premature newborns selected from June, 2007 to June, 2008 from the Maternity of the Federal University of Pernambuco (UFPE). Biological, socioeconomic and demographic data was collected through medical records and interviews with progeny. Newborns were evaluated by the Assessment Guide of Visual Ability in Infants. RESULTS: Most of the newborns were male at a gestational period between 33 weeks and 36 weeks and 6 days, showed a good visual behavior development for the age researched, and most of the families showed good socioeconomical and demographic profile. Besides, it was possible to detect ocular signs in 19% of sample, that were referred to an Ophthalmology Service. CONCLUSION: This study results point out the method like an important key in the early detection and visual screening for premature nursling since the first month of life and it led us to believe that clinical view for occupational therapy intervention must be focused not only on biological risks but also at the influence environment in newborn performance.

Neuronal Ceroid-lipofuscinosis with prominent chorea and without visual manifestations: a case report

A case of neuronal ceroid-lipofuscinosis (NCL) is reported in a 11-year-old girl, whose main symptoms were progressive dementia since the age of 4 years and choreic movements since age 10. Seizures, myoclonus and visual deterioration were absent and optic fundi were normal. A cerebral biopsy disclosed two basic types of stored substance in the cytoplasm of neurons: a) severely balloned nerve cells in cortical layers HI and V contained a non-autofluorescent material, which stained with PAS and Sudan Black B in frozen, but not in paraffin sections; ultrastructurally, these neurons showed abundant corpuscles similar to the membranous cytoplasmic bodies of Tay-Sachs disease and, in smaller amounts, also zebra bodies; b) slightly distended or non-distended neurons in all layers contained lipopigment granules, which were autofluorescent, PAS-positive and sudanophil in both frozen and paraffin sections; their ultrastructure was closely comparable to that of lipofuscin. Similar bodies were found in the swollen segments of axons and in a few astrocytes and endothelial cells. The histochemical and ultrastructural demonstration of large amounts of lipopigments allows a presumptive classification of the case as NCL. However, the presence of involuntary movements, the absence of visual disturbances and the unusual ultrastructural features place the patient into a small heterogeneous group within the NCL. A better classification of such unique instances of the disease must await elucidation of the basic enzymatic defects.

Lesões esportivas em atletas com deficiencia visual

Universidade Estadual de Campinas . Faculdade de Educação Física