Robust Multimodal Person Identification With Limited Training Data

This paper presents a novel method of audio-visual feature-level fusion for person identification where both the speech and facial modalities may be corrupted, and there is a lack of prior knowledge about the corruption. Furthermore, we assume there are limited amount of training data for each modality (e.g., a short training speech segment and a single training facial image for each person). A new multimodal feature representation and a modified cosine similarity are introduced to combine and compare bimodal features with limited training data, as well as vastly differing data rates and feature sizes. Optimal feature selection and multicondition training are used to reduce the mismatch between training and testing, thereby making the system robust to unknown bimodal corruption. Experiments have been carried out on a bimodal dataset created from the SPIDRE speaker recognition database and AR face recognition database with variable noise corruption of speech and occlusion in the face images. The system's speaker identification performance on the SPIDRE database, and facial identification performance on the AR database, is comparable with the literature. Combining both modalities using the new method of multimodal fusion leads to significantly improved accuracy over the unimodal systems, even when both modalities have been corrupted. The new method also shows improved identification accuracy compared with the bimodal systems based on multicondition model training or missing-feature decoding alone.

Multimodal functional and anatomical imaging identifies preclinical microvascular abnormalities in type 1 diabetes mellitus

Structural and functional change in the microcirculation in type 1 diabetes mellitus predicts future end-organ damage and macrovascular events. We explored the utility of novel signal processing techniques to detect and track change in ocular hemodynamics in patients with this disease. 24 patients with uncomplicated type 1 diabetes mellitus, and 18 age-and-sex matched control subjects were studied. Doppler ultrasound was used to interrogate the carotid and ophthalmic arteries and digital photography to image the retinal vasculature. Frequency analysis algorithms were applied to quantify velocity waveform structure and retinal photographic data at baseline and following inhalation of 100% oxygen. Frequency data was compared between groups. No significant differences were found in the resistive index between groups at baseline or following inhaled oxygen. Frequency analysis of the Doppler flow velocity waveforms identified significant differences in bands 3-7 between patients and controls in data captured from the ophthalmic artery (p<0.01 for each band). In response to inhaled oxygen, changes in the frequency band amplitudes were significantly greater in control subjects compared with patients (p<0.05). Only control subjects demonstrated a positive correlation (R=0.61) between change in retinal vessel diameter and frequency band amplitudes derived from ophthalmic artery waveform data. The use of multimodal signal processing techniques applied to Doppler flow velocity waveforms and retinal photographic data identified preclinical change in the ocular microcirculation in patients with uncomplicated diabetes mellitus. An impaired autoregulatory response of the retinal microvasculature may contribute to the future development of retinopathy in such patients.

Spectacles of military urbanism in online representations of the Elite Squad of the military police of Rio de Janeiro: a multimodal approach

This paper presents a multimodal analysis of online self-representations of the Elite Squad of the military police of Rio de Janeiro, the Special Police Operations Battalion BOPE. The analysis is placed within the wider context of a “new military urbanism”, which is evidenced in the ongoing “Pacification” of many of the city’s favelas, in which BOPE plays an active interventionist as well as a symbolic role, and is a kind of solution which clearly fails to address the root causes of violence which lie in poverty and social inequality. The paper first provides a sociocultural account of BOPE’s role in Rio’s public security and then looks at some of the mainly visual mediated discourses the Squad employs in constructing a public image of itself as a modern and efficient, yet at the same time “magical” police force.

Multimodal Critical Discourse Analysis (MCDA)

Critical Discourse Analysis (CDA) has probably made the most comprehensive attempt to develop a theory of the inter-connectedness of discourse, power and ideology and is specifically concerned with the role that discourse plays in main-taining and legitimizing inequality in society. While CDA’s general thrust has been towards the analysis of linguistic structures, some critical discourse analysts have begun to focus on multimodal discourses because of the increasingly impor-tant role these play in many social and political contexts. Still, a great deal of CDA analysis has remained largely monomodal. The principal aim of this chapter is therefore to address this situation and demonstrate in what ways CDA can be deployed to analyse the ways that ideological discourses can be communicated, naturalised and legitimated beyond the linguistic level. The chapter also offers a rationale for a multimodal approach based on Halliday’s Systemic Functional Linguistics (SFL), by which it is directly informed

Cascaded multimodal biometric recognition framework

A practically viable multi-biometric recognition system should not only be stable, robust and accurate but should also adhere to real-time processing speed and memory constraints. This study proposes a cascaded classifier-based framework for use in biometric recognition systems. The proposed framework utilises a set of weak classifiers to reduce the enrolled users' dataset to a small list of candidate users. This list is then used by a strong classifier set as the final stage of the cascade to formulate the decision. At each stage, the candidate list is generated by a Mahalanobis distance-based match score quality measure. One of the key features of the authors framework is that each classifier in the ensemble can be designed to use a different modality thus providing the advantages of a truly multimodal biometric recognition system. In addition, it is one of the first truly multimodal cascaded classifier-based approaches for biometric recognition. The performance of the proposed system is evaluated both for single and multimodalities to demonstrate the effectiveness of the approach.

Sensory imagery in individuals who are blind and sighted: examining unimodal and multimodal forms

Introduction: Previous research has suggested that visual images are more easily generated, more vivid and more memorable than other sensory modalities. This research examined whether or not imagery is experienced in similar ways by people with and without sight. Specifically, the imabeability of visual, auditory and tactile cue words was compared. The degree to which images were multimodal or unimodal was also examined. Method: Twelve participants totally blind from early infancy and 12 sighted participants generated images in response to 53 sensory and non sensory words, rating imageability and the sensory modality, and describing images. From these 53 items, 4 subgroups of words, which stimulated images that were predominantly visual, tactile, auditory and low-imagery, respectively, were created. Results: T-tests comparing imageability ratings from blind and sighted participants found no differences for auditory and tactile words (both p>.1). Nevertheless, whilst participants without sight found auditory and tactile images equally imageable, sighted participants found images in response to tactile cue words harder to generate than visual cue words (mean difference: -0.51, p=.025). Participants with sight were also more likely to develop multisensory images than were participants without sight (both U≥15.0, N1=12, N2=12, p≤.008). Discussion: For both the blind and sighted, auditory and tactile images were rich and varied and similar language was used. Sighted participants were more likely to generate multimodal images. This was particularly the case for tactile words. Nevertheless, cue words that resulted in multisensory images were not necessarily rated as more imageable. The discussion considers whether or not multimodal imagery represent a method of compensating for impoverished unimodal imagery. Implications for Practitioners: Imagery is important not only as a mnemonic in memory rehabilitation, but also everyday uses for things such as autobiographical memory. This research emphasises both the importance of not only auditory and tactile sensory imagery, but also spatial imagery for people without sight.

Primary localized rectal/pararectal gastrointestinal stromal tumors: results of surgical and multimodal therapy from the French Sarcoma group.

BACKGROUND: Rectal and pararectal gastrointestinal stromal tumors (GISTs) are rare. The optimal management strategy for primary localized GISTs remains poorly defined. METHODS: We conducted a retrospective analysis of 41 patients with localized rectal or pararectal GISTs treated between 1991 and 2011 in 13 French Sarcoma Group centers. RESULTS: Of 12 patients who received preoperative imatinib therapy for a median duration of 7 (2-12) months, 8 experienced a partial response, 3 had stable disease, and 1 had a complete response. Thirty and 11 patients underwent function-sparing conservative surgery and abdominoperineal resection, respectively. Tumor resections were mostly R0 and R1 in 35 patients. Tumor rupture occurred in 12 patients. Eleven patients received postoperative imatinib with a median follow-up of 59 (2.4-186) months. The median time to disease relapse was 36 (9.8-62) months. The 5-year overall survival rate was 86.5%. Twenty patients developed local recurrence after surgery alone, two developed recurrence after resection combined with preoperative and/or postoperative imatinib, and eight developed metastases. In univariate analysis, the mitotic index (≤5) and tumor size (≤5 cm) were associated with a significantly decreased risk of local relapse. Perioperative imatinib was associated with a significantly reduced risk of overall relapse and local relapse. CONCLUSIONS: Perioperative imatinib therapy was associated with improved disease-free survival. Preoperative imatinib was effective. Tumor shrinkage has a clear benefit for local excision in terms of feasibility and function preservation. Given the complexity of rectal GISTs, referral of patients with this rare disease to expert centers to undergo a multidisciplinary approach is recommended.

Comprehensive Statewide Transportation Model: Multimodal Investment Analysis Methodology Phase II, CTRE Project 02-126, 2001

The purposes of this report (Phase II of the project) are to specify in mathematical form the individual modules of the conceptual model developed in Phase I, to identify and evaluate sources of data for the model set, and to develop the transport networks necessary to support the models.

La limitation de responsabilité dans le transport multimodal

"Mémoire présenté à la Faculté des études supérieures en vue de l'obtention du grade de L.L.M. en droit des affaires"

Multimodal carrier liability in the U.S. and Canada : towards uniformity of applicable rules?

"Thèse présentée à la Faculté des études supérieures en vue de l'obtention du grade de Docteur en Droit (LL.D.) et à la Faculté de Droit et de Sciences Politiques de l'Université de Nantes en vue de l'obtention du grade de Docteur"

L'augmentation des habiletés sociales chez les enfants de 4-5 ans présentant des comportements perturbateurs et/ou agressifs via l'implantation de deux volets du programme multimodal Fluppy

Rapport d'analyse d'intervention présenté à la Faculté des arts et sciences en vue de l'obtention du grade de Maîtrise ès sciences (M. Sc.) en psychoéducation

Rapport d'intervention d'un programme multimodal auprès d'élèves TDAH de première année

Rapport d'analyse d'intervention présenté à la Faculté des arts et sciences en vue de l'obtention du grade de Maîtrise ès sciences (M. Sc.) en psychoéducation

Multimodal image registration of the scoliotic torso for surgical planning

Background This paper presents a method that registers MRIs acquired in prone position, with surface topography (TP) and X-ray reconstructions acquired in standing position, in order to obtain a 3D representation of a human torso incorporating the external surface, bone structures, and soft tissues. Methods TP and X-ray data are registered using landmarks. Bone structures are used to register each MRI slice using an articulated model, and the soft tissue is confined to the volume delimited by the trunk and bone surfaces using a constrained thin-plate spline. Results The method is tested on 3 pre-surgical patients with scoliosis and shows a significant improvement, qualitatively and using the Dice similarity coefficient, in fitting the MRI into the standing patient model when compared to rigid and articulated model registration. The determinant of the Jacobian of the registration deformation shows higher variations in the deformation in areas closer to the surface of the torso. Conclusions The novel, resulting 3D full torso model can provide a more complete representation of patient geometry to be incorporated in surgical simulators under development that aim at predicting the effect of scoliosis surgery on the external appearance of the patient’s torso.