TESSA - Toolkit for experimentation with multimodal sensory substitution and augmentation

TESSA is a toolkit for experimenting with sensory augmentation. It includes hardware and software to facilitate rapid prototyping of interfaces that can enhance one sense using information gathered from another sense. The toolkit contains a range of sensors (e.g. ultrasonics, temperature sensors) and actuators (e.g. tactors or stereo sound), designed modularly so that inputs and outputs can be easily swapped in and out and customized using TESSA’s graphical user interface (GUI), with “real time” feedback. The system runs on a Raspberry Pi with a built-in touchscreen, providing a compact and portable form that is amenable for field trials. At CHI Interactivity, the audience will have the opportunity to experience sensory augmentation effects using this system, and design their own sensory augmentation interfaces.

Robust multimodal face and fingerprint fusion in the presence of spoofing attacks

Anti-spoofing is attracting growing interest in biometrics, considering the variety of fake materials and new means to attack biometric recognition systems. New unseen materials continuously challenge state-of-the-art spoofing detectors, suggesting for additional systematic approaches to target anti-spoofing. By incorporating liveness scores into the biometric fusion process, recognition accuracy can be enhanced, but traditional sum-rule based fusion algorithms are known to be highly sensitive to single spoofed instances. This paper investigates 1-median filtering as a spoofing-resistant generalised alternative to the sum-rule targeting the problem of partial multibiometric spoofing where m out of n biometric sources to be combined are attacked. Augmenting previous work, this paper investigates the dynamic detection and rejection of livenessrecognition pair outliers for spoofed samples in true multi-modal configuration with its inherent challenge of normalisation. As a further contribution, bootstrap aggregating (bagging) classifiers for fingerprint spoof-detection algorithm is presented. Experiments on the latest face video databases (Idiap Replay- Attack Database and CASIA Face Anti-Spoofing Database), and fingerprint spoofing database (Fingerprint Liveness Detection Competition 2013) illustrate the efficiency of proposed techniques.

The role of text in televideo cybersex

Televideo cybersex provides a unique example of the ways meanings are instantiated, identities constructed, and relationships negotiated across different semiotic modes. This article explores the role of verbal messages in these multimodal exchanges, examining the specific interactional functions they perform. Text, it is argued, plays a rather unique role in this particular kind of interaction. Unlike ordinary face-to-face conversation, in which the body (posture, gestures, gaze) usually plays more of an ancillary role, in televideo cybersex, the bodily performance is primary, with verbal messages functioning to contextualize physical actions. Text is used to help increase the sense of ‘presence’ participants feel, to regulate the rhythm of the unfolding interaction, to help manage the orderly exchange of information, and to create narrative frames within which bodily displays can be interpreted and made coherent.

Multi-level Input Fusion in multi-modal systems

Context-aware multimodal interactive systems aim to adapt to the needs and behavioural patterns of users and offer a way forward for enhancing the efficacy and quality of experience (QoE) in human-computer interaction. The various modalities that constribute to such systems each provide a specific uni-modal response that is integratively presented as a multi-modal interface capable of interpretation of multi-modal user input and appropriately responding to it through dynamically adapted multi-modal interactive flow management , This paper presents an initial background study in the context of the first phase of a PhD research programme in the area of optimisation of data fusion techniques to serve multimodal interactivite systems, their applications and requirements.

Patterns of text layout in Pompeian verse inscriptions

This article explores patterns of formal text layout of the metrical graffiti of Pompeii. After a brief discussion of the importance of formal text layout for linguistic research in general (and its relevance for poetic texts), a representative sample of poetic graffiti is discussed and analysed in detail. It is argued, then, that nature of the surface and sentence structure in particular can take precedence over the ‘default solution’ (coincidence of verse and line structures).

Carmina Saturnia Epigraphica: Einleitung, Text und Kommentar zu den Saturnischen Versinschriften

Edition and commentary of the Latin verse inscription thought to be written in the so-called Saturnian verse

Strengthening argumentation in medical explanations by text planning revision

Explanations are an important by-product of medical decisionsupport activities, as they have proved to favour compliance and correct treatment performance. To achieve this purpose, these texts should have a strong argumentation content and should adapt to emotional, as well as to rational attitudes of the Addressee. This paper describes how Rhetorical Sentence Planning can contribute to this aim: the rulebased plan discourse revision is introduced between Text Planning and Linguistic Realization, and exploits knowledge about the user personality and emotions and about the potential impact of domain items on user compliance and memory recall. The proposed approach originates from analytical and empirical evaluation studies of computer generated explanation texts in the domain of drug prescription. This work was partially supported by a British-Italian Collaboration in Research and Higher Education Project, which involved the Universities of Reading and of Bari, in 1996.

Beyond Belief: Harry Potter in text and film

Despite the fact that the Harry Potter books have won a place in the cultural consciousness and have had enjoyed immense commercial success, they were not accepted within many faith groups at the outset. Using the term ‘religion’ to refers to any system of belief that may be recognised by symbols , I look extracts from three of the novels, alongside their subsequent cinematic adaptations, in order to consider the construction of representations of religion in the films and the contribution made by the films to this debate.

Parchment codices in a new text of Galen

An article proposing that the recently discovered Peri Alupias of Galen contains important evidence about the adoption and use of the codex book form.

Multimodal semantic-associative collateral labelling and indexing of still images

A novel framework for multimodal semantic-associative collateral image labelling, aiming at associating image regions with textual keywords, is described. Both the primary image and collateral textual modalities are exploited in a cooperative and complementary fashion. The collateral content and context based knowledge is used to bias the mapping from the low-level region-based visual primitives to the high-level visual concepts defined in a visual vocabulary. We introduce the notion of collateral context, which is represented as a co-occurrence matrix, of the visual keywords, A collaborative mapping scheme is devised using statistical methods like Gaussian distribution or Euclidean distance together with collateral content and context-driven inference mechanism. Finally, we use Self Organising Maps to examine the classification and retrieval effectiveness of the proposed high-level image feature vector model which is constructed based on the image labelling results.

Semantic-associative visual content labelling and retrieval: A multimodal approach

A novel framework referred to as collaterally confirmed labelling (CCL) is proposed, aiming at localising the visual semantics to regions of interest in images with textual keywords. Both the primary image and collateral textual modalities are exploited in a mutually co-referencing and complementary fashion. The collateral content and context-based knowledge is used to bias the mapping from the low-level region-based visual primitives to the high-level visual concepts defined in a visual vocabulary. We introduce the notion of collateral context, which is represented as a co-occurrence matrix of the visual keywords. A collaborative mapping scheme is devised using statistical methods like Gaussian distribution or Euclidean distance together with collateral content and context-driven inference mechanism. We introduce a novel high-level visual content descriptor that is devised for performing semantic-based image classification and retrieval. The proposed image feature vector model is fundamentally underpinned by the CCL framework. Two different high-level image feature vector models are developed based on the CCL labelling of results for the purposes of image data clustering and retrieval, respectively. A subset of the Corel image collection has been used for evaluating our proposed method. The experimental results to-date already indicate that the proposed semantic-based visual content descriptors outperform both traditional visual and textual image feature models. (C) 2007 Elsevier B.V. All rights reserved.