TESSA - Toolkit for experimentation with multimodal sensory substitution and augmentation

TESSA is a toolkit for experimenting with sensory augmentation. It includes hardware and software to facilitate rapid prototyping of interfaces that can enhance one sense using information gathered from another sense. The toolkit contains a range of sensors (e.g. ultrasonics, temperature sensors) and actuators (e.g. tactors or stereo sound), designed modularly so that inputs and outputs can be easily swapped in and out and customized using TESSA’s graphical user interface (GUI), with “real time” feedback. The system runs on a Raspberry Pi with a built-in touchscreen, providing a compact and portable form that is amenable for field trials. At CHI Interactivity, the audience will have the opportunity to experience sensory augmentation effects using this system, and design their own sensory augmentation interfaces.

Interactive sensory objects for and by people with learning disabilities

This project engages people with learning disabilities as co-researchers and co-designers in the development of multisensory interactive artworks, with the aim of making museums or heritage sites more interesting, meaningful, and fun. This article describes our explorations, within this context, of a range of technologies including squishy circuits, littleBits, and easy-build websites, and presents examples of objects created by the co-researchers such as “sensory boxes” and interactive buckets, baskets, and boots. Public engagement is an important part of the project and includes an annual public event and seminar day, a blog rich with photos and videos of the workshops, and an activities book to give people ideas for creating their own sensory explorations of museums and heritage sites.

Is it me? Self-recognition bias across sensory modalities and its relationship to autistic traits

Background Atypical self-processing is an emerging theme in autism research, suggested by lower self-reference effect in memory, and atypical neural responses to visual self-representations. Most research on physical self-processing in autism uses visual stimuli. However, the self is a multimodal construct, and therefore, it is essential to test self-recognition in other sensory modalities as well. Self-recognition in the auditory modality remains relatively unexplored and has not been tested in relation to autism and related traits. This study investigates self-recognition in auditory and visual domain in the general population and tests if it is associated with autistic traits. Methods Thirty-nine neurotypical adults participated in a two-part study. In the first session, individual participant’s voice was recorded and face was photographed and morphed respectively with voices and faces from unfamiliar identities. In the second session, participants performed a ‘self-identification’ task, classifying each morph as ‘self’ voice (or face) or an ‘other’ voice (or face). All participants also completed the Autism Spectrum Quotient (AQ). For each sensory modality, slope of the self-recognition curve was used as individual self-recognition metric. These two self-recognition metrics were tested for association between each other, and with autistic traits. Results Fifty percent ‘self’ response was reached for a higher percentage of self in the auditory domain compared to the visual domain (t = 3.142; P < 0.01). No significant correlation was noted between self-recognition bias across sensory modalities (τ = −0.165, P = 0.204). Higher recognition bias for self-voice was observed in individuals higher in autistic traits (τ AQ = 0.301, P = 0.008). No such correlation was observed between recognition bias for self-face and autistic traits (τ AQ = −0.020, P = 0.438). Conclusions Our data shows that recognition bias for physical self-representation is not related across sensory modalities. Further, individuals with higher autistic traits were better able to discriminate self from other voices, but this relation was not observed with self-face. A narrow self-other overlap in the auditory domain seen in individuals with high autistic traits could arise due to enhanced perceptual processing of auditory stimuli often observed in individuals with autism.

Sensory enhancement, a pilot perceptual study of subdermal magnetic implants

Subdermal magnetic implants originated as an art form in the world of body modification. To date an in depth scientific analysis of the benefits of this implant has yet to be established. This research explores the concept of sensory extension of the tactile sense utilising this form of implantation. This relatively simple procedure enables the tactile sense to respond to static and alternating magnetic fields. This is not to say that the underlying biology of the system has changed; i.e. the concept does not increase our tactile frequency response range or sensitivity to pressure, but now does invoke a perceptual response to a stimulus that is not innately available to humans. Within this research two social surveys have been conducted in order to ascertain one, the social acceptance of the general notion of human enhancement, and two the perceptual experiences of individuals with the magnetic implants themselves. In terms of acceptance to the notion of sensory improvement (via implantation) ~39% of the general population questioned responded positively with a further ~25% of the respondents answering with the indecisive response. Thus with careful dissemination a large proportion of individuals may adopt this technology much like this if it were to become available for consumers. Interestingly of the responses collected from the magnetic implants survey ~60% of the respondents actually underwent the implant for magnetic vision purposes. The main contribution of this research however comes from a series of psychophysical testing. In which 7 subjects with subdermal magnetic implants, were cross compared with 7 subjects that had similar magnets superficially attached to their dermis. The experimentation examined multiple psychometric thresholds of the candidates including intensity, frequency and temporal. Whilst relatively simple, the experimental setup for the perceptual experimentation conducted was novel in that custom hardware and protocols were created in order to determine the subjective thresholds of the individuals. Abstract iv The overall purpose of this research is to utilise this concept in high stress scenarios, such as driving or piloting; whereby alerts and warnings could be relayed to an operator without intruding upon their other (typically overloaded) exterior senses (i.e. the auditory and visual senses). Hence each of the thresholding experiments were designed with the intention of utilising the results in the design of signals for information transfer. The findings from the study show that the implanted group of subjects significantly outperformed the superficial group in the absolute intensity threshold experiment, i.e. the implanted group required significantly less force than the superficial group in order to perceive the stimulus. The results for the frequency difference threshold showed no significant difference in the two groups tested. Interestingly however at low frequencies, i.e. 20 and 50 Hz, the ability of the subjects tested to discriminate frequencies significantly increased with more complex waveforms i.e. square and sawtooth, when compared against the typically used sinewave. Furthermore a novel protocol for establishing the temporal gap detection threshold during a temporal numerosity study has been established in this thesis. This experiment measured the subjects’ capability to correctly determine the number of concatenated signals presented to them whilst the time between the signals, referred to as pulses, tended to zero. A significant finding was that when altering the length of, the frequency of, and the number of cycles of the pulses, the time between pulses for correct recognition altered. This finding will ultimately aid in the design of the tactile alerts for this method of information transfer. Preliminary development work for the use of this method of input to the body, in an automotive scenario, is also presented within this thesis in the form of a driving simulation. The overall goal of which is to present warning alerts to a driver, such as rear-to-end collision, or excessive speeds on roads, in order to prevent incidents and penalties from occurring. Discussion on the broader utility of this implant has been presented, reflecting on its potential use as a basis for vibrotactile, and sensory substitution, devices. This discussion furthers with postulations on its use as a human machine interface, as well as how a similar implant could be used within the ear as a hearing aid device.

Anticipatory engineering: anticipation in sensory-motor systems of human

In visual tracking experiments, distributions of the relative phase be-tween target and tracer showed positive relative phase indicating that the tracer precedes the target position. We found a mode transition from the reactive to anticipatory mode. The proposed integrated model provides a framework to understand the antici-patory behaviour of human, focusing on the integration of visual and soma-tosensory information. The time delays in visual processing and somatosensory feedback are explicitly treated in the simultaneous differential equations. The anticipatory behaviour observed in the visual tracking experiments can be ex-plained by the feedforward term of target velocity, internal dynamics, and time delay in somatosensory feedback.

Girls talk: authorship and authenticity in the reception of Lena Dunham's Girls

By examining the discourse around Lena Dunham's HBO comedy Girls (2012– present), this article argues that the programme served as a space to think through female authorship, televisual representations and cultural tensions surrounding young womanhood. Central to this discourse was the narrative asserting Girls' and Dunham's 'authenticity', originality and universality, which sought to legitimate her gendered authorship and interest in the comedy of female intimacy within HBO' s masculine prestige channel identity. Charting three cycles of discourse surrounding the programme's debut, this article explores the paratextual framing by promotional concerns, television critics and women' s websites. It highlights how journalists and critics furthered HBO's paratextual framing of Dunham, which was later countered by the networked spaces of niche online media, which used the programme as a space to productively work through industrial and cultural tensions; particularly those surrounding female comic authorship, autobiography and intersectionality.

Optimization of a sponge cake formulation with inulin as fat replacer: structure, physicochemical, and sensory properties

The effects of several fat replacement levels (0%, 35%, 50%, 70%, and 100%) by inulin in sponge cake microstructure and physicochemical properties were studied. Oil substitution for inulin decreased significantly (P < 0.05) batter viscosity, giving heterogeneous bubbles size distributions as it was observed by light microscopy. Using confocal laser scanning microscopy the fat was observed to be located at the bubbles’ interface, enabling an optimum crumb cake structure development during baking. Cryo-SEM micrographs of cake crumbs showed a continuous matrix with embedded starch granules and coated with oil; when fat replacement levels increased, starch granules appeared as detached structures. Cakes with fat replacement up to 70% had a high crumb air cell values; they were softer and rated as acceptable by an untrained sensory panel (n = 51). So, the reformulation of a standard sponge cake recipe to obtain a new product with additional health benefits and accepted by consumers is achieved.

Replacing fat and sugar with inulin in cakes: bubble size distribution, physical and sensory properties

The replacement of fat and sugar in cakes is a challenge as they have an important effect on the structural and sensory properties. Moreover, there is the possibility to incorporate an additional value using novel replacers. In this work, inulin and oligofructose were used as fat and sugar replacers, respectively. Different combinations of replacement levels were investigated: fat replacement (0 and 50 %) and sugar replacement (0, 20, 30, 40 and 50 %). Simulated microbaking was carried out to study bubble size distribution during baking. Batter viscosity and weight loss during baking were also analysed. Cake characteristics were studied in terms of cell crumb structure, height, texture and sensory properties. Fat and sugar replacement gave place to batters with low apparent viscosity values. During heating, bubbles underwent a marked expansion in replaced cakes if compared to the control cake. The low batter stability in fat-replaced samples increased bubble movement, giving place to cakes with bigger cells and less height than the control. Sugar-replaced samples had smaller and fewer cells and lower height than the control. Moreover, sugar replacement decreased hardness and cohesiveness and in- creased springiness, which could be related with a denser crumb and an easily crumbled product. Regarding the sensory analysis, a replacement up to 50 % of fat and 30 % of sugar, separately and simultaneously, did not change remarkably the overall acceptability of the cakes. However, the sponginess and the sweetness could be improved in all the replaced cakes, according to the Just About Right scales.

Should healthy eating programmes incorporate interaction with foods in different sensory modalities? A review of the evidence

Commercial interventions seeking to promote fruit and vegetable consumption by encouraging preschool- and school-aged children to engage with foods with ‘all their senses’ are increasing in number. We review the efficacy of such sensory interaction programmes and consider the components of these that are likely to encourage food acceptance. Repeated exposure to a food's flavour has robust empirical support in terms of its potential to increase food intake. However, children are naturally reluctant to taste new or disliked foods, and parents often struggle to provide sufficient taste opportunities for these foods to be adopted into the child's diet. We therefore explore whether prior exposure to a new food's non-taste sensory properties, such as its smell, sound, appearance or texture, might facilitate the food's introduction into the child's diet, by providing the child with an opportunity to become partially familiar with the food without invoking the distress associated with tasting it. We review the literature pertaining to the benefits associated with exposure to foods through each of the five sensory modalities in turn. We conclude by calling for further research into the potential for familiarisation with the visual, olfactory, somaesthetic and auditory properties of foods to enhance children's willingness to consume a variety of fruits and vegetables.

Intertextuality, authenticity, and gonzo selves in Anya Ulinich’s 'Lena Finkle’s Magic Barrel'

This article traces the intertextual relationships between Anya Ulinich’s graphic novel Lena Finkle’s Magic Barrel, Bernard Malamud’s short story ‘The Magic Barrel’ and a number of works by Philip Roth. Through these relationships and her construction of a number of variations on what Miriam Libicki has called a ‘gonzo self’ Ulinich explores the tensions between life and art, fact and fiction, and autobiography and the novel, mediating the aesthetic imperatives of what Roth has called the ‘written world’ and the ethical obligations of the ‘unwritten world’ in order to arrive at an authentic sense of herself as an artist and writer.

Multi-sensory treatment for children with developmental motor speech disorders

What this paper adds? What is already known on the subject? Multi-sensory treatment approaches have been shown to impact outcome measures positively, such as accuracy of speech movement patterns and speech intelligibility in adults with motor speech disorders, as well as in children with apraxia of speech, autism and cerebral palsy. However, there has been no empirical study using multi-sensory treatment for children with speech sound disorders (SSDs) who demonstrate motor control issues in the jaw and orofacial structures (e.g. jaw sliding, jaw over extension, inadequate lip rounding/retraction and decreased integration of speech movements). What this paper adds? Findings from this study indicate that, for speech production disorders where both the planning and production of spatiotemporal parameters of movement sequences for speech are disrupted, multi-sensory treatment programmes that integrate auditory, visual and tactile–kinesthetic information improve auditory and visual accuracy of speech production. The training (practised in treatment) and test words (not practised in treatment) both demonstrated positive change in most participants, indicating generalization of target features to untrained words. It is inferred that treatment that focuses on integrating multi-sensory information and normalizing parameters of speech movements is an effective method for treating children with SSDs who demonstrate speech motor control issues.

Space, authenticity and utopia in the hip-hop teen dance film

The dance film flourished in the 2000s in the form of the hip-hop teen dance film. Such films as Save the Last Dance (Thomas Carter, 2001), Honey (Billy Woodruff, 2002) and Step Up (Anne Fletcher, 2006) drew on hip-hop’s dominance of the mainstream music industry and combined the teen film’s pre-existing social problem and musical narratives. Yet various tension were created by their interweaving of representations of post-industrial city youth with the utopian sensibilities of the classical Hollywood musical. Their narratives celebrated hip-hop performance, and depicted dance’s ability to bridge cultural boundaries and bring together couples and communities. These films used hip-hop to define space and identity yet often constructed divisions within their soundscapes, limiting hip-hop’s expressive potential. This article explores the cycle’s celebration of, yet struggle with, hip-hop through examining select films’ interactions between soundscape, narrative and form. It will engage with these films’ attempts to marry the representational, narrative and aesthetic meanings of hip-hop culture with the form and ideologies of the musical genre, particularly the tensions and continuities that arise from their engagement with the genre’s utopian qualities identified by Richard Dyer (1985). Yet whilst these films illustrate the tensions and challenges of combining hip-hop culture and the musical genre, they also demonstrate an effective integration of hip-hop soundscape and the dancing body in their depiction of dance, highlighting both form’s aesthetics of layering, rupture and flow (Rose, 1994: 22).

Sensory and consumer science methods used with older adults: a review of current methods and recommendations for the future

Capturing the sensory perception and preferences of older adults, whether healthy or with particular disease states, poses major methodological challenges for the sensory community. Currently a vastly under researched area, it is at the same time a vital area of research as alterations in sensory perception can affect daily dietary food choices, intake, health and wellbeing. Tailored sensory methods are needed that take into account the challenges of working with such populations including poor access leading to low patient numbers (study power), cognitive abilities, use of medications, clinical treatments and context (hospitals and care homes). The objective of this paper was to review current analytical and affective sensory methodologies used with different cohorts of healthy and frail older adults, with focus on food preference and liking. We particularly drew attention to studies concerning general ageing as well as to those considering age-related diseases that have an emphasis on malnutrition and weight loss. Pubmed and Web of Science databases were searched to 2014 for relevant articles in English. From this search 75 papers concerning sensory acuity, 41 regarding perceived intensity and 73 relating to hedonic measures were reviewed. Simpler testing methods, such as directional forced choice tests and paired preference tests need to be further explored to determine whether they lead to more reliable results and better inter-cohort comparisons. Finally, sensory quality and related quality of life for older adults suffering from dementia must be included and not ignored in our future actions.

Sensory stories

Sensory Objects researchers and co-researchers, people with learning disabilities from Liverpool Mencap Access to Heritage Group presented thier interactive sensory objects made for the National Trust Speke Hall.

Seminar: sensory stories retold

The Sensory Stories Retold seminar showcased the first year of the Sensory Objects project, which was based at Speke Hall, a National Trust house in Liverpool. The research team presented their work with interactive demonstrations of their sensory objects and a hands-on workshop for attendees to try making their own. The day featured a discussion led by Marcus Weisen (Jodi Mattes Trust), and a presentation by Ticky Lowe (Access to Heritage) about the Jodi Award Winning Touch Pods project. The event provided an opportunity for 65 museum curators, researchers and disability professionals to discuss and explore museum and heritage engagement, the potential of sensory art-based workshops, the use of electronics in museum interpretation, and multimedia advocacy.