Interactive sensory objects for improving access to heritage

In this project we explore how to enhance the experience and understanding of cultural heritage in museums and heritage sites by creating interactive multisensory objects collaboratively with artists, technologists and people with learning disabilities. We focus here on workshops conducted during the first year of a three year project in which people with learning disabilities each constructed a 'sensory box' to represent their experiences of Speke Hall, a heritage site in the UK. The box is developed further in later workshops which explore aspects of physicality and how to appeal to the entire range of senses, making use of Arduino technology and basic sensors to enable an interactive user experience.

Methods of sensory augmentation through electro-tactile stimulation of the tongue

This paper details an investigation into sensory substitution by means of direct electrical stimulation of the tongue for the purpose of information input to the human brain. In particular, a device has been constructed and a series of trials have been performed in order to demonstrate the efficacy and performance of an electro-tactile array mounted onto the tongue surface for the purpose of sensory augmentation. Tests have shown that by using a low resolution array a computer-human feedback loop can be successfully implemented by humans in order to complete tasks such as object tracking, surface shape identification and shape recognition with no training or prior experience with the device. Comparisons of this technique have been made with visual alternatives and these show that the tongue based tactile array can match such methods in convenience and accuracy in performing simple tasks.

Sensory and instrumental analysis of medium and long shelf-life Charentais cantaloupe melons (Cucumis melo L.) harvested at different maturities

The flavour profiles of two genotypes of Charentais cantaloupe melons (medium shelf-life and long shelf-life), harvested at two distinct maturities (immature and mature fruit), were investigated. Dynamic headspace extraction (DHE), solid-phase extraction (SPE), gas chromatography–mass spectrometry (GC-MS) and gas chromatography–olfactometry/mass spectrometry (GC-O/MS) were used to determine volatile and semi-volatile compounds. Qualitative descriptive analysis (QDA) was used to assess the organoleptic impact of the different melons and the sensory data were correlated with the chemical analysis. There were significant, consistent and substantial differences between the mature and immature fruit for the medium shelf-life genotype, the less mature giving a green, cucumber character and lacking the sweet, fruity character of the mature fruit. However, maturity at harvest had a much smaller impact on the long shelf-life melons and fewer differences were detected. These long shelf-life melons tasted sweet, but lacked fruity flavours, instead exhibiting a musty, earthy character.

Comparison of a trained sensory panel and an electronic tongue in the assessment of bitter dairy protein hydrolsates

The bitter taste elicited by dairy protein hydrolysates (DPH) is a renowned issue for their acceptability by consumers and therefore incorporation into foods. The traditional method of assessment of taste in foods is by sensory analysis but this can be problematic due to the overall unpleasantness of the samples. Thus, there is a growing interest into the use of electronic tongues (e-tongues) as an alternative method to quantify the bitterness in such samples. In the present study the response of the e-tongue to the standard bitter agent caffeine and a range of both casein and whey based hydrolysates was compared to that of a trained sensory panel. Partial least square regression (PLS) was employed to compare the response of the e-tongue and the sensory panel. There was strong correlation shown between the two methods in the analysis of caffeine (R2 of 0.98) and DPH samples with R2 values ranging from 0.94-0.99. This study exhibits potential for the e-tongue to be used in bitterness screening in DPHs to reduce the reliance on expensive and time consuming sensory panels.

Potential sources of mouth drying in beverages fortified with dairy proteins: a comparison of casein and whey-rich ingredients

Oral nutritional supplement drinks (ONS) are beverages high in dairy proteins that are prescribed to individuals at risk of malnutrition. Consumption of ONS is poor in elderly care facilities, with patients commenting that the sensory attributes of these drinks reduce their enjoyment and willingness to consume. Mouth drying is an attribute of ONS found to build with repeated consumption, which may further limit liking of these products. This study investigated the sources of drying sensations by sequential profiling, with a trained sensory panel rating a range of model milk systems and ONS over repeated sips and during after-effects. Sequential profiling found that fortification of milk with both caseinate and whey protein concentrate significantly increased the perception of mouth drying over repeated consumption, increasing by between 35 and 85% over consumption of 40mL. Enrichment of ONS with either whey protein concentrate or milk protein concentrate to a total protein content of 8.7% (wt/wt) resulted in whey and casein levels of 4.3:4.4% and 1.7:7.0% respectively. The product higher in whey protein was substantially more mouth drying, implying that whey proteins may be the most important contributor to mouth drying in ONS. However, efforts to mask mouth drying of protein-fortified milk by increasing sweetness or fat level were unsuccessful at the levels tested. Increasing the viscosity of protein-fortified milk led to a small but significant reduction in mouth drying. However, this approach was not successful when tested within complete ONS. Further analysis is required into the mechanism of protein-derived mouth drying to mask negative sensations and improve the enjoyment and consumption of protein-rich ONS.

Taxonomic free sorting: a successful method with older consumers and a novel approach to preference mapping

Taxonomic free sorting (TFS) is a fast, reliable and new technique in sensory science. The method extends the typical free sorting task where stimuli are grouped according to similarities, by asking respondents to combine their groups two at a time to produce a hierarchy. Previously, TFS has been used for the visual assessment of packaging whereas this study extends the range of potential uses of the technique to incorporate full sensory analysis by the target consumer, which, when combined with hedonic liking scores, was used to generate a novel preference map. Furthermore, to fully evaluate the efficacy of using the sorting method, the technique was evaluated with a healthy older adult consumer group. Participants sorted eight products into groups and described their reason at each stage as they combined those groups, producing a consumer-specific vocabulary. This vocabulary was combined with hedonic data from a separate group of older adults, to give the external preference map. Taxonomic sorting is a simple, fast and effective method for use with older adults, and its combination with liking data can yield a preference map constructed entirely from target consumer data.

Enhanced affective brain representations of chocolate in cravers vs. non-cravers

To examine the neural circuitry involved in food craving, in making food particularly appetitive and thus in driving wanting and eating, we used fMRI to measure the response to the flavour of chocolate, the sight of chocolate and their combination in cravers vs. non-cravers. Statistical parametric mapping (SPM) analyses showed that the sight of chocolate produced more activation in chocolate cravers than non-cravers in the medial orbitofrontal cortex and ventral striatum. For cravers vs. non-cravers, a combination of a picture of chocolate with chocolate in the mouth produced a greater effect than the sum of the components (i.e. supralinearity) in the medial orbitofrontal cortex and pregenual cingulate cortex. Furthermore, the pleasantness ratings of the chocolate and chocolate-related stimuli had higher positive correlations with the fMRI blood oxygenation level-dependent signals in the pregenual cingulate cortex and medial orbitofrontal cortex in the cravers than in the non-cravers. To our knowledge, this is the first study to show that there are differences between cravers and non-cravers in their responses to the sensory components of a craved food in the orbitofrontal cortex, ventral striatum and pregenual cingulate cortex, and that in some of these regions the differences are related to the subjective pleasantness of the craved foods. Understanding individual differences in brain responses to very pleasant foods helps in the understanding of the mechanisms that drive the liking for specific foods and thus intake of those foods.

Enhancing consumer liking of low salt tomato soup over repeated exposure by herb and spice seasonings

There is strong evidence for the link between high dietary sodium and increased risk of cardiovascular disease which drives the need to reduce salt content in foods. In this study, herb and spice blends were used to enhance consumer acceptability of a low salt tomato soup (0.26% w/w). Subjects (n=148) scored their liking of tomato soup samples over five consecutive days. The first and last days were pre-and post-exposure visits where all participants rated three tomato soup samples; standard, low salt and low salt with added herbs and spices. The middle 3 days were the repeated exposure phase where participants were divided into three balanced groups; consuming the standard soup, the low salt soup, or the low salt soup with added herbs and spices. Reducing salt in the tomato soup led to a significant decline in consumer acceptability, and incorporating herbs and spices did not lead to an immediate enhancement in liking. However, inclusion of herbs and spices enhanced the perception of the salty taste of the low salt soup to the same level as the standard. Repeated exposure to the herbs and spice-modified soup led to a significant increase in the overall liking and liking of flavour, texture and aftertaste of the soup, whereas no changes in liking were observed for the standard and low salt tomato soups over repeated exposure. Moreover, a positive trend in increasing the post-exposure liking of the herbs and spices soup was observed. The findings suggest that the use of herbs and spices is a useful approach to reduce salt content in foods; however, herbs and spices should be chosen carefully to complement the food as large contrasts in flavour can polarise consumer liking.

Exposure to foods’ non-taste sensory properties: a nursery intervention to increase children’s willingness to try fruit and vegetables

Activities that engage young children with the sensory properties of foods are popular with nursery schools, despite the lack of evidence for their efficacy in increasing children's consumption of healthy foods. This study provides the first empirical exploration of the effectiveness of a non-taste sensory activity program in a nursery school setting. Ninety-two children aged between 12 and 36 months were allocated to either an intervention group, who took part in looking, listening, feeling and smelling activities with unusual fruits and vegetables every day for four weeks, or to a non-intervention control group. In a subsequent mealtime taste test, children touched and tasted more of the vegetables to which they had been familiarized in their playtime activities than of a matched set of non-exposed foods. The results demonstrate that hands-on activities with unfamiliar fruits and vegetables can enhance children’s willingness to taste these foods, and confirm the potential for such activities to support healthy eating initiatives.

Novel flavours paired with glutamate condition increased intake in older adults in the absence of changes in liking

Previous research on the repeat exposure to a novel flavour combined with monosodium glutamate (MSG) has shown an increase in liking and consumption for the particular flavour. The aim of the current work was to investigate whether this could also be observed in the case of older people, since they are most affected by undernutrition in the developed world and ways to increase consumption of food are of significant importance for this particular age group. For this study, 40 older adults (age 65-88) repeatedly consumed potato soup with two novel flavours (lemongrass and cumin) which were either with or without a high level of MSG (5%w/w). A randomized single blind within-subject design was implemented, where each participant was exposed to both soup flavours three times over 6 days, with one of the soup flavours containing MSG. After three repeat exposures, consumption increased significantly for the soups where the flavours had contained MSG during the repeated exposure (mean weight consumed increased from 123 to 164 g, p=0.017), implying that glutamate conditioned for increased wanting and consumption, despite the fact that the liking for the soup had not increased.

TESSA - Toolkit for experimentation with multimodal sensory substitution and augmentation

TESSA is a toolkit for experimenting with sensory augmentation. It includes hardware and software to facilitate rapid prototyping of interfaces that can enhance one sense using information gathered from another sense. The toolkit contains a range of sensors (e.g. ultrasonics, temperature sensors) and actuators (e.g. tactors or stereo sound), designed modularly so that inputs and outputs can be easily swapped in and out and customized using TESSA’s graphical user interface (GUI), with “real time” feedback. The system runs on a Raspberry Pi with a built-in touchscreen, providing a compact and portable form that is amenable for field trials. At CHI Interactivity, the audience will have the opportunity to experience sensory augmentation effects using this system, and design their own sensory augmentation interfaces.

Interactive sensory objects for and by people with learning disabilities

This project engages people with learning disabilities as co-researchers and co-designers in the development of multisensory interactive artworks, with the aim of making museums or heritage sites more interesting, meaningful, and fun. This article describes our explorations, within this context, of a range of technologies including squishy circuits, littleBits, and easy-build websites, and presents examples of objects created by the co-researchers such as “sensory boxes” and interactive buckets, baskets, and boots. Public engagement is an important part of the project and includes an annual public event and seminar day, a blog rich with photos and videos of the workshops, and an activities book to give people ideas for creating their own sensory explorations of museums and heritage sites.

Is it me? Self-recognition bias across sensory modalities and its relationship to autistic traits

Background Atypical self-processing is an emerging theme in autism research, suggested by lower self-reference effect in memory, and atypical neural responses to visual self-representations. Most research on physical self-processing in autism uses visual stimuli. However, the self is a multimodal construct, and therefore, it is essential to test self-recognition in other sensory modalities as well. Self-recognition in the auditory modality remains relatively unexplored and has not been tested in relation to autism and related traits. This study investigates self-recognition in auditory and visual domain in the general population and tests if it is associated with autistic traits. Methods Thirty-nine neurotypical adults participated in a two-part study. In the first session, individual participant’s voice was recorded and face was photographed and morphed respectively with voices and faces from unfamiliar identities. In the second session, participants performed a ‘self-identification’ task, classifying each morph as ‘self’ voice (or face) or an ‘other’ voice (or face). All participants also completed the Autism Spectrum Quotient (AQ). For each sensory modality, slope of the self-recognition curve was used as individual self-recognition metric. These two self-recognition metrics were tested for association between each other, and with autistic traits. Results Fifty percent ‘self’ response was reached for a higher percentage of self in the auditory domain compared to the visual domain (t = 3.142; P < 0.01). No significant correlation was noted between self-recognition bias across sensory modalities (τ = −0.165, P = 0.204). Higher recognition bias for self-voice was observed in individuals higher in autistic traits (τ AQ = 0.301, P = 0.008). No such correlation was observed between recognition bias for self-face and autistic traits (τ AQ = −0.020, P = 0.438). Conclusions Our data shows that recognition bias for physical self-representation is not related across sensory modalities. Further, individuals with higher autistic traits were better able to discriminate self from other voices, but this relation was not observed with self-face. A narrow self-other overlap in the auditory domain seen in individuals with high autistic traits could arise due to enhanced perceptual processing of auditory stimuli often observed in individuals with autism.

Sensory enhancement, a pilot perceptual study of subdermal magnetic implants

Subdermal magnetic implants originated as an art form in the world of body modification. To date an in depth scientific analysis of the benefits of this implant has yet to be established. This research explores the concept of sensory extension of the tactile sense utilising this form of implantation. This relatively simple procedure enables the tactile sense to respond to static and alternating magnetic fields. This is not to say that the underlying biology of the system has changed; i.e. the concept does not increase our tactile frequency response range or sensitivity to pressure, but now does invoke a perceptual response to a stimulus that is not innately available to humans. Within this research two social surveys have been conducted in order to ascertain one, the social acceptance of the general notion of human enhancement, and two the perceptual experiences of individuals with the magnetic implants themselves. In terms of acceptance to the notion of sensory improvement (via implantation) ~39% of the general population questioned responded positively with a further ~25% of the respondents answering with the indecisive response. Thus with careful dissemination a large proportion of individuals may adopt this technology much like this if it were to become available for consumers. Interestingly of the responses collected from the magnetic implants survey ~60% of the respondents actually underwent the implant for magnetic vision purposes. The main contribution of this research however comes from a series of psychophysical testing. In which 7 subjects with subdermal magnetic implants, were cross compared with 7 subjects that had similar magnets superficially attached to their dermis. The experimentation examined multiple psychometric thresholds of the candidates including intensity, frequency and temporal. Whilst relatively simple, the experimental setup for the perceptual experimentation conducted was novel in that custom hardware and protocols were created in order to determine the subjective thresholds of the individuals. Abstract iv The overall purpose of this research is to utilise this concept in high stress scenarios, such as driving or piloting; whereby alerts and warnings could be relayed to an operator without intruding upon their other (typically overloaded) exterior senses (i.e. the auditory and visual senses). Hence each of the thresholding experiments were designed with the intention of utilising the results in the design of signals for information transfer. The findings from the study show that the implanted group of subjects significantly outperformed the superficial group in the absolute intensity threshold experiment, i.e. the implanted group required significantly less force than the superficial group in order to perceive the stimulus. The results for the frequency difference threshold showed no significant difference in the two groups tested. Interestingly however at low frequencies, i.e. 20 and 50 Hz, the ability of the subjects tested to discriminate frequencies significantly increased with more complex waveforms i.e. square and sawtooth, when compared against the typically used sinewave. Furthermore a novel protocol for establishing the temporal gap detection threshold during a temporal numerosity study has been established in this thesis. This experiment measured the subjects’ capability to correctly determine the number of concatenated signals presented to them whilst the time between the signals, referred to as pulses, tended to zero. A significant finding was that when altering the length of, the frequency of, and the number of cycles of the pulses, the time between pulses for correct recognition altered. This finding will ultimately aid in the design of the tactile alerts for this method of information transfer. Preliminary development work for the use of this method of input to the body, in an automotive scenario, is also presented within this thesis in the form of a driving simulation. The overall goal of which is to present warning alerts to a driver, such as rear-to-end collision, or excessive speeds on roads, in order to prevent incidents and penalties from occurring. Discussion on the broader utility of this implant has been presented, reflecting on its potential use as a basis for vibrotactile, and sensory substitution, devices. This discussion furthers with postulations on its use as a human machine interface, as well as how a similar implant could be used within the ear as a hearing aid device.

Anticipatory engineering: anticipation in sensory-motor systems of human

In visual tracking experiments, distributions of the relative phase be-tween target and tracer showed positive relative phase indicating that the tracer precedes the target position. We found a mode transition from the reactive to anticipatory mode. The proposed integrated model provides a framework to understand the antici-patory behaviour of human, focusing on the integration of visual and soma-tosensory information. The time delays in visual processing and somatosensory feedback are explicitly treated in the simultaneous differential equations. The anticipatory behaviour observed in the visual tracking experiments can be ex-plained by the feedforward term of target velocity, internal dynamics, and time delay in somatosensory feedback.