Gesture-Based, Touch-Free Multi-User Gaming on Wall-Sized, High-Resolution Tiled Displays

Having to carry input devices can be inconvenient when interacting with wall-sized, high-resolution tiled displays. Such displays are typically driven by a cluster of computers. Running existing games on a cluster is non-trivial, and the performance attained using software solutions like Chromium is not good enough. This paper presents a touch-free, multi-user, humancomputer interface for wall-sized displays that enables completely device-free interaction. The interface is built using 16 cameras and a cluster of computers, and is integrated with the games Quake 3 Arena (Q3A) and Homeworld. The two games were parallelized using two different approaches in order to run on a 7x4 tile, 21 megapixel display wall with good performance. The touch-free interface enables interaction with a latency of 116 ms, where 81 ms are due to the camera hardware. The rendering performance of the games is compared to their sequential counterparts running on the display wall using Chromium. Parallel Q3A’s framerate is an order of magnitude higher compared to using Chromium. The parallel version of Homeworld performed on par with the sequential, which did not run at all using Chromium. Informal use of the touch-free interface indicates that it works better for controlling Q3A than Homeworld.

Grapheme-color synaesthesia is associated with a distinct cognitive style

In this study we investigated whether synaesthesia is associated with a particular cognitive style. Cognitive style refers to preferred modes of information processing, such as a verbal style or a visual style. We reasoned that related to the enriched world of experiences created by synaesthesia, its association with enhanced verbal and visual memory, higher imagery and creativity, synaesthetes might show enhanced preference for a verbal as well as for a visual cognitive style compared to non-synaesthetes. In Study 1 we tested a large convenience sample of 1046 participants, who classified themselves as grapheme-color, sound-color, lexical-gustatory, sequence-space, or as non-synaesthetes. To assess cognitive style, we used the revised verbalizer-visualizer questionnaire (VVQ), which involves three independent cognitive style dimensions (verbal style, visual-spatial style, and vivid imagery style). The most important result was that those who reported grapheme-color synaesthesia showed higher ratings on the verbal and vivid imagery style dimensions, but not on the visual-spatial style dimension. In Study 2 we replicated this finding in a laboratory study involving 24 grapheme-color synaesthetes with objectively confirmed synaesthesia and a closely matched control group. Our results indicate that grapheme-color synaesthetes prefer both a verbal and a specific visual cognitive style. We suggest that this enhanced preference, probably together with the greater ease to switch between a verbal and a vivid visual imagery style, may be related to cognitive advantages associated with grapheme color synaesthesia such as enhanced memory performance and creativity.

Developmental aspects of synaesthesia across the adult lifespan

In synaesthesia, stimuli such as sounds, words or letters trigger experiences of colors, shapes or tastes and the consistency of these experiences is a hallmark of this condition. In this study we investigate for the first time whether there are age-related changes in the consistency of synaesthetic experiences. We tested a sample of more than 400 grapheme-color synaesthetes who have color experiences when they see letters and/or digits with a well-established test of consistency. Our results showed a decline in the number of consistent grapheme-color associations across the adult lifespan. We also assessed age-related changes in the breadth of the color spectrum. The results showed that the appearance of primary colors (i.e., red, blue, and green) was mainly age-invariant. However, there was a decline in the occurrence of lurid colors while brown and achromatic tones occurred more often as concurrents in older age. These shifts in the color spectrum suggest that synaesthesia does not simply fade, but rather undergoes more comprehensive changes. We propose that these changes are the result of a combination of both age-related perceptual and memory processing shifts.

Acquiring synaesthesia: insights from training studies

Synaesthesia denotes a condition of remarkable individual differences in experience characterized by specific additional experiences in response to normal sensory input. Synaesthesia seems to (i) run in families which suggests a genetic component, (ii) is associated with marked structural and functional neural differences, and (iii) is usually reported to exist from early childhood. Hence, synaesthesia is generally regarded as a congenital phenomenon. However, most synaesthetic experiences are triggered by cultural artifacts (e.g., letters, musical sounds). Evidence exists to suggest that synaesthetic experiences are triggered by the conceptual representation of their inducer stimuli. Cases were identified for which the specific synaesthetic associations are related to prior experiences and large scale studies show that grapheme-color associations in synaesthesia are not completely random. Hence, a learning component is inherently involved in the development of specific synaesthetic associations. Researchers have hypothesized that associative learning is the critical mechanism. Recently, it has become of scientific and public interest if synaesthetic experiences may be acquired by means of associative training procedures and whether the gains of these trainings are associated with similar cognitive benefits as genuine synaesthetic experiences. In order to shed light on these issues and inform synaesthesia researchers and the general interested public alike, we provide a comprehensive literature review on developmental aspects of synaesthesia and specific training procedures in non-synaesthetes. Under the light of a clear working definition of synaesthesia, we come to the conclusion that synaesthesia can potentially be learned by the appropriate training.

Semantic Representation of Synaesthesia

Synaesthesia has multifaceted consequences for both subjective experience and cognitive performance. Here, I broach the issue of how synaesthesia is represented in semantic memory. I hypothesize that, for example, in grapheme colour synaesthesia, colour is represented as an additional feature in the semantic network that enables the formation of associations that are not present in non-synaesthetes. Thus, synaesthesia provokes richer memory representations which enable learning opportunities that are not present in non-synaesthetes, provides additional memory cues, and may trigger creative ideas.

Enhanced memory ability: insights from synaesthesia

People with synaesthesia show an enhanced memory relative to demographically matched controls. The most obvious explanation for this is that the ‘extra’ perceptual experiences lead to richer encoding and retrieval opportunities of stimuli which induce synaesthesia (typically verbal stimuli). Although there is some evidence for this, it is unlikely to be the whole explanation. For instance, not all stimuli which trigger synaesthesia are better remembered (e.g., digit span) and some stimuli which do not trigger synaesthesia are better remembered. In fact, synaesthetes tend to have better visual memory than verbal memory. We suggest that enhanced memory in synaesthesia is linked to wider changes in cognitive systems at the interface of perception and memory and link this to recent findings in the neuroscience of memory.

Psychophysiological evidence for the genuineness of swimming-style colour synaesthesia

Recently, swimming-style colour synaesthesia was introduced as a new form of synaesthesia. A synaesthetic Stroop test was used to establish its genuineness. Since Stroop interference can occur for any type of overlearned association, in the present study we used a modified Stroop test and psychophysiological synaesthetic conditioning to further establish the genuineness of this form of synaesthesia. We compared the performance of a swimming-style colour synaesthete and a control who was trained on swimming-style colour associations. Our results showed that behavioural aspects of swimming-style colour synaesthesia can be mimicked in a trained control. Importantly, however, our results showed a psychophysiological conditioning effect for the synaesthete only. We discuss the theoretical relevance of swimming-style colour synaesthesia according to different models of synaesthesia. We conclude that swimming-style colour synaesthesia is a genuine form of synaesthesia, can be mimicked behaviourally in non-synaesthetes, and is best explained by a re-entrant feedback model.

Coloured Letters and Numbers (CLaN): A reliable factor-analysis based synaesthesia questionnaire

Synaesthesia is a heterogeneous phenomenon, even when considering one particular sub-type. The purpose of this study was to design a reliable and valid questionnaire for grapheme-colour synaesthesia that captures this heterogeneity. By the means of a large sample of 628 synaesthetes and a factor analysis, we created the Coloured Letters and Numbers (CLaN) questionnaire with 16 items loading on 4 different factors (i.e., localisation, automaticity/attention, deliberate use, and longitudinal changes). These factors were externally validated with tests which are widely used in the field of synaesthesia research. The questionnaire showed good test–retest reliability and construct validity (i.e., internally and externally). Our findings are discussed in the light of current theories and new ideas in synaesthesia research. More generally, the questionnaire is a useful tool which can be widely used in synaesthesia research to reveal the influence of individual differences on various performance measures and will be useful in generating new hypotheses.