Animated virtual agents to cue user attention: comparison of static and dynamic deictic cues on gaze and touch responses

This paper describes an experiment developed to study the performance of virtual agent animated cues within digital interfaces. Increasingly, agents are used in virtual environments as part of the branding process and to guide user interaction. However, the level of agent detail required to establish and enhance efficient allocation of attention remains unclear. Although complex agent motion is now possible, it is costly to implement and so should only be routinely implemented if a clear benefit can be shown. Pevious methods of assessing the effect of gaze-cueing as a solution to scene complexity have relied principally on two-dimensional static scenes and manual peripheral inputs. Two experiments were run to address the question of agent cues on human-computer interfaces. Both experiments measured the efficiency of agent cues analyzing participant responses either by gaze or by touch respectively. In the first experiment, an eye-movement recorder was used to directly assess the immediate overt allocation of attention by capturing the participant’s eyefixations following presentation of a cueing stimulus. We found that a fully animated agent could speed up user interaction with the interface. When user attention was directed using a fully animated agent cue, users responded 35% faster when compared with stepped 2-image agent cues, and 42% faster when compared with a static 1-image cue. The second experiment recorded participant responses on a touch screen using same agent cues. Analysis of touch inputs confirmed the results of gaze-experiment, where fully animated agent made shortest time response with a slight decrease on the time difference comparisons. Responses to fully animated agent were 17% and 20% faster when compared with 2-image and 1-image cue severally. These results inform techniques aimed at engaging users’ attention in complex scenes such as computer games and digital transactions within public or social interaction contexts by demonstrating the benefits of dynamic gaze and head cueing directly on the users’ eye movements and touch responses.

Combining Textual and Visual Cues for Content-based Image Retrieval on the World Wide Web

Some WWW image engines allow the user to form a query in terms of text keywords. To build the image index, keywords are extracted heuristically from HTML documents containing each image, and/or from the image URL and file headers. Unfortunately, text-based image engines have merely retro-fitted standard SQL database query methods, and it is difficult to include images cues within such a framework. On the other hand, visual statistics (e.g., color histograms) are often insufficient for helping users find desired images in a vast WWW index. By truly unifying textual and visual statistics, one would expect to get better results than either used separately. In this paper, we propose an approach that allows the combination of visual statistics with textual statistics in the vector space representation commonly used in query by image content systems. Text statistics are captured in vector form using latent semantic indexing (LSI). The LSI index for an HTML document is then associated with each of the images contained therein. Visual statistics (e.g., color, orientedness) are also computed for each image. The LSI and visual statistic vectors are then combined into a single index vector that can be used for content-based search of the resulting image database. By using an integrated approach, we are able to take advantage of possible statistical couplings between the topic of the document (latent semantic content) and the contents of images (visual statistics). This allows improved performance in conducting content-based search. This approach has been implemented in a WWW image search engine prototype.

Parallel Auditory Filtering By Sustained and Transient Channels Separates Coarticulated Vowels and Consonants

A neural model of peripheral auditory processing is described and used to separate features of coarticulated vowels and consonants. After preprocessing of speech via a filterbank, the model splits into two parallel channels, a sustained channel and a transient channel. The sustained channel is sensitive to relatively stable parts of the speech waveform, notably synchronous properties of the vocalic portion of the stimulus it extends the dynamic range of eighth nerve filters using coincidence deteectors that combine operations of raising to a power, rectification, delay, multiplication, time averaging, and preemphasis. The transient channel is sensitive to critical features at the onsets and offsets of speech segments. It is built up from fast excitatory neurons that are modulated by slow inhibitory interneurons. These units are combined over high frequency and low frequency ranges using operations of rectification, normalization, multiplicative gating, and opponent processing. Detectors sensitive to frication and to onset or offset of stop consonants and vowels are described. Model properties are characterized by mathematical analysis and computer simulations. Neural analogs of model cells in the cochlear nucleus and inferior colliculus are noted, as are psychophysical data about perception of CV syllables that may be explained by the sustained transient channel hypothesis. The proposed sustained and transient processing seems to be an auditory analog of the sustained and transient processing that is known to occur in vision.

Stream segregation on a single electrode as a function of pulse rate in cochlear implant listeners.

While cochlear implants (CIs) usually provide high levels of speech recognition in quiet, speech recognition in noise remains challenging. To overcome these difficulties, it is important to understand how implanted listeners separate a target signal from interferers. Stream segregation has been studied extensively in both normal and electric hearing, as a function of place of stimulation. However, the effects of pulse rate, independent of place, on the perceptual grouping of sequential sounds in electric hearing have not yet been investigated. A rhythm detection task was used to measure stream segregation. The results of this study suggest that while CI listeners can segregate streams based on differences in pulse rate alone, the amount of stream segregation observed decreases as the base pulse rate increases. Further investigation of the perceptual dimensions encoded by the pulse rate and the effect of sequential presentation of different stimulation rates on perception could be beneficial for the future development of speech processing strategies for CIs.

Deformation of stem cell nuclei by nanotopographical cues.

Cells sense cues in their surrounding microenvironment. These cues are converted into intracellular signals and transduced to the nucleus in order for the cell to respond and adapt its function. Within the nucleus, structural changes occur that ultimately lead to changes in the gene expression. In this study, we explore the structural changes of the nucleus of human mesenchymal stem cells as an effect of topographical cues. We use a controlled nanotopography to drive shape changes to the cell nucleus, and measure the changes with both fluorescence microscopy and a novel light scattering technique. The nucleus changes shape dramatically in response to the nanotopography, and in a manner dependent on the mechanical properties of the substrate. The kinetics of the nuclear deformation follows an unexpected trajectory. As opposed to a gradual shape change in response to the topography, once the cytoskeleton attains an aligned and elongation morphology on the time scale of several hours, the nucleus changes shape rapidly and intensely.

Myosin VIIA, important for human auditory function, is necessary for Drosophila auditory organ development.

BACKGROUND: Myosin VIIA (MyoVIIA) is an unconventional myosin necessary for vertebrate audition [1]-[5]. Human auditory transduction occurs in sensory hair cells with a staircase-like arrangement of apical protrusions called stereocilia. In these hair cells, MyoVIIA maintains stereocilia organization [6]. Severe mutations in the Drosophila MyoVIIA orthologue, crinkled (ck), are semi-lethal [7] and lead to deafness by disrupting antennal auditory organ (Johnston's Organ, JO) organization [8]. ck/MyoVIIA mutations result in apical detachment of auditory transduction units (scolopidia) from the cuticle that transmits antennal vibrations as mechanical stimuli to JO. PRINCIPAL FINDINGS: Using flies expressing GFP-tagged NompA, a protein required for auditory organ organization in Drosophila, we examined the role of ck/MyoVIIA in JO development and maintenance through confocal microscopy and extracellular electrophysiology. Here we show that ck/MyoVIIA is necessary early in the developing antenna for initial apical attachment of the scolopidia to the articulating joint. ck/MyoVIIA is also necessary to maintain scolopidial attachment throughout adulthood. Moreover, in the adult JO, ck/MyoVIIA genetically interacts with the non-muscle myosin II (through its regulatory light chain protein and the myosin binding subunit of myosin II phosphatase). Such genetic interactions have not previously been observed in scolopidia. These factors are therefore candidates for modulating MyoVIIA activity in vertebrates. CONCLUSIONS: Our findings indicate that MyoVIIA plays evolutionarily conserved roles in auditory organ development and maintenance in invertebrates and vertebrates, enhancing our understanding of auditory organ development and function, as well as providing significant clues for future research.

Different stimuli, different spatial codes: a visual map and an auditory rate code for oculomotor space in the primate superior colliculus.

Maps are a mainstay of visual, somatosensory, and motor coding in many species. However, auditory maps of space have not been reported in the primate brain. Instead, recent studies have suggested that sound location may be encoded via broadly responsive neurons whose firing rates vary roughly proportionately with sound azimuth. Within frontal space, maps and such rate codes involve different response patterns at the level of individual neurons. Maps consist of neurons exhibiting circumscribed receptive fields, whereas rate codes involve open-ended response patterns that peak in the periphery. This coding format discrepancy therefore poses a potential problem for brain regions responsible for representing both visual and auditory information. Here, we investigated the coding of auditory space in the primate superior colliculus(SC), a structure known to contain visual and oculomotor maps for guiding saccades. We report that, for visual stimuli, neurons showed circumscribed receptive fields consistent with a map, but for auditory stimuli, they had open-ended response patterns consistent with a rate or level-of-activity code for location. The discrepant response patterns were not segregated into different neural populations but occurred in the same neurons. We show that a read-out algorithm in which the site and level of SC activity both contribute to the computation of stimulus location is successful at evaluating the discrepant visual and auditory codes, and can account for subtle but systematic differences in the accuracy of auditory compared to visual saccades. This suggests that a given population of neurons can use different codes to support appropriate multimodal behavior.

Sensory information and associative cues used in food detection by wild vervet monkeys.

Understanding animals' spatial perception is a critical step toward discerning their cognitive processes. The spatial sense is multimodal and based on both the external world and mental representations of that world. Navigation in each species depends upon its evolutionary history, physiology, and ecological niche. We carried out foraging experiments on wild vervet monkeys (Chlorocebus pygerythrus) at Lake Nabugabo, Uganda, to determine the types of cues used to detect food and whether associative cues could be used to find hidden food. Our first and second set of experiments differentiated between vervets' use of global spatial cues (including the arrangement of feeding platforms within the surrounding vegetation) and/or local layout cues (the position of platforms relative to one another), relative to the use of goal-object cues on each platform. Our third experiment provided an associative cue to the presence of food with global spatial, local layout, and goal-object cues disguised. Vervets located food above chance levels when goal-object cues and associative cues were present, and visual signals were the predominant goal-object cues that they attended to. With similar sample sizes and methods as previous studies on New World monkeys, vervets were not able to locate food using only global spatial cues and local layout cues, unlike all five species of platyrrhines thus far tested. Relative to these platyrrhines, the spatial location of food may need to stay the same for a longer time period before vervets encode this information, and goal-object cues may be more salient for them in small-scale space.

Baby on board: olfactory cues indicate pregnancy and fetal sex in a non-human primate.

Olfactory cues play an integral, albeit underappreciated, role in mediating vertebrate social and reproductive behaviour. These cues fluctuate with the signaller's hormonal condition, coincident with and informative about relevant aspects of its reproductive state, such as pubertal onset, change in season and, in females, timing of ovulation. Although pregnancy dramatically alters a female's endocrine profiles, which can be further influenced by fetal sex, the relationship between gestation and olfactory cues is poorly understood. We therefore examined the effects of pregnancy and fetal sex on volatile genital secretions in the ring-tailed lemur (Lemur catta), a strepsirrhine primate possessing complex olfactory mechanisms of reproductive signalling. While pregnant, dams altered and dampened their expression of volatile chemicals, with compound richness being particularly reduced in dams bearing sons. These changes were comparable in magnitude with other, published chemical differences among lemurs that are salient to conspecifics. Such olfactory 'signatures' of pregnancy may help guide social interactions, potentially promoting mother-infant recognition, reducing intragroup conflict or counteracting behavioural mechanisms of paternity confusion; cues that also advertise fetal sex may additionally facilitate differential sex allocation.

Schema-driven construction of future autobiographical traumatic events: the future is much more troubling than the past.

Research on future episodic thought has produced compelling theories and results in cognitive psychology, cognitive neuroscience, and clinical psychology. In experiments aimed to integrate these with basic concepts and methods from autobiographical memory research, 76 undergraduates remembered past and imagined future positive and negative events that had or would have a major impact on them. Correlations of the online ratings of visual and auditory imagery, emotion, and other measures demonstrated that individuals used the same processes to the same extent to remember past and construct future events. These measures predicted the theoretically important metacognitive judgment of past reliving and future "preliving" in similar ways. On standardized tests of reactions to traumatic events, scores for future negative events were much higher than scores for past negative events. The scores for future negative events were in the range that would qualify for a diagnosis of posttraumatic stress disorder (PTSD); the test was replicated (n = 52) to check for order effects. Consistent with earlier work, future events had less sensory vividness. Thus, the imagined symptoms of future events were unlikely to be caused by sensory vividness. In a second experiment, to confirm this, 63 undergraduates produced numerous added details between 2 constructions of the same negative future events; deficits in rated vividness were removed with no increase in the standardized tests of reactions to traumatic events. Neuroticism predicted individuals' reactions to negative past events but did not predict imagined reactions to future events. This set of novel methods and findings is interpreted in the contexts of the literatures of episodic future thought, autobiographical memory, PTSD, and classic schema theory.

The spatiotemporal dynamics of autobiographical memory: neural correlates of recall, emotional intensity, and reliving.

We sought to map the time course of autobiographical memory retrieval, including brain regions that mediate phenomenological experiences of reliving and emotional intensity. Participants recalled personal memories to auditory word cues during event-related functional magnetic resonance imaging (fMRI). Participants pressed a button when a memory was accessed, maintained and elaborated the memory, and then gave subjective ratings of emotion and reliving. A novel fMRI approach based on timing differences capitalized on the protracted reconstructive process of autobiographical memory to segregate brain areas contributing to initial access and later elaboration and maintenance of episodic memories. The initial period engaged hippocampal, retrosplenial, and medial and right prefrontal activity, whereas the later period recruited visual, precuneus, and left prefrontal activity. Emotional intensity ratings were correlated with activity in several regions, including the amygdala and the hippocampus during the initial period. Reliving ratings were correlated with activity in visual cortex and ventromedial and inferior prefrontal regions during the later period. Frontopolar cortex was the only brain region sensitive to emotional intensity across both periods. Results were confirmed by time-locked averages of the fMRI signal. The findings indicate dynamic recruitment of emotion-, memory-, and sensory-related brain regions during remembering and their dissociable contributions to phenomenological features of the memories.

Activation in mesolimbic and visuospatial neural circuits elicited by smoking cues: evidence from functional magnetic resonance imaging.

OBJECTIVE: The authors sought to increase understanding of the brain mechanisms involved in cigarette addiction by identifying neural substrates modulated by visual smoking cues in nicotine-deprived smokers. METHOD: Event-related functional magnetic resonance imaging (fMRI) was used to detect brain activation after exposure to smoking-related images in a group of nicotine-deprived smokers and a nonsmoking comparison group. Subjects viewed a pseudo-random sequence of smoking images, neutral nonsmoking images, and rare targets (photographs of animals). Subjects pressed a button whenever a rare target appeared. RESULTS: In smokers, the fMRI signal was greater after exposure to smoking-related images than after exposure to neutral images in mesolimbic dopamine reward circuits known to be activated by addictive drugs (right posterior amygdala, posterior hippocampus, ventral tegmental area, and medial thalamus) as well as in areas related to visuospatial attention (bilateral prefrontal and parietal cortex and right fusiform gyrus). In nonsmokers, no significant differences in fMRI signal following exposure to smoking-related and neutral images were detected. In most regions studied, both subject groups showed greater activation following presentation of rare target images than after exposure to neutral images. CONCLUSIONS: In nicotine-deprived smokers, both reward and attention circuits were activated by exposure to smoking-related images. Smoking cues are processed like rare targets in that they activate attentional regions. These cues are also processed like addictive drugs in that they activate mesolimbic reward regions.

Properties of word cues for autobiographical memory.

A sample of 124 words were used to cue autobiographical memories in 120 adults varying in age from 20 to 73 years. Individual words reliably cued autobiographical memories of different ages with different speeds. For all age groups, words rated high in imagery produced older memories and faster reaction times.

Rhyme and Reason: Analyses of Dual Retrieval Cues

If and only if each single cue uniquely defines its target, a independence model based on fragment theory can predict the strength of a combined dual cue from the strengths of its single cue components. If the single cues do not each uniquely define their target, no single monotonic function can predict the strength of the dual cue from its components; rather, what matters is the number of possible targets. The probability of generating a target word was .19 for rhyme cues, .14 for category cues, and .97 for rhyme-plus-category dual cues. Moreover, some pairs of cues had probabilities of producing their targets of .03 when used individually and 1.00 when used together, whereas other pairs had moderate probabilities individually and together. The results, which are interpreted in terms of multiple constraints limiting the number of responses, show why rhymes, which play a minimal role in laboratory studies of memory, are common in real-world mnemonics.