A helmet mounted display system with active gaze control for visual telepresence

The objective of a Visual Telepresence System is to provide the operator with a high fidelity image from a remote stereo camera pair linked to a pan/tilt device such that the operator may reorient the camera position by use of head movement. Systems such as these which utilise virtual reality style helmet mounted displays have a number of limitations. The geometry of the camera positions and of the displays is generally fixed and is most suitable only for viewing elements of a scene at a particular distance. To address such limitations, a prototype system has been developed where the geometry of the displays and cameras is dynamically controlled by the eye movement of the operator. This paper explores why it is necessary to actively adjust the display system as well as the cameras and justifies the use of mechanical adjustment of the displays as an alternative to adjustment by electronic or image processing methods. The electronic and mechanical design is described including optical arrangements and control algorithms. The performance and accuracy of the system is assessed with respect to eye movement.

ERP correlates of shared control mechanisms involved in saccade preparation and in covert attention

We investigated whether attention shifts and eye movement preparation are mediated by shared control mechanisms, as claimed by the premotor theory of attention. ERPs were recorded in three tasks where directional cues presented at the beginning of each trial instructed participants to direct their attention to the cued side without eye movements (Covert task), to prepare an eye movement in the cued direction without attention shifts (Saccade task) or both (Combined task). A peripheral visual Go/Nogo stimulus that was presented 800 ms after cue onset signalled whether responses had to be executed or withheld. Lateralised ERP components triggered during the cue–target interval, which are assumed to reflect preparatory control mechanisms that mediate attentional orienting, were very similar across tasks. They were also present in the Saccade task, which was designed to discourage any concomitant covert attention shifts. These results support the hypothesis that saccade preparation and attentional orienting are implemented by common control structures. There were however systematic differences in the impact of eye movement programming and covert attention on ERPs triggered in response to visual stimuli at cued versus uncued locations. It is concluded that, although the preparatory processes underlying saccade programming and covert attentional orienting may be based on common mechanisms, they nevertheless differ in their spatially specific effects on visual information processing.

Manual response preparation and saccade programming are linked to attention shifts: ERP evidence for covert attentional orienting and spatially specific modulations of visual processing

The premotor theory of attention claims that attentional shifts are triggered during response programming, regardless of which response modality is involved. To investigate this claim, event-related brain potentials (ERPs) were recorded while participants covertly prepared a left or right response, as indicated by a precue presented at the beginning of each trial. Cues signalled a left or right eye movement in the saccade task, and a left or right manual response in the manual task. The cued response had to be executed or withheld following the presentation of a Go/Nogo stimulus. Although there were systematic differences between ERPs triggered during covert manual and saccade preparation, lateralised ERP components sensitive to the direction of a cued response were very similar for both tasks, and also similar to the components previously found during cued shifts of endogenous spatial attention. This is consistent with the claim that the control of attention and of covert response preparation are closely linked. N1 components triggered by task-irrelevant visual probes presented during the covert response preparation interval were enhanced when these probes were presented close to cued response hand in the manual task, and at the saccade target location in the saccade task. This demonstrates that both manual and saccade preparation result in spatially specific modulations of visual processing, in line with the predictions of the premotor theory.

Dynamic vergence telepresence system incorporating active gaze control

Visual Telepresence system which utilize virtual reality style helmet mounted displays have a number of limitations. The geometry of the camera positions and of the display is fixed and is most suitable only for viewing elements of a scene at a particular distance. In such a system, the operator's ability to gaze around without use of head movement is severely limited. A trade off must be made between a poor viewing resolution or a narrow width of viewing field. To address these limitations a prototype system where the geometry of the displays and cameras is dynamically controlled by the eye movement of the operator has been developed. This paper explores the reasons why is necessary to actively adjust both the display system and the cameras and furthermore justifies the use of mechanical adjustment of the displays as an alternative to adjustment by electronic or image processing methods. The electronic and mechanical design is described including optical arrangements and control algorithms, An assessment of the performance of the system against a fixed camera/display system when operators are assigned basic tasks involving depth and distance/size perception. The sensitivity to variations in transient performance of the display and camera vergence is also assessed.

The time-course of morphological constraints: a study of plurals inside derived words

The avoidance of regular but not irregular plurals inside compounds (e.g. *rats eater vs. mice eater) has been one of the most widely studied morphological phenomena in the psycholinguistics literature. To examine whether the constraints that are responsible for this contrast have any general significance beyond compounding, we investigated derived word forms containing regular and irregular plurals in two experiments. Experiment 1 was an offline acceptability judgment task, and experiment 2 measured eye movements during reading derived words containing regular and irregular plurals and uninflected base nouns. The results from both experiments show that the constraint against regular plurals inside compounds generalizes to derived words. We argue that this constraint cannot be reduced to phonological properties, but is instead morphological in nature. The eye-movement data provide detailed information on the time-course of processing derived word forms indicating that early stages of processing are affected by a general constraint that disallows inflected words from feeding derivational processes, and that the more specific constraint against regular plurals comes in at a subsequent later stage of processing. We argue that these results are consistent with stage-based models of language processing.

The timing of island effects in nonnative sentence processing

Using the eye-movement monitoring technique in two reading comprehension experiments, we investigated the timing of constraints on wh-dependencies (so-called ‘island’ constraints) in native and nonnative sentence processing. Our results show that both native and nonnative speakers of English are sensitive to extraction islands during processing, suggesting that memory storage limitations affect native and nonnative comprehenders in essentially the same way. Furthermore, our results show that the timing of island effects in native compared to nonnative sentence comprehension is affected differently by the type of cue (semantic fit versus filled gaps) signalling whether dependency formation is possible at a potential gap site. Whereas English native speakers showed immediate sensitivity to filled gaps but not to lack of semantic fit, proficient German-speaking learners of L2 English showed the opposite sensitivity pattern. This indicates that initial wh-dependency formation in nonnative processing is based on semantic feature-matching rather than being structurally mediated as in native comprehension.

The role of working memory in the processing of reflexives

We report results from two eye-movement experiments that examined how differences in working memory (WM) capacity affect readers' application of structural constraints on reflexive anaphor resolution during sentence comprehension. We examined whether binding Principle A, a syntactic constraint on the interpretation of reflexives, is reducible to a memory friendly “recency” strategy, and whether WM capacity influences the degree to which readers create anaphoric dependencies ruled out by binding theory. Our results indicate that low and high WM span readers applied Principle A early during processing. However, contrary to previous findings, low span readers also showed immediate intrusion effects of a linearly closer but structurally inaccessible competitor antecedent. We interpret these findings as indicating that although the relative prominence of potential antecedents in WM can affect online anaphor resolution, Principle A is not reducible to a processing or linear distance based “least effort” constraint.

The time-course of morphological processing in a second language

We report findings from psycholinguistic experiments investigating the detailed timing of processing morphologically complex words by proficient adult second (L2) language learners of English in comparison to adult native (L1) speakers of English. The first study employed the masked priming technique to investigate -ed forms with a group of advanced Arabic-speaking learners of English. The results replicate previously found L1/L2 differences in morphological priming, even though in the present experiment an extra temporal delay was offered after the presentation of the prime words. The second study examined the timing of constraints against inflected forms inside derived words in English using the eye-movement monitoring technique and an additional acceptability judgment task with highly advanced Dutch L2 learners of English in comparison to adult L1 English controls. Whilst offline the L2 learners performed native-like, the eye-movement data showed that their online processing was not affected by the morphological constraint against regular plurals inside derived words in the same way as in native speakers. Taken together, these findings indicate that L2 learners are not just slower than native speakers in processing morphologically complex words, but that the L2 comprehension system employs real-time grammatical analysis (in this case, morphological information) less than the L1 system.

The influence of distractors on saccade­ target selection: saccade trajectory effects

It has long been known that the path (trajectory) taken by the eye to land on a target is rarely straight (Yarbus, 1967). Furthermore, the magnitude and direction of this natural tendency for curvature can be modulated by the presence of a competing distractor stimu­ lus presented along with the saccade target. The distractor­related modulation of saccade trajectories provides a subtle measure of the underlying competitive processes involved in saccade target selection. Here we review some of our own studies into the effects distract­ ors have on saccade trajectories, which can be regarded as a way of probing the competit­ ive balance between target and distractor salience.

Coargumenthood and the processing of reflexives

We report three eye-movement experiments and an antecedent choice task investigating the interpretation of reflexives in different syntactic contexts. This included contexts in which the reflexive and a local antecedent were coarguments of the same verbal predicate (John heard that the soldier had injured himself), and also so-called picture noun phrases, either with a possessor (John heard about the soldier’s picture of himself) or without (John heard that the soldier had a picture of himself). While results from the antecedent choice task indicated that comprehenders would choose a nonlocal antecedent (‘John’ above) for reflexives in either type of picture noun phrase, the eye-movement experiments suggested that participants preferred to initially interpret the reflexive in each context as referring to the local antecedent (‘the soldier’), as indexed by longer reading times when it mismatched in gender with the reflexive. We also observed a difference in the time-course of this effect. While it was observed during first-pass processing at the reflexive for coargument reflexives and those in picture noun phrases with a possessor, it was comparatively delayed for reflexives in possessorless picture noun phrases. These results suggest that locality constraints are more strongly weighted cues to retrieval than gender agreement for both coargument reflexives and those inside picture noun phrases. We interpret the observed time-course differences as indexing the relative ease of accessing the local antecedent in different syntactic contexts.

Word length and landing position effects during reading in children and adults

The present study examined the effects of word length on children’s eye movement behaviour when other variables were carefully controlled. Importantly, the results showed that word length influenced children’s reading times and fixation positions on words. Furthermore, children exhibited stronger word length effects than adults in gaze durations and refixations. Adults and children generally did not differ in initial landing positions, but did differ in refixation behaviour. Overall, the results indicated that while adults and children show similar effects of word length for early measures of eye movement behaviour, differences emerge in later measures.

Dream characteristics in a Brazilian sample: an online survey focusing on lucid dreaming

During sleep, humans experience the offline images and sensations that we call dreams, which are typically emotional and lacking in rational judgment of their bizarreness. However, during lucid dreaming (LD), subjects know that they are dreaming, and may control oneiric content. Dreaming and LD features have been studied in North Americans, Europeans and Asians, but not among Brazilians, the largest population in Latin America. Here we investigated dreams and LD characteristics in a Brazilian sample (n=3,427; median age=25 years) through an online survey. The subjects reported recalling dreams at least once a week (76%), and that dreams typically depicted actions (93%), known people (92%), sounds/voices (78%), and colored images (76%). The oneiric content was associated with plans for the upcoming days (37%), memories of the previous day (13%), or unrelated to the dreamer (30%). Nightmares usually depicted anxiety/fear (65%), being stalked (48%), or other unpleasant sensations(47%). These data corroborate Freudian notion of day residue in dreams, and suggest that dreams and nightmares are simulations of life situations that are related to our psychobiological integrity. Regarding LD, we observed that 77% of the subjects experienced LD at least once in life (44% up to 10 episodes ever), and for 48% LD subjectively lasted less than 1 min. LD frequency correlated weakly with dream recall frequency (r =0.20,p< 0.01), and LD control was rare (29%). LD occurrence was facilitated when subjects did not need to wake up early (38%), a situation that increases rapid eye movement sleep (REMS) duration, or when subjects were under stress (30%), which increases REMS transitions into waking. These results indicate that LD is relatively ubiquitous but rare, unstable, difficult to control, and facilitated by increases in REMS duration and transitions to wake state. Together with LD incidence in USA, Europe and Asia, our data from Latin America strengthen the notion that LD is a general phenomenon of the human species.

Neurobiology and clinical implications of lucid dreaming

Several lines of evidence converge to the idea that rapid eye movement sleep (REMS) is a good model to foster our understanding of psychosis. Both REMS and psychosis course with internally generated perceptions and lack of rational judgment, which is attributed to a hyperlimbic activity along with hypofrontality. Interestingly, some individuals can become aware of dreaming during REMS, a particular experience known as lucid dreaming (LD), whose neurobiological basis is still controversial. Since the frontal lobe plays a role in self-consciousness, working memory and attention, here we hypothesize that LD is associated with increased frontal activity during REMS. A possible way to test this hypothesis is to check whether transcranial magnetic or electric stimulation of the frontal region during REMS triggers LD. We further suggest that psychosis and LD are opposite phenomena: LD as a physiological awakening while dreaming due to frontal activity, and psychosis as a pathological intrusion of dream features during wake state due to hypofrontality. We further suggest that LD research may have three main clinical implications. First, LD could be important to the study of consciousness, including its pathologies and other altered states. Second, LD could be used as a therapy for recurrent nightmares, a common symptom of depression and post-traumatic stress disorder. Finally, LD may allow for motor imagery during dreaming with possible improvement of physical rehabilitation. In all, we believe that LD research may clarify multiple aspects of brain functioning in its physiological, altered and pathological states.

An investigation of Hebbian phase sequences as assembly graphs

Hebb proposed that synapses between neurons that fire synchronously are strengthened, forming cell assemblies and phase sequences. The former, on a shorter scale, are ensembles of synchronized cells that function transiently as a closed processing system; the latter, on a larger scale, correspond to the sequential activation of cell assemblies able to represent percepts and behaviors. Nowadays, the recording of large neuronal populations allows for the detection of multiple cell assemblies. Within Hebb's theory, the next logical step is the analysis of phase sequences. Here we detected phase sequences as consecutive assembly activation patterns, and then analyzed their graph attributes in relation to behavior. We investigated action potentials recorded from the adult rat hippocampus and neocortex before, during and after novel object exploration (experimental periods). Within assembly graphs, each assembly corresponded to a node, and each edge corresponded to the temporal sequence of consecutive node activations. The sum of all assembly activations was proportional to firing rates, but the activity of individual assemblies was not. Assembly repertoire was stable across experimental periods, suggesting that novel experience does not create new assemblies in the adult rat. Assembly graph attributes, on the other hand, varied significantly across behavioral states and experimental periods, and were separable enough to correctly classify experimental periods (Naïve Bayes classifier; maximum AUROCs ranging from 0.55 to 0.99) and behavioral states (waking, slow wave sleep, and rapid eye movement sleep; maximum AUROCs ranging from 0.64 to 0.98). Our findings agree with Hebb's view that assemblies correspond to primitive building blocks of representation, nearly unchanged in the adult, while phase sequences are labile across behavioral states and change after novel experience. The results are compatible with a role for phase sequences in behavior and cognition.

Diferenças de gênero na produção de associações livres de palavras através do ciclo sono-vigília

Although several studies, have shown differences in cognitive performance between men and women, it not yet known whether these differences occur in tasks involving free association of words (WA). Studies across the sleep-wake cycle (SWC) suggest that rapid eye movement sleep (REM) favors semantic flexibility, in comparison with pre-sleep waking (Pre-WK), slow-wave sleep (SWS) and post-sleep waking (Post-WK). The present work has two aims: (1) to evaluate the semantic distances of word pairs produced by AP, comparing men and women, (2) to evaluate semantic distance in word pairs produced by free association across the SWC in young adults of both sexes. To achieve aim (1), we applied a task of WA in 68 adult volunteers during waking (52 women and 16 men). The WA task consisted of writing the first word that came to mind after viewing another word offered as a stimulus (root Word). To achieve aim (2), we performed polysomnography to identify specific stages of the SWC. The experimental subjects were then awakened (if they were asleep) and were immediately given a WA task. The task was administered to 2 groups of 10 subjects each (G1 and G2). G1 subjects were stimulated with the same set of root words after waking from various states of SWC, while G2 subjects received sets of different root words at each state of the SWC. In the absence of a Portuguese corpus suitable for the measurement of semantic distances, the words collected in our experiments were translated to English, and semantically quantified within a systematic and representative corpus of that language (Wordnet). This procedure removed the polysemies typical of Portuguese, but preserved the semantic macrostructure common to both languages. During waking, we found that semantic distances are significantly lower in WA produced by women, in comparison with the distances observed in men. Through the SWC, there were no statistically significant differences in G1. In G2 women, we detected a significant increase of semantic distances upon being awakened from SWS. In contrast, G2 men showed a significant increase in semantic distances upon being awakened from REM. The results of the first experiment are consistent with the notion that women have a more concrete reasoning than men. The results of the second experiment indicate that men awakened from REM present more flexibility in word association than when being awakened from other states. In contrast, women showed more flexible word association after being awakened from SWS, in compared with other states. The results indicate that the cognitive flexibility attributed to different states of the SWC shows gender dependency