Sistema de auto calibração visual para robots do ISePorto, baseado em EKF

Os sistemas de perceção existentes nos robôs autónomos, hoje em dia, são bastante complexos. A informação dos vários sensores, existentes em diferentes partes do robôs, necessitam de estar relacionados entre si face ao referencial do robô ou do mundo. Para isso, o conhecimento da atitude (posição e rotação) entre os referenciais dos sensores e o referencial do robô é um fator critico para o desempenho do mesmo. O processo de calibração dessas posições e translações é chamado calibração dos parâmetros extrínsecos. Esta dissertação propõe o desenvolvimento de um método de calibração autónomo para robôs como câmaras direcionais, como é o caso dos robôs da equipa ISePorto. A solução proposta consiste na aquisição de dados da visão, giroscópio e odometria durante uma manobra efetuada pelo robô em torno de um alvo com um padrão conhecido. Esta informação é então processada em conjunto através de um Extended Kalman Filter (EKF) onde são estimados necessários para relacionar os sensores existentes no robô em relação ao referencial do mesmo. Esta solução foi avaliada com recurso a vários testes e os resultados obtidos foram bastante similares aos obtidos pelo método manual, anteriormente utilizado, com um aumento significativo em rapidez e consistência.

Analysis of exploratory eye movements in patients with schizophrenia during visual scanning of projective tests' figures

OBJECTIVE: Compare pattern of exploratory eye movements during visual scanning of the Rorschach and TAT test cards in people with schizophrenia and controls. METHOD: 10 participants with schizophrenia and 10 controls matched by age, schooling and intellectual level participated in the study. Severity of symptoms was evaluated with the Positive and Negative Syndrome Scale. Test cards were divided into three groups: TAT cards with scenes content, TAT cards with interaction content (TAT-faces), and Rorschach cards with abstract images. Eye movements were analyzed for: total number, duration and location of fixation; and length of saccadic movements. RESULTS: Different pattern of eye movement was found, with schizophrenia participants showing lower number of fixations but longer fixation duration in Rorschach cards and TAT-faces. The biggest difference was observed in Rorschach, followed by TAT-faces and TAT-scene cards. CONCLUSIONS: Results suggest alteration in visual exploration mechanisms possibly related to integration of abstract visual information.

Olfactory and/or visual cues for spatial navigation through ontogeny: olfactory cues enable the use of visual cues

This study analyzed the spatial memory capacities of rats in darkness with visual and/or olfactory cues through ontogeny. Tests were conducted with the homing board, where rats had to find the correct escape hole. Four age groups (24 days, 48 days, 3-6 months, and 12 months) were trained in 3 conditions: (a) 3 identical light cues; (b) 5 different olfactory cues; and (c) both types of cues, followed by removal of the olfactory cues. Results indicate that immature rats first take into account olfactory information but are unable to orient with only the help of discrete visual cues. Olfaction enables the use of visual information by 48-day-old rats. Visual information predominantly supports spatial cognition in adult and 12-month-old rats. Results point out cooperation between vision and olfaction for place navigation during ontogeny in rats.

Selective integration of auditory-visual looming cues by humans.

An object's motion relative to an observer can confer ethologically meaningful information. Approaching or looming stimuli can signal threats/collisions to be avoided or prey to be confronted, whereas receding stimuli can signal successful escape or failed pursuit. Using movement detection and subjective ratings, we investigated the multisensory integration of looming and receding auditory and visual information by humans. While prior research has demonstrated a perceptual bias for unisensory and more recently multisensory looming stimuli, none has investigated whether there is integration of looming signals between modalities. Our findings reveal selective integration of multisensory looming stimuli. Performance was significantly enhanced for looming stimuli over all other multisensory conditions. Contrasts with static multisensory conditions indicate that only multisensory looming stimuli resulted in facilitation beyond that induced by the sheer presence of auditory-visual stimuli. Controlling for variation in physical energy replicated the advantage for multisensory looming stimuli. Finally, only looming stimuli exhibited a negative linear relationship between enhancement indices for detection speed and for subjective ratings. Maximal detection speed was attained when motion perception was already robust under unisensory conditions. The preferential integration of multisensory looming stimuli highlights that complex ethologically salient stimuli likely require synergistic cooperation between existing principles of multisensory integration. A new conceptualization of the neurophysiologic mechanisms mediating real-world multisensory perceptions and action is therefore supported.

How the visual brain encodes and keeps track of time.

Time is embedded in any sensory experience: the movements of a dance, the rhythm of a piece of music, the words of a speaker are all examples of temporally structured sensory events. In humans, if and how visual cortices perform temporal processing remains unclear. Here we show that both primary visual cortex (V1) and extrastriate area V5/MT are causally involved in encoding and keeping time in memory and that this involvement is independent from low-level visual processing. Most importantly we demonstrate that V1 and V5/MT come into play simultaneously and seem to be functionally linked during interval encoding, whereas they operate serially (V1 followed by V5/MT) and seem to be independent while maintaining temporal information in working memory. These data help to refine our knowledge of the functional properties of human visual cortex, highlighting the contribution and the temporal dynamics of V1 and V5/MT in the processing of the temporal aspects of visual information.

Coarse Global Visual Localization of a Mobile Robot

Localization, which is the ability of a mobile robot to estimate its position within its environment, is a key capability for autonomous operation of any mobile robot. This thesis presents a system for indoor coarse and global localization of a mobile robot based on visual information. The system is based on image matching and uses SIFT features as natural landmarks. Features extracted from training images arestored in a database for use in localization later. During localization an image of the scene is captured using the on-board camera of the robot, features are extracted from the image and the best match is searched from the database. Feature matching is done using the k-d tree algorithm. Experimental results showed that localization accuracy increases with the number of training features used in the training database, while, on the other hand, increasing number of features tended to have a negative impact on the computational time. For some parts of the environment the error rate was relatively high due to a strong correlation of features taken from those places across the environment.

Contrast sensitivity threshold measured by sweep-visual evoked potential in term and preterm infants at 3 and 10 months of age

Although healthy preterm infants frequently seem to be more attentive to visual stimuli and to fix on them longer than full-term infants, no difference in visual acuity has been reported compared to term infants. We evaluated the contrast sensitivity (CS) function of term (N = 5) and healthy preterm (N = 11) infants at 3 and 10 months of life using sweep-visual evoked potentials. Two spatial frequencies were studied: low (0.2 cycles per degrees, cpd) and medium (4.0 cpd). The mean contrast sensitivity (expressed in percentage of contrast) of the preterm infants at 3 months was 55.4 for the low spatial frequency (0.2 cpd) and 43.4 for the medium spatial frequency (4.0 cpd). At 10 months the low spatial CS was 52.7 and the medium spatial CS was 9.9. The results for the term infants at 3 months were 55.1 for the low spatial frequency and 34.5 for the medium spatial frequency. At 10 months the equivalent values were 54.3 and 14.4, respectively. No difference was found using the Mann-Whitney rank sum T-test between term and preterm infants for the low frequency at 3 or 10 months or for the medium spatial frequency at 3 or 10 months. The development of CS for the medium spatial frequency was equally fast for term and preterm infants. As also observed for visual acuity, CS was equivalent among term and preterm infants, suggesting that visual experience does not modify the development of the primary visual pathway. An earlier development of synapses in higher cortical visual areas of preterm infants could explain the better use of visual information observed behaviorally in these infants.

Auditory and visual event-related potential alterations in fragile X syndrome

Le syndrome du X fragile (SXF) est la première cause héréditaire de déficience intellectuelle et également la première cause monogénique d’autisme. Le SXF est causé par l'expansion de la répétition du nucléotide CGG sur le gène FMR1, ce qui empêche l’expression de la protéine FMRP. L’absence du FMRP mène à une altération du développement structurel et fonctionnel de la synapse, ce qui empêche la maturation des synapses induite par l’activité et l’élagage synaptique, qui sont essentiels pour le développement cérébral et cognitif. Nous avons investigué les potentiels reliés aux événements (PRE) évoqués par des stimulations fondamentales auditives et visuelles dans douze adolescents et jeunes adultes (10-22) atteints du SXF, ainsi que des participants contrôles appariés en âge chronologique et développemental. Les résultats indiquent un profil des PRE altéré, notamment l’augmentation de l’amplitude de N1 auditive, par rapport aux deux groupes contrôle, ainsi que l’augmentation des amplitudes de P2 et N2 auditifs et de la latence de N2 auditif. Chez les patients SXF, le traitement sensoriel semble être davantage perturbé qu’immature. En outre, la modalité auditive semble être plus perturbée que la modalité visuelle. En combinaison avec des résultats anatomique du cerveau, des mécanismes biochimiques et du comportement, nos résultats suggèrent une hyperexcitabilité du système nerveux dans le SXF.

Cholinergic enhancement of perceptual learning : behavioral, physiological, and neuro-pharmacological study in the rat primary visual cortex

Les cortices sensoriels sont des régions cérébrales essentielles pour la perception. En particulier, le cortex visuel traite l’information visuelle en provenance de la rétine qui transite par le thalamus. Les neurones sont les unités fonctionnelles qui transforment l'information sensorielle en signaux électriques, la transfèrent vers le cortex et l'intègrent. Les neurones du cortex visuel sont spécialisés et analysent différents aspects des stimuli visuels. La force des connections entre les neurones peut être modulée par la persistance de l'activité pré-synaptique et induit une augmentation ou une diminution du signal post-synaptique à long terme. Ces modifications de la connectivité synaptique peuvent induire la réorganisation de la carte corticale, c’est à dire la représentation de ce stimulus et la puissance de son traitement cortical. Cette réorganisation est connue sous le nom de plasticité corticale. Elle est particulièrement active durant la période de développement, mais elle s’observe aussi chez l’adulte, par exemple durant l’apprentissage. Le neurotransmetteur acétylcholine (ACh) est impliqué dans de nombreuses fonctions cognitives telles que l’apprentissage ou l’attention et il est important pour la plasticité corticale. En particulier, les récepteurs nicotiniques et muscariniques du sous-type M1 et M2 sont les récepteurs cholinergiques impliqués dans l’induction de la plasticité corticale. L’objectif principal de la présente thèse est de déterminer les mécanismes de plasticité corticale induits par la stimulation du système cholinergique au niveau du télencéphale basal et de définir les effets sur l’amélioration de la perception sensorielle. Afin d’induire la plasticité corticale, j’ai jumelé des stimulations visuelles à des injections intracorticales d’agoniste cholinergique (carbachol) ou à une stimulation du télencéphale basal (neurones cholinergiques qui innervent le cortex visuel primaire). J'ai analysé les potentiels évoqués visuels (PEVs) dans le cortex visuel primaire des rats pendant 4 à 8 heures après le couplage. Afin de préciser l’action de l’ACh sur l’activité des PEVs dans V1, j’ai injecté individuellement l’antagoniste des récepteurs muscariniques, nicotiniques, α7 ou NMDA avant l’infusion de carbachol. La stimulation du système cholinergique jumelée avec une stimulation visuelle augmente l’amplitude des PEVs durant plus de 8h. Le blocage des récepteurs muscarinique, nicotinique et NMDA abolit complètement cette amélioration, tandis que l’inhibition des récepteurs α7 a induit une augmentation instantanée des PEVs. Ces résultats suggèrent que l'ACh facilite à long terme la réponse aux stimuli visuels et que cette facilitation implique les récepteurs nicotiniques, muscariniques et une interaction avec les récepteur NMDA dans le cortex visuel. Ces mécanismes sont semblables à la potentiation à long-terme, évènement physiologique lié à l’apprentissage. L’étape suivante était d’évaluer si l’effet de l’amplification cholinergique de l’entrée de l’information visuelle résultait non seulement en une modification de l’activité corticale mais aussi de la perception visuelle. J’ai donc mesuré l’amélioration de l’acuité visuelle de rats adultes éveillés exposés durant 10 minutes par jour pendant deux semaines à un stimulus visuel de type «réseau sinusoïdal» couplé à une stimulation électrique du télencéphale basal. L’acuité visuelle a été mesurée avant et après le couplage des stimulations visuelle et cholinergique à l’aide d’une tâche de discrimination visuelle. L’acuité visuelle du rat pour le stimulus d’entrainement a été augmentée après la période d’entrainement. L’augmentation de l’acuité visuelle n’a pas été observée lorsque la stimulation visuelle seule ou celle du télencéphale basal seul, ni lorsque les fibres cholinergiques ont été lésées avant la stimulation visuelle. Une augmentation à long terme de la réactivité corticale du cortex visuel primaire des neurones pyramidaux et des interneurones GABAergiques a été montrée par l’immunoréactivité au c-Fos. Ainsi, lorsque couplé à un entrainement visuel, le système cholinergique améliore les performances visuelles pour l’orientation et ce probablement par l’optimisation du processus d’attention et de plasticité corticale dans l’aire V1. Afin d’étudier les mécanismes pharmacologiques impliqués dans l’amélioration de la perception visuelle, j’ai comparé les PEVs avant et après le couplage de la stimulation visuelle/cholinergique en présence d’agonistes/antagonistes sélectifs. Les injections intracorticales des différents agents pharmacologiques pendant le couplage ont montré que les récepteurs nicotiniques et M1 muscariniques amplifient la réponse corticale tandis que les récepteurs M2 muscariniques inhibent les neurones GABAergiques induisant un effet excitateur. L’infusion d’antagoniste du GABA corrobore l’hypothèse que le système inhibiteur est essentiel pour induire la plasticité corticale. Ces résultats démontrent que l’entrainement visuel jumelé avec la stimulation cholinergique améliore la plasticité corticale et qu’elle est contrôlée par les récepteurs nicotinique et muscariniques M1 et M2. Mes résultats suggèrent que le système cholinergique est un système neuromodulateur qui peut améliorer la perception sensorielle lors d’un apprentissage perceptuel. Les mécanismes d’amélioration perceptuelle induits par l’acétylcholine sont liés aux processus d’attention, de potentialisation à long-terme et de modulation de la balance d’influx excitateur/inhibiteur. En particulier, le couplage de l’activité cholinergique avec une stimulation visuelle augmente le ratio de signal / bruit et ainsi la détection de cibles. L’augmentation de la concentration cholinergique corticale potentialise l’afférence thalamocorticale, ce qui facilite le traitement d’un nouveau stimulus et diminue la signalisation cortico-corticale minimisant ainsi la modulation latérale. Ceci est contrôlé par différents sous-types de récepteurs cholinergiques situés sur les neurones GABAergiques ou glutamatergiques des différentes couches corticales. La présente thèse montre qu’une stimulation électrique dans le télencéphale basal a un effet similaire à l’infusion d’agoniste cholinergique et qu’un couplage de stimulations visuelle et cholinergique induit la plasticité corticale. Ce jumelage répété de stimulations visuelle/cholinergique augmente la capacité de discrimination visuelle et améliore la perception. Cette amélioration est corrélée à une amplification de l’activité neuronale démontrée par immunocytochimie du c-Fos. L’immunocytochimie montre aussi une différence entre l’activité des neurones glutamatergiques et GABAergiques dans les différentes couches corticales. L’injection pharmacologique pendant la stimulation visuelle/cholinergique suggère que les récepteurs nicotiniques, muscariniques M1 peuvent amplifier la réponse excitatrice tandis que les récepteurs M2 contrôlent l’activation GABAergique. Ainsi, le système cholinergique activé au cours du processus visuel induit des mécanismes de plasticité corticale et peut ainsi améliorer la capacité perceptive. De meilleures connaissances sur ces actions ouvrent la possibilité d’accélérer la restauration des fonctions visuelles lors d’un déficit ou d’amplifier la fonction cognitive.

Interface issues in the ESCHER visual database editor

Presentation at the 1997 Dagstuhl Seminar "Evaluation of Multimedia Information Retrieval", Norbert Fuhr, Keith van Rijsbergen, Alan F. Smeaton (eds.), Dagstuhl Seminar Report 175, 14.04. - 18.04.97 (9716). - Abstract: This presentation will introduce ESCHER, a database editor which supports visualization in non-standard applications in engineering, science, tourism and the entertainment industry. It was originally based on the extended nested relational data model and is currently extended to include object-relational properties like inheritance, object types, integrity constraints and methods. It serves as a research platform into areas such as multimedia and visual information systems, QBE-like queries, computer-supported concurrent work (CSCW) and novel storage techniques. In its role as a Visual Information System, a database editor must support browsing and navigation. ESCHER provides this access to data by means of so called fingers. They generalize the cursor paradigm in graphical and text editors. On the graphical display, a finger is reflected by a colored area which corresponds to the object a finger is currently pointing at. In a table more than one finger may point to objects, one of which is the active finger and is used for navigating through the table. The talk will mostly concentrate on giving examples for this type of navigation and will discuss some of the architectural needs for fast object traversal and display. ESCHER is available as public domain software from our ftp site in Kassel. The portable C source can be easily compiled for any machine running UNIX and OSF/Motif, in particular our working environments IBM RS/6000 and Intel-based LINUX systems. A porting to Tcl/Tk is under way.

Coding of visual space during motor preparation: approaching objects rapidly modulate corticospinal excitability in hand-centered coordinates

Defensive behaviors, such as withdrawing your hand to avoid potentially harmful approaching objects, rely on rapid sensorimotor transformations between visual and motor coordinates. We examined the reference frame for coding visual information about objects approaching the hand during motor preparation. Subjects performed a simple visuomanual task while a task-irrelevant distractor ball rapidly approached a location either near to or far from their hand. After the distractor ball appearance, single pulses of transcranial magnetic stimulation were delivered over the subject's primary motor cortex, eliciting motor evoked potentials (MEPs) in their responding hand. MEP amplitude was reduced when the ball approached near the responding hand, both when the hand was on the left and the right of the midline. Strikingly, this suppression occurred very early, at 70-80ms after ball appearance, and was not modified by visual fixation location. Furthermore, it was selective for approaching balls, since static visual distractors did not modulate MEP amplitude. Together with additional behavioral measurements, we provide converging evidence for automatic hand-centered coding of visual space in the human brain.

Intermittently-visual tracking experiments reveal the roles of error-correction and predictive mechanisms in the human visual-motor control system

Prediction mechanism is necessary for human visual motion to compensate a delay of sensory-motor system. In a previous study, “proactive control” was discussed as one example of predictive function of human beings, in which motion of hands preceded the virtual moving target in visual tracking experiments. To study the roles of the positional-error correction mechanism and the prediction mechanism, we carried out an intermittently-visual tracking experiment where a circular orbit is segmented into the target-visible regions and the target-invisible regions. Main results found in this research were following. A rhythmic component appeared in the tracer velocity when the target velocity was relatively high. The period of the rhythm in the brain obtained from environmental stimuli is shortened more than 10%. The shortening of the period of rhythm in the brain accelerates the hand motion as soon as the visual information is cut-off, and causes the precedence of hand motion to the target motion. Although the precedence of the hand in the blind region is reset by the environmental information when the target enters the visible region, the hand motion precedes the target in average when the predictive mechanism dominates the error-corrective mechanism.

Bio-inspired autonomous visual vertical control of a quadrotor UAV

Near ground maneuvers, such as hover, approach and landing, are key elements of autonomy in unmanned aerial vehicles. Such maneuvers have been tackled conventionally by measuring or estimating the velocity and the height above the ground often using ultrasonic or laser range finders. Near ground maneuvers are naturally mastered by flying birds and insects as objects below may be of interest for food or shelter. These animals perform such maneuvers efficiently using only the available vision and vestibular sensory information. In this paper, the time-to-contact (Tau) theory, which conceptualizes the visual strategy with which many species are believed to approach objects, is presented as a solution for Unmanned Aerial Vehicles (UAV) relative ground distance control. The paper shows how such an approach can be visually guided without knowledge of height and velocity relative to the ground. A control scheme that implements the Tau strategy is developed employing only visual information from a monocular camera and an inertial measurement unit. To achieve reliable visual information at a high rate, a novel filtering system is proposed to complement the control system. The proposed system is implemented on-board an experimental quadrotor UAV and shown not only to successfully land and approach ground, but also to enable the user to choose the dynamic characteristics of the approach. The methods presented in this paper are applicable to both aerial and space autonomous vehicles.

Bioinspired autonomous visual vertical control of a quadrotor unmanned aerial vehicle

Near-ground maneuvers, such as hover, approach, and landing, are key elements of autonomy in unmanned aerial vehicles. Such maneuvers have been tackled conventionally by measuring or estimating the velocity and the height above the ground, often using ultrasonic or laser range finders. Near-ground maneuvers are naturally mastered by flying birds and insects because objects below may be of interest for food or shelter. These animals perform such maneuvers efficiently using only the available vision and vestibular sensory information. In this paper, the time-tocontact (tau) theory, which conceptualizes the visual strategy with which many species are believed to approach objects, is presented as a solution for relative ground distance control for unmanned aerial vehicles. The paper shows how such an approach can be visually guided without knowledge of height and velocity relative to the ground. A control scheme that implements the tau strategy is developed employing only visual information from a monocular camera and an inertial measurement unit. To achieve reliable visual information at a high rate, a novel filtering system is proposed to complement the control system. The proposed system is implemented onboard an experimental quadrotor unmannedaerial vehicle and is shown to not only successfully land and approach ground, but also to enable the user to choose the dynamic characteristics of the approach. The methods presented in this paper are applicable to both aerial and space autonomous vehicles.

Aspectos linguístico-ergonômicos em rótulos: avaliação da linguagem verbo-visual de rótulos de embalagens para alimentos achocolatados

We have been living in a world of packed products. The package and the labels support the companies to communicate with the customers in addition to give protection, storage and convenience in proportion to the products that move in the price list. The labels mainly add up a value which helps the companies differ their products and increase the value of the brands among the final customers. However, the information given in the label are not clear sometimes. It displays a verbal-visual defective language resulted from a poor visibility, legibleness and comprehensibleness of the verbal and visual marks. The aim of this research is to verify, according to the costumers‟ view, the level of the clarity in the informative texts, harmony and ergonomic conformity of the package labels in the chocolate powder of the Claralate brand, considering the linguistic aspects presented on the labels. The criteria to evaluate the chocolate package selected were based on the linguistic field: the organization and the structure of the text derided from the classification of the textual genre; the clarity and the comprehension of the language utilized on those labels. From the ergonomic view, the informative and ergonomic conformity, based on the following requirements: legibility, symbols, characters, reading fields and intermission of the written lines. Therefore, the research done july 2007 and added july 2011 had a structured questionnaire in the interview put to the 118 customers of the chocolate package that go shopping in one of the two supermarkets in Floriano, Piauí São Jorge and/or Super Quaresma. The main results of the investigation show that the linguistic aspects in the informative texts of the labels provide the customers‟ expectancy partially, while the consideration of the informative ergonomic analyzed can contribute to the improvement of the information and consequent visual progress of those, on the labels of chocolate package investigated. As recommendation towards the maker of the product, the outcome of the research indicates: harmonize the proportion of the letters and numbers; enlarge the letters size; make the visual information more comprehensive determined by the reading field; put the expiry date in a better visual place