Cortical 3D face and object recognition using 2D projections

Empirical studies concerning face recognition suggest that faces may be stored in memory by a few canonical representations. In cortical area V1 exist double-opponent colour blobs, also simple, complex and end-stopped cells which provide input for a multiscale line/edge representation, keypoints for dynamic feature routine, and saliency maps for Focus-of-Attention.

Biologically Inspired Vision for Indoor Robot Navigation

Ultrasonic, infrared, laser and other sensors are being applied in robotics. Although combinations of these have allowed robots to navigate, they are only suited for specific scenarios, depending on their limitations. Recent advances in computer vision are turning cameras into useful low-cost sensors that can operate in most types of environments. Cameras enable robots to detect obstacles, recognize objects, obtain visual odometry, detect and recognize people and gestures, among other possibilities. In this paper we present a completely biologically inspired vision system for robot navigation. It comprises stereo vision for obstacle detection, and object recognition for landmark-based navigation. We employ a novel keypoint descriptor which codes responses of cortical complex cells. We also present a biologically inspired saliency component, based on disparity and colour.

Contribuições para a localização e mapeamento em robótica através da identificação visual de lugares

Tese de doutoramento, Informática (Engenharia Informática), Universidade de Lisboa, Faculdade de Ciências, 2015

An exploration of pre-attentive visual discrimination using event-related potentials

The Mismatch Negativity (MMN) has been characterised as a ‘pre-attentive’ component of an Event-Related Potential (ERP) that is related to discriminatory processes. Although well established in the auditory domain, characteristics of the MMN are less well characterised in the visual domain. The five main studies presented in this thesis examine visual cortical processing using event-related potentials. Novel methodologies have been used to elicit visual detection and discrimination components in the absence of a behavioural task. Developing paradigms in which a behavioural task is not required may have important clinical applications for populations, such as young children, who cannot comply with the demands of an active task. The ‘pre-attentive’ nature of visual MMN has been investigated by modulating attention. Generators and hemispheric lateralisation of visual MMN have been investigated by using pertinent clinical groups. A three stimulus visual oddball paradigm was used to explore the elicitation of visual discrimination components to a change in the orientation of stimuli in the absence of a behavioural task. Monochrome stimuli based on pacman figures were employed that differed from each other only in terms of the orientation of their elements. One such stimulus formed an illusory figure in order to capture the participant’s attention, either in place of, or alongside, a behavioural task. The elicitation of a P3a to the illusory figure but not to the standard or deviant stimuli provided evidence that the illusory figure captured attention. A visual MMN response was recorded in a paradigm with no task demands. When a behavioural task was incorporated into the paradigm, a P3b component was elicited consistent with the allocation of attentional resources to the task. However, visual discrimination components were attenuated revealing that the illusory figure was unable to command all attentional resources from the standard deviant transition. The results are the first to suggest that the visual MMN is modulated by attention. Using the same three stimulus oddball paradigm, generators of visual MMN were investigated by recording potentials directly from the cortex of an adolescent undergoing pre-surgical evaluation for resection of a right anterior parietal lesion. To date no other study has explicitly recorded activity related to the visual MMN intracranially using an oddball paradigm in the absence of a behavioural task. Results indicated that visual N1 and visual MMN could be temporally and spatially separated, with visual MMN being recorded more anteriorly than N1. The characteristic abnormality in retinal projections in albinism afforded the opportunity to investigate each hemisphere in relative isolation and was used, for the first time, as a model to investigate lateralisation of visual MMN and illusory contour processing. Using the three stimulus oddball paradigm, no visual MMN was elicited in this group, and so no conclusions regarding the lateralisation of visual MMN could be made. Results suggested that both hemispheres were equally capable of processing an illusory figure. As a method of presenting visual test stimuli without conscious perception, a continuous visual stream paradigm was developed that used a briefly presented checkerboard stimulus combined with masking for exploring stimulus detection below and above subjective levels of perception. A correlate of very early cortical processing at a latency of 60-80 ms (CI) was elicited whether stimuli were reported as seen or unseen. Differences in visual processing were only evident at a latency of 90 ms (CII) implying that this component may represent a correlate of visual consciousness/awareness. Finally, an oddball sequence was introduced into the visual stream masking paradigm to investigate whether visual MMN responses could be recorded without conscious perception. The stimuli comprised of black and white checkerboard elements differing only in terms of their orientation to form an x or a +. Visual MMN was not recorded when participants were unable to report seeing the stimulus. Results therefore suggest that behavioural identification of the stimuli was required for the elicitation of visual MMN and that visual MMN may require some attentional resources. On the basis of these studies it is concluded that visual MMN is not entirely independent of attention. Further, the combination of clinical and non-clinical investigations provides a unique opportunity to study the characterisation and localisation of putative mechanisms related to conscious and non-conscious visual processing.

Why is “blindsight” blind? A new perspective on primary visual cortex, recurrent activity and visual awareness

The neuropsychological phenomenon of blindsight has been taken to suggest that the primary visual cortex (V1) plays a unique role in visual awareness, and that extrastriate activation needs to be fed back to V1 in order for the content of that activation to be consciously perceived. The aim of this review is to evaluate this theoretical framework and to revisit its key tenets. Firstly, is blindsight truly a dissociation of awareness and visual detection? Secondly, is there sufficient evidence to rule out the possibility that the loss of awareness resulting from a V1 lesion simply reflects reduced extrastriate responsiveness, rather than a unique role of V1 in conscious experience? Evaluation of these arguments and the empirical evidence leads to the conclusion that the loss of phenomenal awareness in blindsight may not be due to feedback activity in V1 being the hallmark awareness. On the basis of existing literature, an alternative explanation of blindsight is proposed. In this view, visual awareness is a “global” cognitive function as its hallmark is the availability of information to a large number of perceptual and cognitive systems; this requires inter-areal long-range synchronous oscillatory activity. For these oscillations to arise, a specific temporal profile of neuronal activity is required, which is established through recurrent feedback activity involving V1 and the extrastriate cortex. When V1 is lesioned, the loss of recurrent activity prevents inter-areal networks on the basis of oscillatory activity. However, as limited amount of input can reach extrastriate cortex and some extrastriate neuronal selectivity is preserved, computations involving comparison of neural firing rates within a cortical area remain possible. This enables “local” read-out from specific brain regions, allowing for the detection and discrimination of basic visual attributes. Thus blindsight is blind due to lack of “global” long-range synchrony, and it functions via “local” neural readout from extrastriate areas.

Development of paper-based color test-strip for drug detection in aquatic environment: Application to oxytetracycline

The wide use of antibiotics in aquaculture has led to the emergence of resistant microbial species. It should be avoided/minimized by controlling the amount of drug employed in fish farming. For this purpose, the present work proposes test-strip papers aiming at the detection/semi-quantitative determination of organic drugs by visual comparison of color changes, in a similar analytical procedure to that of pH monitoring by universal pH paper. This is done by establishing suitable chemical changes upon cellulose, attributing the paper the ability to react with the organic drug and to produce a color change. Quantitative data is also enabled by taking a picture and applying a suitable mathematical treatment to the color coordinates given by the HSL system used by windows. As proof of concept, this approach was applied to oxytetracycline (OXY), one of the antibiotics frequently used in aquaculture. A bottom-up modification of paper was established, starting by the reaction of the glucose moieties on the paper with 3-triethoxysilylpropylamine (APTES). The so-formed amine layer allowed binding to a metal ion by coordination chemistry, while the metal ion reacted after with the drug to produce a colored compound. The most suitable metals to carry out such modification were selected by bulk studies, and the several stages of the paper modification were optimized to produce an intense color change against the concentration of the drug. The paper strips were applied to the analysis of spiked environmental water, allowing a quantitative determination for OXY concentrations as low as 30 ng/mL. In general, this work provided a simple, method to screen and discriminate tetracycline drugs, in aquaculture, being a promising tool for local, quick and cheap monitoring of drugs.

Novel optical PVC probes for on-site detection/determination of fluoroquinolones in a solid/liquid interface: Application to the determination of Norfloxacin in aquaculture water

A novel optical disposable probe for screening fluoroquinolones in fish farming waters is presented, having Norfloxacin (NFX) as target compound. The colorimetric reaction takes place in the solid/liquid interface consisting of a plasticized PVC layer carrying the colorimetric reagent and the sample solution. NFX solutions dropped on top of this solid-sensory surface provided a colour change from light yellow to dark orange. Several metals were tested as colorimetric reagents and Fe(III) was selected. The main parameters affecting the obtained colour were assessed and optimised in both liquid and solid phases. The corresponding studies were conducted by visible spectrophotometry and digital image acquisition. The three coordinates of the HSL model system of the collected image (Hue, Saturation and Lightness) were obtained by simple image management (enabled in any computer). The analytical response of the optimised solid-state optical probe against concentration was tested for several mathematical transformations of the colour coordinates. Linear behaviour was observed for logarithm NFX concentration against Hue+Lightness. Under this condition, the sensor exhibited a limit of detection below 50 μM (corresponding to about 16 mg/mL). Visual inspection also enabled semi-quantitative information. The selectivity was ensured against drugs from other chemical groups than fluoroquinolones. Finally, similar procedure was used to prepare an array of sensors for NFX, consisting on different metal species. Cu(II), Mn(II) and aluminon were selected for this purpose. The sensor array was used to detect NFX in aquaculture water, without any prior sample manipulation.

Calibration Method for Underwater Visual Ground-Truth System

This work presents an automatic calibration method for a vision based external underwater ground-truth positioning system. These systems are a relevant tool in benchmarking and assessing the quality of research in underwater robotics applications. A stereo vision system can in suitable environments such as test tanks or in clear water conditions provide accurate position with low cost and flexible operation. In this work we present a two step extrinsic camera parameter calibration procedure in order to reduce the setup time and provide accurate results. The proposed method uses a planar homography decomposition in order to determine the relative camera poses and the determination of vanishing points of detected lines in the image to obtain the global pose of the stereo rig in the reference frame. This method was applied to our external vision based ground-truth at the INESC TEC/Robotics test tank. Results are presented in comparison with an precise calibration performed using points obtained from an accurate 3D LIDAR modelling of the environment.

Octopus Standard Automated Perimetry, Pulsar Perimetry, Moorfields Motion Displacement Test and Heidelberg Retinal Tomography in Glaucoma Detection - A Comparison of Diagnostic Accuracy

Purpose: To investigate the accuracy of 4 clinical instruments in the detection of glaucomatous damage. Methods: 102 eyes of 55 test subjects (Age mean = 66.5yrs, range = [39; 89]) underwent Heidelberg Retinal Tomography (HRTIII), (disc area<2.43); and standard automated perimetry (SAP) using Octopus (Dynamic); Pulsar (TOP); and Moorfields Motion Displacement Test (MDT) (ESTA strategy). Eyes were separated into three groups 1) Healthy (H): IOP<21mmHg and healthy discs (clinical examination), 39 subjects, 78 eyes; 2) Glaucoma suspect (GS): Suspicious discs (clinical examination), 12 subjects, 15 eyes; 3) Glaucoma (G): progressive structural or functional loss, 14 subjects, 20 eyes. Clinical diagnostic precision was examined using the cut-off associated with the p<5% normative limit of MD (Octopus/Pulsar), PTD (MDT) and MRA (HRT) analysis. The sensitivity, specificity and accuracy were calculated for each instrument. Results: See table Conclusions: Despite the advantage of defining glaucoma suspects using clinical optic disc examination, the HRT did not yield significantly higher accuracy than functional measures. HRT, MDT and Octopus SAP yielded higher accuracy than Pulsar perimetry, although results did not reach statistical significance. Further studies are required to investigate the structure-function correlations between these instruments.

Towards high-quality simultaneous EEG-fMRI at 7T: Detection and reduction of EEG artifacts due to head motion.

The enhanced functional sensitivity offered by ultra-high field imaging may significantly benefit simultaneous EEG-fMRI studies, but the concurrent increases in artifact contamination can strongly compromise EEG data quality. In the present study, we focus on EEG artifacts created by head motion in the static B0 field. A novel approach for motion artifact detection is proposed, based on a simple modification of a commercial EEG cap, in which four electrodes are non-permanently adapted to record only magnetic induction effects. Simultaneous EEG-fMRI data were acquired with this setup, at 7T, from healthy volunteers undergoing a reversing-checkerboard visual stimulation paradigm. Data analysis assisted by the motion sensors revealed that, after gradient artifact correction, EEG signal variance was largely dominated by pulse artifacts (81-93%), but contributions from spontaneous motion (4-13%) were still comparable to or even larger than those of actual neuronal activity (3-9%). Multiple approaches were tested to determine the most effective procedure for denoising EEG data incorporating motion sensor information. Optimal results were obtained by applying an initial pulse artifact correction step (AAS-based), followed by motion artifact correction (based on the motion sensors) and ICA denoising. On average, motion artifact correction (after AAS) yielded a 61% reduction in signal power and a 62% increase in VEP trial-by-trial consistency. Combined with ICA, these improvements rose to a 74% power reduction and an 86% increase in trial consistency. Overall, the improvements achieved were well appreciable at single-subject and single-trial levels, and set an encouraging quality mark for simultaneous EEG-fMRI at ultra-high field.

Developmental and individual differences in novelty and error detection in 10-year-old children and young adults /

Event-related potentials were recorded from 10-year-old children and young adults in order to examine the developmental dififerences in two frontal lobe functions: detection of novel stimuli during an auditory novelty oddball task, and error detection during a visual flanker task. All participants showed a parietally-maximal P3 in response to auditory stimuli. In children, novel stimuli generated higher P3 amplitudes at the frontal site compared with target stimuli, whereas target stimuli generated higher P3 amplitudes at the parietal site compared with novel stimuli. Adults, however, had higher P3 amplitude to novel tones compared with target tones at each site. Children also had greater P3 amplitude at more parietal sites than adults during the novelty oddball and flanker tasks. Furthermore, children and adults did not show a significant reduction in P3 amplitude from the first to second novel stimulus presentation. No age differences were found with respect to P3 latency to novel and target stimuli. These findings suggest that the detection of novel and target stimuli is mature in 10-year-olds. Error trials typically elicit a negative ERP deflection (the ERN) with a frontal-central scalp distribution that may reflect response monitoring. There is also evidence of a positive ERP peak (the Pe) with a posterior scalp distribution which may reflect subjective recognition of a response. Both children and adults showed an ERN and Pe maximal at frontal-central sites. Children committed more errors, had smaller ERN across sites, and had a larger Pe at the parietal site than adults. This suggests that response monitoring is still immature in 10-year-olds whereas recognition of and emotional responses to errors may be similar in children and adults.

Real-Time Automatic Object Classification and Tracking using Genetic Programming and NVIDIA R CUDA TM

Genetic Programming (GP) is a widely used methodology for solving various computational problems. GP's problem solving ability is usually hindered by its long execution times. In this thesis, GP is applied toward real-time computer vision. In particular, object classification and tracking using a parallel GP system is discussed. First, a study of suitable GP languages for object classification is presented. Two main GP approaches for visual pattern classification, namely the block-classifiers and the pixel-classifiers, were studied. Results showed that the pixel-classifiers generally performed better. Using these results, a suitable language was selected for the real-time implementation. Synthetic video data was used in the experiments. The goal of the experiments was to evolve a unique classifier for each texture pattern that existed in the video. The experiments revealed that the system was capable of correctly tracking the textures in the video. The performance of the system was on-par with real-time requirements.

The effects of manipulated augmented sensory feedback on error detection when using a touch screen

The purpose of the present study was to determine which augmented sensory modality would best develop subjective error-detection capabilities of learners performing a spatial-temporal task when using a touch screen monitor. Participants were required to learn a 5-digit key-pressing task in a goal time of 2550 ms over 100 acquisition trials on a touch screen. Participants were randomized into 1 of 4 groups: 1) visual-feedback (colour change of button when selected), 2) auditory-feedback (click sound when button was selected), 3) visual-auditory feedback (both colour change and click sound when button was selected), and 4) no-feedback (no colour change or click sound when button was selected). Following each trial, participants were required to provide a subjective estimate regarding their performance time in relation to the actual time it took for them complete the 5-digit sequence. A no-KR retention test was conducted approximately 24-hours after the last completed acquisition trial. Results showed that practicing a timing task on a touch screen augmented with both visual and auditory information may have differentially impacted motor skill acquisition such that removal of one or both sources of augmented feedback did not result in a severe detriment to timing performance or error detection capabilities of the learner. The present study reflects the importance of multimodal augmented feedback conditions to maximize cognitive abilities for developing a stronger motor memory for subjective error-detection and correction capabilities.

Validation d’une situation de stimulation suscitant des comportements stéréotypés et des intérêts restreints chez le jeune enfant autiste

Les comportements stéréotypés et les intérêts restreints sont des comportements à valeur diagnostique dans l’autisme. Pourtant, il y a des lacunes en clinique, dans la façon de détecter ces comportements, considérant l’absence d’instruments standardisés les suscitant et en recherche, dans la façon de documenter ces comportements pour arriver à les définir de façon opérationnelle. Cette thèse a pour objectif de mieux documenter, par une situation d’observation, les comportements stéréotypés et les intérêts restreints en bas âge dans l’autisme, et de permettre l’utilisation de cette situation en clinique. Deux étapes préliminaires ont permis de documenter les comportements stéréotypés et les intérêts restreints en bas âge dans l’autisme. La première, l’élaboration d’un questionnaire sur les comportements stéréotypés et les intérêts restreints et les objets qui les déclenchent complété par des experts dans le domaine. Ce questionnaire a permis de construire la grille de cotation et la situation de stimulation. La seconde la construction d’une grille de cotation qui apporte une définition opérationnelle des comportements stéréotypés et des intérêts restreints en bas âge dans l’autisme et vise à les colliger. L’étape principale de la présente recherche consiste en l’élaboration d’une situation de stimulation suscitant des comportements stéréotypés et des intérêts restreints par l’exposition à des objets qui les déclenchent. Cette situation a permis de documenter, par observation, les comportements stéréotypés et les intérêts restreints en bas âge dans l’autisme. La validation de la situation de stimulation a été appliquée auprès de deux groupes d’enfants âgés de 24 à 72 mois appariés en âge chronologique, 21 enfants portant un diagnostic d’autisme et 24 enfants au développement typique Les résultats montrent que la situation de stimulation est un instrument suffisamment sensible pour détecter des comportements stéréotypés et des intérêts restreints en bas âge dans l’autisme et d’identifier des objets d’intérêt. En effet, lors de l’exposition à la situation de stimulation, les enfants autistes se distinguent des enfants typiques sur la base du nombre et de la durée des comportements stéréotypés et des intérêts restreints qu’ils présentent. Les enfants autistes montrent une fréquence significativement plus élevé pour les CSIR suivants: maniérismes des mains et des doigts, crispation des doigts, sautillement, doigts dans la bouche, objets dans la bouche, exploration visuelle: regard rapproché, met les objets en mouvement non circulaire. Les enfants autistes se distinguent également des enfants typiques sur la base de l’exploration des objets, en fréquence et en durée, significativement, pour les objets: Bateau: marteau et balles et lettres et chiffres. Cette étude est la première qui passe par un protocole d’observation systématique, pour documenter les comportements stéréotypés et les intérêts restreints, ainsi que les objets qui les déclenchent, des objets d’intérêt, en bas âge dans l’autisme. Cette situation pourrait ultimement faire partie du processus d’évaluation diagnostique ou de dépistage de l’autisme permettant d’identifier en bas âge des enfants autistes ou à risque d’autisme.

Encodage des forces tactiles dans le cortex somatosensoriel primaire

Les deux fonctions principales de la main sont la manipulation d’objet et l’exploration tactile. La détection du glissement, rapportée par les mécanorécepteurs de la peau glabre, est essentielle pour l’exécution de ces deux fonctions. Durant la manipulation d’objet, la détection rapide du micro-glissement (incipient slip) amène la main à augmenter la force de pince pour éviter que l’objet ne tombe. À l’opposé, le glissement est un aspect essentiel à l’exploration tactile puisqu’il favorise une plus grande acuité tactile. Pour ces deux actions, les forces normale et tangentielle exercées sur la peau permettent de décrire le glissement mais également ce qui arrive juste avant qu’il y ait glissement. Toutefois, on ignore comment ces forces contrôlées par le sujet pourraient être encodées au niveau cortical. C’est pourquoi nous avons enregistré l’activité unitaire des neurones du cortex somatosensoriel primaire (S1) durant l’exécution de deux tâches haptiques chez les primates. Dans la première tâche, deux singes devaient saisir une pastille de métal fixe et y exercer des forces de cisaillement sans glissement dans une de quatre directions orthogonales. Des 144 neurones enregistrés, 111 (77%) étaient modulés à la direction de la force de cisaillement. L’ensemble de ces vecteurs préférés s’étendait dans toutes les directions avec un arc variant de 50° à 170°. Plus de 21 de ces neurones (19%) étaient également modulés à l’intensité de la force de cisaillement. Bien que 66 neurones (59%) montraient clairement une réponse à adaptation lente et 45 autres (41%) une réponse à adaptation rapide, cette classification ne semblait pas expliquer la modulation à l’intensité et à la direction de la force de cisaillement. Ces résultats montrent que les neurones de S1 encodent simultanément la direction et l’intensité des forces même en l’absence de glissement. Dans la seconde tâche, deux singes ont parcouru différentes surfaces avec le bout des doigts à la recherche d’une cible tactile, sans feedback visuel. Durant l’exploration, les singes, comme les humains, contrôlaient les forces et la vitesse de leurs doigts dans une plage de valeurs réduite. Les surfaces à haut coefficient de friction offraient une plus grande résistance tangentielle à la peau et amenaient les singes à alléger la force de contact, normale à la peau. Par conséquent, la somme scalaire des composantes normale et tangentielle demeurait constante entre les surfaces. Ces observations démontrent que les singes contrôlent les forces normale et tangentielle qu’ils appliquent durant l’exploration tactile. Celles-ci sont également ajustées selon les propriétés de surfaces telles que la texture et la friction. Des 230 neurones enregistrés durant la tâche d’exploration tactile, 96 (42%) ont montré une fréquence de décharge instantanée reliée aux forces exercées par les doigts sur la surface. De ces neurones, 52 (54%) étaient modulés avec la force normale ou la force tangentielle bien que l’autre composante orthogonale avait peu ou pas d’influence sur la fréquence de décharge. Une autre sous-population de 44 (46%) neurones répondait au ratio entre la force normale et la force tangentielle indépendamment de l’intensité. Plus précisément, 29 (30%) neurones augmentaient et 15 (16%) autres diminuaient leur fréquence de décharge en relation avec ce ratio. Par ailleurs, environ la moitié de tous les neurones (112) étaient significativement modulés à la direction de la force tangentielle. De ces neurones, 59 (53%) répondaient à la fois à la direction et à l’intensité des forces. L’exploration de trois ou quatre différentes surfaces a permis d’évaluer l’impact du coefficient de friction sur la modulation de 102 neurones de S1. En fait, 17 (17%) neurones ont montré une augmentation de leur fréquence de décharge avec l’augmentation du coefficient de friction alors que 8 (8%) autres ont montré le comportement inverse. Par contre, 37 (36%) neurones présentaient une décharge maximale sur une surface en particulier, sans relation linéaire avec le coefficient de friction des surfaces. La classification d’adaptation rapide ou lente des neurones de S1 n’a pu être mise en relation avec la modulation aux forces et à la friction. Ces résultats montrent que la fréquence de décharge des neurones de S1 encode l’intensité des forces normale et tangentielle, le ratio entre les deux composantes et la direction du mouvement. Ces résultats montrent que le comportement d’une importante sous-population des neurones de S1 est déterminé par les forces normale et tangentielle sur la peau. La modulation aux forces présentée ici fait le pont entre les travaux évaluant les propriétés de surfaces telles que la rugosité et les études touchant à la manipulation d’objets. Ce système de référence s’applique en présence ou en absence de glissement entre la peau et la surface. Nos résultats quant à la modulation des neurones à adaptation rapide ou lente nous amènent à suggérer que cette classification découle de la manière que la peau est stimulée. Nous discuterons aussi de la possibilité que l’activité des neurones de S1 puisse inclure une composante motrice durant ces tâches sensorimotrices. Finalement, un nouveau cadre de référence tridimensionnel sera proposé pour décrire et rassembler, dans un même continuum, les différentes modulations aux forces normale et tangentielle observées dans S1 durant l’exploration tactile.