A study on tackling visual odometry by a transformer architecture

This dissertation describes a deepening study about Visual Odometry problem tackled with transformer architectures. The existing VO algorithms are based on heavily hand-crafted features and are not able to generalize well to new environments. To train them, we need carefully fine-tune the hyper-parameters and the network architecture. We propose to tackle the VO problem with transformer because it is a general-purpose architecture and because it was designed to transformer sequences of data from a domain to another one, which is the case of the VO problem. Our first goal is to create synthetic dataset using BlenderProc2 framework to mitigate the problem of the dataset scarcity. The second goal is to tackle the VO problem by using different versions of the transformer architecture, which will be pre-trained on the synthetic dataset and fine-tuned on the real dataset, KITTI dataset. Our approach is defined as follows: we use a feature-extractor to extract features embeddings from a sequence of images, then we feed this sequence of embeddings to the transformer architecture, finally, an MLP is used to predict the sequence of camera poses.

An exploration of varieties of visual attention: ERP findings

A set of five tasks was designed to examine dynamic aspects of visual attention: selective attention to color, selective attention to pattern, dividing and switching attention between color and pattern, and selective attention to pattern with changing target. These varieties of visual attention were examined using the same set of stimuli under different instruction sets; thus differences between tasks cannot be attributed to differences in the perceptual features of the stimuli. ERP data are presented for each of these tasks. A within-task analysis of different stimulus types varying in similarity to the attended target feature revealed that an early frontal selection positivity (FSP) was evident in selective attention tasks, regardless of whether color was the attended feature. The scalp distribution of a later posterior selection negativity (SN) was affected by whether the attended feature was color or pattern. The SN was largely unaffected by dividing attention across color and pattern. A large widespread positivity was evident in most conditions, consisting of at least three subcomponents which were differentially affected by the attention conditions. These findings are discussed in relation to prior research and the time course of visual attention processes in the brain. (C) 1999 Elsevier Science B.V. All rights reserved.

Evaluating SVM and MLDA in the extraction of discriminant regions for mental state prediction

Pattern recognition methods have been successfully applied in several functional neuroimaging studies. These methods can be used to infer cognitive states, so-called brain decoding. Using such approaches, it is possible to predict the mental state of a subject or a stimulus class by analyzing the spatial distribution of neural responses. In addition it is possible to identify the regions of the brain containing the information that underlies the classification. The Support Vector Machine (SVM) is one of the most popular methods used to carry out this type of analysis. The aim of the current study is the evaluation of SVM and Maximum uncertainty Linear Discrimination Analysis (MLDA) in extracting the voxels containing discriminative information for the prediction of mental states. The comparison has been carried out using fMRI data from 41 healthy control subjects who participated in two experiments, one involving visual-auditory stimulation and the other based on bimanual fingertapping sequences. The results suggest that MLDA uses significantly more voxels containing discriminative information (related to different experimental conditions) to classify the data. On the other hand, SVM is more parsimonious and uses less voxels to achieve similar classification accuracies. In conclusion, MLDA is mostly focused on extracting all discriminative information available, while SVM extracts the information which is sufficient for classification. (C) 2009 Elsevier Inc. All rights reserved.

Binocular rivalry and the cerebral hemispheres with a note on the correlates and constitution of visual consciousness

In addressing the scientific study of consciousness, Crick and Koch state, "It is probable that at any moment some active neuronal processes in your head correlate with consciousness, while others do not: what is the difference between them?" (1998, p. 97). Evidence from electrophysiological and brain-imaging studies of binocular rivalry supports the premise of this statement and answers to some extent, the question posed. I discuss these recent developments and outline the rationale and experimental evidence for the interhemispheric switch hypothesis of perceptual rivalry. According to this model, the perceptual alternations of rivalry reflect hemispheric alternations, suggesting that visual consciousness of rivalling stimuli may be unihemispheric at any one time (Miller et al., 2000). However, in this paper, I suggest that interhemispheric switching could involve alternating unihemispheric attentional selection of neuronal processes for access to visual consciousness. On this view, visual consciousness during rivalry could be bihemispheric because the processes constitutive of attentional selection may be distinct from those constitutive of visual consciousness. This is a special case of the important distinction between the neuronal correlates and constitution of visual consciousness.

Developmental changes in the cone visual pigments of black bream Acanthopagrus butcheri

The spectral absorption characteristics of the visual pigments in the photoreceptors of the black bream Acanthopagrus butcheri Munro (Sparidae, Teleostei), were measured using microspectrophotometry. A single cohort of fish aged 5-172 days post-hatch (dph), aquarium-reared adults and wild-caught juveniles were investigated. During the larval stage and in juveniles younger than 100 dph, two classes of visual pigment were found, with wavelengths of maximum absorbance (lambda(max)) at approximately 425 nm and 535 nm. Following double cone formation, from 40 dph onwards, the short wavelength-sensitive pigment was recorded in single cones and the longer wavelength-sensitive pigment in double cones. From 100 dph, a gradual shift in the lambda(max) towards longer wavelengths was observed in both cone types. By 160 dph, and in adults, all single cones had a lambda(max) at approximately 475 nm while the lambda(max) in double cones ranged from 545 to 575 nm. The relationships between the lambda(max) and the ratio of bandwidth:lambda(max), for changes in either chromophore or opsin, were modelled mathematically for the long-wavelength-sensitive visual pigments. Comparing our data with the models indicated that changes in lambda(max) were not mediated by a switch from an A(1) to A(2) chromophore, rather a change in opsin expression was most likely. The shifts in the lambda(max) of the visual pigments occur at a stage when the juvenile fish begin feeding in deeper, tannin-stained estuarine waters, which transmit predominantly longer wavelengths, so the spectral sensitivity changes may represent an adaptation by the fish to the changing light environment.

Visual Biology of Hawaiian Coral Reef Fishes. I. Ocular Transmission and Visual Pigments

The visual biology of Hawaiian reef fishes was explored by examining their eyes for spectral sensitivity of their visual pigments and for transmission of light through the ocular media to the retina. The spectral absorption curves for the visual pigments of 38 species of Hawaiian fish were recorded using microspectrophotometry. The peak absorption wavelength (lambda(max)) of the rods varied from 477-502 nm and the lambda(max) of individual species conformed closely to values for the same species previously reported using a whole retina extraction procedure. The visual pigments of single cone photoreceptors were categorized, dependent on their lambda(max)-values, as ultraviolet (347-376 nm), violet (398-431 nm) or blue (439-498 nm) sensitive cones. Eight species possessed ultraviolet-sensitive cones and 14 species violet-sensitive cones. Thus, 47% of the species examined displayed photosensitivity to the short-wavelength region of the spectrum. Both identical and nonidentical paired and double cones were found with blue sensitivity or green absorption peaks (> 500 nm). Spectrophotometry of the lens, cornea, and humors for 195 species from 49 families found that the spectral composition of the light transmitted to the retina was most often limited by the lens (73% of species examined). Except for two unusual species with humor-limited eyes, Acanthocybium solandri (Scombridae) and the priacanthid fish, Heteropriacanthus cruentatus, the remainder had corneal-limited eyes. The wavelength at which 50% of the light was blocked (T50) was classified according to a system modified from Douglas and McGuigan (1989) as Type I, T50 < = 355 nm, (32 species); Type IIa, 355 < T50 < = 380 nm (30 species); Type IIb, 380 < T50 405 nm (84 species). Possession of UV-transmitting ocular media follows both taxonomic and functional lines and, if the ecology of the species is considered, is correlated with the short-wavelength visual pigments found in the species. Three types of short-wavelength vision in fishes are hypothesized: UV-sensitive, UV-specialized, and violet-specialized. UV-sensitive eyes lack UV blockers (Type I and IIa) and can sense UV light with the secondary absorption peak or beta peak of their longer wavelength visual pigments but do not possess specialized UV receptor cells and, therefore, probably lack UV hue discrimination. UV-specialized eyes allow transmission of UV light to the retina (Type I and IIa) and also possess UV-sensitive cone receptors with peak absorption between 300 and 400 nm. Given the appropriate perceptual mechanisms, these species could possess true UV-color vision and hue discrimination. Violet-specialized eyes extend into Type IIb eyes and possess violet-sensitive cone cells. UV-sensitive eyes are found throughout the fishes from at least two species of sharks to modern bony fishes. Eyes with specialized short-wavelength sensitivity are common in tropical reef fishes and must be taken into consideration when performing research involving the visual perception systems of these fishes. Because most glass and plastics are UV-opaque, great care must be taken to ensure that aquarium dividers, specimen holding containers, etc., are UV-transparent or at least to report the types of materials in use.

Perceção visual na avaliação diagnóstica em mamografia: uma revisão sistemática

Introdução – Na avaliação diagnóstica em mamografia, o desempenho do radiologista pode estar sujeito a erros de diagnóstico. Objetivo – Descrever a importância da perceção visual na análise da mamografia, identificando os principais fatores que contribuem para a perceção visual do radiologista e que condicionam a acuidade diagnóstica. Metodologia – Estudo descritivo baseado numa revisão sistemática de literatura através da PubMed e da Science Direct. Foram incluídos 42 artigos que respeitavam, pelo menos, um dos critérios de inclusão no estudo. Para a seleção das referências foi utilizada a metodologia PRISMA, constituída por 4 fases: identificação, seleção preliminar, elegibilidade e estudos incluídos. Resultados – Na avaliação diagnóstica em mamografia, a perceção visual está intimamente relacionada com: 1) diferentes parâmetros visuais e da motilidade ocular (acuidade visual, sensibilidade ao contraste e à luminância e movimentos oculares); 2) com condições de visualização de uma imagem (iluminância da sala e luminância do monitor); e 3) fadiga ocular provocada pela observação diária consecutiva de imagens. Conclusões – A perceção visual pode ser influenciada por 3 categorias de erros observados: erros de pesquisa (lesões não são fixadas pela fóvea), erros de reconhecimento (lesões fixadas, mas não durante o tempo suficiente) e erros de decisão (lesões fixadas, mas não identificadas como suspeitas). Os estudos analisados sobre perceção visual, atenção visual e estratégia visual, bem como os estudos sobre condições de visualização não caracterizam a função visual dos observadores. Para uma avaliação correta da perceção visual em mamografia deverão ser efetuados estudos que correlacionem a função visual com a qualidade diagnóstica. ABSTRACT - Introduction – Diagnostic evaluation in mammography could be influenced by the radiologist performance that could be under diagnostic errors. Aims – To describe the importance of radiologist visual perception in mammographic diagnostic evaluation and to identify the main factors that contribute to diagnostic accuracy. Methods – In this systematic review 42 references were included based on inclusion criteria (PubMed and Science Direct). PRISMA method was used to select the references following 4 steps: identification, screening, eligibility and included references. Results – Visual perception in mammography diagnostic evaluation is related with: 1) visual parameters and ocular motility (visual acuity, contrast sensitivity and luminance and ocular movements); 2) image visualization environment (room iluminance and monitor luminance); and 3) eyestrain caused by image daily consecutive observation. Conclusions – Visual perception can be influenced by three errors categories: search errors (lesions are never looked at with high-resolution foveal vision), recognition errors (lesions are looked at, but not long enough to detect or recognize) and decision errors (lesions are looked at for long periods of time but are still missed). The reviewed studies concerning visual perception, visual attention, visual strategies and image visualization environment do not describe observer’s visual function. An accurate evaluation of visual perception in mammography must include visual function analysis.

Observação do desenvolvimento de linguagem e funções auditiva e visual em lactentes

OBJETIVO: Investigar o desenvolvimento da linguagem e das funções auditiva e visual em lactentes de creche, a partir da avaliação realizada por educadores. MÉTODOS: Foram avaliados 115 lactentes, nos anos de 1998 a 2001, usuários de uma creche da área da saúde de uma universidade do Estado de São Paulo. Foi utilizado o "Protocolo da Observação do Desenvolvimento de Linguagem e das Funções Auditiva e Visual", com 39 provas no total, para a avaliação dos lactentes de 3 até 12 meses de idade. A aplicação desse Protocolo foi feita pelas educadoras da creche, devidamente treinadas. Utilizou-se o teste de Qui-quadrado ou Exato de Fisher. O nível de significância adotado foi de 5%. RESULTADOS: Os lactentes apresentaram um padrão diferente no desenvolvimento da linguagem quanto ao início do balbucio e das primeiras palavras, bem como na função visual, quanto à imitação e uso de jogos gestuais e de seguir ordem com uso de gestos. CONCLUSÕES: O ambiente creche propicia condições para um outro padrão de desenvolvimento de linguagem e das funções auditiva e visual. Ações de prevenção na creche devem integrar as áreas de saúde e educação num objetivo comum.

An Interface to Retrieve Personal Memories Using an Iconic Visual Language

Relevant past events can be remembered when visualizing related pictures. The main difficulty is how to find these photos in a large personal collection. Query definition and image annotation are key issues to overcome this problem. The former is relevant due to the diversity of the clues provided by our memory when recovering a past moment and the later because images need to be annotated with information regarding those clues to be retrieved. Consequently, tools to recover past memories should deal carefully with these two tasks. This paper describes a user interface designed to explore pictures from personal memories. Users can query the media collection in several ways and for this reason an iconic visual language to define queries is proposed. Automatic and semi-automatic annotation is also performed using the image content and the audio information obtained when users show their images to others. The paper also presents the user interface evaluation based on tests with 58 participants.

Expert opinion on best practice guidelines and competency framework for visual screening in children

PURPOSE: Screening programs to detect visual abnormalities in children vary among countries. The aim of this study is to describe experts' perception of best practice guidelines and competency framework for visual screening in children. METHODS: A qualitative focus group technique was applied during the Portuguese national orthoptic congress to obtain the perception of an expert panel of 5 orthoptists and 2 ophthalmologists with experience in visual screening for children (mean age 53.43 years, SD ± 9.40). The panel received in advance a script with the description of three tuning competencies dimensions (instrumental, systemic, and interpersonal) for visual screening. The session was recorded in video and audio. Qualitative data were analyzed using a categorical technique. RESULTS: According to experts' views, six tests (35.29%) have to be included in a visual screening: distance visual acuity test, cover test, bi-prism or 4/6(Δ) prism, fusion, ocular movements, and refraction. Screening should be performed according to the child age before and after 3 years of age (17.65%). The expert panel highlighted the influence of the professional experience in the application of a screening protocol (23.53%). They also showed concern about the false negatives control (23.53%). Instrumental competencies were the most cited (54.09%), followed by interpersonal (29.51%) and systemic (16.4%). CONCLUSIONS: Orthoptists should have professional experience before starting to apply a screening protocol. False negative results are a concern that has to be more thoroughly investigated. The proposed framework focuses on core competencies highlighted by the expert panel. Competencies programs could be important do develop better screening programs.

Longe da vista, longe do coração: o design como “âncora” visual e elemento agregador da atividade museológica

To become an open to outer space, the "museum" acquired new forms and new expressions. The complexity of museological activity thus leads to new representations that alter the initial image of the museum as a building with objects. Their 'boundaries' are now less sharp, not only in relation to the spatial relationship, but also to its temporal dimension, creating an additional challenge which is the recognition of the museum itself. The design, while transdisciplinary activity, thereby assumes a key role in the communication of the museums in its visual representation and recognition of their action. The present study results from a survey conducted in 2010 to 364 Portuguese museums (from a universe of 849 museums), presenting an analysis to its base elements of visual expression of identity (name, logo, symbol, and color).

Avaliação por fMRI do córtex visual, motor e auditivo através de estimulação sensoriomotora e sonora em desportistas invisuais e desportistas sem deficiência visual

Mestrado em Radiações Aplicadas às Tecnologias da Saúde - Ramo de especialização: Imagem por Ressonância Magnética

Application of Visual-Inertial SLAM for 3D Mapping of Underground Environments

The underground scenarios are one of the most challenging environments for accurate and precise 3d mapping where hostile conditions like absence of Global Positioning Systems, extreme lighting variations and geometrically smooth surfaces may be expected. So far, the state-of-the-art methods in underground modelling remain restricted to environments in which pronounced geometric features are abundant. This limitation is a consequence of the scan matching algorithms used to solve the localization and registration problems. This paper contributes to the expansion of the modelling capabilities to structures characterized by uniform geometry and smooth surfaces, as is the case of road and train tunnels. To achieve that, we combine some state of the art techniques from mobile robotics, and propose a method for 6DOF platform positioning in such scenarios, that is latter used for the environment modelling. A visual monocular Simultaneous Localization and Mapping (MonoSLAM) approach based on the Extended Kalman Filter (EKF), complemented by the introduction of inertial measurements in the prediction step, allows our system to localize himself over long distances, using exclusively sensors carried on board a mobile platform. By feeding the Extended Kalman Filter with inertial data we were able to overcome the major problem related with MonoSLAM implementations, known as scale factor ambiguity. Despite extreme lighting variations, reliable visual features were extracted through the SIFT algorithm, and inserted directly in the EKF mechanism according to the Inverse Depth Parametrization. Through the 1-Point RANSAC (Random Sample Consensus) wrong frame-to-frame feature matches were rejected. The developed method was tested based on a dataset acquired inside a road tunnel and the navigation results compared with a ground truth obtained by post-processing a high grade Inertial Navigation System and L1/L2 RTK-GPS measurements acquired outside the tunnel. Results from the localization strategy are presented and analyzed.

Combining sparse and dense methods in 6D Visual Odometry

13th International Conference on Autonomous Robot Systems (Robotica), 2013, Lisboa