Multi-scale cortical keypoint representation for attention and object detection

Keypoints (junctions) provide important information for focus-of-attention (FoA) and object categorization/recognition. In this paper we analyze the multi-scale keypoint representation, obtained by applying a linear and quasi-continuous scaling to an optimized model of cortical end-stopped cells, in order to study its importance and possibilities for developing a visual, cortical architecture.We show that keypoints, especially those which are stable over larger scale intervals, can provide a hierarchically structured saliency map for FoA and object recognition. In addition, the application of non-classical receptive field inhibition to keypoint detection allows to distinguish contour keypoints from texture (surface) keypoints.

Face recognition by cortical multi-scale line and edge representations

Empirical studies concerning face recognition suggest that faces may be stored in memory by a few canonical representations. Models of visual perception are based on image representations in cortical area V1 and beyond, which contain many cell layers for feature extraction. Simple, complex and end-stopped cells provide input for line, edge and keypoint detection. Detected events provide a rich, multi-scale object representation, and this representation can be stored in memory in order to identify objects. In this paper, the above context is applied to face recognition. The multi-scale line/edge representation is explored in conjunction with keypoint-based saliency maps for Focus-of-Attention. Recognition rates of up to 96% were achieved by combining frontal and 3/4 views, and recognition was quite robust against partial occlusions.

Face normalization using multi-scale cortical keypoints

Empirical studies concerning face recognition suggest that faces may be stored in memory by a few canonical representations. Models of visual perception are based on image representations in cortical area V1 and beyond, which contain many cell layers for feature extractions. Simple, complex and end-stopped cells tuned to different spatial frequencies (scales) and/or orientations provide input for line, edge and keypoint detection. This yields a rich, multi-scale object representation that can be stored in memory in order to identify objects. The multi-scale, keypoint-based saliency maps for Focus-of-Attention can be explored to obtain face detection and normalization, after which face recognition can be achieved using the line/edge representation. In this paper, we focus only on face normalization, showing that multi-scale keypoints can be used to construct canonical representations of faces in memory.

Cortical object segregation and categorization by multi-scale line and edge coding

In this paper we present an improved scheme for line and edge detection in cortical area V1, based on responses of simple and complex cells, truly multi-scale with no free parameters. We illustrate the multi-scale representation for visual reconstruction, and show how object segregation can be achieved with coarse-to-finescale groupings. A two-level object categorization scenario is tested in which pre-categorization is based on coarse scales only, and final categorization on coarse plus fine scales. Processing schemes are discussed in the framework of a complete cortical architecture.

Multi-scale keypoint hierarchy for Focus-of-Attention and object detection

Hypercolumns in area V1 contain frequency- and orientation-selective simple and complex cells for line (bar) and edge coding, plus end-stopped cells for key- point (vertex) detection. A single-scale (single-frequency) mathematical model of single and double end-stopped cells on the basis of Gabor filter responses was developed by Heitger et al. (1992 Vision Research 32 963-981). We developed an improved model by stabilising keypoint detection over neighbouring micro- scales.

Building the what and where systems: multi-scale lines, edges and keypoints

Computer vision for realtime applications requires tremendous computational power because all images must be processed from the first to the last pixel. Ac tive vision by probing specific objects on the basis of already acquired context may lead to a significant reduction of processing. This idea is based on a few concepts from our visual cortex (Rensink, Visual Cogn. 7, 17-42, 2000): (1) our physical surround can be seen as memory, i.e. there is no need to construct detailed and complete maps, (2) the bandwidth of the what and where systems is limited, i.e. only one object can be probed at any time, and (3) bottom-up, low-level feature extraction is complemented by top-down hypothesis testing, i.e. there is a rapid convergence of activities in dendritic/axonal connections.

Invariant multi-scale object categorisation and recognition

Object recognition requires that templates with canonical views are stored in memory. Such templates must somehow be normalised. In this paper we present a novel method for obtaining 2D translation, rotation and size invariance. Cortical simple, complex and end-stopped cells provide multi-scale maps of lines, edges and keypoints. These maps are combined such that objects are characterised. Dynamic routing in neighbouring neural layers allows feature maps of input objects and stored templates to converge. We illustrate the construction of group templates and the invariance method for object categorisation and recognition in the context of a cortical architecture, which can be applied in computer vision.

Fine arts edutainment: the amateur painter

A new scheme for painterly rendering (NPR) has been developed. This scheme is based on visual perception, in particular themulti-scale line/edge representation in the visual cortex. The Amateur Painter (TAP) is the user interface on top of the rendering scheme. It allows to (semi)automatically create paintings from photographs, with different types of brush strokes and colour manipulations. In contrast to similar painting tools, TAP has a set of menus that reflects the procedure followed by a normal painter. In addition, menus and options have been designed such that they are very intuitive, avoiding a jungle of sub-menus with options from image processing that children and laymen do not understand. Our goal is to create a tool that is extremely easy to use, with the possibility that the user becomes interested in painting techniques, styles, and fine arts in general.

A vision fronted with a new disparity model

We are developing a frontend that is based on the image representation in the visual cortex and plausible processing schemes. This frontend consists of multiscale line/edge and keypoint (vertex) detection, using models of simple, complex and end-stopped cells. This frontend is being extended by a new disparity model. Assuming that there is no neural inverse tangent operator, we do not exploit Gabor phase information. Instead, we directly use simple cell (Gabor) responses at positions where lines and edges are detected.

Integrated multi-scale architecture of the cortex with application to computer vision

Tese de dout., Engenharia Electrónica e de Computadores, Faculdade de Ciência e Tecnologia, Universidade do Algarve, 2007

Análisis multiescala y multiorientación de imágenes mediante un banco de filtros de Gabor-2D.

Resumen basado en el del autor. Resumen en español e inglés. Notas al finalîp. 243-244

Encoding mechanisms based on fast oscillations in the retina of the cat and their dependencies on anesthesia

Processing in the visual system starts in the retina. Its complex network of cells with different properties enables for parallel encoding and transmission of visual information to the lateral geniculate nucleus (LGN) and to the cortex. In the retina, it has been shown that responses are often accompanied by fast synchronous oscillations (30 - 90 Hz) in a stimulus-dependent manner. Studies in the frog, rabbit, cat and monkey, have shown strong oscillatory responses to large stimuli which probably encode global stimulus properties, such as size and continuity (Neuenschwander and Singer, 1996; Ishikane et al., 2005). Moreover, simultaneous recordings from different levels in the visual system have demonstrated that the oscillatory patterning of retinal ganglion cell responses are transmitted to the cortex via the LGN (Castelo-Branco et al., 1998). Overall these results suggest that feedforward synchronous oscillations contribute to visual encoding. In the present study on the LGN of the anesthetized cat, we further investigate the role of retinal oscillations in visual processing by applying complex stimuli, such as natural visual scenes, light spots of varying size and contrast, and flickering checkerboards. This is a necessary step for understanding encoding mechanisms in more naturalistic conditions, as currently most data on retinal oscillations have been limited to simple, flashed and stationary stimuli. Correlation analysis of spiking responses confirmed previous results showing that oscillatory responses in the retina (observed here from the LGN responses) largely depend on the size and stationarity of the stimulus. For natural scenes (gray-level and binary movies) oscillations appeared only for brief moments probably when receptive fields were dominated by large continuous, flat-contrast surfaces. Moreover, oscillatory responses to a circle stimulus could be broken with an annular mask indicating that synchronization arises from relatively local interactions among populations of activated cells in the retina. A surprising finding in this study was that retinal oscillations are highly dependent on halothane anesthesia levels. In the absence of halothane, oscillatory activity vanished independent of the characteristics of the stimuli. The same results were obtained for isoflurane, which has similar pharmacological properties. These new and unexpected findings question whether feedfoward oscillations in the early visual system are simply due to an imbalance between excitation and inhibition in the retinal networks generated by the halogenated anesthetics. Further studies in awake behaving animals are necessary to extend these conclusions

Horizontal projections of area 17 in Cebus monkeys: metric features, and modular and laminar distribution

Metric features and modular and laminar distributions of intrinsic projections of area 17 were studied in Cebus apella. Anterogradely and retrogradely labeled cell appendages were obtained using both saturated pellets and iontophoretic injections of biocytin into the operculum. Laminar and modular distributions of the labeled processes were analyzed using Nissl counterstaining, and/or cytochrome oxidase and/or NADPH-diaphorase histochemistry. We distinguished three labeled cell types: pyramidal, star pyramidal and stellate cells located in supragranular cortical layers (principally in layers IIIa, IIIb α, IIIb ß and IIIc). Three distinct axon terminal morphologies were found, i.e., Ia, Ib and II located in granular and supragranular layers. Both complete and partial segregation of group I axon terminals relative to the limits of the blobs of V1 were found. The results are compatible with recent evidence of incomplete segregation of visual information flow in V1 of Old and New World primates.

Methylmercury intoxication and histochemical demonstration of NADPH-diaphorase activity in the striate cortex of adult cats

The effects of methylmercury (MeHg) on histochemical demonstration of the NADPH-diaphorase (NADPH-d) activity in the striate cortex were studied in 4 adult cats. Two animals were used as control. The contaminated animals received 50 ml milk containing 0.42 µg MeHg and 100 g fish containing 0.03 µg MeHg daily for 2 months. The level of MeHg in area 17 of intoxicated animals was 3.2 µg/g wet weight brain tissue. Two cats were perfused 24 h after the last dose (group 1) and the other animals were perfused 6 months later (group 2). After microtomy, sections were processed for NADPHd histochemistry procedures using the malic enzyme method. Dendritic branch counts were performed from camera lucida drawings for control and intoxicated animals (N = 80). Average, standard deviation and Student t-test were calculated for each data group. The concentrations of mercury (Hg) in milk, fish and brain tissue were measured by acid digestion of samples, followed by reduction of total Hg in the digested sample to metallic Hg using stannous chloride followed by atomic fluorescence analysis. Only group 2 revealed a reduction of the neuropil enzyme activity and morphometric analysis showed a reduction in dendritic field area and in the number of distal dendrite branches of the NADPHd neurons in the white matter (P<0.05). These results suggest that NADPHd neurons in the white matter are more vulnerable to the long-term effects of MeHg than NADPHd neurons in the gray matter.

O fator de crescimento neuronal na infecção por Schistosoma mansoni: estudo molecular, imunoenzimático e morfométrico em modelo permissível e não permissível à infecção

A esquistossomose é uma doença tropical causada, principalmente, pelo trematódeo Schistosoma mansoni, sendo que sua ocorrência afeta, mundialmente, 110 milhões de pessoas. A deposição dos ovos do parasita pode ocorrer, de forma ectópica, no sistema nervoso central (SNC) o qual leva à formação de granulomas com consequente produção do Fator de Crescimento Neuronal (NGF). Uma vez que muitos estudos demonstram a importância do NGF no desenvolvimento das vias corticais visuais, nosso estudo visou avaliar a possível alteração dos níveis de NGF no sistema visual assim como o impacto deste sobre a morfologia de células piramidais em dois modelos animais. A alteração na concentração do fator de crescimento assim como a morfometria neuronal foram avaliadas em animais permissíveis (camundongos) e não permissíveis (ratos) à infecção. Foram utilizados 174 ratos (Hooded Lister) e 135 camundongos albinos criados e mantidos em gaiolas e alimentados ad libitum. Esses animais foram inoculados, logo após o nascimento, com 50 cercárias. Setenta e sete ratos e 73 camundongos foram inoculados com solução salina e constituíram o grupo controle do estudo. Os períodos de infecção abrangeram uma a 48 semanas. Amostras do fígado e córtex visual foram retiradas, extraídas e quantificadas com kit de imunoensaio (ChemiKineTM Nerve Growth Factor (NGF) Sandwich ELISA Kit – Chemicon International). Para a análise morfométrica utilizamos células piramidais da camada IV do córtex visual marcadas através de injeção extracelular com Dextrana-Biotinilada (10.000 kDa). Os resultados foram expressos como média ± desvio padrão. Utilizamos teste t de Student para determinar diferenças estatísticas entre os grupos estudados. O valor médio de NGF encontrado no córtex visual de ratos infectados foi 39,2% maior do que no grupo controle (infectados: 400,9 ± 143,1 pg/mL; controle: 288 ± 31,9 pg/mL; p < 0,0001). Nas amostras de fígado, o aumento foi 28,9% maior no grupo infectado (infectados: 340,9 ± 103,9 pg/mL; p < 0,01; controle: 264,4 ± 38,6 pg/mL). Nenhum aumento significativo foi detectado antes de uma semana de infecção. Entre os camundongos, o aumento de NGF na área visual foi de 94,1% (infectados: 478,4 ± 284 pg/mL; p < 0,01; controle: 246,5 ± 76,8 pg/mL). No fígado destes animais o aumento foi de 138,7% (infectados: 561,8 ± 260,7 pg/mL; p < 0,01; controle: 301,3 ± 134,6 pg/mL). Em camundongos encontramos diferenças significativas quanto aos parâmetros dendríticos avaliados. A quantidade de dendritos foi 11,41% maior no grupo infectado do que no controle (controle: 25,28 ± 5,19; infectados: 28,16 ± 7,45; p < 0,05). O comprimento total dos dendritos também foi afetado (controle: 4.916,52 ± 1.492,65 μm; infectados: 5.460,40 ± 1.214,07 μm; p < 0,05) correspondendo a um aumento de 11,06%. A área total do campo receptor dendrítico sofreu um aumento de 12,99% (controle: 29.346,69 ± 11.298,62 μm2; infectados: 33.158,20 ± 7.758,31; p < 0,05) enquanto que a área somática teve uma redução de 13,61% (controle: 119,38 ± 19,68 μm2; infectados: 103,13 ± 24,69 μm2; p < 0,001). Quando foram avaliados os efeitos do aumento de NGF em ratos infectados não observamos diferenças significativas quanto aos parâmetros dendríticos analisados, em comparação ao grupo controle, com exceção de um aumento na área do corpo neuronal da ordem de 21,18% (controle: 132,20 ± 28,46 μm2; infectados: 160,20 ± 31,63 μm2; p < 0,00001). Este trabalho mostrou que a reação de produção de NGF no SNC durante a infecção por Schistosoma mansoni ocorre em maior magnitude no modelo permissível do que no modelo não permissível. Também demonstramos que, em camundongos, os efeitos sobre a morfologia neuronal é drasticamente afetada quando o organismo é submetido a um aumento na concentração de NGF em decorrência da infecção por Schistosoma mansoni. Diante destes dados, estudos avaliando as possíveis repercussões visuais e também dos efeitos na fisiologia celular causados pela infecção mansônica torna-se necessário para avaliar o real dano causado por este aumento patológico do fator de crescimento neuronal nas vias visuais de mamíferos.