Mind the gaps: cognitive-inspired AI for high-level visual sensemaking. Towards abstract concept image classification

The abundance of visual data and the push for robust AI are driving the need for automated visual sensemaking. Computer Vision (CV) faces growing demand for models that can discern not only what images "represent," but also what they "evoke." This is a demand for tools mimicking human perception at a high semantic level, categorizing images based on concepts like freedom, danger, or safety. However, automating this process is challenging due to entropy, scarcity, subjectivity, and ethical considerations. These challenges not only impact performance but also underscore the critical need for interoperability. This dissertation focuses on abstract concept-based (AC) image classification, guided by three technical principles: situated grounding, performance enhancement, and interpretability. We introduce ART-stract, a novel dataset of cultural images annotated with ACs, serving as the foundation for a series of experiments across four key domains: assessing the effectiveness of the end-to-end DL paradigm, exploring cognitive-inspired semantic intermediaries, incorporating cultural and commonsense aspects, and neuro-symbolic integration of sensory-perceptual data with cognitive-based knowledge. Our results demonstrate that integrating CV approaches with semantic technologies yields methods that surpass the current state of the art in AC image classification, outperforming the end-to-end deep vision paradigm. The results emphasize the role semantic technologies can play in developing both effective and interpretable systems, through the capturing, situating, and reasoning over knowledge related to visual data. Furthermore, this dissertation explores the complex interplay between technical and socio-technical factors. By merging technical expertise with an understanding of human and societal aspects, we advocate for responsible labeling and training practices in visual media. These insights and techniques not only advance efforts in CV and explainable artificial intelligence but also propel us toward an era of AI development that harmonizes technical prowess with deep awareness of its human and societal implications.

Advancing Bag-of-visual-words Representations For Lesion Classification In Retinal Images.

Diabetic Retinopathy (DR) is a complication of diabetes that can lead to blindness if not readily discovered. Automated screening algorithms have the potential to improve identification of patients who need further medical attention. However, the identification of lesions must be accurate to be useful for clinical application. The bag-of-visual-words (BoVW) algorithm employs a maximum-margin classifier in a flexible framework that is able to detect the most common DR-related lesions such as microaneurysms, cotton-wool spots and hard exudates. BoVW allows to bypass the need for pre- and post-processing of the retinographic images, as well as the need of specific ad hoc techniques for identification of each type of lesion. An extensive evaluation of the BoVW model, using three large retinograph datasets (DR1, DR2 and Messidor) with different resolution and collected by different healthcare personnel, was performed. The results demonstrate that the BoVW classification approach can identify different lesions within an image without having to utilize different algorithms for each lesion reducing processing time and providing a more flexible diagnostic system. Our BoVW scheme is based on sparse low-level feature detection with a Speeded-Up Robust Features (SURF) local descriptor, and mid-level features based on semi-soft coding with max pooling. The best BoVW representation for retinal image classification was an area under the receiver operating characteristic curve (AUC-ROC) of 97.8% (exudates) and 93.5% (red lesions), applying a cross-dataset validation protocol. To assess the accuracy for detecting cases that require referral within one year, the sparse extraction technique associated with semi-soft coding and max pooling obtained an AUC of 94.2 ± 2.0%, outperforming current methods. Those results indicate that, for retinal image classification tasks in clinical practice, BoVW is equal and, in some instances, surpasses results obtained using dense detection (widely believed to be the best choice in many vision problems) for the low-level descriptors.

Visual Navigation In The Neotropical Ant Odontomachus Hastatus (formicidae, Ponerinae), A Predominantly Nocturnal, Canopy-dwelling Predator Of The Atlantic Rainforest.

The arboreal ant Odontomachus hastatus nests among roots of epiphytic bromeliads in the sandy forest at Cardoso Island (Brazil). Crepuscular and nocturnal foragers travel up to 8m to search for arthropod prey in the canopy, where silhouettes of leaves and branches potentially provide directional information. We investigated the relevance of visual cues (canopy, horizon patterns) during navigation in O. hastatus. Laboratory experiments using a captive ant colony and a round foraging arena revealed that an artificial canopy pattern above the ants and horizon visual marks are effective orientation cues for homing O. hastatus. On the other hand, foragers that were only given a tridimensional landmark (cylinder) or chemical marks were unable to home correctly. Navigation by visual cues in O. hastatus is in accordance with other diurnal arboreal ants. Nocturnal luminosity (moon, stars) is apparently sufficient to produce contrasting silhouettes from the canopy and surrounding vegetation, thus providing orientation cues. Contrary to the plain floor of the round arena, chemical cues may be important for marking bifurcated arboreal routes. This experimental demonstration of the use of visual cues by a predominantly nocturnal arboreal ant provides important information for comparative studies on the evolution of spatial orientation behavior in ants. This article is part of a Special Issue entitled: Neotropical Behaviour.

Applications Of Visual Evoked Potentials And Fourier-domain Optical Coherence Tomography In Parkinson's Disease: A Controlled Study.

The goal of this cross-sectional observational study was to quantify the pattern-shift visual evoked potentials (VEP) and the thickness as well as the volume of retinal layers using optical coherence tomography (OCT) across a cohort of Parkinson's disease (PD) patients and age-matched controls. Forty-three PD patients and 38 controls were enrolled. All participants underwent a detailed neurological and ophthalmologic evaluation. Idiopathic PD cases were included. Cases with glaucoma or increased intra-ocular pressure were excluded. Patients were assessed by VEP and high-resolution Fourier-domain OCT, which quantified the inner and outer thicknesses of the retinal layers. VEP latencies and the thicknesses of the retinal layers were the main outcome measures. The mean age, with standard deviation (SD), of the PD patients and controls were 63.1 (7.5) and 62.4 (7.2) years, respectively. The patients were predominantly in the initial Hoehn-Yahr (HY) disease stages (34.8% in stage 1 or 1.5, and 55.8 % in stage 2). The VEP latencies and the thicknesses as well as the volumes of the retinal inner and outer layers of the groups were similar. A negative correlation between the retinal thickness and the age was noted in both groups. The thickness of the retinal nerve fibre layer (RNFL) was 102.7 μm in PD patients vs. 104.2 μm in controls. The thicknesses of retinal layers, VEP, and RNFL of PD patients were similar to those of the controls. Despite the use of a representative cohort of PD patients and high-resolution OCT in this study, further studies are required to establish the validity of using OCT and VEP measurements as the anatomic and functional biomarkers for the evaluation of retinal and visual pathways in PD patients.

Ecological Literacy And Beyond: Problem-based Learning For Future Professionals.

Ecological science contributes to solving a broad range of environmental problems. However, lack of ecological literacy in practice often limits application of this knowledge. In this paper, we highlight a critical but often overlooked demand on ecological literacy: to enable professionals of various careers to apply scientific knowledge when faced with environmental problems. Current university courses on ecology often fail to persuade students that ecological science provides important tools for environmental problem solving. We propose problem-based learning to improve the understanding of ecological science and its usefulness for real-world environmental issues that professionals in careers as diverse as engineering, public health, architecture, social sciences, or management will address. Courses should set clear learning objectives for cognitive skills they expect students to acquire. Thus, professionals in different fields will be enabled to improve environmental decision-making processes and to participate effectively in multidisciplinary work groups charged with tackling environmental issues.

Physical Activity Interventions For Children And Youth With Visual Impairments.

The authors conducted a systematic literature review on physical activity interventions for children and youth with visual impairment (VI). Five databases were searched to identify studies involving the population of interest and physical activity practices. After evaluating 2,495 records, the authors found 18 original full-text studies published in English they considered eligible. They identified 8 structured exercise-training studies that yielded overall positive effect on physical-fitness and motor-skill outcomes. Five leisure-time-physical-activity and 5 instructional-strategy interventions were also found with promising proposals to engage and instruct children and youth with VI to lead an active lifestyle. However, the current research on physical activity interventions for children and youth with VI is still limited by an absence of high-quality research designs, low sample sizes, use of nonvalidated outcome measures, and lack of generalizability, which need to be addressed in future studies.

Delivery Of A Urology Online Course Using Moodle Versus Didactic Lectures Methods.

To subjectively and objectively compare an accessible interactive electronic library using Moodle with lectures for urology teaching of medical students. Forty consecutive fourth-year medical students and one urology teacher were exposed to two teaching methods (4 weeks each) in the form of problem-based learning: - lectures and - student-centered group discussion based on Moodle (modular object-oriented dynamic learning environment) full time online delivered (24/7) with video surgeries, electronic urology cases and additional basic principles of the disease process. All 40 students completed the study. While 30% were moderately dissatisfied with their current knowledge base, online learning course delivery using Moodle was considered superior to the lectures by 86% of the students. The study found the following observations: (1) the increment in learning grades ranged from 7.0 to 9.7 for students in the online Moodle course compared to 4.0-9.6 to didactic lectures; (2) the self-reported student involvement in the online course was characterized as large by over 60%; (3) the teacher-student interaction was described as very frequent (50%) and moderately frequent (50%); and (4) more inquiries and requisitions by students as well as peer assisting were observed from the students using the Moodle platform. The Moodle platform is feasible and effective, enthusing medical students to learn, improving immersion in the urology clinical rotation and encouraging the spontaneous peer assisted learning. Future studies should expand objective evaluations of knowledge acquisition and retention.

Método para avaliação da conduta visual de lactentes

The purpose of this study is to introduce a method to evaluate visual functions in infants in the first three months of life. An adaptation of the Guide for the Assessment of Visual Ability in Infants (Gagliardo, 1997) was used. The instrument was a ring with string. It was implemented a pilot study with 33 infants, selected according to the following criteria: neonates well enough to go home within two days of birth; 1 to 3 months of chronological age; monthly evaluation with no absence; subjects living in Campinas/SP metropolitan area. In the first month we observed: visual fixation (93,9%); eye contact (90,9%); horizontal tracking (72,7%); inspects surroundings (97,0%). In the third month, we observed: inspects own hands (42,4%) and increased movements of arms (36,4%). This method allowed the evaluation of visual functions in infants, according to the chronological age. Alterations in this function will facilitate immediate referral to medical services for diagnoses.

Visual demonstration of the ionic strength effect in the classroom: the Debye-Hückel limiting law

The effects of ionic strength on ions in aqueous solutions are quite relevant, especially for biochemical systems, in which proteins and amino acids are involved. The teaching of this topic and more specifically, the Debye-Hückel limiting law, is central in chemistry undergraduate courses. In this work, we present a description of an experimental procedure based on the color change of aqueous solutions of bromocresol green (BCG), driven by addition of electrolyte. The contribution of charge product (z+|z-|) to the Debye-Hückel limiting law is demonstrated when the effects of NaCl and Na2SO4 on the color of BCG solutions are compared.

Estudo da circulação retrobulbar e do campo visual após dose única oral de citrato de sildenafil (Viagra®)

Purpose: To analyze the effects of 100 mg of sildenafil citrate (Viagra®) on the retrobulbar circulation and visual field. Methods: A double masked, placebo controlled study was conducted in 10 males with a mean age of 27.7 + 5.68 years. The right eye of each volunteer underwent orbital color Doppler imaging and automated perimetry (Humphrey, program 30-2, Full-Threshold Strategy) at 3 occasions: baseline, 1 hour after placebo and 1 hour after 100 mg of sildenafil. The foveal threshold and the mean deviation (MD) were analyzed by automated perimetry on the three occasions. Color Doppler imaging allowed the measurement of the peak systolic velocity (PSV), end diastolic velocity (EDV) and Pourcelot index (PI) in the central retinal artery and ophthalmic artery. Results: The foveal threshold and the mean deviation did not show a significant change following the administration of sildenafil. The ophthalmic artery peak systolic velocity and end diastolic velocity significantly increased after the administration of sildenafil (p<0.001). The hemodynamic parameters in the central retinal artery and the ophthalmic artery PI did not significantly change. Conclusions: Sildefanil citrate increased the blood flow velocities in the ophthalmic artery in normal subjects, with no significant changes in the foveal threshold and mean deviation in the automated perimetry.

Utilização de recursos ópticos e equipamentos por escolares com deficiência visual

PURPOSE: To verify perceptions and conduct of students with visual impairment regarding devices and equipment utilized in schooling process. METHODS: A transversal descriptive study on a population of 12-year-old or older students in schooling process, affected by congenital or acquired visual impairment, inserted in the government teaching system of Campinas during the year 2000. An interview quiz, created based on an exploratory study was applied. RESULTS: A group of 26 students, 46% of them with low vision and 53.8% affected by blindness was obtained. Most of the students were from fundamental teaching courses (65.4%), studying in schools with classrooms provided with devices (73.1%). Among the resources used in reading and writing activities, 94.1% of the students reported they used the Braille system and 81.8% reported that the reading subject was dictated by a colleague. Most of the students with low vision wore glasses (91.7%), and 33.3% utilized a magnifying glass as optical devices. Among the non-optical devices, the most common were the environmental ones, getting closer to the blackboard (75.0%) and to the window (66.7%) for better lighting. CONCLUSIONS: It became evident that students with low vision eye-sight made use of devices meant for bearers of blindness, such as applying the Braille system. A reduced number of low vision students making use of optical and non-optical devices applicable to their problems were observed, indicating a probable unawareness of their visual potential and the appropriate devices to improve efficiency.

Curva de aprendizado e flutuações a curto e longo prazos com perimetria de freqüência dupla (FDT)

OBJECTIVE: To evaluate the learning effect, short-term fluctuation and long-term fluctuation in healthy subjects undergoing frequency doubling perimetry (FDP). METHODS: Twenty healthy young subjects underwent FDT (program N30, full threshold) in one eye (right). Each subject was tested once in the first three sessions and three times in the fourth session. Both short- and long-term fluctuations were studied as the average fluctuation of all the tested points or as a point-to-point fluctuation. To study the learning effect, the MDs values of the first session were compared to the second, third and fourth sessions. RESULTS: In the short-term analysis (3 examination done in the last session), the total mean sensitivity was 31.91 ± 1.20 dB and the mean MD and PSD were 0.84 ± 1.85 and 3.73 ± 1.55 dB, respectively. The average short-term fluctuation was 1.72 ± 0.38 dB. When the four examination, performed at different visits, were compared, the average mean sensitivity of all sessions and the average long-term fluctuation were 31.75 ± 1.11 and 2.16 ± 0.26 dB, respectively. The MD averages of the first, second, third and fourth tests were 0.11 ± 2.14 dB, 0.47 ± 1.64 dB, 1.16 ± 1.62 dB and 0.98 ± 1.92 dB respectively. The MD difference between the first and the third and between the first and the fourth examinations were statistically significant (p<0.05). CONCLUSION: The threshold sensitivity detected by FDP is influenced by both short- and long-term fluctuations. We observed a mild learning effect that shoud be taken into account whenever a patient undergoes this test for the first time.

Comportamento visual e perfil socioeconômico e demográfico de recém-nascidos prematuros da Maternidade do Hospital das Clínicas de Pernambuco - UFPE

OBJECTIVE: To characterize the behavior of premature newborns in the first year of chronological age. METHODS: This is a cross-sectional descriptive study, bound to a longitudinal study titled: Comparison of visual behavior on the first quarter of year of life of premature nursling born at two maternities of Recife/PE. The sample was composed by 52 premature newborns selected from June, 2007 to June, 2008 from the Maternity of the Federal University of Pernambuco (UFPE). Biological, socioeconomic and demographic data was collected through medical records and interviews with progeny. Newborns were evaluated by the Assessment Guide of Visual Ability in Infants. RESULTS: Most of the newborns were male at a gestational period between 33 weeks and 36 weeks and 6 days, showed a good visual behavior development for the age researched, and most of the families showed good socioeconomical and demographic profile. Besides, it was possible to detect ocular signs in 19% of sample, that were referred to an Ophthalmology Service. CONCLUSION: This study results point out the method like an important key in the early detection and visual screening for premature nursling since the first month of life and it led us to believe that clinical view for occupational therapy intervention must be focused not only on biological risks but also at the influence environment in newborn performance.

Neuronal Ceroid-lipofuscinosis with prominent chorea and without visual manifestations: a case report

A case of neuronal ceroid-lipofuscinosis (NCL) is reported in a 11-year-old girl, whose main symptoms were progressive dementia since the age of 4 years and choreic movements since age 10. Seizures, myoclonus and visual deterioration were absent and optic fundi were normal. A cerebral biopsy disclosed two basic types of stored substance in the cytoplasm of neurons: a) severely balloned nerve cells in cortical layers HI and V contained a non-autofluorescent material, which stained with PAS and Sudan Black B in frozen, but not in paraffin sections; ultrastructurally, these neurons showed abundant corpuscles similar to the membranous cytoplasmic bodies of Tay-Sachs disease and, in smaller amounts, also zebra bodies; b) slightly distended or non-distended neurons in all layers contained lipopigment granules, which were autofluorescent, PAS-positive and sudanophil in both frozen and paraffin sections; their ultrastructure was closely comparable to that of lipofuscin. Similar bodies were found in the swollen segments of axons and in a few astrocytes and endothelial cells. The histochemical and ultrastructural demonstration of large amounts of lipopigments allows a presumptive classification of the case as NCL. However, the presence of involuntary movements, the absence of visual disturbances and the unusual ultrastructural features place the patient into a small heterogeneous group within the NCL. A better classification of such unique instances of the disease must await elucidation of the basic enzymatic defects.

Cromomicose cerebral: registro de um caso

A case of brain abscess and meningitis due to pigmented fungi is reported. The patient was a 59-year-old white male, who had enjoyed excellent health until October 1977, when he developed headache, later accompanied by paresthesias and weakness in the left-sided extremities. These symptoms worsened progressively and in November of that year he had to quit his job. From February 1978 on he became inactive and anorexic. Intense continuous headache was associated with frequent episodes of vomiting. He gradually became tor-porous, and according to his relatives, suffered from visual and possibly auditory deficiency. On examination, he was malnourished and dehydrated, with decubitus ulcers. Temperature was 38,5°C. A left-sided spastic hemiplegia and prominent meningorradicular signs were noted. The CSF was examined six times between May 17th and June 1st and showed variable hypercytosis (143 to 4,437 leucocytes/ cu mm) with predominance of neutrophils (up tp 95%), low glucose and high protein concentrations. No microorganisms were identified. Electroencephalographic study disclosed a low background activity especially in left temporal areas. Despite supportive care and antibiotic therapy he lapsed into coma. Carotid angiography was normal on June 1st. He remained in deep coma until his death on June 6th, 1978. Necropsy was limited to the brain, which weighed 1,550 g after fixation and showed diffuse intense edema and hyperemia. On coronal sectioning an encapsulated abscess was found in the right basal ganglia, which also involved the internal capsule, and measured 1.5 cm in diameter. Microscopical examination disclosed large numbers of brownish fungi, appearing both as oval yeasts and as septate hyphae in the thick fibrous capsule and in the necrotic content of the abscess. The same organisms were demonstrated in moderate numbers in the leptomeninges of the medulla oblongata and , less frequently, of the hippo-campal region and cerebellum.