Continual learning for computer vision applications

One of the most visionary goals of Artificial Intelligence is to create a system able to mimic and eventually surpass the intelligence observed in biological systems including, ambitiously, the one observed in humans. The main distinctive strength of humans is their ability to build a deep understanding of the world by learning continuously and drawing from their experiences. This ability, which is found in various degrees in all intelligent biological beings, allows them to adapt and properly react to changes by incrementally expanding and refining their knowledge. Arguably, achieving this ability is one of the main goals of Artificial Intelligence and a cornerstone towards the creation of intelligent artificial agents. Modern Deep Learning approaches allowed researchers and industries to achieve great advancements towards the resolution of many long-standing problems in areas like Computer Vision and Natural Language Processing. However, while this current age of renewed interest in AI allowed for the creation of extremely useful applications, a concerningly limited effort is being directed towards the design of systems able to learn continuously. The biggest problem that hinders an AI system from learning incrementally is the catastrophic forgetting phenomenon. This phenomenon, which was discovered in the 90s, naturally occurs in Deep Learning architectures where classic learning paradigms are applied when learning incrementally from a stream of experiences. This dissertation revolves around the Continual Learning field, a sub-field of Machine Learning research that has recently made a comeback following the renewed interest in Deep Learning approaches. This work will focus on a comprehensive view of continual learning by considering algorithmic, benchmarking, and applicative aspects of this field. This dissertation will also touch on community aspects such as the design and creation of research tools aimed at supporting Continual Learning research, and the theoretical and practical aspects concerning public competitions in this field.

Neurogeometry of stereo vision

This work aims to develop a neurogeometric model of stereo vision, based on cortical architectures involved in the problem of 3D perception and neural mechanisms generated by retinal disparities. First, we provide a sub-Riemannian geometry for stereo vision, inspired by the work on the stereo problem by Zucker (2006), and using sub-Riemannian tools introduced by Citti-Sarti (2006) for monocular vision. We present a mathematical interpretation of the neural mechanisms underlying the behavior of binocular cells, that integrate monocular inputs. The natural compatibility between stereo geometry and neurophysiological models shows that these binocular cells are sensitive to position and orientation. Therefore, we model their action in the space R3xS2 equipped with a sub-Riemannian metric. Integral curves of the sub-Riemannian structure model neural connectivity and can be related to the 3D analog of the psychophysical association fields for the 3D process of regular contour formation. Then, we identify 3D perceptual units in the visual scene: they emerge as a consequence of the random cortico-cortical connection of binocular cells. Considering an opportune stochastic version of the integral curves, we generate a family of kernels. These kernels represent the probability of interaction between binocular cells, and they are implemented as facilitation patterns to define the evolution in time of neural population activity at a point. This activity is usually modeled through a mean field equation: steady stable solutions lead to consider the associated eigenvalue problem. We show that three-dimensional perceptual units naturally arise from the discrete version of the eigenvalue problem associated to the integro-differential equation of the population activity.

Computer vision in aseptic pharmaceutical manufacturing: a deep learning-based approach to vial inspection and pack formation during a lyophilizer loading cycle

Vision systems are powerful tools playing an increasingly important role in modern industry, to detect errors and maintain product standards. With the enlarged availability of affordable industrial cameras, computer vision algorithms have been increasingly applied in industrial manufacturing processes monitoring. Until a few years ago, industrial computer vision applications relied only on ad-hoc algorithms designed for the specific object and acquisition setup being monitored, with a strong focus on co-designing the acquisition and processing pipeline. Deep learning has overcome these limits providing greater flexibility and faster re-configuration. In this work, the process to be inspected consists in vials’ pack formation entering a freeze-dryer, which is a common scenario in pharmaceutical active ingredient packaging lines. To ensure that the machine produces proper packs, a vision system is installed at the entrance of the freeze-dryer to detect eventual anomalies with execution times compatible with the production specifications. Other constraints come from sterility and safety standards required in pharmaceutical manufacturing. This work presents an overview about the production line, with particular focus on the vision system designed, and about all trials conducted to obtain the final performance. Transfer learning, alleviating the requirement for a large number of training data, combined with data augmentation methods, consisting in the generation of synthetic images, were used to effectively increase the performances while reducing the cost of data acquisition and annotation. The proposed vision algorithm is composed by two main subtasks, designed respectively to vials counting and discrepancy detection. The first one was trained on more than 23k vials (about 300 images) and tested on 5k more (about 75 images), whereas 60 training images and 52 testing images were used for the second one.

Graph-based Vision-Language Model for Multi-Modal Retrieval of Fashion Products

Artificial Intelligence is reshaping the field of fashion industry in different ways. E-commerce retailers exploit their data through AI to enhance their search engines, make outfit suggestions and forecast the success of a specific fashion product. However, it is a challenging endeavour as the data they possess is huge, complex and multi-modal. The most common way to search for fashion products online is by matching keywords with phrases in the product's description which are often cluttered, inadequate and differ across collections and sellers. A customer may also browse an online store's taxonomy, although this is time-consuming and doesn't guarantee relevant items. With the advent of Deep Learning architectures, particularly Vision-Language models, ad-hoc solutions have been proposed to model both the product image and description to solve this problems. However, the suggested solutions do not exploit effectively the semantic or syntactic information of these modalities, and the unique qualities and relations of clothing items. In this work of thesis, a novel approach is proposed to address this issues, which aims to model and process images and text descriptions as graphs in order to exploit the relations inside and between each modality and employs specific techniques to extract syntactic and semantic information. The results obtained show promising performances on different tasks when compared to the present state-of-the-art deep learning architectures.

Advancing Bag-of-visual-words Representations For Lesion Classification In Retinal Images.

Diabetic Retinopathy (DR) is a complication of diabetes that can lead to blindness if not readily discovered. Automated screening algorithms have the potential to improve identification of patients who need further medical attention. However, the identification of lesions must be accurate to be useful for clinical application. The bag-of-visual-words (BoVW) algorithm employs a maximum-margin classifier in a flexible framework that is able to detect the most common DR-related lesions such as microaneurysms, cotton-wool spots and hard exudates. BoVW allows to bypass the need for pre- and post-processing of the retinographic images, as well as the need of specific ad hoc techniques for identification of each type of lesion. An extensive evaluation of the BoVW model, using three large retinograph datasets (DR1, DR2 and Messidor) with different resolution and collected by different healthcare personnel, was performed. The results demonstrate that the BoVW classification approach can identify different lesions within an image without having to utilize different algorithms for each lesion reducing processing time and providing a more flexible diagnostic system. Our BoVW scheme is based on sparse low-level feature detection with a Speeded-Up Robust Features (SURF) local descriptor, and mid-level features based on semi-soft coding with max pooling. The best BoVW representation for retinal image classification was an area under the receiver operating characteristic curve (AUC-ROC) of 97.8% (exudates) and 93.5% (red lesions), applying a cross-dataset validation protocol. To assess the accuracy for detecting cases that require referral within one year, the sparse extraction technique associated with semi-soft coding and max pooling obtained an AUC of 94.2 ± 2.0%, outperforming current methods. Those results indicate that, for retinal image classification tasks in clinical practice, BoVW is equal and, in some instances, surpasses results obtained using dense detection (widely believed to be the best choice in many vision problems) for the low-level descriptors.

Endocytosis Of Tight Junctions Caveolin Nitrosylation Dependent Is Improved By Cocoa Via Opioid Receptor On Rpe Cells In Diabetic Conditions.

Retinal pigment epithelium cells, along with tight junction (TJ) proteins, constitute the outer blood retinal barrier (BRB). Contradictory findings suggest a role for the outer BRB in the pathogenesis of diabetic retinopathy (DR). The aim of this study was to investigate whether the mechanisms involved in these alterations are sensitive to nitrosative stress, and if cocoa or epicatechin (EC) protects from this damage under diabetic (DM) milieu conditions. Cells of a human RPE line (ARPE-19) were exposed to high-glucose (HG) conditions for 24 hours in the presence or absence of cocoa powder containing 0.5% or 60.5% polyphenol (low-polyphenol cocoa [LPC] and high-polyphenol cocoa [HPC], respectively). Exposure to HG decreased claudin-1 and occludin TJ expressions and increased extracellular matrix accumulation (ECM), whereas levels of TNF-α and inducible nitric oxide synthase (iNOS) were upregulated, accompanied by increased nitric oxide levels. This nitrosative stress resulted in S-nitrosylation of caveolin-1 (CAV-1), which in turn increased CAV-1 traffic and its interactions with claudin-1 and occludin. This cascade was inhibited by treatment with HPC or EC through δ-opioid receptor (DOR) binding and stimulation, thereby decreasing TNF-α-induced iNOS upregulation and CAV-1 endocytosis. The TJ functions were restored, leading to prevention of paracellular permeability, restoration of resistance of the ARPE-19 monolayer, and decreased ECM accumulation. The detrimental effects on TJs in ARPE-19 cells exposed to DM milieu occur through a CAV-1 S-nitrosylation-dependent endocytosis mechanism. High-polyphenol cocoa or EC exerts protective effects through DOR stimulation.

Epiretinal Membrane Formation Associated With Idiopathic Macular Telangiectasia: Case Report.

A 46-year-old woman complained of blurred and distorted vision in both eyes. Ophthalmic examination showed that visual acuity was 20/200 for the right eye and counting fingers left eye. Fundoscopy revealed perimacular hemorrhages, aneurismal dilatation of the vessels in the posterior pole, and a white and elevated lesion adjacent to vascular changes. We report a case of idiopathic macular telangiectasia and epiretinal membrane that occurs concomitantly. To our knowledge, this is the first report that describes an association between idiopathic macular telangiectasia and epiretinal membrane formation.

Medication Adherence And Quality Of Life Among The Elderly With Diabetic Retinopathy.

one hundred (n=100) elderly outpatients with diabetic retinopathy taking antihypertensives and/or oral antidiabetics/insulin were interviewed. Adherence was evaluated by the adherence proportion and its association with the care taken in administrating medications and by the Morisky Scale. The National Eye Institute Visual Functioning Questionnaire (NEI VFQ-25) was used to evaluate HRQoL. most (58%) reported the use of 80% or more of the prescribed dose and care in utilizing the medication. The item stopping the drug when experiencing an adverse event, from the Morisky Scale, explained 12.8% and 13.5% of the variability of adherence proportion to antihypertensives and oral antidiabetics/insulin, respectively. there was better HRQoL in the Color Vision, Driving and Social Functioning domains of the NEI VFQ-25. Individuals with lower scores on the NEI VFQ-25 and higher scores on the Morisky Scale presented greater chance to be nonadherent to the pharmacological treatment of diabetes and hypertension.

Formação de conceitos em crianças cegas: questões teóricas e implicações educacionais

Concept formation depends on language and thought, that promote the integration of information coming from the senses. It is postulated that changes in the person, the objects and events to be known suggest flexible models of concept teaching. It is assumed that the same considerations apply to teaching concepts to blind pupils. Specificities of this process are discussed, including the role of touch as resource, although not as a direct substitute to vision, and the notion of representation as a basis for the elaboration of pedagogical resources for the blind student.

O aluno portador de visão subnormal na escola regular: desafio para o professor?

Purpose:1) To check self-knowledge and needs for orientation among regular class teachers working with low vision students; 2) To gather information to assist the training on visual deficiency of regular class teachers. Methods: A survey was conducted for the academic year of 1999 among those teachers working in public schools, Campinas/SP/Brazil, of which 11 were municipal and 9 state schools, respectively 79.0% and 90.0% of these schools. A self-administered questionnaire was used as data collection instrument. Results: The sample was composed of 50 teachers with a regular class experience averaging 20 years. Most of them, 94.0%, said that they had no specific preparation in the area of low vision. Only 18 teachers declared to have received some kind of information/orientation in order to work with their low vision students and of those only 15 teachers mentioned the kind of orientation received. The whole group of 50 declared interest in receiving information. From the information/orientation requested 66.0% mentioned extended working class materials, 50.0% visual performance and eye disease of their students and 46.0% visual acuity/visual field. Conclusion: It was detected that teachers of regular classes received none or little information about their low vision students but demonstrated interest in its obtention. It was also shown that those teachers are not prepared to work with visually impaired children.

Avaliação e conduta em escolares portadores de visão subnormal atendidos em sala de recursos

Purpose: To identify improvement in visual performance of low vision students after assessment and management conducted at the Low Vision Service of State University of Campinas (UNICAMP). Method: Fourteen low vision students aged six to 30 years, attended in a room with resources for visual deficiency in Americana and Santa Bárbara d'Oeste -- SP during 1998 received complete ophthalmologic examination, specialized low vision assessment and educational intervention. Results: The most prevalent cause of vision loss was operated congenital cataract with four cases (28.6%), followed by congenital bilateral toxoplasmic macular scars and eye malformation, both with two cases (14.3%) cases each. Eight students (57.2%) had acuity classified as severe vision loss, four (28.6%) profound, one (7.1%) moderate and one (7.1%) nearly normal vision. Twelve (85.7%) were behind expected school grade. Optical aids were prescribed for 12 (85.8%) students but only 7 (58.3%) acquired the aids thus improving significantly their school performance. Conclusion: All students improved school performance even considering that 12 (85.7%) had severe to profound vision loss. As a group their performance could even be better if the optical aid prescriptions were acquired by all. This indicates the need of a social work to support such needs. For good results at school and effective student inclusion a partnership between school, family and specialized education is necessary. We recommend to promote the benefits of the resource room.

Utilização de recursos ópticos e equipamentos por escolares com deficiência visual

PURPOSE: To verify perceptions and conduct of students with visual impairment regarding devices and equipment utilized in schooling process. METHODS: A transversal descriptive study on a population of 12-year-old or older students in schooling process, affected by congenital or acquired visual impairment, inserted in the government teaching system of Campinas during the year 2000. An interview quiz, created based on an exploratory study was applied. RESULTS: A group of 26 students, 46% of them with low vision and 53.8% affected by blindness was obtained. Most of the students were from fundamental teaching courses (65.4%), studying in schools with classrooms provided with devices (73.1%). Among the resources used in reading and writing activities, 94.1% of the students reported they used the Braille system and 81.8% reported that the reading subject was dictated by a colleague. Most of the students with low vision wore glasses (91.7%), and 33.3% utilized a magnifying glass as optical devices. Among the non-optical devices, the most common were the environmental ones, getting closer to the blackboard (75.0%) and to the window (66.7%) for better lighting. CONCLUSIONS: It became evident that students with low vision eye-sight made use of devices meant for bearers of blindness, such as applying the Braille system. A reduced number of low vision students making use of optical and non-optical devices applicable to their problems were observed, indicating a probable unawareness of their visual potential and the appropriate devices to improve efficiency.

Flicker perimetry in healthy subjects: influence of age and gender, learning effect and short-term fluctuation

PURPOSE: To determine the mean critical fusion frequency and the short-term fluctuation, to analyze the influence of age, gender, and the learning effect in healthy subjects undergoing flicker perimetry. METHODS: Study 1 - 95 healthy subjects underwent flicker perimetry once in one eye. Mean critical fusion frequency values were compared between genders, and the influence of age was evaluated using linear regression analysis. Study 2 - 20 healthy subjects underwent flicker perimetry 5 times in one eye. The first 3 sessions were separated by an interval of 1 to 30 days, whereas the last 3 sessions were performed within the same day. The first 3 sessions were used to investigate the presence of a learning effect, whereas the last 3 tests were used to calculate short-term fluctuation. RESULTS: Study 1 - Linear regression analysis demonstrated that mean global, foveal, central, and critical fusion frequency per quadrant significantly decreased with age (p<0.05).There were no statistically significant differences in mean critical fusion frequency values between males and females (p>0.05), with the exception of the central area and inferonasal quadrant (p=0.049 and p=0.011, respectively), where the values were lower in females. Study 2 - Mean global (p=0.014), central (p=0.008), and peripheral (p=0.03) critical fusion frequency were significantly lower in the first session compared to the second and third sessions. The mean global short-term fluctuation was 5.06±1.13 Hz, the mean interindividual and intraindividual variabilities were 11.2±2.8% and 6.4±1.5%, respectively. CONCLUSION: This study suggests that, in healthy subjects, critical fusion frequency decreases with age, that flicker perimetry is associated with a learning effect, and that a moderately high short-term fluctuation is expected.

Ibopamina 2% vs. sobrecarga hídrica como teste provocativo para glaucoma

PURPOSE: To compare the 2% ibopamine provocative test with the water drinking test as a provocative test for glaucoma. METHODS: Primary open-angle glaucoma patients and normal individuals were selected from CEROF-Universidade Federal de Goiânia UFG, and underwent the 2% ibopamine provocative test and the water drinking test in a randomized fashion, at least 1 week apart. Intraocular pressure (IOP) before and after both tests, Bland-Altman graph, sensitivity and specificity (as mesured by ROC curves) were obtained for both methods. RESULTS: Forty-seven eyes from 25 patients were included (27 eyes from 15 glaucoma patients and 20 eyes from 10 normal individuals), with a mean age of 54.2 ± 12.7 years. The mean MD of glaucoma patients was -2.8 ± 2.11 dB. There was no statistically difference in the baseline IOP (p=0.8) comparing glaucoma patients, but positive after the provocative tests (p=0.03), and in the IOP variation (4.4 ± 1.3 mmHg for ibopamine and 3.2 ± 2.2 mmHg for water drinking test, p=0.01). There was no difference in all studied parameters for normal individuals. The Bland-Altman graph showed high dispersion comparing both methods. The areas under the ROC curve were 0.987 for the ibopamine provocative test, and 0.807 for the water-drinking test. CONCLUSION: In this selected subgroup of glaucoma patients with early visual field defect, the ibopamine provocative test has shown better sensitivity/specificity than the water drinking test.

Ocular findings in Brazilian identical twins with Cohen syndrome: case report

A case of identical male twins with Cohen syndrome who present multiple ophthalmic findings is reported. The patients were identical 16 year-old twin boys who showed down slanting eyelids, mild ptosis, high-grade myopia, small cortical lens opacities, posterior subcapsular cataracts, myotic and corectopic pupils with poor dilation due to focal iris atrophy and retinochoroidal dystrophy. Ophthalmologists must be aware of the ocular and systemic findings of Cohen syndrome in the evaluation of young patients with mental retardation and visual impairment.