From vision to actions: Towards adaptive and autonomous humanoid robots

Although robotics research has seen advances over the last decades robots are still not in widespread use outside industrial applications. Yet a range of proposed scenarios have robots working together, helping and coexisting with humans in daily life. In all these a clear need to deal with a more unstructured, changing environment arises. I herein present a system that aims to overcome the limitations of highly complex robotic systems, in terms of autonomy and adaptation. The main focus of research is to investigate the use of visual feedback for improving reaching and grasping capabilities of complex robots. To facilitate this a combined integration of computer vision and machine learning techniques is employed. From a robot vision point of view the combination of domain knowledge from both imaging processing and machine learning techniques, can expand the capabilities of robots. I present a novel framework called Cartesian Genetic Programming for Image Processing (CGP-IP). CGP-IP can be trained to detect objects in the incoming camera streams and successfully demonstrated on many different problem domains. The approach requires only a few training images (it was tested with 5 to 10 images per experiment) is fast, scalable and robust yet requires very small training sets. Additionally, it can generate human readable programs that can be further customized and tuned. While CGP-IP is a supervised-learning technique, I show an integration on the iCub, that allows for the autonomous learning of object detection and identification. Finally this dissertation includes two proof-of-concepts that integrate the motion and action sides. First, reactive reaching and grasping is shown. It allows the robot to avoid obstacles detected in the visual stream, while reaching for the intended target object. Furthermore the integration enables us to use the robot in non-static environments, i.e. the reaching is adapted on-the- fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. The second integration highlights the capabilities of these frameworks, by improving the visual detection by performing object manipulation actions.

Occupational chronic solvent encephalopathy in Finland 1995 - 2007 : incidence and diagnostic methods

The occurrence of occupational chronic solvent encephalopathy (CSE) seems to decrease, but still every year reveals new cases. To prevent CSE and early retirement of solvent-exposed workers, actions should focus on early CSE detection and diagnosis. Identifying the work tasks and solvent exposure associated with high risk for CSE is crucial. Clinical and exposure data of all the 128 cases diagnosed with CSE as an occupational disease in Finland during 1995-2007 was collected from the patient records at the Finnish Institute of Occupational Health (FIOH) in Helsinki. The data on the number of exposed workers in Finland were gathered from the Finnish Job-exposure Matrix (FINJEM) and the number of employed from the national workforce survey. We analyzed the work tasks and solvent exposure of CSE patients and the findings in brain magnetic resonance imaging (MRI), quantitative electroencephalography (QEEG), and event-related potentials (ERP). The annual number of new cases diminished from 18 to 3, and the incidence of CSE decreased from 8.6 to 1.2 / million employed per year. The highest incidence of CSE was in workers with their main exposure to aromatic hydrocarbons; during 1995-2006 the incidence decreased from 1.2 to 0.3 / 1 000 exposed workers per year. The work tasks with the highest incidence of CSE were floor layers and lacquerers, wooden surface finishers, and industrial, metal, or car painters. Among 71 CSE patients, brain MRI revealed atrophy or white matter hyperintensities or both in 38% of the cases. Atrophy which was associated with duration of exposure was most frequently located in the cerebellum and in the frontal or parietal brain areas. QEEG in a group of 47 patients revealed increased power of the theta band in the frontal brain area. In a group of 86 patients, the P300 amplitude of auditory ERP was decreased, but at individual level, all the amplitude values were classified as normal. In 11 CSE patients and 13 age-matched controls, ERP elicited by a multimodal paradigm including an auditory, a visual detection, and a recognition memory task under single and dual-task conditions corroborated the decrease of auditory P300 amplitude in CSE patients in single-task condition. In dual-task conditions, the auditory P300 component was, more often in patients than in controls, unrecognizable. Due to the paucity and non-specificity of the findings, brain MRI serves mainly for differential diagnostics in CSE. QEEG and auditory P300 are insensitive at individual level and not useful in the clinical diagnostics of CSE. A multimodal ERP paradigm may, however, provide a more sensitive method to diagnose slight cognitive disturbances such as CSE.

Working memory in Alzheimer’s disease and mild cognitive impairment (MCI) : assessment and intervention

L’objectif principal de cette thèse est d’examiner et d’intervenir auprès des déficits de la mémoire de travail (MdeT) à l’intérieur de deux populations cliniques : la maladie d’Alzheimer (MA) et le trouble cognitif léger (TCL). La thèse se compose de trois articles empiriques. Le but de la première expérimentation était d’examiner les déficits de MdeT dans le vieillissement normal, le TCL et la MA à l’aide de deux versions de l’empan complexe : l’empan de phrases et l’empan arithmétique. De plus, l’effet de «l’oubli» (forgetting) a été mesuré en manipulant la longueur de l’intervalle de rétention. Les résultats aux tâches d’empan complexe indiquent que la MdeT est déficitaire chez les individus atteints de TCL et encore plus chez les gens ayant la MA. Les données recueillies supportent également le rôle de l’oubli à l’intérieur de la MdeT. L’augmentation de l’intervalle de rétention exacerbait le déficit dans la MA et permettait de prédire un pronostic négatif dans le TCL. L’objectif de la deuxième étude était d’examiner la faisabilité d’un programme d’entraînement cognitif à l’ordinateur pour la composante de contrôle attentionnel à l’intérieur de la MdeT. Cette étude a été réalisée auprès de personnes âgées saines et de personnes âgées avec TCL. Les données de cette expérimentation ont révélé des effets positifs de l’entraînement pour les deux groupes de personnes. Toutefois, l’absence d’un groupe contrôle a limité l’interprétation des résultats. Sur la base de ces données, la troisième expérimentation visait à implémenter une étude randomisée à double-insu avec groupe contrôle d’un entraînement du contrôle attentionnel chez des personnes TCL avec atteinte exécutive. Ce protocole impliquait un paradigme de double-tâche composé d’une tâche de détection visuelle et d’une tâche de jugement alpha-arithmétique. Alors que le groupe contrôle pratiquait simplement la double-tâche sur six périodes d’une heure chacune, le groupe expérimental recevait un entraînement de type priorité variable dans lequel les participants devaient gérer leur contrôle attentionnel en variant la proportion de ressources attentionnelles allouée à chaque tâche. Les résultats montrent un effet significatif de l’intervention sur une des deux tâches impliquées (précision à la tâche de détection visuelle) ainsi qu’une tendance au transfert à une autre tâche d’attention divisée, mais peu d’effets de généralisation à d’autres tâches d’attention. En résumé, les données originales rapportées dans la présente thèse démontrent un déficit de la MdeT dans les maladies neurodégénératives liées à l’âge, avec un gradient entre le TCL et la MA. Elles suggèrent également une préservation de la plasticité des capacités attentionnelles chez les personnes à risque de développer une démence.

Mise à jour de la Base de Données Topographiques du Québec à l'aide d'images à très haute résolution spatiale et du progiciel Sigma0 : le cas des voies de communication

Le Ministère des Ressources Naturelles et de la Faune (MRNF) a mandaté la compagnie de géomatique SYNETIX inc. de Montréal et le laboratoire de télédétection de l’Université de Montréal dans le but de développer une application dédiée à la détection automatique et la mise à jour du réseau routier des cartes topographiques à l’échelle 1 : 20 000 à partir de l’imagerie optique à haute résolution spatiale. À cette fin, les mandataires ont entrepris l’adaptation du progiciel SIGMA0 qu’ils avaient conjointement développé pour la mise à jour cartographique à partir d’images satellitales de résolution d’environ 5 mètres. Le produit dérivé de SIGMA0 fut un module nommé SIGMA-ROUTES dont le principe de détection des routes repose sur le balayage d’un filtre le long des vecteurs routiers de la cartographie existante. Les réponses du filtre sur des images couleurs à très haute résolution d’une grande complexité radiométrique (photographies aériennes) conduisent à l’assignation d’étiquettes selon l’état intact, suspect, disparu ou nouveau aux segments routiers repérés. L’objectif général de ce projet est d’évaluer la justesse de l’assignation des statuts ou états en quantifiant le rendement sur la base des distances totales détectées en conformité avec la référence ainsi qu’en procédant à une analyse spatiale des incohérences. La séquence des essais cible d’abord l’effet de la résolution sur le taux de conformité et dans un second temps, les gains escomptés par une succession de traitements de rehaussement destinée à rendre ces images plus propices à l’extraction du réseau routier. La démarche globale implique d’abord la caractérisation d’un site d’essai dans la région de Sherbrooke comportant 40 km de routes de diverses catégories allant du sentier boisé au large collecteur sur une superficie de 2,8 km2. Une carte de vérité terrain des voies de communication nous a permis d’établir des données de référence issues d’une détection visuelle à laquelle sont confrontés les résultats de détection de SIGMA-ROUTES. Nos résultats confirment que la complexité radiométrique des images à haute résolution en milieu urbain bénéficie des prétraitements telles que la segmentation et la compensation d’histogramme uniformisant les surfaces routières. On constate aussi que les performances présentent une hypersensibilité aux variations de résolution alors que le passage entre nos trois résolutions (84, 168 et 210 cm) altère le taux de détection de pratiquement 15% sur les distances totales en concordance avec la référence et segmente spatialement de longs vecteurs intacts en plusieurs portions alternant entre les statuts intact, suspect et disparu. La détection des routes existantes en conformité avec la référence a atteint 78% avec notre plus efficace combinaison de résolution et de prétraitements d’images. Des problèmes chroniques de détection ont été repérés dont la présence de plusieurs segments sans assignation et ignorés du processus. Il y a aussi une surestimation de fausses détections assignées suspectes alors qu’elles devraient être identifiées intactes. Nous estimons, sur la base des mesures linéaires et des analyses spatiales des détections que l’assignation du statut intact devrait atteindre 90% de conformité avec la référence après divers ajustements à l’algorithme. La détection des nouvelles routes fut un échec sans égard à la résolution ou au rehaussement d’image. La recherche des nouveaux segments qui s’appuie sur le repérage de points potentiels de début de nouvelles routes en connexion avec les routes existantes génère un emballement de fausses détections navigant entre les entités non-routières. En lien avec ces incohérences, nous avons isolé de nombreuses fausses détections de nouvelles routes générées parallèlement aux routes préalablement assignées intactes. Finalement, nous suggérons une procédure mettant à profit certaines images rehaussées tout en intégrant l’intervention humaine à quelques phases charnières du processus.

Perceptual shift in bilingualism: brain potentials reveal plasticity in pre-attentive colour perception

The validity of the linguistic relativity principle continues to stimulate vigorous debate and research. The debate has recently shifted from the behavioural investigation arena to a more biologically grounded field, in which tangible physiological evidence for language effects on perception can be obtained. Using brain potentials in a colour oddball detection task with Greek and English speakers, a recent study suggests that language effects may exist at early stages of perceptual integration [Thierry, G., Athanasopoulos, P., Wiggett, A., Dering, B., & Kuipers, J. (2009). Unconscious effects of language-specific terminology on pre-attentive colour perception. Proceedings of the National Academy of Sciences, 106, 4567–4570]. In this paper, we test whether in Greek speakers exposure to a new cultural environment (UK) with contrasting colour terminology from their native language affects early perceptual processing as indexed by an electrophysiological correlate of visual detection of colour luminance. We also report semantic mapping of native colour terms and colour similarity judgements. Results reveal convergence of linguistic descriptions, cognitive processing, and early perception of colour in bilinguals. This result demonstrates for the first time substantial plasticity in early, pre-attentive colour perception and has important implications for the mechanisms that are involved in perceptual changes during the processes of language learning and acculturation.

Diurnal variation in transect counts of birds in a cerrado landscape in the state of São Paulo, Brazil

The general statement that birds are recorded more often on morning than on afternoon counts is quite common and widespread among ornithologists. Although many investigators have reported temporal variations in bird detections using Point Counts in temperate regions, few researches regarding the same objectives have been conducted in Neotropical habitats or used transect counts as field method. We used transect counts to test the hypothesis that birds are evenly recorded between times of day in a predominantly open Cerrado landscape in southeastern Brazil. Although not always significantly, the number of species and individuals were consistently greater during the morning counts, which corroborates the fact that birds can be more detectable during this time of day. However, a few families as well as a small percentage of species were more likely to be recorded during either one of the two periods we analyzed. Our results suggest that morning counts should detect higher number of both species and individuals in our study area, but specific taxa show distinct patterns of detection which should be acknowledged prior to sampling.

Whole-body MRI of multiple myeloma: comparison of different MRI sequences in assessment of different growth patterns

PURPOSE: To determine sensitivity, specificity and inter-observer variability of different whole-body MRI (WB-MRI) sequences in patients with multiple myeloma (MM). METHODS AND MATERIALS: WB-MRI using a 1.5T MRI scanner was performed in 23 consecutive patients (13 males, 10 females; mean age 63+/-12 years) with histologically proven MM. All patients were clinically classified according to infiltration (low-grade, n=7; intermediate-grade, n=7; high-grade, n=9) and to the staging system of Durie and Salmon PLUS (stage I, n=12; stage II, n=4; stage III, n=7). The control group consisted of 36 individuals without malignancy (25 males, 11 females; mean age 57+/-13 years). Two observers independently evaluated the following WB-MRI sequences: T1w-TSE (T1), T2w-TIRM (T2), and the combination of both sequences, including a contrast-enhanced T1w-TSE with fat-saturation (T1+/-CE/T2). They had to determine growth patterns (focal and/or diffuse) and the MRI sequence that provided the highest confidence level in depicting the MM lesions. Results were calculated on a per-patient basis. RESULTS: Visual detection of MM was as follows: T1, 65% (sensitivity)/85% (specificity); T2, 76%/81%; T1+/-CE/T2, 67%/88%. Inter-observer variability was as follows: T1, 0.3; T2, 0.55; T1+/-CE/T2, 0.55. Sensitivity improved depending on infiltration grade (T1: 1=60%; 2=36%; 3=83%; T2: 1=70%; 2=71%; 3=89%; T1+/-CE/T2: 1=50%; 2=50%; 3=89%) and clinical stage (T1: 1=58%; 2=63%; 3=79%; T2: 1=58%; 2=88%; 3=100%; T1+/-CE/T2: 1=50%; 2=63%; 3=100%). T2w-TIRM sequences achieved the best reliability in depicting the MM lesions (65% in the mean of both readers). CONCLUSIONS: T2w-TIRM sequences achieved the highest level of sensitivity and best reliability, and thus might be valuable for initial assessment of MM. For an exact staging and grading the examination protocol should encompass unenhanced and enhanced T1w-MRI sequences, in addition to T2w-TIRM.

A new feature extraction method for signal classification applied to cat spinal cord signals

In the spinal cord of the anesthetized cat, spontaneous cord dorsum potentials (CDPs) appear synchronously along the lumbo-sacral segments. These CDPs have different shapes and magnitudes. Previous work has indicated that some CDPs appear to be specially associated with the activation of spinal pathways that lead to primary afferent depolarization and presynaptic inhibition. Visual detection and classification of these CDPs provides relevant information on the functional organization of the neural networks involved in the control of sensory information and allows the characterization of the changes produced by acute nerve and spinal lesions. We now present a novel feature extraction approach for signal classification, applied to CDP detection. The method is based on an intuitive procedure. We first remove by convolution the noise from the CDPs recorded in each given spinal segment. Then, we assign a coefficient for each main local maximum of the signal using its amplitude and distance to the most important maximum of the signal. These coefficients will be the input for the subsequent classification algorithm. In particular, we employ gradient boosting classification trees. This combination of approaches allows a faster and more accurate discrimination of CDPs than is obtained by other methods.

Colour changes by laser irradiation of reddish building limestones

We have used X-ray photoelectron spectroscopy (XPS) as a novel method to investigate the causes of colour changes in a reddish limestone under irradiation by a Q-switched Nd:YAG 1064 nm laser. We irradiated clean dry and wet surfaces of Pidramuelle Roja, a building stone frequently used in the Asturian heritage, at fluences ranging from 0.12 to 1.47 J cm−2. We measured the colour coordinates and undertook XPS analysis of the state of oxidation of iron both before and after irradiation. Visible colour changes and potential aesthetic damage occurred on dry surfaces from a fluence of 0.31 J cm−2, with the stone showing a greening effect and very intense darkening. The colour change on dry surfaces was considerably higher than on wet surfaces, which at the highest fluence (1.47 J cm−2) was also above the human visual detection threshold. The use of XPS demonstrated that the change in colour (chroma and hue) is associated with a reduction in the iron oxidation state on dry surfaces during laser irradiation. This points out to a potential routinary use of XPS to analyse causes of colour changes during laser cleaning in other types of coloured building stones.

Data compilation of marine pelagic organism biomasses, swimming velocities, clearance and respiration rates

Jellyfishes have functionally replaced several overexploited commercial stocks of planktivorous fishes. This is paradoxical, because they use a primitive prey capture mechanism requiring direct contact with the prey, whereas fishes use more efficient visual detection. We have compiled published data to show that, in spite of their primitive life-style, jellyfishes exhibit similar instantaneous prey clearance and respiration rates as their fish competitors and similar potential for growth and reproduction. To achieve this production, they have evolved large, water-laden bodies that increase prey contact rates. Although larger bodies are less efficient for swimming, optimization analysis reveals that large collectors are advantageous if they move through the water sufficiently slowly.

Seeing light vs dark lines: psychophysical performance is based on separate channels, limited by noise and uncertainty

Visual detection performance (d') is usually an accelerating function of stimulus contrast, which could imply a smooth, threshold-like nonlinearity in the sensory response. Alternatively, Pelli (1985 Journal of the Optical Society of America A 2 1508 - 1532) developed the 'uncertainty model' in which responses were linear with contrast, but the observer was uncertain about which of many noisy channels contained the signal. Such internal uncertainty effectively adds noise to weak signals, and predicts the nonlinear psychometric function. We re-examined these ideas by plotting psychometric functions (as z-scores) for two observers (SAW, PRM) with high precision. The task was to detect a single, vertical, blurred line at the fixation point, or identify its polarity (light vs dark). Detection of a known polarity was nearly linear for SAW but very nonlinear for PRM. Randomly interleaving light and dark trials reduced performance and rendered it non-linear for SAW, but had little effect for PRM. This occurred for both single-interval and 2AFC procedures. The whole pattern of results was well predicted by our Monte Carlo simulation of Pelli's model, with only two free parameters. SAW (highly practised) had very low uncertainty. PRM (with little prior practice) had much greater uncertainty, resulting in lower contrast sensitivity, nonlinear performance, and no effect of external (polarity) uncertainty. For SAW, identification was about v2 better than detection, implying statistically independent channels for stimuli of opposite polarity, rather than an opponent (light - dark) channel. These findings strongly suggest that noise and uncertainty, rather than sensory nonlinearity, limit visual detection.

Inter-individual variability in optimal current direction for transcranial magnetic stimulation of the motor cortex

We evaluated inter-individual variability in optimal current direction for biphasic transcranial magnetic stimulation (TMS) of the motor cortex. Motor threshold for first dorsal interosseus was detected visually at eight coil orientations in 45° increments. Each participant (n = 13) completed two experimental sessions. One participant with low test–retest correlation (Pearson's r < 0.5) was excluded. In four subjects, visual detection of motor threshold was compared to EMG detection; motor thresholds were very similar and highly correlated (0.94–0.99). Similar with previous studies, stimulation in the majority of participants was most effective when the first current pulse flowed towards postero-lateral in the brain. However, in four participants, the optimal coil orientation deviated from this pattern. A principal component analysis using all eight orientations suggests that in our sample the optimal orientation of current direction was normally distributed around the postero-lateral orientation with a range of 63° (S.D. = 13.70°). Whenever the intensity of stimulation at the target site is calculated as a percentage from the motor threshold, in order to minimize intensity and side-effects it may be worthwhile to check whether rotating the coil 45° from the traditional posterior–lateral orientation decreases motor threshold.

Seawater carbonate chemistry and brown dottyback (Pseudochromis fuscus) movement and feeding behaviour during experiments, 2011

Changes in olfactory-mediated behaviour caused by elevated CO2 levels in the ocean could affect recruitment to reef fish populations because larval fish become more vulnerable to predation. However, it is currently unclear how elevated CO2 will impact the other key part of the predator-prey interaction - the predators. We investigated the effects of elevated CO2 and reduced pH on olfactory preferences, activity levels and feeding behaviour of a common coral reef meso-predator, the brown dottyback (Pseudochromis fuscus). Predators were exposed to either current-day CO2 levels or one of two elevated CO2 levels (~600 µatm or ~950 µatm) that may occur by 2100 according to climate change predictions. Exposure to elevated CO2 and reduced pH caused a shift from preference to avoidance of the smell of injured prey, with CO2treated predators spending approximately 20% less time in a water stream containing prey odour compared with controls. Furthermore, activity levels of fish was higher in the high CO2 treatment and feeding activity was lower for fish in the mid CO2treatment; indicating that future conditions may potentially reduce the ability of the fish to respond rapidly to fluctuations in food availability. Elevated activity levels of predators in the high CO2 treatment, however, may compensate for reduced olfactory ability, as greater movement facilitated visual detection of food. Our findings show that, at least for the species tested to date, both parties in the predator-prey relationship may be affected by ocean acidification. Although impairment of olfactory-mediated behaviour of predators might reduce the risk of predation for larval fishes, the magnitude of the observed effects of elevated CO2 acidification appear to be more dramatic for prey compared to predators. Thus, it is unlikely that the altered behaviour of predators is sufficient to fully compensate for the effects of ocean acidification on prey mortality.

Mapping and modulating spatial attention asymmetries in young and older adults

Healthy young adults demonstrate a group-level, systematic preference for stimuli presented in the left side of space relative to the right (‘pseudoneglect’) (Bowers & Heilman, 1980). This results in an overestimation of features such as size, brightness, numerosity and spatial frequency in the left hemispace, probably as a result of right cerebral hemisphere dominance for visuospatial attention. This spatial attention asymmetry is reduced in the healthy older population, and can be shifted entirely into right hemispace under certain conditions. Although this rightward shift has been consistently documented in behavioural experiments, there is very little neuroimaging evidence to explain this effect at a neuroanatomical level. In this thesis, I used behavioural methodology and electroencephalography (EEG) to map spatial attention asymmetries in young and older adults. I then use transcranial direct current stimulation (tDCS) to modulate these spatial biases, with the aim of assessing age-related differences in response to tDCS. In the first of three experiments presented in this thesis, I report in Chapter Two that five different spatial attention tasks provide consistent intra-task measures of spatial bias in young adults across two testing days. There were, however, no inter-task correlations between the five tasks, indicating that pseudoneglect is at least partially driven by task-dependent patterns of neural activity. In Chapter Three, anodal tDCS was applied separately to the left (P5) and right (P6) posterior parietal cortex (PPC) in young and older adults, with an aim to improve the detection of stimuli appearing in the contralateral visual field. There were no age differences in response to tDCS, but there were significant differences depending on baseline performance. Relative to a sham tDCS protocol, tDCS applied to the right PPC resulted in maintained visual detection across both visual fields in adults who were good at the task at baseline. In contrast, left PPC tDCS resulted in reduced detection sensitivity across both visual fields in poor performers. Finally, in Chapter Four, I report a right-hemisphere lateralisation of EEG activity in young adults that was present for long (but not short) landmark task lines. In contrast, older adults demonstrated no lateralised activity for either line length, thus providing novel evidence of an age-related reduction of hemispheric asymmetry in older adults. The results of this thesis provide evidence of a highly complex set of factors that underlie spatial attention asymmetries in healthy young and older adults.

Lip detection for audio-visual speech recognition in-car environment

Acoustically, car cabins are extremely noisy and as a consequence audio-only, in-car voice recognition systems perform poorly. As the visual modality is immune to acoustic noise, using the visual lip information from the driver is seen as a viable strategy in circumventing this problem by using audio visual automatic speech recognition (AVASR). However, implementing AVASR requires a system being able to accurately locate and track the drivers face and lip area in real-time. In this paper we present such an approach using the Viola-Jones algorithm. Using the AVICAR [1] in-car database, we show that the Viola- Jones approach is a suitable method of locating and tracking the driver’s lips despite the visual variability of illumination and head pose for audio-visual speech recognition system.