Calibration Method for Underwater Visual Ground-Truth System

This work presents an automatic calibration method for a vision based external underwater ground-truth positioning system. These systems are a relevant tool in benchmarking and assessing the quality of research in underwater robotics applications. A stereo vision system can in suitable environments such as test tanks or in clear water conditions provide accurate position with low cost and flexible operation. In this work we present a two step extrinsic camera parameter calibration procedure in order to reduce the setup time and provide accurate results. The proposed method uses a planar homography decomposition in order to determine the relative camera poses and the determination of vanishing points of detected lines in the image to obtain the global pose of the stereo rig in the reference frame. This method was applied to our external vision based ground-truth at the INESC TEC/Robotics test tank. Results are presented in comparison with an precise calibration performed using points obtained from an accurate 3D LIDAR modelling of the environment.

Hand gesture recognition for human computer interaction : a comparative study of different image features

Hand gesture recognition for human computer interaction, being a natural way of human computer interaction, is an area of active research in computer vision and machine learning. This is an area with many different possible applications, giving users a simpler and more natural way to communicate with robots/systems interfaces, without the need for extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them to convey information or for device control. For that, vision-based hand gesture interfaces require fast and extremely robust hand detection, and gesture recognition in real time. In this study we try to identify hand features that, isolated, respond better in various situations in human-computer interaction. The extracted features are used to train a set of classifiers with the help of RapidMiner in order to find the best learner. A dataset with our own gesture vocabulary consisted of 10 gestures, recorded from 20 users was created for later processing. Experimental results show that the radial signature and the centroid distance are the features that when used separately obtain better results, with an accuracy of 91% and 90,1% respectively obtained with a Neural Network classifier. These to methods have also the advantage of being simple in terms of computational complexity, which make them good candidates for real-time hand gesture recognition.

Generic system for human-computer gesture interaction: applications on sign language recognition and robotic soccer refereeing

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for human-computer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of visionbased interaction systems could be the same for all applications and thus facilitate the implementation. For hand posture recognition, a SVM (Support Vector Machine) model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM (Hidden Markov Model) model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications. To validate the proposed framework two applications were implemented. The first one is a real-time system able to interpret the Portuguese Sign Language. The second one is an online system able to help a robotic soccer game referee judge a game in real time.

Generic system for human-computer gesture interaction

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for humancomputer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of vision-based interaction systems can be the same for all applications and thus facilitate the implementation. In order to test the proposed solutions, three prototypes were implemented. For hand posture recognition, a SVM model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications.

Accurate performance of a rat model of schizophrenia in the water maze depends on visual cue availability and stability: a distortion in cognitive mapping abilities?

Rats were treated postnatally (PND 5-16) with BSO (l-buthionine-(S,R)-sulfoximine) in an animal model of schizophrenia based on transient glutathione deficit. The BSO treated rats were impaired in patrolling a maze or a homing table when adult, yet demonstrated preserved escape learning, place discrimination and reversal in a water maze task [37]. In the present work, BSO rats' performance in the water maze was assessed in conditions controlling for the available visual cues. First, in a completely curtained environment with two salient controlled cues, BSO rats showed little accuracy compared to control rats. Secondly, pre-trained BSO rats were impaired in reaching the familiar spatial position when curtains partially occluded different portions of the room environment in successive sessions. The apparently preserved place learning in a classical water maze task thus appears to require the stability and the richness of visual landmarks from the surrounding environment. In other words, the accuracy of BSO rats in place and reversal learning is impaired in a minimal cue condition or when the visual panorama changes between trials. However, if the panorama remains rich and stable between trials, BSO rats are equally efficient in reaching a familiar position or in learning a new one. This suggests that the BSO accurate performance in the water maze does not satisfy all the criteria for a cognitive map based navigation on the integration of polymodal cues. It supports the general hypothesis of a binding deficit in BSO rats.

Pedicle screw insertion: robotic assistance versus conventional C-arm fluoroscopy.

Bone-mounted robotic guidance for pedicle screw placement has been recently introduced, aiming at increasing accuracy. The aim of this prospective study was to compare this novel approach with the conventional fluoroscopy assisted freehand technique (not the two- or three-dimensional fluoroscopy-based navigation). Two groups were compared: 11 patients, constituting the robotical group, were instrumented with 64 pedicle screws; 23 other patients, constituting the fluoroscopic group, were also instrumented with 64 pedicle screws. Screw position was assessed by two independent observers on postoperative CT-scans using the Rampersaud A to D classification. No neurological complications were noted. Grade A (totally within pedicle margins) accounted for 79% of the screws in the robotically assisted and for 83% of the screws in the fluoroscopic group respectively (p = 0.8). Grade C and D screws, considered as misplacements, accounted for 4.7% of all robotically inserted screws and 7.8% of the fluoroscopically inserted screws (p = 0.71). The current study did not allow to state that robotically assisted screw placement supersedes the conventional fluoroscopy assisted technique, although the literature is more optimistic about the former.

Probabilistic sonar scan matching for an AUV

This paper proposes MSISpIC, a probabilistic sonar scan matching algorithm for the localization of an autonomous underwater vehicle (AUV). The technique uses range scans gathered with a Mechanical Scanning Imaging Sonar (MSIS), the robot displacement estimated through dead-reckoning using a Doppler velocity log (DVL) and a motion reference unit (MRU). The proposed method is an extension of the pIC algorithm. An extended Kalman filter (EKF) is used to estimate the robot-path during the scan in order to reference all the range and bearing measurements as well as their uncertainty to a scan fixed frame before registering. The major contribution consists of experimentally proving that probabilistic sonar scan matching techniques have the potential to improve the DVL-based navigation. The algorithm has been tested on an AUV guided along a 600 m path within an abandoned marina underwater environment with satisfactory results

Assessment of speed of human locomotion using a differential satellite global positioning system.

PURPOSE: The objective was to explore whether a satellite-based navigation system, global positioning system used in differential mode (DGPS), could accurately assess the speed of running in humans. METHODS: A subject was equipped with a portable GPS receptor coupled to a receiver for differential corrections, while running outdoors on a straight asphalt road at 27 different speeds. Actual speed (reference method) was assessed by chronometry. RESULTS: The accuracy of speed prediction had a standard deviation (SD) of 0.08 km x h(-1) for walking, 0.11 km x h(-1) for running, yielding a coefficient of variation (SD/mean) of 1.38% and 0.82%, respectively. There was a highly significant linear relationship between actual and DGPS speed assessment (r2 = 0.999) with little bias in the prediction equation, because the slope of the regression line was close to unity (0.997). CONCLUSION: the DGPS technique appears to be a valid and inconspicuous tool for "on line" monitoring of the speed of displacement of individuals located on any field on earth, for prolonged periods of time and unlimited distance, but only in specific environmental conditions ("open sky"). Furthermore, the accuracy of speed assessment using the differential GPS mode was improved by a factor of 10 as compared to non-differential GPS.

Using the optical mouse sensor as a two-euro counterfeit coin detector

In this paper, the sensor of an optical mouse is presented as a counterfeit coin detector applied to the two-Euro case. The detection process is based on the short distance image acquisition capabilities of the optical mouse sensor where partial images of the coin under analysis are compared with some partial reference coin images for matching. Results show that, using only the vision sense, the counterfeit acceptance and rejection rates are very similar to those of a trained user and better than those of an untrained user.

Discriminating crop, weeds and soil surface with a terrestrial LIDAR sensor

In this study, the evaluation of the accuracy and performance of a light detection and ranging (LIDAR) sensor for vegetation using distance and reflection measurements aiming to detect and discriminate maize plants and weeds from soil surface was done. The study continues a previous work carried out in a maize field in Spain with a LIDAR sensor using exclusively one index, the height profile. The current system uses a combination of the two mentioned indexes. The experiment was carried out in a maize field at growth stage 12–14, at 16 different locations selected to represent the widest possible density of three weeds: Echinochloa crus-galli (L.) P.Beauv., Lamium purpureum L., Galium aparine L.and Veronica persica Poir.. A terrestrial LIDAR sensor was mounted on a tripod pointing to the inter-row area, with its horizontal axis and the field of view pointing vertically downwards to the ground, scanning a vertical plane with the potential presence of vegetation. Immediately after the LIDAR data acquisition (distances and reflection measurements), actual heights of plants were estimated using an appropriate methodology. For that purpose, digital images were taken of each sampled area. Data showed a high correlation between LIDAR measured height and actual plant heights (R2 = 0.75). Binary logistic regression between weed presence/absence and the sensor readings (LIDAR height and reflection values) was used to validate the accuracy of the sensor. This permitted the discrimination of vegetation from the ground with an accuracy of up to 95%. In addition, a Canonical Discrimination Analysis (CDA) was able to discriminate mostly between soil and vegetation and, to a far lesser extent, between crop and weeds. The studied methodology arises as a good system for weed detection, which in combination with other principles, such as vision-based technologies, could improve the efficiency and accuracy of herbicide spraying.

Yksittäistuotannon robotisointi konenäön avulla

Työn tavoitteena oli uuden konenäköpohjaisen hitsausrobottiaseman käyttöönotto ja järjestelmän kehittäminen siten, että voitiin mahdollistaa vähintään vaadittu 70 %:n kaariaikasuhde ohjelmien sisällä. Aseman käyttöönotolla pyrittiin tehostamaan hitsaustyötä ja helpottamaan tuotanto-paineita osavalmistuksen paneeli-linjalla. Hitsausasemalla parannetaan myös työntekijöiden työergonomiaa sekä koko työn imagoa. Kirjallisen osan tarkoituksena oli tutkia robotisoitua hitsausta ja sen tuomia etuja, verrata jo olemassa olevia ohjelmointimenetelmiä uuteen ohjelmointitapaan ja selvittää robottien tarkkuuksia. Kirjallisuusosan jälkimmäisessä osassa tutustuttiin konenäköön ja sen laitteistoihin sekä sovelluksiin. Kirjallisuus-tutkimuksessa selvisi, että uusi ohjelmointimenetelmä on selvä parannus kyseiseen sovellukseen. Käytännön osassa on esitelty konenäköpohjainen hitsausrobottiasema sen suunnittelusta, käyttöönottoon ja tuotantotesteihin asti. Käytännön osassa on lisäksi esitelty asema osana koko automatisointiprojektia. Lisäksi on kerrottu järjestelmän toiminnasta sekä makroista, joihin robotin toiminta perustuu. Lopuksi on tehty katsaus maailmalla olevista vastaavista järjestelmistä ja niiden teknologioista sekä verrattiin niitä tähän uuteen järjestelmään. Tavoitteena olleeseen 70% kaariaikasuhteeseen ohjelmien sisällä päästiin jo lyhyen koejakson aikana. Käyttöönotettu konenäköpohjainen hitsausrobottiasema on tiettävästi ensimmäinen hitsausasema maailmassa, jota ei tarvitse ohjelmoida etukäteen. Ohjelmointiin kuluva aika on minimaalinen, koska operaattori ohjelmoi robotin sen hitsatessa tuotetta. Kokeellinen osa osoitti, että käyttöönotettu konenäköpohjainen hitsausrobottiasema toimii, kuten se oli suunniteltukin. Käyttöönoton yhteydessä huomattiin monia kehitettäviä asioita, joilla järjestelmästä on mahdollista saada vieläkin tehokkaampi.

Konenäköavusteinen robotiikka elintarviketeollisuudessa

Elintarvikeala on muita toimialoja kannattomampana jäänyt automatisoinnin kehityksessä jälkeen. Varsinkin pienet - sekä keskisuuret yritykset ovat olleet haluttomia panostamaan automatisointiin. Monet automatisoitavissa olevat sovellutukset ovat jääneet toteuttamatta ja ne tehdään edelleen manuaalisesti. Automaatioyritykset ovat nähneet tämän elintarviketeollisuudessa vallitsevan potentiaalin, mutta prosessit ovat olleet liian hankalia automatisoida kilpailukykyisesti. Teknologioiden kehityttyä on asia kuitenkin muuttunut ja elintarviketeollisuus on täynnä mahdollisuuksia.

Vuokamallisten kartonkipakkausten laadunvarmistuslaitteiston kehittäminen

Diplomityön tavoitteena oli kehittää vuokamallisille kartonkipakkauksille laadunvarmistuslaitteisto. Kirjallisen osan alussa esiteltiin vuokamallisten kartonkipakkausten valmistusprosessia. Tästä siirryttiin laatuasioihin, jossa tärkeimmät asiat olivat kartonkivuokien valmistuksessa esiintyvät laatupoikkeamat ja konenäkö. Tutkimusosan alussa esitellään Lappeenrannan teknillisessä yliopistossa kehitetty kartonkivuokien valmistuslinjasto. Tämän jälkeen vaatimuslistan pohjalta suunnitellaan kyseiseen linjastoon sopiva automaattinen laadunvalvontalaite, johon sisältyy myös kartonkivuokien siirtolaite. Suunnitteluprosessi aloitettiin koekuvaamalla kartonkivuokia erilaisilla kameroilla ja valaistusmenetelmillä. Koekuvausten perusteella valittiin konenäkölaitteisto. Tämän jälkeen toiminnoista luotiin periaatepiirroksia, joista kehitettiin varsinainen suunnitelma. Työn tuloksena saatiin suunnitelma konenäköön perustuvan automaattisen laadunvalvontalaitteen rakentamiselle.

Automatic traffic sign inventory- and condition analysis

This thesis researches automatic traffic sign inventory and condition analysis using machine vision and pattern recognition methods. Automatic traffic sign inventory and condition analysis can be used to more efficient road maintenance, improving the maintenance processes, and to enable intelligent driving systems. Automatic traffic sign detection and classification has been researched before from the viewpoint of self-driving vehicles, driver assistance systems, and the use of signs in mapping services. Machine vision based inventory of traffic signs consists of detection, classification, localization, and condition analysis of traffic signs. The produced machine vision system performance is estimated with three datasets, from which two of have been been collected for this thesis. Based on the experiments almost all traffic signs can be detected, classified, and located and their condition analysed. In future, the inventory system performance has to be verified in challenging conditions and the system has to be pilot tested.

Étude électrophysiologique de la suppression interoculaire : implication pour l'amblyopie

L’amblyopie est un trouble développemental de la vision binoculaire. Elle est typiquement caractérisée par des atteintes de l’acuité visuelle et de la stéréoscopie. Toutefois, de plus en plus d’études indiquent la présence d’atteintes plus étendues telles que les difficultés d’attention visuelle ou de lecture. L’amblyopie est généralement expliquée par une suppression interoculaire au niveau cortical, considérée comme chronique ou permanente à l’extérieur de la période développementale. Or, un nombre croissant d’études suggèrent que des interactions binoculaires normales seraient présentes chez les amblyopes adultes. Dans une première étude, nous avons tenté d’identifier un marqueur électrophysiologique de la vision binoculaire. Nous avons enregistré des potentiels évoqués visuels chez des observateurs normaux à qui l’on a induit une dysfonction binoculaire. Les interactions binoculaires étaient caractérisées à l’aide de patrons (facilitation, moyennage et suppression) en comparant les réponses monoculaires et binoculaires. De plus, ces interactions étaient quantifiées à partir d’index d’intégration continus en soustrayant la somme des réponses monoculaires de la réponse binoculaire. Les résultats indiquaient que les patrons d’interaction n’étaient pas optimaux pour estimer les performances stéréoscopiques. Ces dernières étaient, en revanche, mieux expliquées par notre index d’intégration binoculaire. Ainsi, cette étude suggère que l’électrophysiologie est un bon prédicteur de la vision binoculaire. Dans une deuxième étude, nous avons examiné les corrélats neuronaux et comportementaux de la suppression interoculaire chez des amblyopes adultes et des observateurs normaux. Des potentiels évoqués visuels stationnaires ont été enregistrés en utilisant un paradigme de suppression par flash. La suppression était modulée par un changement de contraste du stimulus flash (10, 20, 30, ou 100%), ou le suppresseur, qui était présenté soit dans l’œil dominant ou non-dominant (ou amblyope). Sur le plan comportemental, la suppression interoculaire était observée indépendamment de l’œil stimulé par le flash chez les contrôles. Au contraire, chez les amblyopes, la suppression était asymétrique (c’est-à-dire supérieure lorsqu’elle provenait de l’œil dominant), ce qui suggérait une suppression chronique. De manière intéressante, l’œil amblyope a supprimé l’œil dominant à haut niveau de contraste. Sur le plan électrophysiologique, l’effet de suppression interoculaire observé à la région occipitale était équivalent dans chaque groupe. Toutefois, les réponses électrophysiologiques à la région frontale chez les amblyopes n’étaient pas modulées comme celles des contrôles; la suppression de l’œil amblyope était manifeste même à bas contraste. Nous résultats supportent ainsi l’existence d’interaction binoculaire fonctionnelle chez les amblyopes adultes ainsi que l’implication d’un réseau cortical étendu dans la suppression interoculaire. En somme, l’amblyopie est une condition complexe dont les atteintes corticales et les déficits fonctionnels semblent globaux. L’amblyopie ne doit plus être considérée comme limitée à une dysfonction de l’aire visuelle primaire. La suppression interoculaire semble un point central de cette problématique, mais encore beaucoup d’études seront nécessaires afin de déterminer l’ensemble des mécanismes impliqués dans celle-ci.