Visual sensor fusion for active security in robotic industrial environments

This work presents a method of information fusion involving data captured by both a standard CCD camera and a ToF camera to be used in the detection of the proximity between a manipulator robot and a human. Both cameras are assumed to be located above the work area of an industrial robot. The fusion of colour images and time of light information makes it possible to know the 3D localization of objects with respect to a world coordinate system. At the same time this allows to know their colour information. Considering that ToF information given by the range camera contains innacuracies including distance error, border error, and pixel saturation, some corrections over the ToF information are proposed and developed to improve the results. The proposed fusion method uses the calibration parameters of both cameras to reproject 3D ToF points, expressed in a common coordinate system for both cameras and a robot arm, in 2D colour images. In addition to this, using the 3D information, the motion detection in a robot industrial environment is achieved, and the fusion of information is applied to the foreground objects previously detected. This combination of information results in a matrix that links colour and 3D information, giving the possibility of characterising the object by its colour in addition to its 3D localization. Further development of these methods will make it possible to identify objects and their position in the real world, and to use this information to prevent possible collisions between the robot and such objects.

Representing 3D shape and location

The 3D shape of an object and its 3D location have traditionally thought of as very separate entities, although both can be described within a single 3D coordinate frame. Here, 3D shape and location are considered as two aspects of a view-based approach to representing depth, avoiding the use of 3D coordinate frames.

Visual sensor fusion for active security in robotic industrial environments

This work presents a method of information fusion involving data captured by both a standard charge-coupled device (CCD) camera and a time-of-flight (ToF) camera to be used in the detection of the proximity between a manipulator robot and a human. Both cameras are assumed to be located above the work area of an industrial robot. The fusion of colour images and time-of-flight information makes it possible to know the 3D localization of objects with respect to a world coordinate system. At the same time, this allows to know their colour information. Considering that ToF information given by the range camera contains innacuracies including distance error, border error, and pixel saturation, some corrections over the ToF information are proposed and developed to improve the results. The proposed fusion method uses the calibration parameters of both cameras to reproject 3D ToF points, expressed in a common coordinate system for both cameras and a robot arm, in 2D colour images. In addition to this, using the 3D information, the motion detection in a robot industrial environment is achieved, and the fusion of information is applied to the foreground objects previously detected. This combination of information results in a matrix that links colour and 3D information, giving the possibility of characterising the object by its colour in addition to its 3D localisation. Further development of these methods will make it possible to identify objects and their position in the real world and to use this information to prevent possible collisions between the robot and such objects.

Video stimuli reduce object-directed imitation accuracy: a novel two-person motion-tracking approach

Imitation is an important form of social behavior, and research has aimed to discover and explain the neural and kinematic aspects of imitation. However, much of this research has featured single participants imitating in response to pre-recorded video stimuli. This is in spite of findings that show reduced neural activation to video vs. real life movement stimuli, particularly in the motor cortex. We investigated the degree to which video stimuli may affect the imitation process using a novel motion tracking paradigm with high spatial and temporal resolution. We recorded 14 positions on the hands, arms, and heads of two individuals in an imitation experiment. One individual freely moved within given parameters (moving balls across a series of pegs) and a second participant imitated. This task was performed with either simple (one ball) or complex (three balls) movement difficulty, and either face-to-face or via a live video projection. After an exploratory analysis, three dependent variables were chosen for examination: 3D grip position, joint angles in the arm, and grip aperture. A cross-correlation and multivariate analysis revealed that object-directed imitation task accuracy (as represented by grip position) was reduced in video compared to face-to-face feedback, and in complex compared to simple difficulty. This was most prevalent in the left-right and forward-back motions, relevant to the imitator sitting face-to-face with the actor or with a live projected video of the same actor. The results suggest that for tasks which require object-directed imitation, video stimuli may not be an ecologically valid way to present task materials. However, no similar effects were found in the joint angle and grip aperture variables, suggesting that there are limits to the influence of video stimuli on imitation. The implications of these results are discussed with regards to previous findings, and with suggestions for future experimentation.

Environmental factors and features that influence visual search in a 3D WIMP interface

The challenge of moving past the classic Window Icons Menus Pointer (WIMP) interface, i.e. by turning it ‘3D’, has resulted in much research and development. To evaluate the impact of 3D on the ‘finding a target picture in a folder’ task, we built a 3D WIMP interface that allowed the systematic manipulation of visual depth, visual aides, semantic category distribution of targets versus non-targets; and the detailed measurement of lower-level stimuli features. Across two separate experiments, one large sample web-based experiment, to understand associations, and one controlled lab environment, using eye tracking to understand user focus, we investigated how visual depth, use of visual aides, use of semantic categories, and lower-level stimuli features (i.e. contrast, colour and luminance) impact how successfully participants are able to search for, and detect, the target image. Moreover in the lab-based experiment, we captured pupillometry measurements to allow consideration of the influence of increasing cognitive load as a result of either an increasing number of items on the screen, or due to the inclusion of visual depth. Our findings showed that increasing the visible layers of depth, and inclusion of converging lines, did not impact target detection times, errors, or failure rates. Low-level features, including colour, luminance, and number of edges, did correlate with differences in target detection times, errors, and failure rates. Our results also revealed that semantic sorting algorithms significantly decreased target detection times. Increased semantic contrasts between a target and its neighbours correlated with an increase in detection errors. Finally, pupillometric data did not provide evidence of any correlation between the number of visible layers of depth and pupil size, however, using structural equation modelling, we demonstrated that cognitive load does influence detection failure rates when there is luminance contrasts between the target and its surrounding neighbours. Results suggest that WIMP interaction designers should consider stimulus-driven factors, which were shown to influence the efficiency with which a target icon can be found in a 3D WIMP interface.

Cross-situational learning of object-word mapping using Neural Modeling Fields

The issue of how children learn the meaning of words is fundamental to developmental psychology. The recent attempts to develop or evolve efficient communication protocols among interacting robots or Virtual agents have brought that issue to a central place in more applied research fields, such as computational linguistics and neural networks, as well. An attractive approach to learning an object-word mapping is the so-called cross-situational learning. This learning scenario is based on the intuitive notion that a learner can determine the meaning of a word by finding something in common across all observed uses of that word. Here we show how the deterministic Neural Modeling Fields (NMF) categorization mechanism can be used by the learner as an efficient algorithm to infer the correct object-word mapping. To achieve that we first reduce the original on-line learning problem to a batch learning problem where the inputs to the NMF mechanism are all possible object-word associations that Could be inferred from the cross-situational learning scenario. Since many of those associations are incorrect, they are considered as clutter or noise and discarded automatically by a clutter detector model included in our NMF implementation. With these two key ingredients - batch learning and clutter detection - the NMF mechanism was capable to infer perfectly the correct object-word mapping. (C) 2009 Elsevier Ltd. All rights reserved.

A non-parametric vessel detection method for complex vascular structures

Modern medical imaging techniques enable the acquisition of in vivo high resolution images of the vascular system. Most common methods for the detection of vessels in these images, such as multiscale Hessian-based operators and matched filters, rely on the assumption that at each voxel there is a single cylinder. Such an assumption is clearly violated at the multitude of branching points that are easily observed in all, but the Most focused vascular image studies. In this paper, we propose a novel method for detecting vessels in medical images that relaxes this single cylinder assumption. We directly exploit local neighborhood intensities and extract characteristics of the local intensity profile (in a spherical polar coordinate system) which we term as the polar neighborhood intensity profile. We present a new method to capture the common properties shared by polar neighborhood intensity profiles for all the types of vascular points belonging to the vascular system. The new method enables us to detect vessels even near complex extreme points, including branching points. Our method demonstrates improved performance over standard methods on both 2D synthetic images and 3D animal and clinical vascular images, particularly close to vessel branching regions. (C) 2008 Elsevier B.V. All rights reserved.

Minimizing the object dimensions in circle and sphere packing problems

Given a fixed set of identical or different-sized circular items, the problem we deal with consists on finding the smallest object within which the items can be packed. Circular, triangular, squared, rectangular and also strip objects are considered. Moreover, 2D and 3D problems are treated. Twice-differentiable models for all these problems are presented. A strategy to reduce the complexity of evaluating the models is employed and, as a consequence, instances with a large number of items can be considered. Numerical experiments show the flexibility and reliability of the new unified approach. (C) 2007 Elsevier Ltd. All rights reserved.

Face Detection and Facial Feature Localization for multi-pose faces and complex backgroundimages

The objective of this thesis work, is to propose an algorithm to detect the faces in a digital image with complex background. A lot of work has already been done in the area of face detection, but drawback of some face detection algorithms is the lack of ability to detect faces with closed eyes and open mouth. Thus facial features form an important basis for detection. The current thesis work focuses on detection of faces based on facial objects. The procedure is composed of three different phases: segmentation phase, filtering phase and localization phase. In segmentation phase, the algorithm utilizes color segmentation to isolate human skin color based on its chrominance properties. In filtering phase, Minkowski addition based object removal (Morphological operations) has been used to remove the non-skin regions. In the last phase, Image Processing and Computer Vision methods have been used to find the existence of facial components in the skin regions.This method is effective on detecting a face region with closed eyes, open mouth and a half profile face. The experiment’s results demonstrated that the detection accuracy is around 85.4% and the detection speed is faster when compared to neural network method and other techniques.

Detection and thinning of edges in previously known directions

This work presents a study on the generation of digital masks aiming at edge detection with previously known directions. This solution is important when edge direction is available either from a direction histogram or from a prediction based on camera and object models. A modification in the non-maximum suppression method of thinning is also presented enabling the comparison of local maxima for any edge directions. Results with a synthetic image and with crops of a CBERS satellite images are presented showing an example with its application in road detection, provided that directions are previously known.

Proposta metodológica para o imageamento digital e modelagem virtual 3d de um bloco de rochas travertinas

In this paper we present the methodological procedures involved in the digital imaging in mesoscale of a block of travertines rock of quaternary age, originating from the city of Acquasanta, located in the Apennines, Italy. This rocky block, called T-Block, was stored in the courtyard of the Laboratório Experimental Petróleo "Kelsen Valente" (LabPetro), of Universidade Estadual de Campinas (UNICAMP), so that from it were performed Scientific studies, mainly for research groups universities and research centers working in brazilian areas of reservoir characterization and 3D digital imaging. The purpose of this work is the development of a Model Solid Digital, from the use of non-invasive techniques of digital 3D imaging of internal and external surfaces of the T-Block. For the imaging of the external surfaces technology has been used LIDAR (Light Detection and Range) and the imaging surface Interior was done using Ground Penetrating Radar (GPR), moreover, profiles were obtained with a Gamma Ray Gamae-spectômetro laptop. The goal of 3D digital imaging involved the identification and parameterization of surface geological and sedimentary facies that could represent heterogeneities depositional mesoscale, based on study of a block rocky with dimensions of approximately 1.60 m x 1.60 m x 2.70 m. The data acquired by means of terrestrial laser scanner made available georeferenced spatial information of the surface of the block (X, Y, Z), and varying the intensity values of the return laser beam and high resolution RGB data (3 mm x 3 mm), total points acquired 28,505,106. This information was used as an aid in the interpretation of radargrams and are ready to be displayed in rooms virtual reality. With the GPR was obtained 15 profiles of 2.3 m and 2 3D grids, each with 24 sections horizontal of 1.3 and 14 m vertical sections of 2.3 m, both the Antenna 900 MHz to about 2600 MHz antenna. Finally, the use of GPR associated with Laser Scanner enabled the identification and 3D mapping of 3 different radarfácies which were correlated with three sedimentary facies as had been defined at the outset. The 6 profiles showed gamma a low amplitude variation in the values of radioactivity. This is likely due to the fact of the sedimentary layers profiled have the same mineralogical composition, being composed by carbonate sediments, with no clay in siliciclastic pellitic layers or other mineral carrier elements radioactive

Employing 2D Projections for Fast Visual Exploration of Large Fiber Tracking Data

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Tensor3D: A computer graphics program to simulate 3D real-time deformation and visualization of geometric bodies

Tensor3D is a geometric modeling program with the capacity to simulate and visualize in real-time the deformation, specified through a tensor matrix and applied to triangulated models representing geological bodies. 3D visualization allows the study of deformational processes that are traditionally conducted in 2D, such as simple and pure shears. Besides geometric objects that are immediately available in the program window, the program can read other models from disk, thus being able to import objects created with different open-source or proprietary programs. A strain ellipsoid and a bounding box are simultaneously shown and instantly deformed with the main object. The principal axes of strain are visualized as well to provide graphical information about the orientation of the tensor's normal components. The deformed models can also be saved, retrieved later and deformed again, in order to study different steps of progressive strain, or to make this data available to other programs. The shape of stress ellipsoids and the corresponding Mohr circles defined by any stress tensor can also be represented. The application was written using the Visualization ToolKit, a powerful scientific visualization library in the public domain. This development choice, allied to the use of the Tcl/Tk programming language, which is independent on the host computational platform, makes the program a useful tool for the study of geometric deformations directly in three dimensions in teaching as well as research activities. (C) 2007 Elsevier Ltd. All rights reserved.

Classificação automática de padrões projetados por um sistema de luz estruturada

One of the main problems in Computer Vision and Close Range Digital Photogrammetry is 3D reconstruction. 3D reconstruction with structured light is one of the existing techniques and which still has several problems, one of them the identification or classification of the projected targets. Approaching this problem is the goal of this paper. An area based method called template matching was used for target classification. This method performs detection of area similarity by correlation, which measures the similarity between the reference and search windows, using a suitable correlation function. In this paper the modified cross covariance function was used, which presented the best results. A strategy was developed for adaptative resampling of the patterns, which solved the problem of deformation of the targets due to object surface inclination. Experiments with simulated and real data were performed in order to assess the efficiency of the proposed methodology for target detection. The results showed that the proposed classification strategy works properly, identifying 98% of targets in plane surfaces and 93% in oblique surfaces.

Detection of cavitated approximal surfaces using cone beam CT and intraoral receptors

Objectives: The aim of this study was to compare cone beam CT (CBCT) in a small field of view (FOV) with a solid-state sensor and a photostimulable phosphor plate system for detection of cavitated approximal surfaces. Methods: 257 non-filled approximal surfaces from human permanent premolars and molars were recorded by two intraoral digital receptors, a storage phosphor plate (Digora Optime, Soredex) and a solid-state CMOS sensor (Digora Toto, Soredex), and scanned in a cone beam CT unit (3D Accuitomo FPD80, Morita) with a FOV of 4 cm and a voxel size of 0.08 mm. Image sections were carried out in the axial and mesiodistal tooth planes. Six observers recorded surface cavitation in all images. Validation of the true absence or presence of surface cavitation was performed by inspecting the surfaces under strong light with the naked eye. Differences in sensitivity, specificity and agreement were estimated by analysing the binary data in a generalized linear model using an identity link function. Results: A significantly higher sensitivity was obtained by all observers with CBCT (p,0.001), which was not compromised by a lower specificity. Therefore, a significantly higher overall agreement was obtained with CBCT (p,0.001). There were no significant differences between the Digora Optime phosphor plate system and the Digora Toto CMOS sensor for any parameter. Conclusions: CBCT was much more accurate in the detection of surface cavitation in approximal surfaces than intraoral receptors. The differences are interpreted as clinically significant. A CBCT examination performed for other reasons should also be assessed for approximal surface cavities in teeth without restorations. © 2013 The British Institute of Radiology.