Evaluation of a ToF Camera for Remote Surveying of Underground Cavities Excavated by Jet Boring

Cigar Lake is a high-grade uranium deposit, located in northern Saskatchewan, Canada. In order to extract the uranium ore remotely, thus ensuring minimal radiation dose to workers and also to access the ore from stable ground, the Jet Boring System (JBS) was developed by Cameco Corporation. This system uses a high-powered water jet to remotely excavate cavities. Survey data is required to determine the final shape, volume, and location of the cavity for mine planning purposes and construction. This paper provides an overview of the challenges involved in remotely surveying a JBS-mined cavity and studies the potential use of a time-of-flight (ToF) camera for remote cavity surveying. It reports on data collected and analyzed from inside an experimental environment as well as on real data acquired on site from the Cigar Lake and Rabbit Lake mines.

Increasing Realism and Supporting Content Planning for Dynamic Scenes in a Mixed Reality System incorporating a Time-of-Flight Camera

For broadcasting purposes MIXED REALITY, the combination of real and virtual scene content, has become ubiquitous nowadays. Mixed Reality recording still requires expensive studio setups and is often limited to simple color keying. We present a system for Mixed Reality applications which uses depth keying and provides threedimensional mixing of real and artificial content. It features enhanced realism through automatic shadow computation which we consider a core issue to obtain realism and a convincing visual perception, besides the correct alignment of the two modalities and correct occlusion handling. Furthermore we present a possibility to support placement of virtual content in the scene. Core feature of our system is the incorporation of a TIME-OF-FLIGHT (TOF)-camera device. This device delivers real-time depth images of the environment at a reasonable resolution and quality. This camera is used to build a static environment model and it also allows correct handling of mutual occlusions between real and virtual content, shadow computation and enhanced content planning. The presented system is inexpensive, compact, mobile, flexible and provides convenient calibration procedures. Chroma-keying is replaced by depth-keying which is efficiently performed on the GRAPHICS PROCESSING UNIT (GPU) by the usage of an environment model and the current ToF-camera image. Automatic extraction and tracking of dynamic scene content is herewith performed and this information is used for planning and alignment of virtual content. An additional sustainable feature is that depth maps of the mixed content are available in real-time, which makes the approach suitable for future 3DTV productions. The presented paper gives an overview of the whole system approach including camera calibration, environment model generation, real-time keying and mixing of virtual and real content, shadowing for virtual content and dynamic object tracking for content planning.

Automatic Identification of Large Fragments in a Pile of Broken Rock Using a Time-of-Flight Camera

This paper presents a solution to part of the problem of making robotic or semi-robotic digging equipment less dependant on human supervision. A method is described for identifying rocks of a certain size that may affect digging efficiency or require special handling. The process involves three main steps. First, by using range and intensity data from a time-of-flight (TOF) camera, a feature descriptor is used to rank points and separate regions surrounding high scoring points. This allows a wide range of rocks to be recognized because features can represent a whole or just part of a rock. Second, these points are filtered to extract only points thought to belong to the large object. Finally, a check is carried out to verify that the resultant point cloud actually represents a rock. Results are presented from field testing on piles of fragmented rock. Note to Practitioners—This paper presents an algorithm to identify large boulders in a pile of broken rock as a step towards an autonomous mining dig planner. In mining, piles of broken rock can contain large fragments that may need to be specially handled. To assess rock piles for excavation, we make use of a TOF camera that does not rely on external lighting to generate a point cloud of the rock pile. We then segment large boulders from its surface by using a novel feature descriptor and distinguish between real and false boulder candidates. Preliminary field experiments show promising results with the algorithm performing nearly as well as human test subjects.

Visual sensor fusion for active security in robotic industrial environments

This work presents a method of information fusion involving data captured by both a standard CCD camera and a ToF camera to be used in the detection of the proximity between a manipulator robot and a human. Both cameras are assumed to be located above the work area of an industrial robot. The fusion of colour images and time of light information makes it possible to know the 3D localization of objects with respect to a world coordinate system. At the same time this allows to know their colour information. Considering that ToF information given by the range camera contains innacuracies including distance error, border error, and pixel saturation, some corrections over the ToF information are proposed and developed to improve the results. The proposed fusion method uses the calibration parameters of both cameras to reproject 3D ToF points, expressed in a common coordinate system for both cameras and a robot arm, in 2D colour images. In addition to this, using the 3D information, the motion detection in a robot industrial environment is achieved, and the fusion of information is applied to the foreground objects previously detected. This combination of information results in a matrix that links colour and 3D information, giving the possibility of characterising the object by its colour in addition to its 3D localization. Further development of these methods will make it possible to identify objects and their position in the real world, and to use this information to prevent possible collisions between the robot and such objects.

Visual sensor fusion for active security in robotic industrial environments

This work presents a method of information fusion involving data captured by both a standard charge-coupled device (CCD) camera and a time-of-flight (ToF) camera to be used in the detection of the proximity between a manipulator robot and a human. Both cameras are assumed to be located above the work area of an industrial robot. The fusion of colour images and time-of-flight information makes it possible to know the 3D localization of objects with respect to a world coordinate system. At the same time, this allows to know their colour information. Considering that ToF information given by the range camera contains innacuracies including distance error, border error, and pixel saturation, some corrections over the ToF information are proposed and developed to improve the results. The proposed fusion method uses the calibration parameters of both cameras to reproject 3D ToF points, expressed in a common coordinate system for both cameras and a robot arm, in 2D colour images. In addition to this, using the 3D information, the motion detection in a robot industrial environment is achieved, and the fusion of information is applied to the foreground objects previously detected. This combination of information results in a matrix that links colour and 3D information, giving the possibility of characterising the object by its colour in addition to its 3D localisation. Further development of these methods will make it possible to identify objects and their position in the real world and to use this information to prevent possible collisions between the robot and such objects.

Calibración de cámaras de tiempo de vuelo: ajuste adaptativo del tiempo de integración y análisis de la frecuencia de modulación

La percepción de profundidad se hace imprescindible en muchas tareas de manipulación, control visual y navegación de robots. Las cámaras de tiempo de vuelo (ToF: Time of Flight) generan imágenes de rango que proporcionan medidas de profundidad en tiempo real. No obstante, el parámetro distancia que calculan estas cámaras es fuertemente dependiente del tiempo de integración que se configura en el sensor y de la frecuencia de modulación empleada por el sistema de iluminación que integran. En este artículo, se presenta una metodología para el ajuste adaptativo del tiempo de integración y un análisis experimental del comportamiento de una cámara ToF cuando se modifica la frecuencia de modulación. Este método ha sido probado con éxito en algoritmos de control visual con arquitectura ‘eye-in-hand’ donde el sistema sensorial está compuesto por una cámara ToF. Además, la misma metodología puede ser aplicada en otros escenarios de trabajo.

Profile Of Phenolic Compounds Of Brazilian Virgin Olive Oils By Rapid Resolution Liquid Chromatography Coupled To Electrospray Ionisation Time-of-flight Mass Spectrometry (rrlc-esi-tof-ms).

In recent years, agronomical researchers began to cultivate several olive varieties in different regions of Brazil to produce virgin olive oil (VOO). Because there has been no reported data regarding the phenolic profile of the first Brazilian VOO, the aim of this work was to determine phenolic contents of these samples using rapid-resolution liquid chromatography coupled to electrospray ionisation time-of-flight mass spectrometry. 25 VOO samples from Arbequina, Koroneiki, Arbosana, Grappolo, Manzanilla, Coratina, Frantoio and MGS Mariense varieties from three different Brazilian states and two crops were analysed. It was possible to quantify 19 phenolic compounds belonging to different classes. The results indicated that Brazilian VOOs have high total phenolic content because the values were comparable with those from high-quality VOOs produced in other countries. VOOs from Coratina, Arbosana and Grappolo presented the highest total phenolic content. These data will be useful in the development and improvement of Brazilian VOO.

Surface foam film waves studied with high-speed linescan camera

Dynamic foam films have been investigated using an improved experimental set-up with a CCD high-speed linescan camera in conjunction with the Scheludko micro-interferometric cell for studying the drainage and rupture of liquid foam films. The improved experimental set-up increased the sensibility of detection of the local thickness heterogeneities and domains during the film evolution. The evolution of the foam films up to the formation of black spots was recorded in the time intervals of 50ms. The wavelengths of the propagating surface waves and their frequencies were determined experimentally. The experimental results show that the current quasi-static hydrodynamic theory does not properly describe the wave dynamics with inter-domain channels. However, the thermodynamic condition for formation of black spots in the foam films was met by the experimental results. (c) 2005 Elsevier B.V. All rights reserved.

People Detection and Tracking using Depth Camera

In this paper we present a method for real-time detection and tracking of people in video captured by a depth camera. For each object to be assessed, an ordered sequence of values that represents the distances between its center of mass to the boundary points is calculated. The recognition is based on the analysis of the total distance value between the above sequence and some pre-defined human poses, after apply the Dynamic Time Warping. This similarity approach showed robust results in people detection.

Sensor element for a metal-insulator-semiconductor camera system (MISCam)

We discuss the operation of a new type of optical sensor (MISCam) based on a metal-insulator-semiconductor (MIS) structure. The operation principle relies on light-induced changes of the band bending and barrier height at the interface between semiconductor and insulator. An image is obtained from the quenching of the ac signal in analogy to the principle of the laser-scanned photodiode (LSP). Lateral resolution depends on the semiconductor material chosen. We have characterised the MIS structures by C-V, I-V, and spectral response measurements testing different types of insulators like a-Si3N4, SiO2, and AlN. The presence of slow interface charges allows for image memory. Colour sensors can be realised by controlling sign and magnitude of the electric fields in the base and the interface region.

Monte Carlo simulation of the basic features of the GE Millennium MG single photon emission computed tomography gamma camera

Objective - To describe and validate the simulation of the basic features of GE Millennium MG gamma camera using the GATE Monte Carlo platform. Material and methods - Crystal size and thickness, parallel-hole collimation and a realistic energy acquisition window were simulated in the GATE platform. GATE results were compared to experimental data in the following imaging conditions: a point source of 99mTc at different positions during static imaging and tomographic acquisitions using two different energy windows. The accuracy between the events expected and detected by simulation was obtained with the Mann–Whitney–Wilcoxon test. Comparisons were made regarding the measurement of sensitivity and spatial resolution, static and tomographic. Simulated and experimental spatial resolutions for tomographic data were compared with the Kruskal–Wallis test to assess simulation accuracy for this parameter. Results - There was good agreement between simulated and experimental data. The number of decays expected when compared with the number of decays registered, showed small deviation (≤0.007%). The sensitivity comparisons between static acquisitions for different distances from source to collimator (1, 5, 10, 20, 30cm) with energy windows of 126–154 keV and 130–158 keV showed differences of 4.4%, 5.5%, 4.2%, 5.5%, 4.5% and 5.4%, 6.3%, 6.3%, 5.8%, 5.3%, respectively. For the tomographic acquisitions, the mean differences were 7.5% and 9.8% for the energy window 126–154 keV and 130–158 keV. Comparison of simulated and experimental spatial resolutions for tomographic data showed no statistically significant differences with 95% confidence interval. Conclusions - Adequate simulation of the system basic features using GATE Monte Carlo simulation platform was achieved and validated.

People counting system using existing surveillance video camera

The Casa da Música Foundation, responsible for the management of Casa da Música do Porto building, has the need to obtain statistical data related to the number of building’s visitors. This information is a valuable tool for the elaboration of periodical reports concerning the success of this cultural institution. For this reason it was necessary to develop a system capable of returning the number of visitors for a requested period of time. This represents a complex task due to the building’s unique architectural design, characterized by very large doors and halls, and the sudden large number of people that pass through them in moments preceding and proceeding the different activities occurring in the building. To achieve the technical solution for this challenge, several image processing methods, for people detection with still cameras, were first studied. The next step was the development of a real time algorithm, using OpenCV libraries and computer vision concepts,to count individuals with the desired accuracy. This algorithm includes the scientific and technical knowledge acquired in the study of the previous methods. The themes developed in this thesis comprise the fields of background maintenance, shadow and highlight detection, and blob detection and tracking. A graphical interface was also built, to help on the development, test and tunning of the proposed system, as a complement to the work. Furthermore, tests to the system were also performed, to certify the proposed techniques against a set of limited circumstances. The results obtained revealed that the algorithm was successfully applied to count the number of people in complex environments with reliable accuracy.

Controlo e aquisição de dados no aparelho Posch ToF-SIMS

Dissertação apresentada na Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa para obtenção do grau de Mestre em Engenharia Física