Implementation of a robust coded structured light technique for dynamic 3D measurements

This paper presents the implementation details of a coded structured light system for rapid shape acquisition of unknown surfaces. Such techniques are based on the projection of patterns onto a measuring surface and grabbing images of every projection with a camera. Analyzing the pattern deformations that appear in the images, 3D information of the surface can be calculated. The implemented technique projects a unique pattern so that it can be used to measure moving surfaces. The structure of the pattern is a grid where the color of the slits are selected using a De Bruijn sequence. Moreover, since both axis of the pattern are coded, the cross points of the grid have two codewords (which permits to reconstruct them very precisely), while pixels belonging to horizontal and vertical slits have also a codeword. Different sets of colors are used for horizontal and vertical slits, so the resulting pattern is invariant to rotation. Therefore, the alignment constraint between camera and projector considered by a lot of authors is not necessary

An approach to coded structured light to obtain three dimensional information

The human visual ability to perceive depth looks like a puzzle. We perceive three-dimensional spatial information quickly and efficiently by using the binocular stereopsis of our eyes and, what is mote important the learning of the most common objects which we achieved through living. Nowadays, modelling the behaviour of our brain is a fiction, that is why the huge problem of 3D perception and further, interpretation is split into a sequence of easier problems. A lot of research is involved in robot vision in order to obtain 3D information of the surrounded scene. Most of this research is based on modelling the stereopsis of humans by using two cameras as if they were two eyes. This method is known as stereo vision and has been widely studied in the past and is being studied at present, and a lot of work will be surely done in the future. This fact allows us to affirm that this topic is one of the most interesting ones in computer vision. The stereo vision principle is based on obtaining the three dimensional position of an object point from the position of its projective points in both camera image planes. However, before inferring 3D information, the mathematical models of both cameras have to be known. This step is known as camera calibration and is broadly describes in the thesis. Perhaps the most important problem in stereo vision is the determination of the pair of homologue points in the two images, known as the correspondence problem, and it is also one of the most difficult problems to be solved which is currently investigated by a lot of researchers. The epipolar geometry allows us to reduce the correspondence problem. An approach to the epipolar geometry is describes in the thesis. Nevertheless, it does not solve it at all as a lot of considerations have to be taken into account. As an example we have to consider points without correspondence due to a surface occlusion or simply due to a projection out of the camera scope. The interest of the thesis is focused on structured light which has been considered as one of the most frequently used techniques in order to reduce the problems related lo stereo vision. Structured light is based on the relationship between a projected light pattern its projection and an image sensor. The deformations between the pattern projected into the scene and the one captured by the camera, permits to obtain three dimensional information of the illuminated scene. This technique has been widely used in such applications as: 3D object reconstruction, robot navigation, quality control, and so on. Although the projection of regular patterns solve the problem of points without match, it does not solve the problem of multiple matching, which leads us to use hard computing algorithms in order to search the correct matches. In recent years, another structured light technique has increased in importance. This technique is based on the codification of the light projected on the scene in order to be used as a tool to obtain an unique match. Each token of light is imaged by the camera, we have to read the label (decode the pattern) in order to solve the correspondence problem. The advantages and disadvantages of stereo vision against structured light and a survey on coded structured light are related and discussed. The work carried out in the frame of this thesis has permitted to present a new coded structured light pattern which solves the correspondence problem uniquely and robust. Unique, as each token of light is coded by a different word which removes the problem of multiple matching. Robust, since the pattern has been coded using the position of each token of light with respect to both co-ordinate axis. Algorithms and experimental results are included in the thesis. The reader can see examples 3D measurement of static objects, and the more complicated measurement of moving objects. The technique can be used in both cases as the pattern is coded by a single projection shot. Then it can be used in several applications of robot vision. Our interest is focused on the mathematical study of the camera and pattern projector models. We are also interested in how these models can be obtained by calibration, and how they can be used to obtained three dimensional information from two correspondence points. Furthermore, we have studied structured light and coded structured light, and we have presented a new coded structured light pattern. However, in this thesis we started from the assumption that the correspondence points could be well-segmented from the captured image. Computer vision constitutes a huge problem and a lot of work is being done at all levels of human vision modelling, starting from a)image acquisition; b) further image enhancement, filtering and processing, c) image segmentation which involves thresholding, thinning, contour detection, texture and colour analysis, and so on. The interest of this thesis starts in the next step, usually known as depth perception or 3D measurement.

Catadioptric stereo based on structured light projection

La percepció per visió es millorada quan es pot gaudir d'un camp de visió ampli. Aquesta tesi es concentra en la percepció visual de la profunditat amb l'ajuda de càmeres omnidireccionals. La percepció 3D s'obté generalment en la visió per computadora utilitzant configuracions estèreo amb el desavantatge del cost computacional elevat a l'hora de buscar els elements visuals comuns entre les imatges. La solució que ofereix aquesta tesi és l'ús de la llum estructurada per resoldre el problema de relacionar les correspondències. S'ha realitzat un estudi sobre els sistemes de visió omnidireccional. S'han avaluat vàries configuracions estèreo i s'ha escollit la millor. Els paràmetres del model són difícils de mesurar directament i, en conseqüència, s'ha desenvolupat una sèrie de mètodes de calibració. Els resultats obtinguts són prometedors i demostren que el sensor pot ésser utilitzat en aplicacions per a la percepció de la profunditat com serien el modelatge de l'escena, la inspecció de canonades, navegació de robots, etc.

A multi-wavelet based technique for calculating dense 2D disparity maps from stereo

A vision based approach for calculating accurate 3D models of the objects is presented. Generally industrial visual inspection systems capable of accurate 3D depth estimation rely on extra hardware tools like laser scanners or light pattern projectors. These tools improve the accuracy of depth estimation but also make the vision system costly and cumbersome. In the proposed algorithm, depth and dimensional accuracy of the produced 3D depth model depends on the existing reference model instead of the information from extra hardware tools. The proposed algorithm is a simple and cost effective software based approach to achieve accurate 3D depth estimation with minimal hardware involvement. The matching process uses the well-known coarse to fine strategy, involving the calculation of matching points at the coarsest level with consequent refinement up to the finest level. Vector coefficients of the wavelet transform-modulus are used as matching features, where wavelet transform-modulus maxima defines the shift invariant high-level features with phase pointing to the normal of the feature surface. The technique addresses the estimation of optimal corresponding points and the corresponding 2D disparity maps leading to the creation of accurate depth perception model.

Haptically enabled interactivity and immersive virtual assembly

Virtual training systems are attracting paramount attention from the manufacturing industries due to their potential advantages over the conventional training practices such as general assembly. Within this virtual training realm for general assembly, a haptically enabled interactive and immersive virtual reality (HIVEx) system is presented. The idea is to imitate real assembly training scenarios by providing comprehensive user interaction as well as by enforcing physical constraints within the virtual environment through the use of haptics technology. The developed system employs a modular system approach providing flexibility of reconfiguration and scalability as well as better utilization of the current multi-core computer architecture. The user interacts with the system using haptics device and data glove while fully immersed into the virtual environment with depth perception. An evaluation module, incorporated into the system, automatically logs and evaluates the information through the simulation providing user performance and improvements over time. A ruggedized portable version of the system is also developed and presented with full system capabilities allowing easy relocation with different factory environments. A number of training scenarios has been developed with varying degree of complexity to exploit the potential of the presented system. The presented system can be employed for teaching and training of existing assembly processes as well as the design of new optimised assembly operations. Furthermore, the presented system can assist in optimizing existing practices by evaluating the effectiveness and the level of knowledge transfer involved in the process. Within the aforementioned conceptual. framework, a working prototype is developed.

Emerging technology for remote IED render safe using the OzBot

The Centre for Intelligent Systems Research (CISR) based within Deakin University’s Geelong campus has been developing technology specifically for remote render-safe of IED since being awarded a CTD contact in 2006. During this time, research engineers have worked with key defence and industry stakeholders to develop a series of robotic platforms tasked with immersing a soldier in his or her remote environment. Utilising Haptics (force feedback technology), stereovision (binocular video stream for depth perception) and intuitive user controls, the robots have been engineered to deliver maximum effectiveness while allowing minimal training liability. In Victoria, CISR’s OzBot series of mobile platforms have been used by the Victorian Police in a first-responder capacity, exploiting the 30-sec system boot-up and man-portable design to get eyes-on-target at the soonest possible moment. The CISR robotics group has been working on technologies that reduce operator fatigue, minimise training liability and maintenance, developing simulation technologies for increased training availability and develop mobile platforms with increased range, payload, manipulator reach and capability. This paper describes some of the technologies, methods and systems developed by CISR in the field of IED neutralisation with the aim of increasing military awareness of Australian capability.

Seeking one's heading through eye movements

A study of eye movements during simulated travel toward a grove of four stationary trees revealed that observers looked most at pairs of trees that converged or decelerated apart. Such pairs specify that one's direction of travel, called heading, is to the outside of the near member of the pair. Observers looked at these trees more than those that accelerated apart; such pairs do not offer trustworthy heading information. Observers also looked at gaps between trees less often when they converged or diverged apart, and heading can never be between such pairs. Heading responses were in accord with eye movements. In general, if observers responded accurately, they had looked at trees that converged or decelerated apart; if they were inaccurate, they had not. Results support the notion that observers seek out their heading through eye movements, saccading to and fixating on the most informative locations in the field of view.

Early impairment of cognitive functions in Parkinson's disease

Background: Impairment in non-motor functions such as disturbances of some executive functions are also common events in Parkinson's disease patients. Objective: To verify the performance of Parkinson's disease patients in activities requiring visuoconstructive and visuospatial skills. Method: Thirty elderly patients with mild or moderate stages of Parkinson's disease were studied. The assessment of the clinical condition was based on the unified Parkinson's disease rating scale (56.28; SD=33.48), Hoehn and Yahr (2.2; SD=0.83), Schwab and England (78.93%), clock drawing test (7.36; SD=2.51), and mini-mental state examination (26.48; SD=10.11). Pearson's correlation and stepwise multiple regression were used for statistical analyses. Results: The patients presented deterioration in visuospatial and visuoconstructive skills. Conclusion: The clock drawing test proved to be a useful predictive tool for identifying early cognitive impairment in these individuals.

Treino prévio reduz o tempo de execução de tarefas visuo-espaciais em ambiente virtual

Objective. To evaluate the influence of previous adaptation to different computational environments in visuo-spacial tasks performance of healthy individuals. Method. Healthy volunteers (n = 30), 15 male, mean age 25.3 ± 3.3 years, were divided in three groups: the first group, considered control, was not adapted to the proposed environments; the second group was adapted to a closed environment (stable and expected), and the third group was adapted to an open environment A (unexpected). The proposed task was to go through two open environments B and C (maze). The dependent variables Time and Error were considered for the analysis. Results. It was observed that during the adaptation phase, in the Time variable, the groups presented a progressive improvement in the performance to each task (p = 0.0036). The group adapted in the A open environment, showed a tendency to be faster in the execution of B and C open environments tasks, than the group adapted in the closed environment (p = 0.068). Conclusion. The study suggests that subjects adapted to visuo-spacial tasks execution involving unknown and no guided situations, present a tendency to a better time performance in these tasks when compared to subjects adapted in fixed and guided situations.

Surgical Performance During Laparoscopic Radical Nephrectomy Is Improved With Training in a Porcine Model

Background and Purpose: Becoming proficient in laparoscopic surgery is dependent on the acquisition of specialized skills that can only be obtained from specific training. This training could be achieved in various ways using inanimate models, animal models, or live patient surgery-each with its own pros and cons. Currently, there are substantial data that support the benefits of animal model training in the initial learning of laparoscopy. Nevertheless, whether these benefits extent themselves to moderately experienced surgeons is uncertain. The purpose of this study was to determine if training using a porcine model results in a quantifiable gain in laparoscopic skills for moderately experienced laparoscopic surgeons. Materials and Methods: Six urologists with some laparoscopic experience were asked to perform a radical nephrectomy weekly for 10 weeks in a porcine model. The procedures were recorded, and surgical performance was assessed by two experienced laparoscopic surgeons using a previously published surgical performance assessment tool. The obtained data were then submitted to statistical analysis. Results: With training, blood loss was reduced approximately 45% when comparing the averages of the first and last surgical procedures (P = 0.006). Depth perception showed an improvement close to 35% (P = 0.041), and dexterity showed an improvement close to 25% (P = 0.011). Total operative time showed trends of improvement, although it was not significant (P = 0.158). Autonomy, efficiency, and tissue handling were the only aspects that did not show any noteworthy change (P = 0.202, P = 0.677, and P = 0.456, respectively). Conclusions: These findings suggest that there are quantifiable gains in laparoscopic skills obtained from training in an animal model. Our results suggest that these benefits also extend to more advanced stages of the learning curve, but it is unclear how far along the learning curve training with animal models provides a clear benefit for the performance of laparoscopic procedures. Future studies are necessary to confirm these findings and better understand the impact of this learning tool on surgical practice.

Verhaltensphysiologische Analyse der visuellen Wahrnehmung räumlicher Tiefe beim Goldfisch

Die Frage, wie es zur visuellen Wahrnehmung räumlicher Tiefe kommt, wenn das Retinabild nur zweidimensional ist, gehört zu den grundlegenden Proble-men der Hirnforschung. Für Tiere, die sich aktiv in ihrer Umgebung bewegen, herrscht ein großer Selektionsdruck Entfernungen und Größen richtig einzu-schätzen. Ziel der vorliegenden Arbeit war es, herauszufinden, ob und wie gut Goldfische Objekte allein aufgrund des Abstandes unterscheiden können und woraus sie Information über den Abstand gewinnen. Hierzu wurde ein Ver-suchsaufbau mit homogen weißem Hintergrund entworfen, in dem die Akkom-modation als Entfernungsinformationen verwendet werden kann, weniger je-doch die Bewegungsparallaxe. Die Goldfische lernten durch operante Konditio-nierung einen Stimulus (schwarze Kreisscheibe) in einem bestimmten Abstand zu wählen, während ein anderer, gleichgroßer Stimulus so entfernt wie möglich präsentiert wurde. Der Abstand zwischen den Stimuli wurde dann verringert, bis die Goldfische keine sichere Wahl für den Dressurstimulus mehr treffen konnten. Die Unterscheidungsleistung der Goldfische wurde mit zunehmendem Abstand des Dressurstimulus immer geringer. Eine Wiederholung der Versuche mit unscharfen Stimu¬lus¬kon¬turen brachte keine Verschlechterung in der Unter-scheidung, was Akkommodation wenig wahrscheinlich macht. Um die Größen-konstanz beim Goldfisch zu testen, wurden die Durchmesser der unterschiedlich entfernten Stimuli so angepasst, dass sie für den Goldfisch die gleiche Retina-bildgröße hatten. Unter diesen Bedingungen waren die Goldfische nicht in der Lage verschieden entfernte Stimuli zu unterscheiden und somit Größenkonstanz zu leisten. Es fand demnach keine echte Entfernungsbestimmung oder Tiefen-wahrneh¬mung statt. Die Unterscheidung der verschieden entfernten Stimuli erfolgte allein durch deren Abbildungsgröße auf der Retina. Dass die Goldfische bei diesem Experiment nicht akkommodieren, wurde durch Infrarot-Photoretinoskopie gezeigt. Somit lässt sich Akkommodation für die Entfer-nungsbestimmung in diesen Versuchen ausschließen. Für diese Leistung und die Größenkonstanz ist vermutlich die Bewegungsparallaxe entscheidend.

A new 3D modelling paradigm for discrete model

Until few years ago, 3D modelling was a topic confined into a professional environment. Nowadays technological innovations, the 3D printer among all, have attracted novice users to this application field. This sudden breakthrough was not supported by adequate software solutions. The 3D editing tools currently available do not assist the non-expert user during the various stages of generation, interaction and manipulation of 3D virtual models. This is mainly due to the current paradigm that is largely supported by two-dimensional input/output devices and strongly affected by obvious geometrical constraints. We have identified three main phases that characterize the creation and management of 3D virtual models. We investigated these directions evaluating and simplifying the classic editing techniques in order to propose more natural and intuitive tools in a pure 3D modelling environment. In particular, we focused on freehand sketch-based modelling to create 3D virtual models, interaction and navigation in a 3D modelling environment and advanced editing tools for free-form deformation and objects composition. To pursuing these goals we wondered how new gesture-based interaction technologies can be successfully employed in a 3D modelling environments, how we could improve the depth perception and the interaction in 3D environments and which operations could be developed to simplify the classical virtual models editing paradigm. Our main aims were to propose a set of solutions with which a common user can realize an idea in a 3D virtual model, drawing in the air just as he would on paper. Moreover, we tried to use gestures and mid-air movements to explore and interact in 3D virtual environment, and we studied simple and effective 3D form transformations. The work was carried out adopting the discrete representation of the models, thanks to its intuitiveness, but especially because it is full of open challenges.

Assessment of US Air Force student pilots with intermittent monofixation syndrome on a non-stereoptic dependent flight maneuver in pilot training

The United States Air Force School of Aerospace Medicine (USAFSAM) and Aeromedical Consult Service (ACS) have developed waiver criteria for pilots with subtle substandard depth perception. This is to allow United States Air Force (USAF) pilots with mild depth perception deficiency to continue flying duties while limiting the risk to flight safety and ensuring the availability of costly human resources. From 1999 to 2005, 166 aviators were given waivers for intermittent monofixation syndrome (IMFS). Of these, 96 were student pilots who performed slightly worse at stereoptic dependent flight maneuvers than student pilots (8,907) with normal depth perception (Lowry, 2006).^ This study's purpose is to evaluate the performance of the extended-trail maneuver, a non-stereoptic dependent flying maneuver, as executed by a cohort of 12 United States Air Force student pilots with intermittent monofixation syndrome versus the cohort of 100 student pilots with normal depth perception. These subjects are extracted from the cohorts examined by Lowry (2006) and the null hypothesis predicts no statistical difference in the performance of the non-stereoptic dependant flight maneuver extended-trail between student pilots with intermittent monofixation syndrome and those without the condition. ^

Analysis of quality of experience in 3D video systems

Esta tesis presenta un estudio exhaustivo sobre la evaluación de la calidad de experiencia (QoE, del inglés Quality of Experience) percibida por los usuarios de sistemas de vídeo 3D, analizando el impacto de los efectos introducidos por todos los elementos de la cadena de procesamiento de vídeo 3D. Por lo tanto, se presentan varias pruebas de evaluación subjetiva específicamente diseñadas para evaluar los sistemas considerados, teniendo en cuenta todos los factores perceptuales relacionados con la experiencia visual tridimensional, tales como la percepción de profundidad y la molestia visual. Concretamente, se describe un test subjetivo basado en la evaluación de degradaciones típicas que pueden aparecer en el proceso de creación de contenidos de vídeo 3D, por ejemplo debidas a calibraciones incorrectas de las cámaras o a algoritmos de procesamiento de la señal de vídeo (p. ej., conversión de 2D a 3D). Además, se presenta el proceso de generación de una base de datos de vídeos estereoscópicos de alta calidad, disponible gratuitamente para la comunidad investigadora y que ha sido utilizada ampliamente en diferentes trabajos relacionados con vídeo 3D. Asimismo, se presenta otro estudio subjetivo, realizado entre varios laboratorios, con el que se analiza el impacto de degradaciones causadas por la codificación de vídeo, así como diversos formatos de representación de vídeo 3D. Igualmente, se describen tres pruebas subjetivas centradas en el estudio de posibles efectos causados por la transmisión de vídeo 3D a través de redes de televisión sobre IP (IPTV, del inglés Internet Protocol Television) y de sistemas de streaming adaptativo de vídeo. Para estos casos, se ha propuesto una innovadora metodología de evaluación subjetiva de calidad vídeo, denominada Content-Immersive Evaluation of Transmission Impairments (CIETI), diseñada específicamente para evaluar eventos de transmisión simulando condiciones realistas de visualización de vídeo en ámbitos domésticos, con el fin de obtener conclusiones más representativas sobre la experiencia visual de los usuarios finales. Finalmente, se exponen dos experimentos subjetivos comparando varias tecnologías actuales de televisores 3D disponibles en el mercado de consumo y evaluando factores perceptuales de sistemas Super Multiview Video (SMV), previstos a ser la tecnología futura de televisores 3D de consumo, gracias a una prometedora visualización de contenido 3D sin necesidad de gafas específicas. El trabajo presentado en esta tesis ha permitido entender los factores perceptuales y técnicos relacionados con el procesamiento y visualización de contenidos de vídeo 3D, que pueden ser de utilidad en el desarrollo de nuevas tecnologías y técnicas de evaluación de la QoE, tanto metodologías subjetivas como métricas objetivas. ABSTRACT This thesis presents a comprehensive study of the evaluation of the Quality of Experience (QoE) perceived by the users of 3D video systems, analyzing the impact of effects introduced by all the elements of the 3D video processing chain. Therefore, various subjective assessment tests are presented, particularly designed to evaluate the systems under consideration, and taking into account all the perceptual factors related to the 3D visual experience, such as depth perception and visual discomfort. In particular, a subjective test is presented, based on evaluating typical degradations that may appear during the content creation, for instance due to incorrect camera calibration or video processing algorithms (e.g., 2D to 3D conversion). Moreover, the process of generation of a high-quality dataset of 3D stereoscopic videos is described, which is freely available for the research community, and has been already widely used in different works related with 3D video. In addition, another inter-laboratory subjective study is presented analyzing the impact of coding impairments and representation formats of stereoscopic video. Also, three subjective tests are presented studying the effects of transmission events that take place in Internet Protocol Television (IPTV) networks and adaptive streaming scenarios for 3D video. For these cases, a novel subjective evaluation methodology, called Content-Immersive Evaluation of Transmission Impairments (CIETI), was proposed, which was especially designed to evaluate transmission events simulating realistic home-viewing conditions, to obtain more representative conclusions about the visual experience of the end users. Finally, two subjective experiments are exposed comparing various current 3D displays available in the consumer market, and evaluating perceptual factors of Super Multiview Video (SMV) systems, expected to be the future technology for consumer 3D displays thanks to a promising visualization of 3D content without specific glasses. The work presented in this thesis has allowed to understand perceptual and technical factors related to the processing and visualization of 3D video content, which may be useful in the development of new technologies and approaches for QoE evaluation, both subjective methodologies and objective metrics.