Designing a Computer Vision System for Underwater Robotic Interventions

This thesis deals with the challenging problem of designing systems able to perceive objects in underwater environments. In the last few decades research activities in robotics have advanced the state of art regarding intervention capabilities of autonomous systems. State of art in fields such as localization and navigation, real time perception and cognition, safe action and manipulation capabilities, applied to ground environments (both indoor and outdoor) has now reached such a readiness level that it allows high level autonomous operations. On the opposite side, the underwater environment remains a very difficult one for autonomous robots. Water influences the mechanical and electrical design of systems, interferes with sensors by limiting their capabilities, heavily impacts on data transmissions, and generally requires systems with low power consumption in order to enable reasonable mission duration. Interest in underwater applications is driven by needs of exploring and intervening in environments in which human capabilities are very limited. Nowadays, most underwater field operations are carried out by manned or remotely operated vehicles, deployed for explorations and limited intervention missions. Manned vehicles, directly on-board controlled, expose human operators to risks related to the stay in field of the mission, within a hostile environment. Remotely Operated Vehicles (ROV) currently represent the most advanced technology for underwater intervention services available on the market. These vehicles can be remotely operated for long time but they need support from an oceanographic vessel with multiple teams of highly specialized pilots. Vehicles equipped with multiple state-of-art sensors and capable to autonomously plan missions have been deployed in the last ten years and exploited as observers for underwater fauna, seabed, ship wrecks, and so on. On the other hand, underwater operations like object recovery and equipment maintenance are still challenging tasks to be conducted without human supervision since they require object perception and localization with much higher accuracy and robustness, to a degree seldom available in Autonomous Underwater Vehicles (AUV). This thesis reports the study, from design to deployment and evaluation, of a general purpose and configurable platform dedicated to stereo-vision perception in underwater environments. Several aspects related to the peculiar environment characteristics have been taken into account during all stages of system design and evaluation: depth of operation and light conditions, together with water turbidity and external weather, heavily impact on perception capabilities. The vision platform proposed in this work is a modular system comprising off-the-shelf components for both the imaging sensors and the computational unit, linked by a high performance ethernet network bus. The adopted design philosophy aims at achieving high flexibility in terms of feasible perception applications, that should not be as limited as in case of a special-purpose and dedicated hardware. Flexibility is required by the variability of underwater environments, with water conditions ranging from clear to turbid, light backscattering varying with daylight and depth, strong color distortion, and other environmental factors. Furthermore, the proposed modular design ensures an easier maintenance and update of the system over time. Performance of the proposed system, in terms of perception capabilities, has been evaluated in several underwater contexts taking advantage of the opportunity offered by the MARIS national project. Design issues like energy power consumption, heat dissipation and network capabilities have been evaluated in different scenarios. Finally, real-world experiments, conducted in multiple and variable underwater contexts, including open sea waters, have led to the collection of several datasets that have been publicly released to the scientific community. The vision system has been integrated in a state of the art AUV equipped with a robotic arm and gripper, and has been exploited in the robot control loop to successfully perform underwater grasping operations.

Precolonial agricultural terracing in Bokoni, South Africa : Typology and an exploratory excavation

Earlier work on the terraced settlements of the Bokoni area (16th to 19th century, Mpumalanga province, South Africa) focussed on the homesteads, their contents, layout and chronology. This paper suggests a terminology and typology of agrarian structures in Bokoni in order to improve comparative approaches in Africa and beyond. The typology and an excavation of the terracing have made possible preliminary conclusions relevant for the further analysis of the terracing and stone-walling in Bokoni. The terracing developed incrementally, whereby cultivation, stone-clearing and terracing were intermixed processes. This is supported both by the organic content in a section of a terrace and by a phytolith analysis. The phytolith analysis furthermore indicated that maize was cultivated on the terraces, but this should be seen as a pilot study only, and presence of maize in Bokoni must be tested with other archaeobotanical methods.

Control of mobile robotic platforms by handheld devices

The intent of this work was to develop a mobile robotic platform that was controlled by a Palm Pilot PDA. Advances in consumer electronics are producing powerful yet small handheld devices. Some of these devices present quasi-PC capabilities for a fraction of the cost; furthermore, they are compact enough that they fit in all but the smallest of platforms. The platform prototype built for testing purposes has a differential-drive configuration to provide simple but agile movement control. The sensor package consisted of two infrared ranging sensors mounted on servomotors that provide a wide area of detection. Building such a platform involved selection of hardware, circuit integration and software development. The software suite selected to develop code for the Palm Pilot was CodeWarrior, a C compiler that can generate code in Palm-native PRC files.

Design and implementation of a relative localization system for ground and aerial robotic teams

The main focus of this thesis is to address the relative localization problem of a heterogenous team which comprises of both ground and micro aerial vehicle robots. This team configuration allows to combine the advantages of increased accessibility and better perspective provided by aerial robots with the higher computational and sensory resources provided by the ground agents, to realize a cooperative multi robotic system suitable for hostile autonomous missions. However, in such a scenario, the strict constraints in flight time, sensor pay load, and computational capability of micro aerial vehicles limits the practical applicability of popular map-based localization schemes for GPS denied navigation. Therefore, the resource limited aerial platforms of this team demand simpler localization means for autonomous navigation. Relative localization is the process of estimating the formation of a robot team using the acquired inter-robot relative measurements. This allows the team members to know their relative formation even without a global localization reference, such as GPS or a map. Thus a typical robot team would benefit from a relative localization service since it would allow the team to implement formation control, collision avoidance, and supervisory control tasks, independent of a global localization service. More importantly, a heterogenous team such as ground robots and computationally constrained aerial vehicles would benefit from a relative localization service since it provides the crucial localization information required for autonomous operation of the weaker agents. This enables less capable robots to assume supportive roles and contribute to the more powerful robots executing the mission. Hence this study proposes a relative localization-based approach for ground and micro aerial vehicle cooperation, and develops inter-robot measurement, filtering, and distributed computing modules, necessary to realize the system. The research study results in three significant contributions. First, the work designs and validates a novel inter-robot relative measurement hardware solution which has accuracy, range, and scalability characteristics, necessary for relative localization. Second, the research work performs an analysis and design of a novel nonlinear filtering method, which allows the implementation of relative localization modules and attitude reference filters on low cost devices with optimal tuning parameters. Third, this work designs and validates a novel distributed relative localization approach, which harnesses the distributed computing capability of the team to minimize communication requirements, achieve consistent estimation, and enable efficient data correspondence within the network. The work validates the complete relative localization-based system through multiple indoor experiments and numerical simulations. The relative localization based navigation concept with its sensing, filtering, and distributed computing methods introduced in this thesis complements system limitations of a ground and micro aerial vehicle team, and also targets hostile environmental conditions. Thus the work constitutes an essential step towards realizing autonomous navigation of heterogenous teams in real world applications.

A ROS-based Workspace Control and Trajectory Planner for a seven degrees of freedom Robotic arm

In this Bachelor Thesis I want to provide readers with tools and scripts for the control of a 7DOF manipulator, backed up by some theory of Robotics and Computer Science, in order to better contextualize the work done. In practice, we will see most common software, and developing environments, used to cope with our task: these include ROS, along with visual simulation by VREP and RVIZ, and an almost "stand-alone" ROS extension called MoveIt!, a very complete programming interface for trajectory planning and obstacle avoidance. As we will better appreciate and understand in the introduction chapter, the capability of detecting collision objects through a camera sensor, and re-plan to the desired end-effector pose, are not enough. In fact, this work is implemented in a more complex system, where recognition of particular objects is needed. Through a package of ROS and customized scripts, a detailed procedure will be provided on how to distinguish a particular object, retrieve its reference frame with respect to a known one, and then allow navigation to that target. Together with technical details, the aim is also to report working scripts and a specific appendix (A) you can refer to, if desiring to put things together.

A systematic review of economic evaluations of the use of robotic assisted laparoscopy in surgery compared with open or laparoscopic surgery

Peer reviewed

A systematic review of economic evaluations of the use of robotic assisted laparoscopy in surgery compared with open or laparoscopic surgery

Peer reviewed

A systematic review of economic evaluations of the use of robotic assisted laparoscopy in surgery compared with open or laparoscopic surgery

Peer reviewed

A smart surgical robotic for cochlear implantation

In this paper a surgical robotic device for cochlear implantation surgery is described that is able to discriminate tissue interfaces and other controlling parameters ahead of a drill tip. The advantage in surgery is that tissues at interfaces can be preserved. The smart tool is able to control interaction with respect to the flexing tissue to avoid penetration control the extent of protrusion with respect to the real-time position of the tissue. To interpret drilling conditions, and conditions leading up to breakthrough at a tissue interface, the sensing scheme used enables discrimination between the variety of conditions posed in the drilling environment. The result is a robust fully autonomous system able to respond to tissue type, behaviour and deflection in real-time. The paper describes the robotic tool that has been designed to be used in the surgical environment where it has been used in the operating room.

(Appendix) Petrographic record from excavation profiles in Old-Luebeck (Liubice), north Germany

Die Frage einer Küstensenkung spielt seit Jahrzehnten bei der geologischen Erforschung des norddeutschen und niederländischen Flachlandes eine große Rolle. Der Schwerpunkt der bisherigen Arbeiten lag im südlichen Nordseeraum. Gerade dieses Gebiet bereitet aber wegen seines Aufbaues aus jungen, teilweise stark setzungsfähigen Gesteinen, des von Ort zu Ort und im Laufe der Zeit unterschiedlichen Einflusses der Gezeiten sowie wegen mancher anderer Bedingungen sehr große methodische Schwierigkeiten. Deshalb überrascht es nicht, daß die einzelnen Untersuchungen noch in den letzten beiden Jahrzehnten für die verschiedenen Abschnitte zu sehr abweichenden Folgerungen kamen, so daß die heute bekannten Einzeltatsachen nur ein recht widerspruchsvolles Gesamtbild ergeben. Da eine Klärung der Probleme durch eine Neubearbeitung an der Nordsee zunächst wenig erfolgversprechend erscheint, wird die Aufmerksamkeit unvermeidlich auf die Verhältnisse im südwestlichen Ostseeraum gelenkt. Hier bestehen in mancher Hinsicht einfachere Bedingungen, die eine derartige Untersuchung begünstigen. Vor allem die Nachbarschaft des skandinavischen Hebungsgebietes am gleichen Wasserbecken, die sehr geringen Gezeitenwirkungen und die hier weit verbreiteten Sandschüttungen bieten bessere Vergleichsmöglichkeiten und schalten viele Fehlerquellen aus. In der voliegenden Arbeit wurden die litorinazeitlichen und die nachlitorinazeitlichen Vertikalbewegungen des Landes und die eustatischen Schwankungen des Wasserstandes im südwestlichen Ostseeraum mit geologisch-morphologischen Methoden und durch die Auswertung von Lage und Verbreitung ur- und frühgeschichtlicher Funde sowie unter Auswertung von pollenanalytisch datierten Bohrprofilen und Pegelaufzeichnungen untersucht.

Robotic Outcome Assessment in a Non-Human Primate Model of Middle Cerebral Artery Stroke

Stroke is a prevalent disorder with immense socioeconomic impact. A variety of chronic neurological deficits result from stroke. In particular, sensorimotor deficits are a significant barrier to achieving post-stroke independence. Unfortunately, the majority of pre-clinical studies that show improved outcomes in animal stroke models have failed in clinical trials. Pre-clinical studies using non-human primate (NHP) stroke models prior to initiating human trials are a potential step to improving translation from animal studies to clinical trials. Robotic assessment tools represent a quantitative, reliable, and reproducible means to assess reaching behaviour following stroke in both humans and NHPs. We investigated the use of robotic technology to assess sensorimotor impairments in NHPs following middle cerebral artery occlusion (MCAO). Two cynomolgus macaques underwent transient MCAO for 90 minutes. Approximately 1.5 years following the procedure these NHPs and two non-stroke control monkeys were trained in a reaching task with both arms in the KINARM exoskeleton. This robot permits elbow and shoulder movements in the horizontal plane. The task required NHPs to make reaching movements from a centrally positioned start target to 1 of 8 peripheral targets uniformly distributed around the first target. We analyzed four movement parameters: reaction time, movement time (MT), initial direction error (IDE), and number of speed maxima to characterize sensorimotor deficiencies. We hypothesized reduced performance in these attributes during a neurobehavioural task with the paretic limb of NHPs following MCAO compared to controls. Reaching movements in the non-affected limbs of control and experimental NHPs showed bell-shaped velocity profiles. In contrast, the reaching movements with the affected limbs were highly variable. We found distinctive patterns in MT, IDE, and number of speed peaks between control and experimental monkeys and between limbs of NHPs with MCAO. NHPs with MCAO demonstrated more speed peaks, longer MTs, and greater IDE in their paretic limb compared to controls. These initial results qualitatively match human stroke subjects’ performance, suggesting that robotic neurobehavioural assessment in NHPs with stroke is feasible and could have translational relevance in subsequent human studies. Further studies will be necessary to replicate and expand on these preliminary findings.