Model and control of tendon actuated robots

The use of tendons for the transmission of the forces and the movements in robotic devices has been investigated from several researchers all over the world. The interest in this kind of actuation modality is based on the possibility of optimizing the position of the actuators with respect to the moving part of the robot, in the reduced weight, high reliability, simplicity in the mechanic design and, finally, in the reduced cost of the resulting kinematic chain. After a brief discussion about the benefits that the use of tendons can introduce in the motion control of a robotic device, the design and control aspects of the UB Hand 3 anthropomorphic robotic hand are presented. In particular, the tendon-sheaths transmission system adopted in the UB Hand 3 is analyzed and the problem of force control and friction compensation is taken into account. The implementation of a tendon based antagonistic actuated robotic arm is then investigated. With this kind of actuation modality, and by using transmission elements with nonlinear force/compression characteristic, it is possible to achieve simultaneous stiffness and position control, improving in this way the safety of the device during the operation in unknown environments and in the case of interaction with other robots or with humans. The problem of modeling and control of this type of robotic devices is then considered and the stability analysis of proposed controller is reported. At the end, some tools for the realtime simulation of dynamic systems are presented. This realtime simulation environment has been developed with the aim of improving the reliability of the realtime control applications both for rapid prototyping of controllers and as teaching tools for the automatic control courses.

Safe and Collaborative Navigation & Interaction With Mobile Manipulators in Domestic Appliance Test Labs

Safe collaboration between a robot and human operator forms a critical requirement for deploying a robotic system into a manufacturing and testing environment. In this dissertation, the safety requirement for is developed and implemented for the navigation system of the mobile manipulators. A methodology for human-robot co-existence through a 3d scene analysis is also investigated. The proposed approach exploits the advance in computing capability by relying on graphic processing units (GPU’s) for volumetric predictive human-robot contact checking. Apart from guaranteeing safety of operators, human-robot collaboration is also fundamental when cooperative activities are required, as in appliance test automation floor. To achieve this, a generalized hierarchical task controller scheme for collision avoidance is developed. This allows the robotic arm to safely approach and inspect the interior of the appliance without collision during the testing procedure. The unpredictable presence of the operators also forms dynamic obstacle that changes very fast, thereby requiring a quick reaction from the robot side. In this aspect, a GPU-accelarated distance field is computed to speed up reaction time to avoid collision between human operator and the robot. An automated appliance testing also involves robotized laundry loading and unloading during life cycle testing. This task involves Laundry detection, grasp pose estimation and manipulation in a container, inside the drum and during recovery grasping. A wrinkle and blob detection algorithms for grasp pose estimation are developed and grasp poses are calculated along the wrinkle and blobs to efficiently perform grasping task. By ranking the estimated laundry grasp poses according to a predefined cost function, the robotic arm attempt to grasp poses that are more comfortable from the robot kinematic side as well as collision free on the appliance side. This is achieved through appliance detection and full-model registration and collision free trajectory execution using online collision avoidance.

Mars delivery service - development of the electro-mechanical systems of the Sample Fetch Rover for the Mars Sample Return Campaign

This thesis describes the development of the Sample Fetch Rover (SFR), studied for Mars Sample Return (MSR), an international campaign carried out in cooperation between the National Aeronautics and Space Administration (NASA) and the European Space Agency (ESA). The focus of this document is the design of the electro-mechanical systems of the rover. After placing this work into the general context of robotic planetary exploration and summarising the state of the art for what concerns Mars rovers, the architecture of the Mars Sample Return Campaign is presented. A complete overview of the current SFR architecture is provided, touching upon all the main subsystems of the spacecraft. For each area, it is discussed what are the design drivers, the chosen solutions and whether they use heritage technology (in particular from the ExoMars Rover) or new developments. This research focuses on two topics of particular interest, due to their relevance for the mission and the novelty of their design: locomotion and sample acquisition, which are discussed in depth. The early SFR locomotion concepts are summarised, covering the initial trade-offs and discarded designs for higher traverse performance. Once a consolidated architecture was reached, the locomotion subsystem was developed further, defining the details of the suspension, actuators, deployment mechanisms and wheels. This technology is presented here in detail, including some key analysis and test results that support the design and demonstrate how it responds to the mission requirements. Another major electro-mechanical system developed as part of this work is the one dedicated to sample tube acquisition. The concept of operations of this machinery was defined to be robust against the unknown conditions that characterise the mission. The design process led to a highly automated robotic system which is described here in its main components: vision system, robotic arm and tube storage.

Design and Control of Robotic Hands

The application of dexterous robotic hands out of research laboratories has been limited by the intrinsic complexity that these devices present. This is directly reflected as an economically unreasonable cost and a low overall reliability. Within the research reported in this thesis it is shown how the problem of complexity in the design of robotic hands can be tackled, taking advantage of modern technologies (i.e. rapid prototyping), leading to innovative concepts for the design of the mechanical structure, the actuation and sensory systems. The solutions adopted drastically reduce the prototyping and production costs and increase the reliability, reducing the number of parts required and averaging their single reliability factors. In order to get guidelines for the design process, the problem of robotic grasp and manipulation by a dual arm/hand system has been reviewed. In this way, the requirements that should be fulfilled at hardware level to guarantee successful execution of the task has been highlighted. The contribution of this research from the manipulation planning side focuses on the redundancy resolution that arise in the execution of the task in a dexterous arm/hand system. In literature the problem of coordination of arm and hand during manipulation of an object has been widely analyzed in theory but often experimentally demonstrated in simplified robotic setup. Our aim is to cover the lack in the study of this topic and experimentally evaluate it in a complex system as a anthropomorphic arm hand system.

Coordinated Control of Robotic Swarms in Unknown Environments

This thesis gathers the work carried out by the author in the last three years of research and it concerns the study and implementation of algorithms to coordinate and control a swarm of mobile robots moving in unknown environments. In particular, the author's attention is focused on two different approaches in order to solve two different problems. The first algorithm considered in this work deals with the possibility of decomposing a main complex task in many simple subtasks by exploiting the decentralized implementation of the so called \emph{Null Space Behavioral} paradigm. This approach to the problem of merging different subtasks with assigned priority is slightly modified in order to handle critical situations that can be detected when robots are moving through an unknown environment. In fact, issues can occur when one or more robots got stuck in local minima: a smart strategy to avoid deadlock situations is provided by the author and the algorithm is validated by simulative analysis. The second problem deals with the use of concepts borrowed from \emph{graph theory} to control a group differential wheel robots by exploiting the Laplacian solution of the consensus problem. Constraints on the swarm communication topology have been introduced by the use of a range and bearing platform developed at the Distributed Intelligent Systems and Algorithms Laboratory (DISAL), EPFL (Lausanne, CH) where part of author's work has been carried out. The control algorithm is validated by demonstration and simulation analysis and, later, is performed by a team of four robots engaged in a formation mission. To conclude, the capabilities of the algorithm based on the local solution of the consensus problem for differential wheel robots are demonstrated with an application scenario, where nine robots are engaged in a hunting task.

Scene Mapping and Understanding by Robotic Vision

The first mechanical Automaton concept was found in a Chinese text written in the 3rd century BC, while Computer Vision was born in the late 1960s. Therefore, visual perception applied to machines (i.e. the Machine Vision) is a young and exciting alliance. When robots came in, the new field of Robotic Vision was born, and these terms began to be erroneously interchanged. In short, we can say that Machine Vision is an engineering domain, which concern the industrial use of Vision. The Robotic Vision, instead, is a research field that tries to incorporate robotics aspects in computer vision algorithms. Visual Servoing, for example, is one of the problems that cannot be solved by computer vision only. Accordingly, a large part of this work deals with boosting popular Computer Vision techniques by exploiting robotics: e.g. the use of kinematics to localize a vision sensor, mounted as the robot end-effector. The remainder of this work is dedicated to the counterparty, i.e. the use of computer vision to solve real robotic problems like grasping objects or navigate avoiding obstacles. Will be presented a brief survey about mapping data structures most widely used in robotics along with SkiMap, a novel sparse data structure created both for robotic mapping and as a general purpose 3D spatial index. Thus, several approaches to implement Object Detection and Manipulation, by exploiting the aforementioned mapping strategies, will be proposed, along with a completely new Machine Teaching facility in order to simply the training procedure of modern Deep Learning networks.

Robotic Approach in the Treatment of Locally Advanced (Stage IIIA-pN2) non Small Cell Lung Cancer after Induction Therapy

Introduction: Despite there are already many studies on robotic surgery as minimally invasive approach for non-small cell lung cancer (NSCLC) patients, the use of this technique for stage III disease is still poorly described. These are the preliminary results of our prospective study on safety and effectiveness of robotic approach in patients with locally advanced NSCLC, in terms of postoperative complications and oncological outcome. Methods: Since 2016, we prospectively investigated, using standardized questionnaire and protocol, 21 consecutive patients with NSCLC stage IIIA-pN2 (diagnosed by EBUS-TBNA) who underwent lobectomy and radical lymph node dissection with robotic approach after induction treatment. Then, we performed a matched case-control study with 54 patients treated with open surgery during the same period of time, with similar age, clinical and pathological tumor stage. Results: The individual matched population was composed of 14 robot-assisted thoracic surgery and 14 patients who underwent open surgery. The median time range of resection was inferior in the open group compared to robotic lobectomy (148 vs 229 minutes; P=0.002). Lymph nodes resection and positivity were not statistically significantly different (p=0.66 and p=0.73 respectively). No difference was observed also for PFS (P=0.99) or OS (P=0.94). Conclusions: Our preliminary results demonstrated that the early outcomes and oncological results of N2-patients after robotic lobectomy were similar to open surgery. Considering the advantages of minimally invasive surgery, robotic assisted lobectomy should be a safe approach also to patients with local advanced disease.

Design of experiment in drug development: optimal allocations in multi-arm clinical trials and design space for multi-step processes

In this thesis, we deal with the design of experiments in the drug development process, focusing on the design of clinical trials for treatment comparisons (Part I) and the design of preclinical laboratory experiments for proteins development and manufacturing (Part II). In Part I we propose a multi-purpose design methodology for sequential clinical trials. We derived optimal allocations of patients to treatments for testing the efficacy of several experimental groups by also taking into account ethical considerations. We first consider exponential responses for survival trials and we then present a unified framework for heteroscedastic experimental groups that encompasses the general ANOVA set-up. The very good performance of the suggested optimal allocations, in terms of both inferential and ethical characteristics, are illustrated analytically and through several numerical examples, also performing comparisons with other designs proposed in the literature. Part II concerns the planning of experiments for processes composed of multiple steps in the context of preclinical drug development and manufacturing. Following the Quality by Design paradigm, the objective of the multi-step design strategy is the definition of the manufacturing design space of the whole process and, as we consider the interactions among the subsequent steps, our proposal ensures the quality and the safety of the final product, by enabling more flexibility and process robustness in the manufacturing.

Robotic Sensing and Manipulation of Deformable Linear Objects with Learning-based methods

Nowadays robotic applications are widespread and most of the manipulation tasks are efficiently solved. However, Deformable-Objects (DOs) still represent a huge limitation for robots. The main difficulty in DOs manipulation is dealing with the shape and dynamics uncertainties, which prevents the use of model-based approaches (since they are excessively computationally complex) and makes sensory data difficult to interpret. This thesis reports the research activities aimed to address some applications in robotic manipulation and sensing of Deformable-Linear-Objects (DLOs), with particular focus to electric wires. In all the works, a significant effort was made in the study of an effective strategy for analyzing sensory signals with various machine learning algorithms. In the former part of the document, the main focus concerns the wire terminals, i.e. detection, grasping, and insertion. First, a pipeline that integrates vision and tactile sensing is developed, then further improvements are proposed for each module. A novel procedure is proposed to gather and label massive amounts of training images for object detection with minimal human intervention. Together with this strategy, we extend a generic object detector based on Convolutional-Neural-Networks for orientation prediction. The insertion task is also extended by developing a closed-loop control capable to guide the insertion of a longer and curved segment of wire through a hole, where the contact forces are estimated by means of a Recurrent-Neural-Network. In the latter part of the thesis, the interest shifts to the DLO shape. Robotic reshaping of a DLO is addressed by means of a sequence of pick-and-place primitives, while a decision making process driven by visual data learns the optimal grasping locations exploiting Deep Q-learning and finds the best releasing point. The success of the solution leverages on a reliable interpretation of the DLO shape. For this reason, further developments are made on the visual segmentation.

Neural states in parietal areas during arm reaching

Since the first subdivisions of the brain into macro regions, it has always been thought a priori that, given the heterogeneity of neurons, different areas host specific functions and process unique information in order to generate a behaviour. Moreover, the various sensory inputs coming from different sources (eye, skin, proprioception) flow from one macro area to another, being constantly computed and updated. Therefore, especially for non-contiguous cortical areas, it is not expected to find the same information. From this point of view, it would be inconceivable that the motor and the parietal cortices, diversified by the information encoded and by the anatomical position in the brain, could show very similar neural dynamics. With the present thesis, by analyzing the population activity of parietal areas V6A and PEc with machine learning methods, we argue that a simplified view of the brain organization do not reflect the actual neural processes. We reliably detected a number of neural states that were tightly linked to distinct periods of the task sequence, i.e. the planning and execution of movement and the holding of target as already observed in motor cortices. The states before and after the movement could be further segmented into two states related to different stages of movement planning and arm posture processing. Rather unexpectedly, we found that activity during the movement could be parsed into two states of equal duration temporally linked to the acceleration and deceleration phases of the arm. Our findings suggest that, at least during arm reaching in 3D space, the posterior parietal cortex (PPC) shows low-level population neural dynamics remarkably similar to those found in the motor cortices. In addition, the present findings suggest that computational processes in PPC could be better understood if studied using a dynamical system approach rather than studying a mosaic of single units.

A phase II, single arm study of CarbopLatin plus Etoposide with Bevacizumab and Atezolizumab in patients with exTEnded-disease small-cell lung cancer (SCLC) – CeLEBrATE trial

Small cell lung cancer (SCLC) is an aggressive neuroendocrine tumor diagnosed at extended disease SCLC (ES-SCLC) stage in about 70% of cases. The new standard of treatment for patients with ES-SCLC is a combination of platinum-etoposide chemotherapy and atezolizumab or durvalumab, two programmed cell death ligand 1 (PD-L1) inhibitory monoclonal antibodies (mAb). However, the benefit derived from the addition of PD-L1 inhibitors to chemotherapy in ES-SCLC was limited and restricted to a subset of patients. The vascular endothelial growth factor (VEGF) is the most important pro-angiogenic factor implicated in cancer angiogenesis, which is abundant in SCLC and associated with poor prognosis. Antiangiogenic agents, such as bevacizumab, a humanized mAb against VEGF, added to platinum-etoposide chemotherapy improved progression-free survival in SCLC in two trials, but it did not translate into a benefit in overall survival. Nevertheless, VEGF has also acts as a mediator of an immunosuppressive microenvironment and its inhibition can revert the immune-suppressive tumor microenvironment and potentially enhance the efficacy of immunotherapies. Based on available preclinical data, we hypothesized that VEGF inhibition by bevacizumab could improve atezolizumab efficacy in a synergistic way and designed a phase II single-arm trial of bevacizumab in combination with carboplatin, etoposide, and atezolizumab as first-line treatment in ES-SCLC. The trial, which is still ongoing, enrolled 53 patients, including those with treated or untreated asymptomatic brain metastases (provided criteria are met), who received atezolizumab, bevacizumab, carboplatin and etoposide for 4-6 cycles (induction phase), followed by maintenance with atezolizumab and bevacizumab for a maximum of 18 total cycles or until disease progression, patient refusal, unacceptable toxicity. The evaluation of efficacy of the experimental combination in terms of 1-year overall survival rate is not yet mature (primary objective of the trial). The combination was feasible and the toxicity profile manageable (secondary objective of the trial).

Relative influence of gaze and static arm position in the Superior Parietal Lobule of the macaque monkey during reaching

The superior parietal lobule (SPL) of macaques is classically described as an associative cortex implicated in visuospatial perception, planning and control of reaching and grasping movements (De Vitis et al., 2019; Galletti et al., 2003, 2018, 2022; Fattori et al., 2017; Hadjidimitrakis et al., 2015). These processes are the result of the integration of signals related to different sensory modalities. During a goal-directed action, eye and limb information are combined to ensure that the hand is transported at the gazed target location and the arm is maintained steady in the final position. The SPL areas V6A, PEc and PE contain cells sensitive to the direction of gaze and limb position but less is known about the degree of independent encoding of these signals. In this thesis, we evaluated the influence of eye and arm position information upon single neuron activity of areas V6A, PEc and PE during the holding period after the execution of arm reaching movement, when the gaze and hand are both still at the reach target. Two male macaques (Macaca fascicularis) performed a reaching task while single unit activity was recorded from areas V6A, PEc and PE. We found that neurons in all these areas were modulated by eye and static arm positions with a joint encoding of gaze and somatosensory signals in V6A and PEc and a mostly separate processing of the two signals in PE. The elaboration of this information reflects the functional gradient found in the SPL with the caudal sector characterized by visuo-somatic properties in comparison to the rostral sector dominated by somatosensory signals. This evidence well agree also with the recent reallocation of areas V6A and PEc in Brodmann’s area 7 depending on their similar structural and functional features with respect to PE belonging to Brodmann’s area 5 (Gamberini et al., 2020).

A robotic platform for precision agriculture and applications

Agricultural techniques have been improved over the centuries to match with the growing demand of an increase in global population. Farming applications are facing new challenges to satisfy global needs and the recent technology advancements in terms of robotic platforms can be exploited. As the orchard management is one of the most challenging applications because of its tree structure and the required interaction with the environment, it was targeted also by the University of Bologna research group to provide a customized solution addressing new concept for agricultural vehicles. The result of this research has blossomed into a new lightweight tracked vehicle capable of performing autonomous navigation both in the open-filed scenario and while travelling inside orchards for what has been called in-row navigation. The mechanical design concept, together with customized software implementation has been detailed to highlight the strengths of the platform and some further improvements envisioned to improve the overall performances. Static stability testing has proved that the vehicle can withstand steep slopes scenarios. Some improvements have also been investigated to refine the estimation of the slippage that occurs during turning maneuvers and that is typical of skid-steering tracked vehicles. The software architecture has been implemented using the Robot Operating System (ROS) framework, so to exploit community available packages related to common and basic functions, such as sensor interfaces, while allowing dedicated custom implementation of the navigation algorithm developed. Real-world testing inside the university’s experimental orchards have proven the robustness and stability of the solution with more than 800 hours of fieldwork. The vehicle has also enabled a wide range of autonomous tasks such as spraying, mowing, and on-the-field data collection capabilities. The latter can be exploited to automatically estimate relevant orchard properties such as fruit counting and sizing, canopy properties estimation, and autonomous fruit harvesting with post-harvesting estimations.

On the motion planning & control of nonlinear robotic systems

In the last decades, we saw a soaring interest in autonomous robots boosted not only by academia and industry, but also by the ever in- creasing demand from civil users. As a matter of fact, autonomous robots are fast spreading in all aspects of human life, we can see them clean houses, navigate through city traffic, or harvest fruits and vegetables. Almost all commercial drones already exhibit unprecedented and sophisticated skills which makes them suitable for these applications, such as obstacle avoidance, simultaneous localisation and mapping, path planning, visual-inertial odometry, and object tracking. The major limitations of such robotic platforms lie in the limited payload that can carry, in their costs, and in the limited autonomy due to finite battery capability. For this reason researchers start to develop new algorithms able to run even on resource constrained platforms both in terms of computation capabilities and limited types of endowed sensors, focusing especially on very cheap sensors and hardware. The possibility to use a limited number of sensors allowed to scale a lot the UAVs size, while the implementation of new efficient algorithms, performing the same task in lower time, allows for lower autonomy. However, the developed robots are not mature enough to completely operate autonomously without human supervision due to still too big dimensions (especially for aerial vehicles), which make these platforms unsafe for humans, and the high probability of numerical, and decision, errors that robots may make. In this perspective, this thesis aims to review and improve the current state-of-the-art solutions for autonomous navigation from a purely practical point of view. In particular, we deeply focused on the problems of robot control, trajectory planning, environments exploration, and obstacle avoidance.