Optimal Nonlinear Control and Estimation for a Reusable Launch Vehicle During Reentry Phase

In this paper a nonlinear optimal controller has been designed for aerodynamic control during the reentry phase of the Reusable Launch Vehicle (RLV). The controller has been designed based on a recently developed technique Optimal Dynamic Inversion (ODI). For full state feedback the controller has required full information about the system states. In this work an Extended Kalman filter (EKF) is developed to estimate the states. The vehicle (RLV) has been has been consider as a nonlinear Six-Degree-Of-Freedom (6-DOF) model. The simulation results shows that EKF gives a very good estimation of the states and it is working well with ODI. The resultant trajectories are very similar to those obtained by perfect state feedback using ODI only.

Unmanned Aerial Vehicles UAVs attitude, height, motion estimation and control using visual systems

This paper presents an implementation of an aircraft pose and motion estimator using visual systems as the principal sensor for controlling an Unmanned Aerial Vehicle (UAV) or as a redundant system for an Inertial Measure Unit (IMU) and gyros sensors. First, we explore the applications of the unified theory for central catadioptric cameras for attitude and heading estimation, explaining how the skyline is projected on the catadioptric image and how it is segmented and used to calculate the UAV’s attitude. Then we use appearance images to obtain a visual compass, and we calculate the relative rotation and heading of the aerial vehicle. Additionally, we show the use of a stereo system to calculate the aircraft height and to measure the UAV’s motion. Finally, we present a visual tracking system based on Fuzzy controllers working in both a UAV and a camera pan and tilt platform. Every part is tested using the UAV COLIBRI platform to validate the different approaches, which include comparison of the estimated data with the inertial values measured onboard the helicopter platform and the validation of the tracking schemes on real flights.

Visual Tracking, Pose Estimation, and Control for Aerial Vehicles

El principal objetivo de esta tesis es dotar a los vehículos aéreos no tripulados (UAVs, por sus siglas en inglés) de una fuente de información adicional basada en visión. Esta fuente de información proviene de cámaras ubicadas a bordo de los vehículos o en el suelo. Con ella se busca que los UAVs realicen tareas de aterrizaje o inspección guiados por visión, especialmente en aquellas situaciones en las que no haya disponibilidad de estimar la posición del vehículo con base en GPS, cuando las estimaciones de GPS no tengan la suficiente precisión requerida por las tareas a realizar, o cuando restricciones de carga de pago impidan añadir sensores a bordo de los vehículos. Esta tesis trata con tres de las principales áreas de la visión por computador: seguimiento visual y estimación visual de la pose (posición y orientación), que a su vez constituyen la base de la tercera, denominada control servo visual, que en nuestra aplicación se enfoca en el empleo de información visual para controlar los UAVs. Al respecto, esta tesis se ocupa de presentar propuestas novedosas que permitan solucionar problemas relativos al seguimiento de objetos mediante cámaras ubicadas a bordo de los UAVs, se ocupa de la estimación de la pose de los UAVs basada en información visual obtenida por cámaras ubicadas en el suelo o a bordo, y también se ocupa de la aplicación de las técnicas propuestas para solucionar diferentes problemas, como aquellos concernientes al seguimiento visual para tareas de reabastecimiento autónomo en vuelo o al aterrizaje basado en visión, entre otros. Las diversas técnicas de visión por computador presentadas en esta tesis se proponen con el fin de solucionar dificultades que suelen presentarse cuando se realizan tareas basadas en visión con UAVs, como las relativas a la obtención, en tiempo real, de estimaciones robustas, o como problemas generados por vibraciones. Los algoritmos propuestos en esta tesis han sido probados con información de imágenes reales obtenidas realizando pruebas on-line y off-line. Diversos mecanismos de evaluación han sido empleados con el propósito de analizar el desempeño de los algoritmos propuestos, entre los que se incluyen datos simulados, imágenes de vuelos reales, estimaciones precisas de posición empleando el sistema VICON y comparaciones con algoritmos del estado del arte. Los resultados obtenidos indican que los algoritmos de visión por computador propuestos tienen un desempeño que es comparable e incluso mejor al de algoritmos que se encuentran en el estado del arte. Los algoritmos propuestos permiten la obtención de estimaciones robustas en tiempo real, lo cual permite su uso en tareas de control visual. El desempeño de estos algoritmos es apropiado para las exigencias de las distintas aplicaciones examinadas: reabastecimiento autónomo en vuelo, aterrizaje y estimación del estado del UAV. Abstract The main objective of this thesis is to provide Unmanned Aerial Vehicles (UAVs) with an additional vision-based source of information extracted by cameras located either on-board or on the ground, in order to allow UAVs to develop visually guided tasks, such as landing or inspection, especially in situations where GPS information is not available, where GPS-based position estimation is not accurate enough for the task to develop, or where payload restrictions do not allow the incorporation of additional sensors on-board. This thesis covers three of the main computer vision areas: visual tracking and visual pose estimation, which are the bases the third one called visual servoing, which, in this work, focuses on using visual information to control UAVs. In this sense, the thesis focuses on presenting novel solutions for solving the tracking problem of objects when using cameras on-board UAVs, on estimating the pose of the UAVs based on the visual information collected by cameras located either on the ground or on-board, and also focuses on applying these proposed techniques for solving different problems, such as visual tracking for aerial refuelling or vision-based landing, among others. The different computer vision techniques presented in this thesis are proposed to solve some of the frequently problems found when addressing vision-based tasks in UAVs, such as obtaining robust vision-based estimations at real-time frame rates, and problems caused by vibrations, or 3D motion. All the proposed algorithms have been tested with real-image data in on-line and off-line tests. Different evaluation mechanisms have been used to analyze the performance of the proposed algorithms, such as simulated data, images from real-flight tests, publicly available datasets, manually generated ground truth data, accurate position estimations using a VICON system and a robotic cell, and comparison with state of the art algorithms. Results show that the proposed computer vision algorithms obtain performances that are comparable to, or even better than, state of the art algorithms, obtaining robust estimations at real-time frame rates. This proves that the proposed techniques are fast enough for vision-based control tasks. Therefore, the performance of the proposed vision algorithms has shown to be of a standard appropriate to the different explored applications: aerial refuelling and landing, and state estimation. It is noteworthy that they have low computational overheads for vision systems.

Spherical image-based visual servo and structure estimation

This paper presents a formulation of image-based visual servoing (IBVS) for a spherical camera where coordinates are parameterized in terms of colatitude and longitude: IBVSSph. The image Jacobian is derived and simulation results are presented for canonical rotational, translational as well as general motion. Problems with large rotations that affect the planar perspective form of IBVS are not present on the sphere, whereas the desirable robustness properties of IBVS are shown to be retained. We also describe a structure from motion (SfM) system based on camera-centric spherical coordinates and show how a recursive estimator can be used to recover structure. The spherical formulations for IBVS and SfM are particularly suitable for platforms, such as aerial and underwater robots, that move in SE(3).

Aircraft collision avoidance using spherical visual predictive control and single point features

This paper presents practical vision-based collision avoidance for objects approximating a single point feature. Using a spherical camera model, a visual predictive control scheme guides the aircraft around the object along a conical spiral trajectory. Visibility, state and control constraints are considered explicitly in the controller design by combining image and vehicle dynamics in the process model, and solving the nonlinear optimization problem over the resulting state space. Importantly, range is not required. Instead, the principles of conical spiral motion are used to design an objective function that simultaneously guides the aircraft along the avoidance trajectory, whilst providing an indication of the appropriate point to stop the spiral behaviour. Our approach is aimed at providing a potential solution to the See and Avoid problem for unmanned aircraft and is demonstrated through a series.

Close-quarters Quadrotor flying for a pole inspection with position based visual servoing and high-speed vision

This paper presents a 100 Hz monocular position based visual servoing system to control a quadrotor flying in close proximity to vertical structures approximating a narrow, locally linear shape. Assuming the object boundaries are represented by parallel vertical lines in the image, detection and tracking is achieved using Plücker line representation and a line tracker. The visual information is fused with IMU data in an EKF framework to provide fast and accurate state estimation. A nested control design provides position and velocity control with respect to the object. Our approach is aimed at high performance on-board control for applications allowing only small error margins and without a motion capture system, as required for real world infrastructure inspection. Simulated and ground-truthed experimental results are presented.

Transpupillary thermotherapy for choroidal melanoma:Tumor control and visual results in 100 consecutive cases

Objective: The authors evaluated the results of primary transpupillary thermotherapy for choroidal melanoma in 100 cases. Design: Prospective nonrandomized analysis of treatment method. Participants: One hundred patients with choroidal melanoma were studied. Main Outcome Measures: Tumor response, ocular side effects, and visual results. Results: Of 100 consecutive patients with choroidal melanoma treated with transpupillary thermotherapy, the mean tumor basal diameter was 7.1 mm and tumor thickness was 2.8 mm. The tumor margin touched the optic disc in 34 eyes (34%) and was beneath the fovea in 42 eyes (42%). Documented growth was present in 64 eyes (64%), and known clinical risks for growth were present in all of the remaining 36 eyes (36%), with an average of 4 of 5 statistical risk factors for growth per tumor. After a mean of three treatment sessions and 14 months of follow-up, the mean tumor thickness was reduced to 1.4 mm. Treatment was successful in 94 eyes (94%) and failed in 6 eyes (6%). Three patients with amelanotic tumors showed no initial response to thermotherapy, but subsequent intravenous indocyanine green administration during thermotherapy resulted in improved heat absorption and tumor regression to a flat scar. The six eyes classified as treatment failures included four eyes with tumors that showed partial or no response to thermotherapy, thus requiring plaque radiotherapy or enucleation, and two eyes with recurrence, subsequently controlled with additional thermotherapy. After treatment, the visual acuity was the same (within 1 line) or better than the pretreatment visual acuity in 58 eyes (58%) and worse in 42 eyes (42%). The main reasons for poorer vision included treatment through the foveola for subfoveal tumor (25 eyes), retinal traction (10 eyes), retinal vascular obstruction (5 eyes), optic disc edema (1 eye), and unrelated ocular ischemia (1 eye). Temporal location (versus nasal and superior, P = 0.02) and greater distance from the optic disc (P = 0.04) were risks for retinal traction. Conclusions: Transpupillary thermotherapy may be an effective treatment for small posterior choroidal melanoma, especially those near the optic disc and fovea. Despite satisfactory local tumor control, ocular side effects can result in decreased vision. Longer follow- up will be necessary to assess the impact of thermotherapy on ultimate local tumor control and metastatic disease.

Postural control and automaticity in dyslexic children: The relationship between visual information and body sway

Difficulty with literacy acquisition is only one of the symptoms of developmental dyslexia. Dyslexic children also show poor motor coordination and postural control. Those problems could be associated with automaticity, i.e., difficulty in performing a task without dispending a fair amount of conscious efforts. If this is the case, dyslexic children would show difficulties in using "unperceived" sensory cues to control body sway. Therefore, the aim of the study was to examine postural control performance and the coupling between visual information and body sway in dyslexic children. Ten dyslexic children and 10 non-dyslexic children stood upright inside a moving room that remained stationary or oscillated back and forward at frequencies of 0.2 or 0.5 Hz. Body sway magnitude and the relationship between the room's movement and body sway were examined. The results indicated that dyslexic children oscillated more than non-dyslexic children in both stationary and oscillating conditions. Visual manipulation induced body sway in all children but the coupling between visual information and body sway was weaker and more variable in dyslexic children. Based upon these results, we can suggest that dyslexic children use visual information to postural control with the same underlying processes as non-dyslexic children; however, dyslexic children show poorer performance and more variability while relating visual information and motor action even in a task that does not require an active cognitive and conscious motor involvement, which may be a further evidence of automaticity problem. (C) 2011 Elsevier Ltd. All rights reserved.

Physical activity and postural control in the elderly: Coupling between visual information and body sway

Background: Aging is characterized by a decline in the postural control performance, which is based on a coherent and stable coupling between sensory information and motor action. Therefore, changes in postural control in elderlies can be related to changes in this coupling. In addition, it has been observed that physical activity seems to improve postural control performance in elderlies. These improvements can be due to changes in the coupling between sensory information and motor action related to postural control. Objective: the purpose of this study was to verify the coupling between visual information and body sway in active and sedentary elderlies. Methods: Sixteen sedentary elderlies ( SE), 16 active elderlies ( AE) and 16 young adults ( YA) were asked to stand upright inside a moving room in two experimental conditions: ( 1) discrete movement and ( 2) continuous movement of the room. Results: In the continuous condition, the results showed that the coupling between the movement of the room and body sway was stronger and more stable for SE and AE compared to YA. In the discrete condition, SE showed larger body displacement compared to AE and YA. Conclusions: SE have more difficulty to discriminate and to integrate sensory information than AE and YA indicating that physical activity may improve sensory integration. Copyright (C) 2005 S. Karger AG, Basel.

Age-related differences in postural control: effects of the complexity of visual manipulation and sensorimotor contribution to postural performance

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Control framework for dexterous manipulation using dynamic visual servoing and tactile sensors’ feedback

Tactile sensors play an important role in robotics manipulation to perform dexterous and complex tasks. This paper presents a novel control framework to perform dexterous manipulation with multi-fingered robotic hands using feedback data from tactile and visual sensors. This control framework permits the definition of new visual controllers which allow the path tracking of the object motion taking into account both the dynamics model of the robot hand and the grasping force of the fingertips under a hybrid control scheme. In addition, the proposed general method employs optimal control to obtain the desired behaviour in the joint space of the fingers based on an indicated cost function which determines how the control effort is distributed over the joints of the robotic hand. Finally, authors show experimental verifications on a real robotic manipulation system for some of the controllers derived from the control framework.

Inclusion of immersive virtual learning environments and visual control systems to support the learning of students with Asperger syndrome

This paper presents the use of immersive virtual reality systems in the educational intervention with Asperger students. The starting points of this study are features of these students' cognitive style that requires an explicit teaching style supported by visual aids and highly structured environments. The proposed immersive virtual reality system, not only to assess the student's behavior and progress, but also is able to adapt itself to the student's specific needs. Additionally, the immersive reality system is equipped with sensors that can determine certain behaviors of the students. This paper determines the possible inclusion of immersive virtual reality as a support tool and learning strategy in these particular students' intervention. With this objective two task protocols have been defined with which the behavior and interaction situations performed by participant students are recorded. The conclusions from this study talks in favor of the inclusion of these virtual immersive environments as a support tool in the educational intervention of Asperger syndrome students as their social competences and executive functions have improved.