Vision-based detection and tracking of aerial targets for UAV collision avoidance

Machine vision represents a particularly attractive solution for sensing and detecting potential collision-course targets due to the relatively low cost, size, weight, and power requirements of the sensors involved (as opposed to radar). This paper describes the development and evaluation of a vision-based collision detection algorithm suitable for fixed-wing aerial robotics. The system was evaluated using highly realistic vision data of the moments leading up to a collision. Based on the collected data, our detection approaches were able to detect targets at distances ranging from 400m to about 900m. These distances (with some assumptions about closing speeds and aircraft trajectories) translate to an advanced warning of between 8-10 seconds ahead of impact, which approaches the 12.5 second response time recommended for human pilots. We make use of the enormous potential of graphic processing units to achieve processing rates of 30Hz (for images of size 1024-by- 768). Currently, integration in the final platform is under way.

Optimal-stopping control for airborne collision avoidance and return-to-course flight

This paper considers an aircraft collision avoidance design problem that also incorporates design of the aircraft’s return-to-course flight. This control design problem is formulated as a non-linear optimal-stopping control problem; a formulation that does not require a prior knowledge of time taken to perform the avoidance and return-to-course manoeuvre. A dynamic programming solution to the avoidance and return-to-course problem is presented, before a Markov chain numerical approximation technique is described. Simulation results are presented that illustrate the proposed collision avoidance and return-to-course flight approach.

Reactive image-based collision avoidance system for unmanned aircraft systems

Approximately 20 years have passed now since the NTSB issued its original recommendation to expedite development, certification and production of low-cost proximity warning and conflict detection systems for general aviation [1]. While some systems are in place (TCAS [2]), ¡¨see-and-avoid¡¨ remains the primary means of separation between light aircrafts sharing the national airspace. The requirement for a collision avoidance or sense-and-avoid capability onboard unmanned aircraft has been identified by leading government, industry and regulatory bodies as one of the most significant challenges facing the routine operation of unmanned aerial systems (UAS) in the national airspace system (NAS) [3, 4]. In this thesis, we propose and develop a novel image-based collision avoidance system to detect and avoid an upcoming conflict scenario (with an intruder) without first estimating or filtering range. The proposed collision avoidance system (CAS) uses relative bearing ƒÛ and angular-area subtended ƒê , estimated from an image, to form a test statistic AS C . This test statistic is used in a thresholding technique to decide if a conflict scenario is imminent. If deemed necessary, the system will command the aircraft to perform a manoeuvre based on ƒÛ and constrained by the CAS sensor field-of-view. Through the use of a simulation environment where the UAS is mathematically modelled and a flight controller developed, we show that using Monte Carlo simulations a probability of a Mid Air Collision (MAC) MAC RR or a Near Mid Air Collision (NMAC) RiskRatio can be estimated. We also show the performance gain this system has over a simplified version (bearings-only ƒÛ ). This performance gain is demonstrated in the form of a standard operating characteristic curve. Finally, it is shown that the proposed CAS performs at a level comparable to current manned aviations equivalent level of safety (ELOS) expectations for Class E airspace. In some cases, the CAS may be oversensitive in manoeuvring the owncraft when not necessary, but this constitutes a more conservative and therefore safer, flying procedures in most instances.

A spatial collision avoidance strategy for UAVs in a non-cooperative environment

This paper presents a feasible spatial collision avoidance approach for fixed-wing unmanned aerial vehicles (UAVs). The proposed strategy aims to achieve the desired relative bearing in the horizontal plane and relative elevation in the vertical plane so that the host aircraft is able to avoid collision with the intruder aircraft in 3D. The host aircraft will follow a desired trajectory in the collision avoidance course and resume the pre-arranged trajectory after collision is avoided. The approaching stopping condition is determined for the host aircraft to trigger an evasion maneuver to avoid collision in terms of measured heading. A switching controller is designed to achieve the spatial collision avoidance strategy. Simulation results demonstrate that the proposed approach can effectively avoid spatial collision, making it suitable for integration into flight control systems of UAVs.

A gust-attenuation controller for fixed-wing UAVs during collision avoidance course

This paper presents a nonlinear gust-attenuation controller to stabilize velocities, attitudes and angular rates of a fixed-wing unmanned aerial vehicle (UAV) in the presence of wind gusts. The proposed controller aims to achieve a steady-state flight condition such that the host UAV can avoid airspace collision with other UAVs during the cruise flight. Based on the typical UAV model capturing flight aerodynamics, a nonlinear Hinf controller is developed with rapid response property in consideration of actuator constraints. Simulations are conducted for the Shadow UAV to verify performance of the proposed controller. Comparative studies with the proportional-integral derivative (PID) controllers demonstrate that the proposed controller exhibits great performance improvement in a gusty environment, making it suitable for integration into the design of flight control systems for cruise flight with safety guarantees.

Modeling collision avoidance decisions in navigation

Due to grave potential human, environmental and economical consequences of collisions at sea, collision avoidance has become an important safety concern in navigation. To reduce the risk of collisions at sea, appropriate collision avoidance actions need to be taken in accordance with the regulations, i.e., International Regulations for Preventing Collisions at Sea. However, the regulations only provide qualitative rules and guidelines, and therefore it requires navigators to decide on collision avoidance actions quantitatively by using their judgments which often leads to making errors in navigation. To better help navigators in collision avoidance, this paper develops a comprehensive collision avoidance decision making model for providing whether a collision avoidance action is required, when to take action and what action to be taken. The model is developed based on three types of collision avoidance actions, such as course change only, speed change only, and a combination of both. The model has potential to reduce the chance of making human error in navigation by assisting navigators in decision making on collision avoidance actions.

A 3D collision avoidance strategy for UAVs in a non-cooperative environment

This paper presents a feasible 3D collision avoidance approach for fixed-wing unmanned aerial vehicles (UAVs). The proposed strategy aims to achieve the desired relative bearing in the horizontal plane and relative elevation in the vertical plane so that the host aircraft is able to avoid collision with the intruder aircraft in 3D. The host aircraft will follow a desired trajectory in the collision avoidance course and resume the pre-arranged trajectory after collision is avoided. The approaching stopping condition is determined for the host aircraft to trigger an evasion maneuver to avoid collision in terms of measured heading. A switching controller is designed to achieve the spatial collision avoidance strategy. Simulation results demonstrate that the proposed approach can effectively avoid spatial collision, making it suitable for integration into flight control systems of UAVs.

Nonlinear H_infinity control of UAVs for collision avoidance in gusty environments

This paper proposes a nonlinear H_infinity controller for stabilization of velocities, attitudes and angular rates of a fixed-wing unmanned aerial vehicle (UAV) in a windy environment. The suggested controller aims to achieve a steady-state flight condition in the presence of wind gusts such that the host UAV can be maneuvered to avoid collision with other UAVs during cruise flight with safety guarantees. This paper begins with building a proper model capturing flight aerodynamics of UAVs. Then a nonlinear controller is developed with gust attenuation and rapid response properties. Simulations are conducted for the Shadow UAV to verify performance of the proposed con- troller. Comparative studies with the proportional-integral-derivative (PID) controllers demonstrate that the proposed controller exhibits great performance improvement in a gusty environment, making it suitable for integration into the design of flight control systems for cruise flight of UAVs.

Flight guardian: a common avionics architecture for collision avoidance and safe emergency landing for unmanned aerial systems

This paper presents an approach to derive requirements for an avionics architecture that provides onboard sense-and-avoid and autonomous emergency forced landing capabilities to a UAS. The approach is based on two design paradigms that (1) derive requirements analyzing the common functionality between these two functions to then derive requirements for sensors, computing capability, interfaces, etc. (2) consider the risk and safety mitigation associated with these functions to derive certification requirements for the system design. We propose to use the Aircraft Certification Matrix (ACM) approach to tailor the system Development Assurance Levels (DAL) and architecture requirements in accordance with acceptable risk criteria. This architecture is developed under the name “Flight Guardian”. Flight Guardian is an avionics architecture that integrates common sensory elements that are essential components of any UAS that is required to be dependable. The Flight Guardian concept is also applicable to conventionally piloted aircraft, where it will serve to reduce cockpit workload.

Visual servoing approach to collision avoidance for aircraft

This paper presents a reactive Sense and Avoid approach using spherical image-based visual servoing. Avoidance of point targets in the lateral or vertical plane is achieved without requiring an estimate of range. Simulated results for static and dynamic targets are provided using a realistic model of a small fixed wing unmanned aircraft.

Rotorcraft collision avoidance using spherical image-based visual servoing and single point features

This paper presents a reactive collision avoidance method for small unmanned rotorcraft using spherical image-based visual servoing. Only a single point feature is used to guide the aircraft in a safe spiral like trajectory around the target, whilst a spherical camera model ensures the target always remains visible. A decision strategy to stop the avoidance control is derived based on the properties of spiral like motion, and the effect of accurate range measurements on the control scheme is discussed. We show that using a poor range estimate does not significantly degrade the collision avoidance performance, thus relaxing the need for accurate range measurements. We present simulated and experimental results using a small quad rotor to validate the approach.

Aircraft collision avoidance using spherical visual predictive control and single point features

This paper presents practical vision-based collision avoidance for objects approximating a single point feature. Using a spherical camera model, a visual predictive control scheme guides the aircraft around the object along a conical spiral trajectory. Visibility, state and control constraints are considered explicitly in the controller design by combining image and vehicle dynamics in the process model, and solving the nonlinear optimization problem over the resulting state space. Importantly, range is not required. Instead, the principles of conical spiral motion are used to design an objective function that simultaneously guides the aircraft along the avoidance trajectory, whilst providing an indication of the appropriate point to stop the spiral behaviour. Our approach is aimed at providing a potential solution to the See and Avoid problem for unmanned aircraft and is demonstrated through a series.

Decision strategies for automated visual collision avoidance

This paper provides a preliminary analysis of an autonomous uncooperative collision avoidance strategy for unmanned aircraft using image-based visual control. Assuming target detection, the approach consists of three parts. First, a novel decision strategy is used to determine appropriate reference image features to track for safe avoidance. This is achieved by considering the current rules of the air (regulations), the properties of spiral motion and the expected visual tracking errors. Second, a spherical visual predictive control (VPC) scheme is used to guide the aircraft along a safe spiral-like trajectory about the object. Lastly, a stopping decision based on thresholding a cost function is used to determine when to stop the avoidance behaviour. The approach does not require estimation of range or time to collision, and instead relies on tuning two mutually exclusive decision thresholds to ensure satisfactory performance.

Visual control for automated aircraft collision avoidance systems

This thesis presents a new vision-based decision and control strategy for automated aircraft collision avoidance that can be realistically applied to the See and Avoid problem. The effectiveness of the control strategy positions the research as a major contribution toward realising the simultaneous operation of manned and unmanned aircraft within civilian airspace. Key developments include novel classical and visual predictive control frameworks, and a performance evaluation technique aligned with existing aviation practise and applicable to autonomous systems. The overall approach is demonstrated through experimental results on a small multirotor unmanned aircraft, and through high fidelity probabilistic simulation studies.

Intelligent collision avoidance, control and guidance

This paper reviews a variety of advanced signal processing algorithms that have been developed at the University of Southampton as part of the Prometheus (Programme for European traffic flow with highest efficiency and unprecedented safety) programme to achieve an intelligent driver warning system (IDWS). The IDWS includes the detection of road edges, lanes, obstacles and their tracking and identification, estimates of time to collision, and behavioural modelling of drivers for a variety of scenarios. The underlying algorithms are briefly discussed in support of the IDWS.