Ship collision avoidance and COLREGS compliance using simulation-based control behavior selection with predictive hazard assessment

This paper describes a concept for a collision avoidance system for ships, which is based on model predictive control. A finite set of alternative control behaviors are generated by varying two parameters: offsets to the guidance course angle commanded to the autopilot and changes to the propulsion command ranging from nominal speed to full reverse. Using simulated predictions of the trajectories of the obstacles and ship, compliance with the Convention on the International Regulations for Preventing Collisions at Sea and collision hazards associated with each of the alternative control behaviors are evaluated on a finite prediction horizon, and the optimal control behavior is selected. Robustness to sensing error, predicted obstacle behavior, and environmental conditions can be ensured by evaluating multiple scenarios for each control behavior. The method is conceptually and computationally simple and yet quite versatile as it can account for the dynamics of the ship, the dynamics of the steering and propulsion system, forces due to wind and ocean current, and any number of obstacles. Simulations show that the method is effective and can manage complex scenarios with multiple dynamic obstacles and uncertainty associated with sensors and predictions.

Towards the implementation of vision-based UAS sense-and-avoid

Machine vision represents a particularly attractive solution for sensing and detecting potential collision-course targets due to the relatively low cost, size, weight, and power requirements of the sensors involved. This paper describes the development of detection algorithms and the evaluation of a real-time flight ready hardware implementation of a vision-based collision detection system suitable for fixed-wing small/medium size UAS. In particular, this paper demonstrates the use of Hidden Markov filter to track and estimate the elevation (β) and bearing (α) of the target, compares several candidate graphic processing hardware choices, and proposes an image based visual servoing approach to achieve collision avoidance

UAS See-and-Avoid using two different approaches of Fuzzy Control

This work presents two UAS See and Avoid approaches using Fuzzy Control. We compare the performance of each controller when a Cross-Entropy method is applied to optimase the parameters for one of the controllers. Each controller receive information from an image processing front-end that detect and track targets in the environment. Visual information is then used under a visual servoing approach to perform autonomous avoidance. Experimental flight trials using a small quadrotor were performed to validate and compare the behaviour of both controllers

Airborne collision scenario flight tests : impact of angle measurement errors on reactive vision-based avoidance control

The future emergence of many types of airborne vehicles and unpiloted aircraft in the national airspace means collision avoidance is of primary concern in an uncooperative airspace environment. The ability to replicate a pilot’s see and avoid capability using cameras coupled with vision based avoidance control is an important part of an overall collision avoidance strategy. But unfortunately without range collision avoidance has no direct way to guarantee a level of safety. Collision scenario flight tests with two aircraft and a monocular camera threat detection and tracking system were used to study the accuracy of image-derived angle measurements. The effect of image-derived angle errors on reactive vision-based avoidance performance was then studied by simulation. The results show that whilst large angle measurement errors can significantly affect minimum ranging characteristics across a variety of initial conditions and closing speeds, the minimum range is always bounded and a collision never occurs.

The Smart Skies Project

Smart Skies is an international research project exploring the development and demonstration of future aviation technologies which facilitate the more efficient utilisation of airspace for both manned and unmanned aircraft. These technologies include autonomous vision-based collision avoidance systems, autonomous airspace separation management systems and a mobile ground-based radar system to support non-segregated UAS operations within the NAS. This presentation will provide an introduction to the key programs of research, detail results from recent flight trial activities and will outline future directions for the project.

Test architecture for prototyping automated dynamic airspace control

The automation of various aspects of air traffic management has many wide-reaching benefits including: reducing the workload for Air Traffic Controllers; increasing the flexibility of operations (both civil and military) within the airspace system through facilitating automated dynamic changes to en-route flight plans; ensuring safe aircraft separation for a complex mix of airspace users within a highly complex and dynamic airspace management system architecture. These benefits accumulate to increase the efficiency and flexibility of airspace use(1). Such functions are critical for the anticipated increase in volume of manned and unmanned aircraft traffic. One significant challenge facing the advancement of airspace automation lies in convincing air traffic regulatory authorities that the level of safety achievable through the use of automation concepts is comparable to, or exceeds, the accepted safety performance of the current system.

Airborne vision-based collision-detection system

Machine vision represents a particularly attractive solution for sensing and detecting potential collision-course targets due to the relatively low cost, size, weight, and power requirements of vision sensors (as opposed to radar and TCAS). This paper describes the development and evaluation of a real-time vision-based collision detection system suitable for fixed-wing aerial robotics. Using two fixed-wing UAVs to recreate various collision-course scenarios, we were able to capture highly realistic vision (from an onboard camera perspective) of the moments leading up to a collision. This type of image data is extremely scarce and was invaluable in evaluating the detection performance of two candidate target detection approaches. Based on the collected data, our detection approaches were able to detect targets at distances ranging from 400m to about 900m. These distances (with some assumptions about closing speeds and aircraft trajectories) translate to an advanced warning of between 8-10 seconds ahead of impact, which approaches the 12.5 second response time recommended for human pilots. We overcame the challenge of achieving real-time computational speeds by exploiting the parallel processing architectures of graphics processing units found on commercially-off-the-shelf graphics devices. Our chosen GPU device suitable for integration onto UAV platforms can be expected to handle real-time processing of 1024 by 768 pixel image frames at a rate of approximately 30Hz. Flight trials using manned Cessna aircraft where all processing is performed onboard will be conducted in the near future, followed by further experiments with fully autonomous UAV platforms.

Modeling perceived collision risk in port water navigation

An increase in the likelihood of navigational collisions in port waters has put focus on the collision avoidance process in port traffic safety. The most widely used on-board collision avoidance system is the automatic radar plotting aid which is a passive warning system that triggers an alert based on the pilot’s pre-defined indicators of distance and time proximities at the closest point of approaches in encounters with nearby vessels. To better help pilot in decision making in close quarter situations, collision risk should be considered as a continuous monotonic function of the proximities and risk perception should be considered probabilistically. This paper derives an ordered probit regression model to study perceived collision risks. To illustrate the procedure, the risks perceived by Singapore port pilots were obtained to calibrate the regression model. The results demonstrate that a framework based on the probabilistic risk assessment model can be used to give a better understanding of collision risk and to define a more appropriate level of evasive actions.

Quadcopter see and avoid using a fuzzy controller

Unmanned Aerial Vehicles (UAVs) industry is a fast growing sector. Nowadays, the market offers numerous possibilities for off-the-shelf UAVs such as quadrotors or fixed-wings. Until UAVs demonstrate advance capabilities such as autonomous collision avoidance they will be segregated and restricted to flight in controlled environments. This work presents a visual fuzzy servoing system for obstacle avoidance using UAVs. To accomplish this task we used the visual information from the front camera. Images are processed off-board and the result send to the Fuzzy Logic controller which then send commands to modify the orientation of the aircraft. Results from flight test are presented with a commercial off-the-shelf platform.

Flight trial of an electro-optical sense-and-avoid system

This paper presents the flight trials of an electro-optical (EO) sense-and-avoid system onboard a Cessna host aircraft (camera aircraft). We focus on the autonomous collision avoidance capability of the sense-and-avoid system; that is, closed-loop integration with the onboard aircraft autopilot. We also discuss the system’s approach to target detection and avoidance control, as well as the methodology of the flight trials. The results demonstrate the ability of the sense-and-avoid system to automatically detect potential conflicting aircraft and engage the host Cessna autopilot to perform an avoidance manoeuvre, all without any human intervention

Sensors for missions

An onboard payload may be seen in most instances as the “Raison d’Etre” for a UAV. It will define its capabilities, usability and hence market value. Large and medium UAV payloads exhibit significant differences in size and computing capability when compared with small UAVs. The latter have stringent size, weight, and power requirements, typically referred as SWaP, while the former still exhibit endless appetite for compute capability. The tendency for this type of UAVs (Global Hawk, Hunter, Fire Scout, etc.) is to increase payload density and hence processing capability. An example of this approach is the Northrop Grumman MQ-8 Fire Scout helicopter, which has a modular payload architecture that incorporates off-the-shelf components. Regardless of the UAV size and capabilities, advances in miniaturization of electronics are enabling the replacement of multiprocessing, power-hungry general-purpose processors for more integrated and compact electronics (e.g., FPGAs). Payloads play a significant role in the quality of ISR (intelligent, surveillance, and reconnaissance) data, and also in how quick that information can be delivered to the end user. At a high level, payloads are important enablers of greater mission autonomy, which is the ultimate aim in every UAV. This section describes common payload sensors and introduces two examples cases in which onboard payloads were used to solve real-world problems. A collision avoidance payload based on electro optical (EO) sensors is first introduced, followed by a remote sensing application for power line inspection and vegetation management.

RPAS integration within an Australian ATM system : what equipment and which airspace

The Australian Civil Aviation Safety Authority (CASA) currently lists more than 100 separate entities or organisations which maintain a UAS Operator Certificate (UOC) [1]. Approved operations are overwhelmingly a permutation of aerial photography, surveillance, survey or spotting and predominantly, are restricted to Visual Line of Sight (VLOS) operations, below 400 feet, and not within 3 NM of an aerodrome. However, demand is increasing for a Remote Piloted Aerial System (RPAS) regulatory regime which facilitates more expansive operations, in particular unsegregated, Beyond Visual Line of Sight (BVLOS) operations. Despite this demand, there is national and international apprehension regarding the necessary levels of airworthiness and operational regulation required to maintain safety and minimise the risk associated with unsegregated operations. Fundamental to addressing these legitimate concerns will be the mechanisms that underpin safe separation and collision avoidance. Whilst a large body of research has been dedicated to investigating on-board, Sense and Avoid (SAA) technology necessary to meet this challenge, this paper focuses on the contribution of the NAS to separation assurance, and how it will support, as well as complicate RPAS integration. The paper collates and presents key, but historically disparate, threads of Australian RPAS and NAS related information, and distils it with a filter focused on minimising RPAS collision risk. Our ongoing effort is motivated by the need to better understand the separation assurance contribution provided by the NAS layers, in the first instance, and subsequently employ this information to identify scenarios where the coincident collision risk is demonstrably low, providing legitimate substantiation for concessions on equipage and airworthiness standards.

Automated centralised separation management with onboard decision support

This paper proposes a novel automated separation management concept in which onboard decision support is integrated within a centralised air traffic separation management system. The onboard decision support system involves a decentralised separation manager that can overrule air traffic management instructions under certain circumstances. This approach allows the advantages of both centralised and decentralised concepts to be combined (and disadvantages of each separation management approach to be mitigated). Simulation studies are used to illustrate the potential benefits of the combined separation management concept.

A hidden Markov model and relative entropy rate approach to vision-based dim target detection for UAV sense-and-avoid

Uninhabited aerial vehicles (UAVs) are a cutting-edge technology that is at the forefront of aviation/aerospace research and development worldwide. Many consider their current military and defence applications as just a token of their enormous potential. Unlocking and fully exploiting this potential will see UAVs in a multitude of civilian applications and routinely operating alongside piloted aircraft. The key to realising the full potential of UAVs lies in addressing a host of regulatory, public relation, and technological challenges never encountered be- fore. Aircraft collision avoidance is considered to be one of the most important issues to be addressed, given its safety critical nature. The collision avoidance problem can be roughly organised into three areas: 1) Sense; 2) Detect; and 3) Avoid. Sensing is concerned with obtaining accurate and reliable information about other aircraft in the air; detection involves identifying potential collision threats based on available information; avoidance deals with the formulation and execution of appropriate manoeuvres to maintain safe separation. This thesis tackles the detection aspect of collision avoidance, via the development of a target detection algorithm that is capable of real-time operation onboard a UAV platform. One of the key challenges of the detection problem is the need to provide early warning. This translates to detecting potential threats whilst they are still far away, when their presence is likely to be obscured and hidden by noise. Another important consideration is the choice of sensors to capture target information, which has implications for the design and practical implementation of the detection algorithm. The main contributions of the thesis are: 1) the proposal of a dim target detection algorithm combining image morphology and hidden Markov model (HMM) filtering approaches; 2) the novel use of relative entropy rate (RER) concepts for HMM filter design; 3) the characterisation of algorithm detection performance based on simulated data as well as real in-flight target image data; and 4) the demonstration of the proposed algorithm's capacity for real-time target detection. We also consider the extension of HMM filtering techniques and the application of RER concepts for target heading angle estimation. In this thesis we propose a computer-vision based detection solution, due to the commercial-off-the-shelf (COTS) availability of camera hardware and the hardware's relatively low cost, power, and size requirements. The proposed target detection algorithm adopts a two-stage processing paradigm that begins with an image enhancement pre-processing stage followed by a track-before-detect (TBD) temporal processing stage that has been shown to be effective in dim target detection. We compare the performance of two candidate morphological filters for the image pre-processing stage, and propose a multiple hidden Markov model (MHMM) filter for the TBD temporal processing stage. The role of the morphological pre-processing stage is to exploit the spatial features of potential collision threats, while the MHMM filter serves to exploit the temporal characteristics or dynamics. The problem of optimising our proposed MHMM filter has been examined in detail. Our investigation has produced a novel design process for the MHMM filter that exploits information theory and entropy related concepts. The filter design process is posed as a mini-max optimisation problem based on a joint RER cost criterion. We provide proof that this joint RER cost criterion provides a bound on the conditional mean estimate (CME) performance of our MHMM filter, and this in turn establishes a strong theoretical basis connecting our filter design process to filter performance. Through this connection we can intelligently compare and optimise candidate filter models at the design stage, rather than having to resort to time consuming Monte Carlo simulations to gauge the relative performance of candidate designs. Moreover, the underlying entropy concepts are not constrained to any particular model type. This suggests that the RER concepts established here may be generalised to provide a useful design criterion for multiple model filtering approaches outside the class of HMM filters. In this thesis we also evaluate the performance of our proposed target detection algorithm under realistic operation conditions, and give consideration to the practical deployment of the detection algorithm onboard a UAV platform. Two fixed-wing UAVs were engaged to recreate various collision-course scenarios to capture highly realistic vision (from an onboard camera perspective) of the moments leading up to a collision. Based on this collected data, our proposed detection approach was able to detect targets out to distances ranging from about 400m to 900m. These distances, (with some assumptions about closing speeds and aircraft trajectories) translate to an advanced warning ahead of impact that approaches the 12.5 second response time recommended for human pilots. Furthermore, readily available graphic processing unit (GPU) based hardware is exploited for its parallel computing capabilities to demonstrate the practical feasibility of the proposed target detection algorithm. A prototype hardware-in- the-loop system has been found to be capable of achieving data processing rates sufficient for real-time operation. There is also scope for further improvement in performance through code optimisations. Overall, our proposed image-based target detection algorithm offers UAVs a cost-effective real-time target detection capability that is a step forward in ad- dressing the collision avoidance issue that is currently one of the most significant obstacles preventing widespread civilian applications of uninhabited aircraft. We also highlight that the algorithm development process has led to the discovery of a powerful multiple HMM filtering approach and a novel RER-based multiple filter design process. The utility of our multiple HMM filtering approach and RER concepts, however, extend beyond the target detection problem. This is demonstrated by our application of HMM filters and RER concepts to a heading angle estimation problem.