Computer vision onboard UAVs for civilian tasks

Computer vision is much more than a technique to sense and recover environmental information from an UAV. It should play a main role regarding UAVs’ functionality because of the big amount of information that can be extracted, its possible uses and applications, and its natural connection to human driven tasks, taking into account that vision is our main interface to world understanding. Our current research’s focus lays on the development of techniques that allow UAVs to maneuver in spaces using visual information as their main input source. This task involves the creation of techniques that allow an UAV to maneuver towards features of interest whenever a GPS signal is not reliable or sufficient, e.g. when signal dropouts occur (which usually happens in urban areas, when flying through terrestrial urban canyons or when operating on remote planetary bodies), or when tracking or inspecting visual targets—including moving ones—without knowing their exact UMT coordinates. This paper also investigates visual serving control techniques that use velocity and position of suitable image features to compute the references for flight control. This paper aims to give a global view of the main aspects related to the research field of computer vision for UAVs, clustered in four main active research lines: visual serving and control, stereo-based visual navigation, image processing algorithms for detection and tracking, and visual SLAM. Finally, the results of applying these techniques in several applications are presented and discussed: this study will encompass power line inspection, mobile target tracking, stereo distance estimation, mapping and positioning.

Unmanned Aerial Vehicles UAVs attitude, height, motion estimation and control using visual systems

This paper presents an implementation of an aircraft pose and motion estimator using visual systems as the principal sensor for controlling an Unmanned Aerial Vehicle (UAV) or as a redundant system for an Inertial Measure Unit (IMU) and gyros sensors. First, we explore the applications of the unified theory for central catadioptric cameras for attitude and heading estimation, explaining how the skyline is projected on the catadioptric image and how it is segmented and used to calculate the UAV’s attitude. Then we use appearance images to obtain a visual compass, and we calculate the relative rotation and heading of the aerial vehicle. Additionally, we show the use of a stereo system to calculate the aircraft height and to measure the UAV’s motion. Finally, we present a visual tracking system based on Fuzzy controllers working in both a UAV and a camera pan and tilt platform. Every part is tested using the UAV COLIBRI platform to validate the different approaches, which include comparison of the estimated data with the inertial values measured onboard the helicopter platform and the validation of the tracking schemes on real flights.

Aerial object following using visual fuzzy servoing

This article presents a visual servoing system to follow a 3D moving object by a Micro Unmanned Aerial Vehicle (MUAV). The presented control strategy is based only on the visual information given by an adaptive tracking method based on the colour information. A visual fuzzy system has been developed for servoing the camera situated on a rotary wing MAUV, that also considers its own dynamics. This system is focused on continuously following of an aerial moving target object, maintaining it with a fixed safe distance and centred on the image plane. The algorithm is validated on real flights on outdoors scenarios, showing the robustness of the proposed systems against winds perturbations, illumination and weather changes among others. The obtained results indicate that the proposed algorithms is suitable for complex controls task, such object following and pursuit, flying in formation, as well as their use for indoor navigation

Rapid prototyping framework for visual control of autonomous micro aerial vehicles

Rapid prototyping environments can speed up the research of visual control algorithms. We have designed and implemented a software framework for fast prototyping of visual control algorithms for Micro Aerial Vehicles (MAV). We have applied a combination of a proxy-based network communication architecture and a custom Application Programming Interface. This allows multiple experimental configurations, like drone swarms or distributed processing of a drone's video stream. Currently, the framework supports a low-cost MAV: the Parrot AR.Drone. Real tests have been performed on this platform and the results show comparatively low figures of the extra communication delay introduced by the framework, while adding new functionalities and flexibility to the selected drone. This implementation is open-source and can be downloaded from www.vision4uav.com/?q=VC4MAV-FW

See-and-avoid quadcopter using fuzzy control optimized by cross-entropy

In this work we present an optimized fuzzy visual servoing system for obstacle avoidance using an unmanned aerial vehicle. The cross-entropy theory is used to optimise the gains of our controllers. The optimization process was made using the ROS-Gazebo 3D simulation with purposeful extensions developed for our experiments. Visual servoing is achieved through an image processing front-end that uses the Camshift algorithm to detect and track objects in the scene. Experimental flight trials using a small quadrotor were performed to validate the parameters estimated from simulation. The integration of cross- entropy methods is a straightforward way to estimate optimal gains achieving excellent results when tested in real flights.

Quadcopter see and avoid using a fuzzy controller

Unmanned Aerial Vehicles (UAVs) industry is a fast growing sector. Nowadays, the market offers numerous possibilities for off-the-shelf UAVs such as quadrotors or fixed-wings. Until UAVs demonstrate advance capabilities such as autonomous collision avoidance they will be segregated and restricted to flight in controlled environments. This work presents a visual fuzzy servoing system for obstacle avoidance using UAVs. To accomplish this task we used the visual information from the front camera. Images are processed off-board and the result send to the Fuzzy Logic controller which then send commands to modify the orientation of the aircraft. Results from flight test are presented with a commercial off-the-shelf platform.

UAS See-and-Avoid using two different approaches of Fuzzy Control

This work presents two UAS See and Avoid approaches using Fuzzy Control. We compare the performance of each controller when a Cross-Entropy method is applied to optimase the parameters for one of the controllers. Each controller receive information from an image processing front-end that detect and track targets in the environment. Visual information is then used under a visual servoing approach to perform autonomous avoidance. Experimental flight trials using a small quadrotor were performed to validate and compare the behaviour of both controllers

Cross-entropy optimization for scaling factors of a fuzzy controller : a see-and-avoid approach for unmanned aerial systems

The Cross-Entropy (CE) is an efficient method for the estimation of rare-event probabilities and combinatorial optimization. This work presents a novel approach of the CE for optimization of a Soft-Computing controller. A Fuzzy controller was designed to command an unmanned aerial system (UAS) for avoiding collision task. The only sensor used to accomplish this task was a forward camera. The CE is used to reach a near-optimal controller by modifying the scaling factors of the controller inputs. The optimization was realized using the ROS-Gazebo simulation system. In order to evaluate the optimization a big amount of tests were carried out with a real quadcopter.

MAVwork : a framework for unified interfacing between micro aerial vehicles and visual controllers

Debugging control software for Micro Aerial Vehicles (MAV) can be risky out of the simulator, especially with professional drones that might harm people around or result in a high bill after a crash. We have designed a framework that enables a software application to communicate with multiple MAVs from a single unified interface. In this way, visual controllers can be first tested on a low-cost harmless MAV and, after safety is guaranteed, they can be moved to the production MAV at no additional cost. The framework is based on a distributed architecture over a network. This allows multiple configurations, like drone swarms or parallel processing of drones' video streams. Live tests have been performed and the results show comparatively low additional communication delays, while adding new functionalities and flexibility. This implementation is open-source and can be downloaded from github.com/uavster/mavwork

Predicting invasion in grassland ecosystems: is exotic dominance the real embarrassment of richness?

Invasions have increased the size of regional species pools, but are typically assumed to reduce native diversity. However, global-scale tests of this assumption have been elusive because of the focus on exotic species richness, rather than relative abundance. This is problematic because low invader richness can indicate invasion resistance by the native community or, alternatively, dominance by a single exotic species. Here, we used a globally replicated study to quantify relationships between exotic richness and abundance in grass-dominated ecosystems in 13 countries on six continents, ranging from salt marshes to alpine tundra. We tested effects of human land use, native community diversity, herbivore pressure, and nutrient limitation on exotic plant dominance. Despite its widespread use, exotic richness was a poor proxy for exotic dominance at low exotic richness, because sites that contained few exotic species ranged from relatively pristine (low exotic richness and cover) to almost completely exotic-dominated ones (low exotic richness but high exotic cover). Both exotic cover and richness were predicted by native plant diversity (native grass richness) and land use (distance to cultivation). Although climate was important for predicting both exotic cover and richness, climatic factors predicting cover (precipitation variability) differed from those predicting richness (maximum temperature and mean temperature in the wettest quarter). Herbivory and nutrient limitation did not predict exotic richness or cover. Exotic dominance was greatest in areas with low native grass richness at the site- or regional-scale. Although this could reflect native grass displacement, a lack of biotic resistance is a more likely explanation, given that grasses comprise the most aggressive invaders. These findings underscore the need to move beyond richness as a surrogate for the extent of invasion, because this metric confounds monodominance with invasion resistance. Monitoring species' relative abundance will more rapidly advance our understanding of invasions.

Herbivores and nutrients control grassland plant diversity via light limitation

Human alterations to nutrient cycles1, 2 and herbivore communities3, 4, 5, 6, 7 are affecting global biodiversity dramatically2. Ecological theory predicts these changes should be strongly counteractive: nutrient addition drives plant species loss through intensified competition for light, whereas herbivores prevent competitive exclusion by increasing ground-level light, particularly in productive systems8, 9. Here we use experimental data spanning a globally relevant range of conditions to test the hypothesis that herbaceous plant species losses caused by eutrophication may be offset by increased light availability due to herbivory. This experiment, replicated in 40 grasslands on 6 continents, demonstrates that nutrients and herbivores can serve as counteracting forces to control local plant diversity through light limitation, independent of site productivity, soil nitrogen, herbivore type and climate. Nutrient addition consistently reduced local diversity through light limitation, and herbivory rescued diversity at sites where it alleviated light limitation. Thus, species loss from anthropogenic eutrophication can be ameliorated in grasslands where herbivory increases ground-level light.

Major histocompatibility complex associations of ankylosing spondylitis are complex and involve further epistasis with ERAP1

Ankylosing spondylitis (AS) is a common, highly heritable, inflammatory arthritis for which HLA-B*27 is the major genetic risk factor, although its role in the aetiology of AS remains elusive. To better understand the genetic basis of the MHC susceptibility loci, we genotyped 7,264 MHC SNPs in 22,647 AS cases and controls of European descent. We impute SNPs, classical HLA alleles and amino-acid residues within HLA proteins, and tested these for association to AS status. Here we show that in addition to effects due to HLA-B*27 alleles, several other HLA-B alleles also affect susceptibility. After controlling for the associated haplotypes in HLA-B, we observe independent associations with variants in the HLA-A, HLA-DPB1 and HLA-DRB1 loci. We also demonstrate that the ERAP1 SNP rs30187 association is not restricted only to carriers of HLA-B*27 but also found in HLA-B*40:01 carriers independently of HLA-B*27 genotype.

Global, regional, and national disability-adjusted life years (DALYs) for 306 diseases and injuries and healthy life expectancy (HALE) for 188 countries, 1990–2013: Quantifying the epidemiological transition

Background The Global Burden of Disease Study 2013 (GBD 2013) aims to bring together all available epidemiological data using a coherent measurement framework, standardised estimation methods, and transparent data sources to enable comparisons of health loss over time and across causes, age–sex groups, and countries. The GBD can be used to generate summary measures such as disability-adjusted life-years (DALYs) and healthy life expectancy (HALE) that make possible comparative assessments of broad epidemiological patterns across countries and time. These summary measures can also be used to quantify the component of variation in epidemiology that is related to sociodemographic development. Methods We used the published GBD 2013 data for age-specific mortality, years of life lost due to premature mortality (YLLs), and years lived with disability (YLDs) to calculate DALYs and HALE for 1990, 1995, 2000, 2005, 2010, and 2013 for 188 countries. We calculated HALE using the Sullivan method; 95% uncertainty intervals (UIs) represent uncertainty in age-specific death rates and YLDs per person for each country, age, sex, and year. We estimated DALYs for 306 causes for each country as the sum of YLLs and YLDs; 95% UIs represent uncertainty in YLL and YLD rates. We quantified patterns of the epidemiological transition with a composite indicator of sociodemographic status, which we constructed from income per person, average years of schooling after age 15 years, and the total fertility rate and mean age of the population. We applied hierarchical regression to DALY rates by cause across countries to decompose variance related to the sociodemographic status variable, country, and time. Findings Worldwide, from 1990 to 2013, life expectancy at birth rose by 6·2 years (95% UI 5·6–6·6), from 65·3 years (65·0–65·6) in 1990 to 71·5 years (71·0–71·9) in 2013, HALE at birth rose by 5·4 years (4·9–5·8), from 56·9 years (54·5–59·1) to 62·3 years (59·7–64·8), total DALYs fell by 3·6% (0·3–7·4), and age-standardised DALY rates per 100 000 people fell by 26·7% (24·6–29·1). For communicable, maternal, neonatal, and nutritional disorders, global DALY numbers, crude rates, and age-standardised rates have all declined between 1990 and 2013, whereas for non–communicable diseases, global DALYs have been increasing, DALY rates have remained nearly constant, and age-standardised DALY rates declined during the same period. From 2005 to 2013, the number of DALYs increased for most specific non-communicable diseases, including cardiovascular diseases and neoplasms, in addition to dengue, food-borne trematodes, and leishmaniasis; DALYs decreased for nearly all other causes. By 2013, the five leading causes of DALYs were ischaemic heart disease, lower respiratory infections, cerebrovascular disease, low back and neck pain, and road injuries. Sociodemographic status explained more than 50% of the variance between countries and over time for diarrhoea, lower respiratory infections, and other common infectious diseases; maternal disorders; neonatal disorders; nutritional deficiencies; other communicable, maternal, neonatal, and nutritional diseases; musculoskeletal disorders; and other non-communicable diseases. However, sociodemographic status explained less than 10% of the variance in DALY rates for cardiovascular diseases; chronic respiratory diseases; cirrhosis; diabetes, urogenital, blood, and endocrine diseases; unintentional injuries; and self-harm and interpersonal violence. Predictably, increased sociodemographic status was associated with a shift in burden from YLLs to YLDs, driven by declines in YLLs and increases in YLDs from musculoskeletal disorders, neurological disorders, and mental and substance use disorders. In most country-specific estimates, the increase in life expectancy was greater than that in HALE. Leading causes of DALYs are highly variable across countries. Interpretation Global health is improving. Population growth and ageing have driven up numbers of DALYs, but crude rates have remained relatively constant, showing that progress in health does not mean fewer demands on health systems. The notion of an epidemiological transition—in which increasing sociodemographic status brings structured change in disease burden—is useful, but there is tremendous variation in burden of disease that is not associated with sociodemographic status. This further underscores the need for country-specific assessments of DALYs and HALE to appropriately inform health policy decisions and attendant actions.

Global, regional, and national comparative risk assessment of 79 behavioural, environmental and occupational, and metabolic risks or clusters of risks in 188 countries, 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013

Background The Global Burden of Disease, Injuries, and Risk Factor study 2013 (GBD 2013) is the first of a series of annual updates of the GBD. Risk factor quantification, particularly of modifiable risk factors, can help to identify emerging threats to population health and opportunities for prevention. The GBD 2013 provides a timely opportunity to update the comparative risk assessment with new data for exposure, relative risks, and evidence on the appropriate counterfactual risk distribution. Methods Attributable deaths, years of life lost, years lived with disability, and disability-adjusted life-years (DALYs) have been estimated for 79 risks or clusters of risks using the GBD 2010 methods. Risk–outcome pairs meeting explicit evidence criteria were assessed for 188 countries for the period 1990–2013 by age and sex using three inputs: risk exposure, relative risks, and the theoretical minimum risk exposure level (TMREL). Risks are organised into a hierarchy with blocks of behavioural, environmental and occupational, and metabolic risks at the first level of the hierarchy. The next level in the hierarchy includes nine clusters of related risks and two individual risks, with more detail provided at levels 3 and 4 of the hierarchy. Compared with GBD 2010, six new risk factors have been added: handwashing practices, occupational exposure to trichloroethylene, childhood wasting, childhood stunting, unsafe sex, and low glomerular filtration rate. For most risks, data for exposure were synthesised with a Bayesian meta-regression method, DisMod-MR 2.0, or spatial-temporal Gaussian process regression. Relative risks were based on meta-regressions of published cohort and intervention studies. Attributable burden for clusters of risks and all risks combined took into account evidence on the mediation of some risks such as high body-mass index (BMI) through other risks such as high systolic blood pressure and high cholesterol. Findings All risks combined account for 57·2% (95% uncertainty interval [UI] 55·8–58·5) of deaths and 41·6% (40·1–43·0) of DALYs. Risks quantified account for 87·9% (86·5–89·3) of cardiovascular disease DALYs, ranging to a low of 0% for neonatal disorders and neglected tropical diseases and malaria. In terms of global DALYs in 2013, six risks or clusters of risks each caused more than 5% of DALYs: dietary risks accounting for 11·3 million deaths and 241·4 million DALYs, high systolic blood pressure for 10·4 million deaths and 208·1 million DALYs, child and maternal malnutrition for 1·7 million deaths and 176·9 million DALYs, tobacco smoke for 6·1 million deaths and 143·5 million DALYs, air pollution for 5·5 million deaths and 141·5 million DALYs, and high BMI for 4·4 million deaths and 134·0 million DALYs. Risk factor patterns vary across regions and countries and with time. In sub-Saharan Africa, the leading risk factors are child and maternal malnutrition, unsafe sex, and unsafe water, sanitation, and handwashing. In women, in nearly all countries in the Americas, north Africa, and the Middle East, and in many other high-income countries, high BMI is the leading risk factor, with high systolic blood pressure as the leading risk in most of Central and Eastern Europe and south and east Asia. For men, high systolic blood pressure or tobacco use are the leading risks in nearly all high-income countries, in north Africa and the Middle East, Europe, and Asia. For men and women, unsafe sex is the leading risk in a corridor from Kenya to South Africa. Interpretation Behavioural, environmental and occupational, and metabolic risks can explain half of global mortality and more than one-third of global DALYs providing many opportunities for prevention. Of the larger risks, the attributable burden of high BMI has increased in the past 23 years. In view of the prominence of behavioural risk factors, behavioural and social science research on interventions for these risks should be strengthened. Many prevention and primary care policy options are available now to act on key risks.