Moving object detection in aerial video based on spatiotemporal saliency

In this paper, the problem of moving object detection in aerial video is addressed. While motion cues have been extensively exploited in the literature, how to use spatial information is still an open problem. To deal with this issue, we propose a novel hierarchical moving target detection method based on spatiotemporal saliency. Temporal saliency is used to get a coarse segmentation, and spatial saliency is extracted to obtain the object’s appearance details in candidate motion regions. Finally, by combining temporal and spatial saliency information, we can get refined detection results. Additionally, in order to give a full description of the object distribution, spatial saliency is detected in both pixel and region levels based on local contrast. Experiments conducted on the VIVID dataset show that the proposed method is efficient and accurate.

Beluga, Delphinapterus leucas, Group Sizes in Cook Inlet, Alaska, Based on Observer Counts and Aerial Video

Belugas, Delphinapterus leucas, groups were videotaped concurrent to observer counts during annual NMFS aerial surveys of Cook Inlet, Alaska, from 1994 to 2000. The videotapes provided permanent records of whale groups that could be examined and compared to group size estimates ade by aerial observers.Examination of the video recordings resulted in 275 counts of 79 whale groups. The McLaren formula was used to account for whales missed while they were underwater (average correction factor 2.03; SD=0.64). A correction for whales missed due to video resolution was developed by using a second, paired video camera that magnified images relative to the standard video. This analysis showed that some whales were missed either because their image size fell below the resolution of hte standard video recording or because two whales surfaced so close to each other that their images appeared to be one large whale. The correction method that resulted depended on knowing the average whale image size in the videotapes. Image sizes were measured for 2,775 whales from 275 different passes over whale groups. Corrected group sizes were calcualted as the product of the original count from video, the correction factor for whales missed underwater, and the correction factor for whales missed due to video resolution (averaged 1.17; SD=0.06). A regression formula was developed to estimate group sizes from aerial observer counts; independent variables were the aerial counts and an interaction term relative to encounter rate (whales per second during the counting of a group), which were regressed against the respective group sizes as calculated from the videotapes. Significant effects of encounter rate, either positive or negative, were found for several observers. This formula was used to estimate group size when video was not available. The estimated group sizes were used in the annual abundance estimates.

Aerial video geo-registration using terrain models from dense and coherent stereo matching

In the context of aerial imagery, one of the first steps toward a coherent processing of the information contained in multiple images is geo-registration, which consists in assigning geographic 3D coordinates to the pixels of the image. This enables accurate alignment and geo-positioning of multiple images, detection of moving objects and fusion of data acquired from multiple sensors. To solve this problem there are different approaches that require, in addition to a precise characterization of the camera sensor, high resolution referenced images or terrain elevation models, which are usually not publicly available or out of date. Building upon the idea of developing technology that does not need a reference terrain elevation model, we propose a geo-registration technique that applies variational methods to obtain a dense and coherent surface elevation model that is used to replace the reference model. The surface elevation model is built by interpolation of scattered 3D points, which are obtained in a two-step process following a classical stereo pipeline: first, coherent disparity maps between image pairs of a video sequence are estimated and then image point correspondences are back-projected. The proposed variational method enforces continuity of the disparity map not only along epipolar lines (as done by previous geo-registration techniques) but also across them, in the full 2D image domain. In the experiments, aerial images from synthetic video sequences have been used to validate the proposed technique.

A Novel Framework for Real-time Traffic Flow Parameter Estimation from Aerial Videos

Thesis (Master's)--University of Washington, 2016-06

Tracking-based moving object detection

We present a novel approach for multi-object detection in aerial videos based on tracking. The proposed method mainly involves three steps. Firstly, the spatial-temporal saliency is employed to detect moving objects. Secondly, the detected objects are tracked by mean shift in the subsequent frames. Finally, the saliency results are fused with the weight map generated by tracking to get refined detection results, and in turn the modified detection results are used to update the tracking models. The proposed algorithm is evaluated on VIVID aerial videos, and the results show that our approach can reliably detect moving objects even in challenging situations. Meanwhile, the proposed method can process videos in real time, without the effect of time delay.

Broughton Archipelago Clam Terrace Survey : final report

During a 1995 aerial video survey of the coastline of Johnstone Strait, an unusual shoreline feature was noted and termed “clam terraces” (inset) because of the terrace-type morphology and the apparent association with high clam productivity on the sandflats. Typical alongshore lengths of the terrace ridges are 20-50m, and across-shore widths are typically 20-40m. An area with an especially high density of clam terraces was noted in the Broughton Archipelago, between Broughton and Gilford Islands of southeastern Queen Charlotte Strait. Clam terraces in this area were inventoried from the aerial video imagery to quantify their distribution. The terraces accounted for over 14 km of shoreline and 365 clam terraces were documented. A three-day field survey by a coastal geomorphologist, archeologist and marine biologist was conducted to document the features and determine their origin. Nine clam terraces were surveyed. The field observations confirmed that: the ridges are comprised of boulder/cobblesized material, ridge crests are typically in the range of 1-1.5m above chart datum, sandflats are comprised almost entirely of shell fragments (barnacles and clams) and sandflats have very high shellfish production. There are an abundance of shell middens in the area (over 175) suggesting that the shellfish associated with the terraces were an important food source of aboriginal peoples. The origin of the ridges is unknown; they appear to be a relict feature in that they are not actively being modified by present-day processes. The ridges may be a relict sea-ice feature, although the mechanics of ridge formation is uncertain. Sand accumulates behind the ridge because the supply rate of the shell fragments exceeds the dispersal rate in these low energy environments. The high density areas of clam terraces correspond to high density areas of shell middens, and it is probable that the clam terraces were subjected to some degree of modification by aboriginal shellfish gatherers over the thousands of years of occupation in the region. (Document contains 39 pages)

Abundance of Belugas, Delphinapterus leucas, in Cook Inlet, Alaska, 1994–2000

Annual abundance estimates of belugas, Delphinapterus leucas, in Cook Inlet were calculated from counts made by aerial observers and aerial video recordings. Whale group-size estimates were corrected for subsurface whales (availability bias) and whales that were at the surface but were missed (detection bias). Logistic regression was used to estimate the probability that entire groups were missed during the systematic surveys, and the results were used to calculate a correction to account for the whales in these missed groups (1.015, CV = 0.03 in 1994–98; 1.021, CV = 0.01 in 1999– 2000). Calculated abundances were 653 (CV = 0.43) in 1994, 491 (CV = 0.44) in 1995, 594 (CV = 0.28) in 1996, 440 (CV = 0.14) in 1997, 347 (CV = 0.29) in 1998, 367 (CV = 0.14) in 1999, and 435 (CV = 0.23, 95% CI=279–679) in 2000. For management purposes the current Nbest = 435 and Nmin = 360. These estimates replace preliminary estimates of 749 for 1994 and 357 for 1999. Monte Carlo simulations indicate a 47% probability that from June 1994 to June 1998 abundance of the Cook Inlet stock of belugas was depleted by 50%. The decline appears to have stopped in 1998.

The investigation of vegetation change using remote sensing to detect and monitor migration of landfill gas

Decomposition of domestic wastes in an anaerobic environment results in the production of landfill gas. Public concern about landfill disposal and particularly the production of landfill gas has been heightened over the past decade. This has been due in large to the increased quantities of gas being generated as a result of modern disposal techniques, and also to their increasing effect on modern urban developments. In order to avert diasters, effective means of preventing gas migration are required. This, in turn requires accurate detection and monitoring of gas in the subsurface. Point sampling techniques have many drawbacks, and accurate measurement of gas is difficult. Some of the disadvantages of these techniques could be overcome by assessing the impact of gas on biological systems. This research explores the effects of landfill gas on plants, and hence on the spectral response of vegetation canopies. Examination of the landfill gas/vegetation relationship is covered, both by review of the literature and statistical analysis of field data. The work showed that, although vegetation health was related to landfill gas, it was not possible to define a simple correlation. In the landfill environment, contribution from other variables, such as soil characteristics, frequently confused the relationship. Two sites are investigated in detail, the sites contrasting in terms of the data available, site conditions, and the degree of damage to vegetation. Gas migration at the Panshanger site was dominantly upwards, affecting crops being grown on the landfill cap. The injury was expressed as an overall decline in plant health. Discriminant analysis was used to account for the variations in plant health, and hence the differences in spectral response of the crop canopy, using a combination of soil and gas variables. Damage to both woodland and crops at the Ware site was severe, and could be easily related to the presence of gas. Air photographs, aerial video, and airborne thematic mapper data were used to identify damage to vegetation, and relate this to soil type. The utility of different sensors for this type of application is assessed, and possible improvements that could lead to more widespread use are identified. The situations in which remote sensing data could be combined with ground survey are identified. In addition, a possible methodology for integrating the two approaches is suggested.

Real-time power line extraction from unmanned aerial system video images

In this paper a real-time vision based power line extraction solution is investigated for active UAV guidance. The line extraction algorithm starts from ridge points detected by steerable filters. A collinear line segments fitting algorithm is followed up by considering global and local information together with multiple collinear measurements. GPU boosted algorithm implementation is also investigated in the experiment. The experimental result shows that the proposed algorithm outperforms two baseline line detection algorithms and is able to fitting long collinear line segments. The low computational cost of the algorithm make suitable for real-time applications.

Rapid prototyping framework for visual control of autonomous micro aerial vehicles

Rapid prototyping environments can speed up the research of visual control algorithms. We have designed and implemented a software framework for fast prototyping of visual control algorithms for Micro Aerial Vehicles (MAV). We have applied a combination of a proxy-based network communication architecture and a custom Application Programming Interface. This allows multiple experimental configurations, like drone swarms or distributed processing of a drone's video stream. Currently, the framework supports a low-cost MAV: the Parrot AR.Drone. Real tests have been performed on this platform and the results show comparatively low figures of the extra communication delay introduced by the framework, while adding new functionalities and flexibility to the selected drone. This implementation is open-source and can be downloaded from www.vision4uav.com/?q=VC4MAV-FW

MAVwork : a framework for unified interfacing between micro aerial vehicles and visual controllers

Debugging control software for Micro Aerial Vehicles (MAV) can be risky out of the simulator, especially with professional drones that might harm people around or result in a high bill after a crash. We have designed a framework that enables a software application to communicate with multiple MAVs from a single unified interface. In this way, visual controllers can be first tested on a low-cost harmless MAV and, after safety is guaranteed, they can be moved to the production MAV at no additional cost. The framework is based on a distributed architecture over a network. This allows multiple configurations, like drone swarms or parallel processing of drones' video streams. Live tests have been performed and the results show comparatively low additional communication delays, while adding new functionalities and flexibility. This implementation is open-source and can be downloaded from github.com/uavster/mavwork

Echolocation call production during aerial and terrestrial locomotion by New Zealand's enigmatic lesser short-tailed bat, Mystacina tuberculata

Linkage of echolocation call production with contraction of flight muscles has been suggested to reduce the energetic cost of flight with echolocation, such that the overall cost is approximately equal to that of flight alone. However, the pattern of call production with limb movement in terrestrially agile bats has never been investigated. We used synchronised high-speed video and audio recordings to determine patterns of association between echolocation call production and limb motion by Mystacina tuberculata Gray 1843 as individuals walked and flew, respectively. Results showed that there was no apparent linkage between call production and limb motion when bats walked. When in flight, two calls were produced per wingbeat, late in the downstroke and early in the upstroke. When bats walked, calls were produced at a higher rate, but at a slightly lower intensity, compared with bats in flight. These results suggest that M. tuberculata do not attempt to reduce the cost of terrestrial locomotion and call production through biomechanical linkage. They also suggest that the pattern of linkage seen when bats are in flight is not universal and that energetic savings cannot necessarily be explained by contraction of muscles associated with the downstroke alone.

Unmanned Aerial Vehicles (UAVs) and artificial intelligence revolutionizing wildlife monitoring and conservation

Surveying threatened and invasive species to obtain accurate population estimates is an important but challenging task that requires a considerable investment in time and resources. Estimates using existing ground-based monitoring techniques, such as camera traps and surveys performed on foot, are known to be resource intensive, potentially inaccurate and imprecise, and difficult to validate. Recent developments in unmanned aerial vehicles (UAV), artificial intelligence and miniaturized thermal imaging systems represent a new opportunity for wildlife experts to inexpensively survey relatively large areas. The system presented in this paper includes thermal image acquisition as well as a video processing pipeline to perform object detection, classification and tracking of wildlife in forest or open areas. The system is tested on thermal video data from ground based and test flight footage, and is found to be able to detect all the target wildlife located in the surveyed area. The system is flexible in that the user can readily define the types of objects to classify and the object characteristics that should be considered during classification.

Motion Segmentation And Meanshift Assisted Contour Refinement For Airborne Video

This paper presents methods for moving object detection in airborne video surveillance. The motion segmentation in the above scenario is usually difficult because of small size of the object, motion of camera, and inconsistency in detected object shape etc. Here we present a motion segmentation system for moving camera video, based on background subtraction. An adaptive background building is used to take advantage of creation of background based on most recent frame. Our proposed system suggests CPU efficient alternative for conventional batch processing based background subtraction systems. We further refine the segmented motion by meanshift based mode association.