Real-time vision, tracking and control

This paper, which serves as an introduction to the mini-symposium on Real-Time Vision, Tracking and Control, provides a broad sketch of visual servoing, the application of real-time vision, tracking and control for robot guidance. It outlines the basic theoretical approaches to the problem, describes a typical architecture, and discusses major milestones, applications and the significant vision sub-problems that must be solved.

Autonomous excavation using a rope shovel

This paper describes automation of the digging cycle of a mining rope shovel which considers autonomous dipper (bucket) filling and determining methods to detect when to disengage the dipper from the bank. Novel techniques to overcome dipper stall and the online estimation of dipper "fullness" are described with in-field experimental results of laser DTM generation, machine automation and digging using a 1/7th scale model rope shovel presented. © 2006 Wiley Periodicals, Inc.

Data muling over underwater wireless sensor networks using an autonomous underwater vehicle

We present algorithms, systems, and experimental results for underwater data muling. In data muling a mobile agent interacts with static agents to upload, download, or transport data to a different physical location. We consider a system comprising an Autonomous Underwater Vehicle (AUV) and many static Underwater Sensor Nodes (USN) networked together optically and acoustically. The AUV can locate the static nodes using vision and hover above the static nodes for data upload. We describe the hardware and software architecture of this underwater system, as well as experimental data. © 2006 IEEE.

Demonstration of image compression in a low-bandwidth wireless camera network

This paper presents an overview of our demonstration of a low-bandwidth, wireless camera network where image compression is undertaken at each node. We briefly introduce the Fleck hardware platform we have developed as well as describe the image compression algorithm which runs on individual nodes. The demo will show real-time image data coming back to base as individual camera nodes are added to the network. Copyright 2007 ACM.

Krill: An exploration in underwater sensor networks

While sensor networks have now become very popular on land, the underwater environment still poses some difficult problems. Communication is one of the difficult challenges under water. There are two options: optical and acoustic. We have designed an optical communication board that allows the Fleck’s to communicate optically. We have tested the resulting underwater sensor nodes in two different applications.

Real-time image streaming over a low-bandwidth wireless camera network

In this paper we describe the recent development of a low-bandwidth wireless camera sensor network. We propose a simple, yet effective, network architecture which allows multiple cameras to be connected to the network and synchronize their communication schedules. Image compression of greater than 90% is performed at each node running on a local DSP coprocessor, resulting in nodes using 1/8th the energy compared to streaming uncompressed images. We briefly introduce the Fleck wireless node and the DSP/camera sensor, and then outline the network architecture and compression algorithm. The system is able to stream color QVGA images over the network to a base station at up to 2 frames per second. © 2007 IEEE.

Virtual fences for controlling cows

We describe a moving virtual fence algorithm for herding cows. Each animal in the herd is given a smart collar consisting of a GPS, PDA, wireless networking and a sound amplifier. Using the GPS, the animal's location can be verified relative to the fence boundary. When approaching the perimeter, the animal is presented with a sound stimulus whose effect is to move away. We have developed the virtual fence control algorithm for moving a herd. We present simulation results and data from experiments with 8 cows equipped with smart collars.

Autonomous deployment and repair of a sensor network using an unmanned aerial vehicle

We describe a sensor network deployment method using autonomous flying robots. Such networks are suitable for tasks such as large-scale environmental monitoring or for command and control in emergency situations. We describe in detail the algorithms used for deployment and for measuring network connectivity and provide experimental data we collected from field trials. A particular focus is on determining gaps in connectivity of the deployed network and generating a plan for a second, repair, pass to complete the connectivity. This project is the result of a collaboration between three robotics labs (CSIRO, USC, and Dartmouth.).

Omnidirectional visual odometry for a planetary rover

Position estimation for planetary rovers has been typically limited to odometry based on proprioceptive measurements such as the integration of distance traveled and measurement of heading change. Here we present and compare two methods of online visual odometry suited for planetary rovers. Both methods use omnidirectional imagery to estimate motion of the rover. One method is based on robust estimation of optical flow and subsequent integration of the flow. The second method is a full structure-from-motion solution. To make the comparison meaningful we use the same set of raw corresponding visual features for each method. The dataset is an sequence of 2000 images taken during a field experiment in the Atacama desert, for which high resolution GPS ground truth is available.

Vision-based detection and tracking of aerial targets for UAV collision avoidance

Machine vision represents a particularly attractive solution for sensing and detecting potential collision-course targets due to the relatively low cost, size, weight, and power requirements of the sensors involved (as opposed to radar). This paper describes the development and evaluation of a vision-based collision detection algorithm suitable for fixed-wing aerial robotics. The system was evaluated using highly realistic vision data of the moments leading up to a collision. Based on the collected data, our detection approaches were able to detect targets at distances ranging from 400m to about 900m. These distances (with some assumptions about closing speeds and aircraft trajectories) translate to an advanced warning of between 8-10 seconds ahead of impact, which approaches the 12.5 second response time recommended for human pilots. We make use of the enormous potential of graphic processing units to achieve processing rates of 30Hz (for images of size 1024-by- 768). Currently, integration in the final platform is under way.

Automated centralised separation management with onboard decision support

This paper proposes a novel automated separation management concept in which onboard decision support is integrated within a centralised air traffic separation management system. The onboard decision support system involves a decentralised separation manager that can overrule air traffic management instructions under certain circumstances. This approach allows the advantages of both centralised and decentralised concepts to be combined (and disadvantages of each separation management approach to be mitigated). Simulation studies are used to illustrate the potential benefits of the combined separation management concept.

Guided chaos : path planning and control for a UAV-forced landing

In recent years, unmanned aerial vehicles (UAVs) have been widely used in combat, and their potential applications in civil and commercial roles are also receiving considerable attention by industry and the research community. There are numerous published reports of UAVs used in Earth science missions [1], fire-fighting [2], and border security [3] trials, with other speculative deployments, including applications in agriculture, communications, and traffic monitoring. However, none of these UAVs can demonstrate an equivalent level of safety to manned aircraft, particularly in the case of an engine failure, which would require an emergency or forced landing. This may be arguably the main factor that has prevented these UAV trials from becoming full-scale commercial operations, as well as restricted operations of civilian UAVs to only within segregated airspace.

Airborne vision-based collision-detection system

Machine vision represents a particularly attractive solution for sensing and detecting potential collision-course targets due to the relatively low cost, size, weight, and power requirements of vision sensors (as opposed to radar and TCAS). This paper describes the development and evaluation of a real-time vision-based collision detection system suitable for fixed-wing aerial robotics. Using two fixed-wing UAVs to recreate various collision-course scenarios, we were able to capture highly realistic vision (from an onboard camera perspective) of the moments leading up to a collision. This type of image data is extremely scarce and was invaluable in evaluating the detection performance of two candidate target detection approaches. Based on the collected data, our detection approaches were able to detect targets at distances ranging from 400m to about 900m. These distances (with some assumptions about closing speeds and aircraft trajectories) translate to an advanced warning of between 8-10 seconds ahead of impact, which approaches the 12.5 second response time recommended for human pilots. We overcame the challenge of achieving real-time computational speeds by exploiting the parallel processing architectures of graphics processing units found on commercially-off-the-shelf graphics devices. Our chosen GPU device suitable for integration onto UAV platforms can be expected to handle real-time processing of 1024 by 768 pixel image frames at a rate of approximately 30Hz. Flight trials using manned Cessna aircraft where all processing is performed onboard will be conducted in the near future, followed by further experiments with fully autonomous UAV platforms.

A biomechanical study of top screw pullout in anterior scoliosis correction constructs

Study Design: Biomechanical testing of vertebral body screw pullout resistance with relevance to top screw pullout in endoscopic anterior scoliosis constructs. Objectives: To analyse the effect of screw positioning and angulation on pullout resistance of vertebral body screws, where the pullout takes place along a curved path as occurs in anterior scoliosis constructs. Summary of Background Data: Top screw pullout is a significant clinical problem in endoscopic anterior scoliosis surgery, with rates of up to 18% reported in the literature. Methods: A custom designed biomechanical test rig was used to perform pullout tests of Medtronic anterior vertebral screws where the pullout occurred along an arc of known radius. Using synthetic bone blocks, a range of pullout radii and screw angulations were tested, in order to determine an ‘optimal’ configuration. The optimal configuration was then compared with standard screw positioning using a series of tests on ovine vertebrae (n=29). Results: Screw angulation has a small but significant effect on pullout resistance, with maximum strength being achieved at 10 degree cephalad angulation. Combining 10 degree cephalad angulation with maximal spacing between the top two screws (maximum pullout radius) increased the pullout resistance by 88% compared to ‘standard’ screw positioning (screws inserted perpendicular to rod at mid-body height). Conclusions: The positioning of the top screw in anterior scoliosis constructs can significantly alter its pullout resistance.

The relationship between deformity correction and clinical outcomes after thoracoscopic scoliosis surgery : a prospective series of 100 patients

The relationship between deformity correction and self-reported patient satisfaction after thoracoscopic anterior scoliosis surgery is unknown. Scoliosis Research Society questionnaire scores, radiographic outcomes, and rib hump correction were prospectively assessed for a group of 100 patients pre-operatively and at two years after surgery. Patients with lower post-op major Cobb angles report significantly higher SRS scores than patients with higher post-op Cobb angles.