Vision-based operations of a large industrial vehicle : autonomous hot metal carrier

Hot metal carriers (HMCs) are large forklift-type vehicles used to move molten metal in aluminum smelters. This paper reports on field experiments that demonstrate that HMCs can operate autonomously and in particular can use vision as a primary sensor to locate the load of aluminum. We present our complete system but focus on the vision system elements and also detail experiments demonstrating reliable operation of the materials handling task. Two key experiments are described, lasting 2 and 5 h, in which the HMC traveled 15 km in total and handled the load 80 times.

Improving robot vision models for object detection through interaction

We propose a method for learning specific object representations that can be applied (and reused) in visual detection and identification tasks. A machine learning technique called Cartesian Genetic Programming (CGP) is used to create these models based on a series of images. Our research investigates how manipulation actions might allow for the development of better visual models and therefore better robot vision. This paper describes how visual object representations can be learned and improved by performing object manipulation actions, such as, poke, push and pick-up with a humanoid robot. The improvement can be measured and allows for the robot to select and perform the `right' action, i.e. the action with the best possible improvement of the detector.

Vision-based control for mining automation

The mining industry is highly suitable for the application of robotics and automation technology since the work is both arduous and dangerous. Visual servoing is a means of integrating noncontact visual sensing with machine control to augment or replace operator based control. This article describes two of our current mining automation projects in order to demonstrate some, perhaps unusual, applications of visual servoing, and also to illustrate some very real problems with robust computer vision

Special issue on robot vision

The International Journal of Robotics Research (IJRR) has a long history of publishing the state-of-the-art in the field of robotic vision. This is the fourth special issue devoted to the topic. Previous special issues were published in 2012 (Volume 31, No. 4), 2010 (Volume 29, Nos 2–3) and 2007 (Volume 26, No. 7, jointly with the International Journal of Computer Vision). In a closely related field was the special issue on Visual Servoing published in IJRR, 2003 (Volume 22, Nos 10–11). These issues nicely summarize the highlights and progress of the past 12 years of research devoted to the use of visual perception for robotics.

A roadmap for interpreting the literature on vision and driving

Over the past several decades there has been a sharp increase in the number of studies focused on the relationship between vision and driving. The intensified attention to this topic has most likely been stimulated by the lack of an evidence basis for determining vision standards for driving licensure and a poor understanding about how vision impairment impacts driver safety and performance. Clinicians depend on the literature on vision and driving to advise visually impaired patients appropriately about driving fitness. Policy makers also depend on the scientific literature in order to develop guidelines that are evidence-based and are thus fair to persons who are visually impaired. Thus it is important for clinicians and policy makers alike to understand how various study designs and measurement methods should be interpreted so that the conclusions and recommendations they make are not overly broad, too narrowly constrained, or even misguided. We offer a methodological framework to guide interpretations of studies on vision and driving that can also serve as a heuristic for researchers in the area. Here, we discuss research designs and general measurement methods for the study of vision as they relate to driver safety, driver performance, and driver-centered (self-reported) outcomes.

Fear of falling in vision impairment

Falls are the leading cause of injury-related morbidity and mortality among older adults. In addition to the resulting physical injury and potential disability after a fall, there are also important psychological consequences, including depression, anxiety, activity restriction, and fear of falling. Fear of falling affects 20 to 43% of community-dwelling older adults and is not limited to those who have previously experienced a fall. About half of older adults who experience fear of falling subsequently restrict their physical and everyday activities, which can lead to functional decline, depression, increased falls risk, and reduced quality of life. Although there is clear evidence that older adults with visual impairment have higher falls risk, only a limited number of studies have investigated fear of falling in older adults with visual impairment and the findings have been mixed. Recent studies suggest increased levels of fear of falling among older adults with various eye conditions, including glaucoma and age-related macular degeneration, whereas other studies have failed to find differences. Interventions, which are still in their infancy in the general population, are also largely unexplored in those with visual impairment. The major aims of this review were to provide an overview of the literature on fear of falling, its measurement, and risk factors among older populations, with specific focus on older adults with visual impairment, and to identify directions for future research in this area.

Applications of real-time machine vision to the control of mining equipment

The mining industry presents us with a number of ideal applications for sensor based machine control because of the unstructured environment that exists within each mine. The aim of the research presented here is to increase the productivity of existing large compliant mining machines by retrofitting with enhanced sensing and control technology. The current research focusses on the automatic control of the swing motion cycle of a dragline and an automated roof bolting system. We have achieved: * closed-loop swing control of an one-tenth scale model dragline; * single degree of freedom closed-loop visual control of an electro-hydraulic manipulator in the lab developed from standard components.

Design and evaluation of decision and control strategies for autonomous vision-based see and avoid systems

This paper details the design and performance assessment of a unique collision avoidance decision and control strategy for autonomous vision-based See and Avoid systems. The general approach revolves around re-positioning a collision object in the image using image-based visual servoing, without estimating range or time to collision. The decision strategy thus involves determining where to move the collision object, to induce a safe avoidance manuever, and when to cease the avoidance behaviour. These tasks are accomplished by exploiting human navigation models, spiral motion properties, expected image feature uncertainty and the rules of the air. The result is a simple threshold based system that can be tuned and statistically evaluated by extending performance assessment techniques derived for alerting systems. Our results demonstrate how autonomous vision-only See and Avoid systems may be designed under realistic problem constraints, and then evaluated in a manner consistent to aviation expectations.

Characteristics of on-road driving by persons with central vision loss: Learning to drive with a bioptic telescope

There is limited research on the driving performance and safety of bioptic drivers and even less regarding the driving skills that are most challenging for those learning to drive with bioptic telescopes. This research consisted of case studies of five trainee bioptic drivers whose driving skills were compared with those of a group of licensed bioptic drivers (n = 23) while they drove along city, suburban, and controlled-access highways in an instrumented dual-brake vehicle. A certified driver rehabilitation specialist was positioned in the front passenger seat to monitor safety and two backseat evaluators independently rated driving using a standardized scoring system. Other aspects of performance were assessed through vehicle instrumentation and video recordings. Results demonstrate that while sign recognition, lane keeping, steering steadiness, gap judgments and speed choices were significantly worse in trainees, some driving behaviors and skills, including pedestrian detection and traffic light recognition were not significantly different to those of the licensed drivers. These data provide useful insights into the skill challenges encountered by a small sample of trainee bioptic drivers which, while not generalizable because of the small sample size, provide valuable insights beyond that of previous studies and can be used as a basis to guide training strategies.

From vision to actions: Towards adaptive and autonomous humanoid robots

Although robotics research has seen advances over the last decades robots are still not in widespread use outside industrial applications. Yet a range of proposed scenarios have robots working together, helping and coexisting with humans in daily life. In all these a clear need to deal with a more unstructured, changing environment arises. I herein present a system that aims to overcome the limitations of highly complex robotic systems, in terms of autonomy and adaptation. The main focus of research is to investigate the use of visual feedback for improving reaching and grasping capabilities of complex robots. To facilitate this a combined integration of computer vision and machine learning techniques is employed. From a robot vision point of view the combination of domain knowledge from both imaging processing and machine learning techniques, can expand the capabilities of robots. I present a novel framework called Cartesian Genetic Programming for Image Processing (CGP-IP). CGP-IP can be trained to detect objects in the incoming camera streams and successfully demonstrated on many different problem domains. The approach requires only a few training images (it was tested with 5 to 10 images per experiment) is fast, scalable and robust yet requires very small training sets. Additionally, it can generate human readable programs that can be further customized and tuned. While CGP-IP is a supervised-learning technique, I show an integration on the iCub, that allows for the autonomous learning of object detection and identification. Finally this dissertation includes two proof-of-concepts that integrate the motion and action sides. First, reactive reaching and grasping is shown. It allows the robot to avoid obstacles detected in the visual stream, while reaching for the intended target object. Furthermore the integration enables us to use the robot in non-static environments, i.e. the reaching is adapted on-the- fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. The second integration highlights the capabilities of these frameworks, by improving the visual detection by performing object manipulation actions.

A Bayesian Framework for the Assessment of Vision-based Weed and Fruit Detection and Classification Algorithms

This paper proposes new metrics and a performance-assessment framework for vision-based weed and fruit detection and classification algorithms. In order to compare algorithms, and make a decision on which one to use fora particular application, it is necessary to take into account that the performance obtained in a series of tests is subject to uncertainty. Such characterisation of uncertainty seems not to be captured by the performance metrics currently reported in the literature. Therefore, we pose the problem as a general problem of scientific inference, which arises out of incomplete information, and propose as a metric of performance the(posterior) predictive probabilities that the algorithms will provide a correct outcome for target and background detection. We detail the framework through which these predicted probabilities can be obtained, which is Bayesian in nature. As an illustration example, we apply the framework to the assessment of performance of four algorithms that could potentially be used in the detection of capsicums (peppers).

Stochastic growth models for analyzing crustacean data

The contemporary methodology for growth models of organisms is based on continuous trajectories and thus it hinders us from modelling stepwise growth in crustacean populations. Growth models for fish are normally assumed to follow a continuous function, but a different type of model is needed for crustacean growth. Crustaceans must moult in order for them to grow. The growth of crustaceans is a discontinuous process due to the periodical shedding of the exoskeleton in moulting. The stepwise growth of crustaceans through the moulting process makes the growth estimation more complex. Stochastic approaches can be used to model discontinuous growth or what are commonly known as "jumps" (Figure 1). However, in stochastic growth model we need to ensure that the stochastic growth model results in only positive jumps. In view of this, we will introduce a subordinator that is a special case of a Levy process. A subordinator is a non-decreasing Levy process, that will assist in modelling crustacean growth for better understanding of the individual variability and stochasticity in moulting periods and increments. We develop the estimation methods for parameter estimation and illustrate them with the help of a dataset from laboratory experiments. The motivational dataset is from the ornate rock lobster, Panulirus ornatus, which can be found between Australia and Papua New Guinea. Due to the presence of sex effects on the growth (Munday et al., 2004), we estimate the growth parameters separately for each sex. Since all hard parts are shed too often, the exact age determination of a lobster can be challenging. However, the growth parameters for the aforementioned moult processes from tank data being able to estimate through: (i) inter-moult periods, and (ii) moult increment. We will attempt to derive a joint density, which is made up of two functions: one for moult increments and the other for time intervals between moults. We claim these functions are conditionally independent given pre-moult length and the inter-moult periods. The variables moult increments and inter-moult periods are said to be independent because of the Markov property or conditional probability. Hence, the parameters in each function can be estimated separately. Subsequently, we integrate both of the functions through a Monte Carlo method. We can therefore obtain a population mean for crustacean growth (e. g. red curve in Figure 1). [GRAPHICS]

A Survey of autonomous vision-based See and Avoid for Unmanned Aircraft Systems

This paper provides a comprehensive review of the vision-based See and Avoid problem for unmanned aircraft. The unique problem environment and associated constraints are detailed, followed by an in-depth analysis of visual sensing limitations. In light of such detection and estimation constraints, relevant human, aircraft and robot collision avoidance concepts are then compared from a decision and control perspective. Remarks on system evaluation and certification are also included to provide a holistic review approach. The intention of this work is to clarify common misconceptions, realistically bound feasible design expectations and offer new research directions. It is hoped that this paper will help us to unify design efforts across the aerospace and robotics communities.

Isolation and expression analysis of multiple isoforms of putative farnesoic acid O-methyltransferase in several crustacean species

Farnesoic acid O-methyltransferase (FaMeT) is the enzyme responsible for the conversion of farnesoic acid (FA) to methyl farnesoate (MF) in the final step of MF synthesis. Multiple isoforms of putative FaMeT were isolated from six crustacean species belonging to the families Portunidae, Penaeidae, Scyllaridae and Parastacidae. The portunid crabs Portunus pelagicus and Scylla serrata code for three forms: short, intermediate and long. Two isoforms (short and long) were isolated from the penaeid prawns Penaeus monodon and Fenneropenaeus merguiensis. Two isoforms were also identified in the scyllarid Thenus orientalis and parastacid Cherax quadricarinatus. Putative FaMeT sequences were also amplified from the genomic DNA of P. pelagicus and compared to the putative FaMeT transcripts expressed. Each putative FaMeT cDNA isoform was represented in the genomic DNA, indicative of a multi-gene family. Various tissues from P. pelagicus were individually screened for putative FaMeT expression using PCR and fragment analysis. Each tissue type expressed all three isoforms of putative FaMeT irrespective of sex or moult stage. Protein domain analysis revealed the presence of a deduced casein kinase II phosphorylation site present only in the long isoform of putative FaMeT.