Applications of real-time machine vision to the control of mining equipment

The mining industry presents us with a number of ideal applications for sensor based machine control because of the unstructured environment that exists within each mine. The aim of the research presented here is to increase the productivity of existing large compliant mining machines by retrofitting with enhanced sensing and control technology. The current research focusses on the automatic control of the swing motion cycle of a dragline and an automated roof bolting system. We have achieved: * closed-loop swing control of an one-tenth scale model dragline; * single degree of freedom closed-loop visual control of an electro-hydraulic manipulator in the lab developed from standard components.

Design and evaluation of decision and control strategies for autonomous vision-based see and avoid systems

This paper details the design and performance assessment of a unique collision avoidance decision and control strategy for autonomous vision-based See and Avoid systems. The general approach revolves around re-positioning a collision object in the image using image-based visual servoing, without estimating range or time to collision. The decision strategy thus involves determining where to move the collision object, to induce a safe avoidance manuever, and when to cease the avoidance behaviour. These tasks are accomplished by exploiting human navigation models, spiral motion properties, expected image feature uncertainty and the rules of the air. The result is a simple threshold based system that can be tuned and statistically evaluated by extending performance assessment techniques derived for alerting systems. Our results demonstrate how autonomous vision-only See and Avoid systems may be designed under realistic problem constraints, and then evaluated in a manner consistent to aviation expectations.

Functional localization of the human color center by decreased water displacement using diffusion-weighted fMRI

Introduction Decreased water displacement following increased neural activity has been observed using diffusion-weighted functional MRI (DfMRI) at high b-values. The physiological mechanisms underlying the diffusion signal change may be unique from the standard blood oxygenation level-dependent (BOLD) contrast and closer to the source of neural activity. Whether DfMRI reflects neural activity more directly than BOLD outside the primary cerebral regions remains unclear. Methods Colored and achromatic Mondrian visual stimuli were statistically contrasted to functionally localize the human color center Area V4 in neurologically intact adults. Spatial and temporal properties of DfMRI and BOLD activation were examined across regions of the visual cortex. Results At the individual level, DfMRI activation patterns showed greater spatial specificity to V4 than BOLD. The BOLD activation patterns were more prominent in the primary visual cortex than DfMRI, where activation was localized to the ventral temporal lobe. Temporally, the diffusion signal change in V4 and V1 both preceded the corresponding hemodynamic response, however the early diffusion signal change was more evident in V1. Conclusions DfMRI may be of use in imaging applications implementing cognitive subtraction paradigms, and where highly precise individual functional localization is required.

Characteristics of on-road driving by persons with central vision loss: Learning to drive with a bioptic telescope

There is limited research on the driving performance and safety of bioptic drivers and even less regarding the driving skills that are most challenging for those learning to drive with bioptic telescopes. This research consisted of case studies of five trainee bioptic drivers whose driving skills were compared with those of a group of licensed bioptic drivers (n = 23) while they drove along city, suburban, and controlled-access highways in an instrumented dual-brake vehicle. A certified driver rehabilitation specialist was positioned in the front passenger seat to monitor safety and two backseat evaluators independently rated driving using a standardized scoring system. Other aspects of performance were assessed through vehicle instrumentation and video recordings. Results demonstrate that while sign recognition, lane keeping, steering steadiness, gap judgments and speed choices were significantly worse in trainees, some driving behaviors and skills, including pedestrian detection and traffic light recognition were not significantly different to those of the licensed drivers. These data provide useful insights into the skill challenges encountered by a small sample of trainee bioptic drivers which, while not generalizable because of the small sample size, provide valuable insights beyond that of previous studies and can be used as a basis to guide training strategies.

From vision to actions: Towards adaptive and autonomous humanoid robots

Although robotics research has seen advances over the last decades robots are still not in widespread use outside industrial applications. Yet a range of proposed scenarios have robots working together, helping and coexisting with humans in daily life. In all these a clear need to deal with a more unstructured, changing environment arises. I herein present a system that aims to overcome the limitations of highly complex robotic systems, in terms of autonomy and adaptation. The main focus of research is to investigate the use of visual feedback for improving reaching and grasping capabilities of complex robots. To facilitate this a combined integration of computer vision and machine learning techniques is employed. From a robot vision point of view the combination of domain knowledge from both imaging processing and machine learning techniques, can expand the capabilities of robots. I present a novel framework called Cartesian Genetic Programming for Image Processing (CGP-IP). CGP-IP can be trained to detect objects in the incoming camera streams and successfully demonstrated on many different problem domains. The approach requires only a few training images (it was tested with 5 to 10 images per experiment) is fast, scalable and robust yet requires very small training sets. Additionally, it can generate human readable programs that can be further customized and tuned. While CGP-IP is a supervised-learning technique, I show an integration on the iCub, that allows for the autonomous learning of object detection and identification. Finally this dissertation includes two proof-of-concepts that integrate the motion and action sides. First, reactive reaching and grasping is shown. It allows the robot to avoid obstacles detected in the visual stream, while reaching for the intended target object. Furthermore the integration enables us to use the robot in non-static environments, i.e. the reaching is adapted on-the- fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. The second integration highlights the capabilities of these frameworks, by improving the visual detection by performing object manipulation actions.

A Bayesian Framework for the Assessment of Vision-based Weed and Fruit Detection and Classification Algorithms

This paper proposes new metrics and a performance-assessment framework for vision-based weed and fruit detection and classification algorithms. In order to compare algorithms, and make a decision on which one to use fora particular application, it is necessary to take into account that the performance obtained in a series of tests is subject to uncertainty. Such characterisation of uncertainty seems not to be captured by the performance metrics currently reported in the literature. Therefore, we pose the problem as a general problem of scientific inference, which arises out of incomplete information, and propose as a metric of performance the(posterior) predictive probabilities that the algorithms will provide a correct outcome for target and background detection. We detail the framework through which these predicted probabilities can be obtained, which is Bayesian in nature. As an illustration example, we apply the framework to the assessment of performance of four algorithms that could potentially be used in the detection of capsicums (peppers).

A Survey of autonomous vision-based See and Avoid for Unmanned Aircraft Systems

This paper provides a comprehensive review of the vision-based See and Avoid problem for unmanned aircraft. The unique problem environment and associated constraints are detailed, followed by an in-depth analysis of visual sensing limitations. In light of such detection and estimation constraints, relevant human, aircraft and robot collision avoidance concepts are then compared from a decision and control perspective. Remarks on system evaluation and certification are also included to provide a holistic review approach. The intention of this work is to clarify common misconceptions, realistically bound feasible design expectations and offer new research directions. It is hoped that this paper will help us to unify design efforts across the aerospace and robotics communities.

Affective sensing + wearable imaging: Vision impairment and implications for spatial enquiry

Recent interest in affect and the body have mobilized a contemporary review of aesthetics and phenomenology within architecture to unpack how environments affect spatial experience. Emerging spatial studies within the neuro-sciences, and their implications for architectural research as raised by architectural theorists Juhani Pallasmaa (2014) and Harry Mallgrave (2013) has been well supported by a raft of scientists and institutions including the prestigious Salk Institute. Although there has been some headway in spatial studies of the vision impaired (Cattaneo et al, 2011) to understand the role of their non-visual systems in assisting navigation and location, little is discussed in terms of their other abilities in sensing particular qualities of space which impinge upon emotion. This paper reviews a collection of studies exploring face vision and echo-location, amongst others, which provide insight into what might be termed affective perception of the vision impaired, and how further interplay between this research and the architectural field can contribute new knowledge regarding space and affect. By engaging with themes from the Aesthetics, Phenomenology and indeed Neuro-science fields, the paper provides background of current and potential cross disciplinary research, and highlights the role wearable technologies can play in enhancing knowledge of affective spatial experience.

Distributed Robotic Vision as a service

The vision sense of standalone robots is limited by line of sight and onboard camera capabilities, but processing video from remote cameras puts a high computational burden on robots. This paper describes the Distributed Robotic Vision Service, DRVS, which implements an on-demand distributed visual object detection service. Robots specify visual information requirements in terms of regions of interest and object detection algorithms. DRVS dynamically distributes the object detection computation to remote vision systems with processing capabilities, and the robots receive high-level object detection information. DRVS relieves robots of managing sensor discovery and reduces data transmission compared to image sharing models of distributed vision. Navigating a sensorless robot from remote vision systems is demonstrated in simulation as a proof of concept.

Retinal photography screening programs to prevent vision loss from diabetic retinopathy in rural and urban Australia: A review

Purpose This review assessed the effectiveness of diabetic retinopathy (DR) screening programs, using retinal photography in Australian urban and rural settings, and considered implications for public health strategy and policy. Methods An electronic search of MEDLINE, PubMed, and Embase for studies published between 1 January 1996 and the 30 June 2013 was undertaken. Key search terms were “diabetic retinopathy,” “screening,” “retinal photography” and “Australia.” Results Twelve peer-reviewed publications were identified. The 14 DR screening programs identified from the 12 publications were successfully undertaken in urban, rural and remote communities across Australia. Locations included a pathology collection center, and Indigenous primary health care and Aboriginal community controlled organizations. Each intervention using retinal photography was highly effective at increasing the number of people who underwent screening for DR. The review identified that prior to commencement of the screening programs a median of 48% (range 16–85%) of those screened had not undergone a retinal examination within the recommended time frame (every year for Indigenous people and every 2 years for non-Indigenous people in Australia). A median of 16% (range 0–45%) of study participants had evidence of DR. Conclusions This review has shown there have been many pilot and demonstration projects in rural and urban Australia that confirm the effectiveness of retinal photography-based screening for DR

Regenerative sustainability for the built environment – from vision to reality: An introductory chapter

Regenerative sustainability is emerging as an alternative discourse around the transition from a ‘mechanistic’ to an ‘ecological’ or living systems worldview. This view helps us to re-conceptualize relationships among humans’ technological, ecological, economic, social and political systems. Through exploration of ‘net positive’ or ‘regenerative’ development lenses and the traditional sustainability literature, the conceptualization and approaches to achieve sustainable development and ecological modernization are expanded to articulate and to explore the evolving sustainability discourse, ‘regenerative sustainability’. This Special Volume of Journal of Cleaner Production (SV) is focused upon various dimensions of regenerative sustainability (e.g. regenerative design, regenerative development, and positive development) applied to the urban built environment at scales, which range from individual buildings, neighborhoods, and urban developments to integrated regional sustainable development. The main focus is on how these approaches and developments are evolving, how they can help us to prevent or adapt to climate change and how these approaches are likely to evolve in the next two to three decades. These approaches are addressed in four themes: (1) reviewing the theoretical development of the discourse of regenerative sustainability, its emerging principles and practices, (2) explaining how it can be measured and monitored, (3) providing encouraging practical pathways and examples of its implementation in multiple cultural and climatic contexts, and (4) mapping obstacles and enablers that must be addressed to help to ensure that more rapid progress is made in implementing the transitions towards an urban built environment that supports genuinely sustainable societies.

A bottom-up integration of vision and actions to create cognitive humanoids

In recent years more and more complex humanoid robots have been developed. On the other hand programming these systems has become more difficult. There is a clear need for such robots to be able to adapt and perform certain tasks autonomously, or even learn by themselves how to act. An important issue to tackle is the closing of the sensorimotor loop. Especially when talking about humanoids the tight integration of perception with actions will allow for improved behaviours, embedding adaptation on the lower-level of the system.

Autonomous UAVs wildlife detection using thermal imaging, predictive navigation and computer vision

There is an increased interest on the use of Unmanned Aerial Vehicles (UAVs) for wildlife and feral animal monitoring around the world. This paper describes a novel system which uses a predictive dynamic application that places the UAV ahead of a user, with a low cost thermal camera, a small onboard computer that identifies heat signatures of a target animal from a predetermined altitude and transmits that target’s GPS coordinates. A map is generated and various data sets and graphs are displayed using a GUI designed for easy use. The paper describes the hardware and software architecture and the probabilistic model for downward facing camera for the detection of an animal. Behavioral dynamics of target movement for the design of a Kalman filter and Markov model based prediction algorithm are used to place the UAV ahead of the user. Geometrical concepts and Haversine formula are applied to the maximum likelihood case in order to make a prediction regarding a future state of the user, thus delivering a new way point for autonomous navigation. Results show that the system is capable of autonomously locating animals from a predetermined height and generate a map showing the location of the animals ahead of the user.

Towards vision-based deep reinforcement learning for robotic motion control

This paper introduces a machine learning based system for controlling a robotic manipulator with visual perception only. The capability to autonomously learn robot controllers solely from raw-pixel images and without any prior knowledge of configuration is shown for the first time. We build upon the success of recent deep reinforcement learning and develop a system for learning target reaching with a three-joint robot manipulator using external visual observation. A Deep Q Network (DQN) was demonstrated to perform target reaching after training in simulation. Transferring the network to real hardware and real observation in a naive approach failed, but experiments show that the network works when replacing camera images with synthetic images.