Robotic explosive charging in mining and construction applications

Describes the development and testing of a robotic system for charging blast holes in underground mining. The automation system supports four main tactical functions: detection of blast holes; teleoperated arm pose control; automatic arm pose control; and human-in-the-loop visual servoing. We present the system architecture, and analyse the major components, Hole detection is crucial for automating the process, and we discuss theoretical and practical aspects in detail. The sensors used are laser range finders and cameras installed in the end effector. For automatic insertion, we consider image processing techniques to support visual servoing the tool to the hole. We also discuss issues surrounding the control of heavy-duty mining manipulators, in particular, friction, stiction, and actuator saturation.

Assessing driver acceptance of intelligent transport systems in the context of railway level crossings

Intelligent Transport Systems (ITS) have the potential to substantially reduce the number of crashes caused by human errors at railway levels crossings. Such systems, however, will only exert an influence on driving behaviour if they are accepted by the driver. This study aimed at assessing driver acceptance of different ITS interventions designed to enhance driver behaviour at railway crossings. Fifty eight participants, divided into three groups, took part in a driving simulator study in which three ITS devices were tested: an in-vehicle visual ITS, an in-vehicle audio ITS, and an on-road valet system. Driver acceptance of each ITS intervention was assessed in a questionnaire guided by the Technology Acceptance Model and the Theory of Planned Behaviour. Overall, results indicated that the strongest intentions to use the ITS devices belonged to participants exposed to the road-based valet system at passive crossings. The utility of both models in explaining drivers’ intention to use the systems is discussed, with results showing greater support for the Theory of Planned Behaviour. Directions for future studies, along with strategies that target attitudes and subjective norms to increase drivers’ behavioural intentions, are also discussed.

Vision-based control for mining automation

The mining industry is highly suitable for the application of robotics and automation technology since the work is both arduous and dangerous. Visual servoing is a means of integrating noncontact visual sensing with machine control to augment or replace operator based control. This article describes two of our current mining automation projects in order to demonstrate some, perhaps unusual, applications of visual servoing, and also to illustrate some very real problems with robust computer vision

Special issue on robot vision

The International Journal of Robotics Research (IJRR) has a long history of publishing the state-of-the-art in the field of robotic vision. This is the fourth special issue devoted to the topic. Previous special issues were published in 2012 (Volume 31, No. 4), 2010 (Volume 29, Nos 2–3) and 2007 (Volume 26, No. 7, jointly with the International Journal of Computer Vision). In a closely related field was the special issue on Visual Servoing published in IJRR, 2003 (Volume 22, Nos 10–11). These issues nicely summarize the highlights and progress of the past 12 years of research devoted to the use of visual perception for robotics.

A simulator evaluation of effects of assistive technologies on driver cognitive load at railway level crossings

Intelligent Transport Systems (ITS) have the potential to substantially reduce the number of crashes caused by human errors at railway levels crossings. However, such systems could overwhelm drivers, generate different types of driver errors and have negative effects on safety at level crossing. The literature shows an increasing interest for new ITS for increasing driver situational awareness at level crossings, as well as evaluations of such new systems on compliance. To our knowledge, the potential negative effects of such technologies have not been comprehensively evaluated yet. This study aimed at assessing the effect of different ITS interventions, designed to enhance driver behaviour at railway crossings, on driver’s cognitive loads. Fifty eight participants took part in a driving simulator study in which three ITS devices were tested: an in-vehicle visual ITS, an in-vehicle audio ITS, and an on-road valet system. Driver cognitive load was objectively and subjectively assessed for each ITS intervention. Objective data were collected from a heart rate monitor and an eye tracker, while subjective data was collected with the NASA-TLX questionnaire. Overall, results indicated that the three trialled technologies did not result in significant changes in cognitive load while approaching crossings.

Design and evaluation of decision and control strategies for autonomous vision-based see and avoid systems

This paper details the design and performance assessment of a unique collision avoidance decision and control strategy for autonomous vision-based See and Avoid systems. The general approach revolves around re-positioning a collision object in the image using image-based visual servoing, without estimating range or time to collision. The decision strategy thus involves determining where to move the collision object, to induce a safe avoidance manuever, and when to cease the avoidance behaviour. These tasks are accomplished by exploiting human navigation models, spiral motion properties, expected image feature uncertainty and the rules of the air. The result is a simple threshold based system that can be tuned and statistically evaluated by extending performance assessment techniques derived for alerting systems. Our results demonstrate how autonomous vision-only See and Avoid systems may be designed under realistic problem constraints, and then evaluated in a manner consistent to aviation expectations.

Featureless vehicle-based visual SLAM with a consumer camera

The Simultaneous Localisation And Mapping (SLAM) problem is one of the major challenges in mobile robotics. Probabilistic techniques using high-end range finding devices are well established in the field, but recent work has investigated vision-only approaches. We present an alternative approach to the leading existing techniques, which extracts approximate rotational and translation velocity information from a vehicle-mounted consumer camera, without tracking landmarks. When coupled with an existing SLAM system, the vision module is able to map a 45 metre long indoor loop and a 1.6 km long outdoor road loop, without any parameter or system adjustment between tests. The work serves as a promising pilot study into ground-based vision-only SLAM, with minimal geometric interpretation of the environment.

Lip detection for audio-visual speech recognition in-car environment

Acoustically, car cabins are extremely noisy and as a consequence audio-only, in-car voice recognition systems perform poorly. As the visual modality is immune to acoustic noise, using the visual lip information from the driver is seen as a viable strategy in circumventing this problem by using audio visual automatic speech recognition (AVASR). However, implementing AVASR requires a system being able to accurately locate and track the drivers face and lip area in real-time. In this paper we present such an approach using the Viola-Jones algorithm. Using the AVICAR [1] in-car database, we show that the Viola- Jones approach is a suitable method of locating and tracking the driver’s lips despite the visual variability of illumination and head pose for audio-visual speech recognition system.

Audio visual automatic speech recognition in vehicles

Acoustically, car cabins are extremely noisy and as a consequence, existing audio-only speech recognition systems, for voice-based control of vehicle functions such as the GPS based navigator, perform poorly. Audio-only speech recognition systems fail to make use of the visual modality of speech (eg: lip movements). As the visual modality is immune to acoustic noise, utilising this visual information in conjunction with an audio only speech recognition system has the potential to improve the accuracy of the system. The field of recognising speech using both auditory and visual inputs is known as Audio Visual Speech Recognition (AVSR). Continuous research in AVASR field has been ongoing for the past twenty-five years with notable progress being made. However, the practical deployment of AVASR systems for use in a variety of real-world applications has not yet emerged. The main reason is due to most research to date neglecting to address variabilities in the visual domain such as illumination and viewpoint in the design of the visual front-end of the AVSR system. In this paper we present an AVASR system in a real-world car environment using the AVICAR database [1], which is publicly available in-car database and we show that the use of visual speech conjunction with the audio modality is a better approach to improve the robustness and effectiveness of voice-only recognition systems in car cabin environments.

The effect of lumbar support on the trunk muscle activity and pelvic tilt in a passenger vehicle car seat

In order to gain a competitive edge in the market, automotive manufacturers and automotive seat suppliers have identified seat ergonomics for further development to improve overall vehicle comfort. Adjustable lumbar support devices have been offered since long as comfort systems in either a 2-way or 4-way adjustable configuration, although their effect on lumbar strain is not well documented. The effect of a lumbar support on posture and muscular strain, and therefore the relationship between discomfort and comfort device parameter settings, requires clarification. The aim of this paper is to study the effect of a 4-way lumbar support on lower trunk and pelvis muscle activity, pelvic tilt and spine curvature during a car seating activity. 10 healthy subjects (5 m/f; age 19-39) performed a seating activity in a passenger vehicle with seven different static lumbar support positions. The lumbar support was tested in 3 different height positions in relation to the seatback surface centreline (high, centre, low), each having 2 depths positions (lumbar prominence). An extra depth position was added for the centre position. Posture data were collected using a VICON MX motion capture system and NORAXON DTS goniometers and inclinometer. A rigid-body model of an adjustable car seat with four-way adjustable lumbar support was constructed in UGS Siemens NX and connected to a musculoskeletal model of a seated-human, modelled in AnyBody. Wireless electromyography (EMG) was used to calibrate the musculoskeletal model and assess the relationship between (a) muscular strain and lumbar prominence (normal to seatback surface) respective to the lumbar height (alongside seatback surface), (b) hip joint moment and lumbar prominence (normal to seatback surface) respective to lumbar height (alongside seatback surface) and (c) pelvic tilt and lumbar prominence (normal to seatback surface) respective to the lumbar height (alongside seatback surface). This study was based on the assumption that the musculoskeletal human model was seated at the correct R-Point (SgRP), determined via the occupant packaging toolkit in the JACK digital human model. The effect of the interaction between the driver/car-seat has been investigated for factors resulting from the presence and adjustment of a 4-way lumbar support. The results obtained show that various seat adjustments, and driver’s lumbar supports can have complex influence on the muscle activation, joint forces and moments, all of which can affect the comfort perception of the driver. This study enables the automotive industry to optimise passenger vehicle seat development and design. It further more supports the evaluation of static postural and dynamic seat comfort in normal everyday driving tasks and can be applied for future car design to reduce investment and improve comfort.

Bus and car travel time on urban networks : integrating Bluetooth and bus vehicle identification data

Travel time in an important transport performance indicator. Different modes of transport (buses and cars) have different mechanical and operational characteristics, resulting in significantly different travel behaviours and complexities in multimodal travel time estimation on urban networks. This paper explores the relationship between bus and car travel time on urban networks by utilising the empirical Bluetooth and Bus Vehicle Identification data from Brisbane. The technologies and issues behind the two datasets are studied. After cleaning the data to remove outliers, the relationship between not-in-service bus and car travel time and the relationship between in-service bus and car travel time are discussed. The travel time estimation models reveal that the not-in-service bus travel time are similar to the car travel time and the in-service bus travel time could be used to estimate car travel time during off-peak hours

The impact of interface modality on police officers' visual behaviour when using an in-vehicle system

Background Standard operating procedures state that police officers should not drive while interacting with their mobile data terminal (MDT) which provides in-vehicle information essential to police work. Such interactions do however occur in practice and represent a potential source of driver distraction. The MDT comprises visual output with manual input via touch screen and keyboard. This study investigated the potential for alternative input and output methods to mitigate driver distraction with specific focus on eye movements. Method Nineteen experienced drivers of police vehicles (one female) from the NSW Police Force completed four simulated urban drives. Three drives included a concurrent secondary task: imitation licence plate search using an emulated MDT. Three different interface methods were examined: Visual-Manual, Visual-Voice, and Audio-Voice (“Visual” and “Audio” = output modality; “Manual” and “Voice” = input modality). During each drive, eye movements were recorded using FaceLAB™ (Seeing Machines Ltd, Canberra, ACT). Gaze direction and glances on the MDT were assessed. Results The Visual-Voice and Visual-Manual interfaces resulted in a significantly greater number of glances towards the MDT than Audio-Voice or Baseline. The Visual-Manual and Visual-Voice interfaces resulted in significantly more glances to the display than Audio-Voice or Baseline. For longer duration glances (>2s and 1-2s) the Visual-Manual interface resulted in significantly more fixations than Baseline or Audio-Voice. The short duration glances (<1s) were significantly greater for both Visual-Voice and Visual-Manual compared with Baseline and Audio-Voice. There were no significant differences between Baseline and Audio-Voice. Conclusion An Audio-Voice interface has the greatest potential to decrease visual distraction to police drivers. However, it is acknowledged that an audio output may have limitations for information presentation compared with visual output. The Visual-Voice interface offers an environment where the capacity to present information is sustained, whilst distraction to the driver is reduced (compared to Visual-Manual) by enabling adaptation of fixation behaviour.

All-environment visual place recognition with SMART

This paper presents Sequence Matching Across Route Traversals (SMART); a generally applicable sequence-based place recognition algorithm. SMART provides invariance to changes in illumination and vehicle speed while also providing moderate pose invariance and robustness to environmental aliasing. We evaluate SMART on vehicles travelling at highly variable speeds in two challenging environments; firstly, on an all-terrain vehicle in an off-road, forest track and secondly, using a passenger car traversing an urban environment across day and night. We provide comparative results to the current state-of-the-art SeqSLAM algorithm and investigate the effects of altering SMART’s image matching parameters. Additionally, we conduct an extensive study of the relationship between image sequence length and SMART’s matching performance. Our results show viable place recognition performance in both environments with short 10-metre sequences, and up to 96% recall at 100% precision across extreme day-night cycles when longer image sequences are used.

Decision strategies for automated visual collision avoidance

This paper provides a preliminary analysis of an autonomous uncooperative collision avoidance strategy for unmanned aircraft using image-based visual control. Assuming target detection, the approach consists of three parts. First, a novel decision strategy is used to determine appropriate reference image features to track for safe avoidance. This is achieved by considering the current rules of the air (regulations), the properties of spiral motion and the expected visual tracking errors. Second, a spherical visual predictive control (VPC) scheme is used to guide the aircraft along a safe spiral-like trajectory about the object. Lastly, a stopping decision based on thresholding a cost function is used to determine when to stop the avoidance behaviour. The approach does not require estimation of range or time to collision, and instead relies on tuning two mutually exclusive decision thresholds to ensure satisfactory performance.

The driving visual field and a history of motor vehicle collision involvement in older drivers: A population-based examination

Purpose We designed a visual field test focused on the field utilized while driving to examine associations between field impairment and motor vehicle collision involvement in 2,000 drivers ≥70 years old. Methods The "driving visual field test" involved measuring light sensitivity for 20 targets in each eye, extending 15° superiorly, 30° inferiorly, 60° temporally and 30° nasally. The target locations were selected on the basis that they fell within the field region utilized when viewing through the windshield of a vehicle or viewing the dashboard while driving. Monocular fields were combined into a binocular field based on the more sensitive point from each eye. Severe impairment in the overall field or a region was defined as average sensitivity in the lowest quartile of sensitivity. At-fault collision involvement for five years prior to enrollment was obtained from state records. Poisson regression was used to calculate crude and adjusted rate ratios examining the association between field impairment and at-fault collision involvement. Results Drivers with severe binocular field impairment in the overall driving visual field had a 40% increased rate of at-fault collision involvement (RR 1.40, 95%CI 1.07-1.83). Impairment in the lower and left fields was associated with elevated collision rates (RR 1.40 95%CI 1.07-1.82 and RR 1.49, 95%CI 1.15-1.92, respectively), whereas impairment in the upper and right field regions was not. Conclusions Results suggest that older drivers with severe impairment in the lower or left region of the driving visual field are more likely to have a history of at-fault collision involvement.