Representation and learning of visual information for pose recognition

Recovering position from sensor information is an important problem in mobile robotics, known as localisation. Localisation requires a map or some other description of the environment to provide the robot with a context to interpret sensor data. The mobile robot system under discussion is using an artificial neural representation of position. Building a geometrical map of the environment with a single camera and artificial neural networks is difficult. Instead it would be simpler to learn position as a function of the visual input. Usually when learning images, an intermediate representation is employed. An appropriate starting point for biologically plausible image representation is the complex cells of the visual cortex, which have invariance properties that appear useful for localisation. The effectiveness for localisation of two different complex cell models are evaluated. Finally the ability of a simple neural network with single shot learning to recognise these representations and localise a robot is examined.

Design and evaluation of an MRI compatible axial compression device for 3D assessment of spinal deformity and flexibility

Magnetic Resonance Imaging (MRI) offers a valuable research tool for the assessment of 3D spinal deformity in AIS, however the horizontal patient position imposed by conventional scanners removes the axial compressive loading on the spine which is an important determinant of deformity shape and magnitude in standing scoliosis patients. The objective of this study was to design, construct and test an MRI compatible compression device for research into the effect of axial loading on spinal deformity using supine MRI scans. The compression device was designed and constructed, consisting of a vest worn by the patient, which was attached via straps to a pneumatically actuated footplate. An applied load of 0.5 x bodyweight was remotely controlled by a unit in the scanner operator’s console. The entire device was constructed using non-metallic components for MRI compatibility. The device was evaluated by performing unloaded and loaded supine MRI scans on a series of 10 AIS patients. The study concluded that an MRI compatible compression device had been successfully designed and constructed, providing a research tool for studies into the effect of axial loading on 3D spinal deformity in scoliosis. The 3D axially loaded MR imaging capability developed in this study will allow future research investigations of the effect of axial loading on spinal rotation, and for imaging the response of scoliotic spinal tissues to axial loading.

3D lung and rib cage reconstruction using low dose CT scans of adolescent idiopathic scoliosis pre and post anterior thoracoscopic correction

Adolescent Idiopathic Scoliosis (AIS) has been associated with reduced pulmonary function believed to be due to a restriction of lung volume by the deformed thoracic cavity. A recent study by our group examined the changes in lung volume pre and post anterior thoracoscopic scoliosis correction using pulmonary function testing (1), however the anatomical changes in ribcage shape and left/right lung volume after thoracoscopic surgery which govern overall respiratory capacity are unknown. The aim of this study was to use 3D rendering from CT scan data to compare lung and ribcage anatomical changes from pre to two years post thoracoscopic anterior scoliosis correction. The study concluded that 3D volumetric reconstruction from CT scans is a powerful means of evaluating changes in pulmonary and thoracic anatomy following surgical AIS correction. Most likely, lung volume changes following thoracoscopic scoliosis correction are multifactorial and affected by changes in height (due to residual growth), ribcage shape, diaphragm positioning, Cobb angle correction in the thoracic spine. Further analysis of the 3D reconstructions will be performed to assess how each of these factors affect lung volume in this patient cohort.

Lip detection for audio-visual speech recognition in-car environment

Acoustically, car cabins are extremely noisy and as a consequence audio-only, in-car voice recognition systems perform poorly. As the visual modality is immune to acoustic noise, using the visual lip information from the driver is seen as a viable strategy in circumventing this problem by using audio visual automatic speech recognition (AVASR). However, implementing AVASR requires a system being able to accurately locate and track the drivers face and lip area in real-time. In this paper we present such an approach using the Viola-Jones algorithm. Using the AVICAR [1] in-car database, we show that the Viola- Jones approach is a suitable method of locating and tracking the driver’s lips despite the visual variability of illumination and head pose for audio-visual speech recognition system.

The Sun gives without ever receiving

A small scale sculpture that contributes towards my ongoing explorations into how our collective ability to sustain (the future) is as much a cultural problematic as it is an economic or technological one. The curatorial brief of the project was a technical one - in that each curated artist was to design a piece in CAD suitable for 3D resin printing - The object should be entirely generated through 3D visualisation and modelling tools and should be machined and shipped within the dimensions of 6cm x 6cm x 6cm. My design for this brief was influenced by recent research I had conducted in Mildura in the Sunraysia irrigated region of NW Victoria. Each name set within the work is an Australian soldier/settler – who, on returning from the ‘Great War’ was duly awarded a ‘block’ in Australia’s new inland irrigated settlements - with the explicit task of clearing it to plant and reap. Through their concerted and well-intentioned efforts, these workers began to profoundly re-shape Australia’s marginal country - inadvertently presaging the bleak future faced today by many of Australia’s inland lands and river systems. Furthermore, through that time's predominant colonial conception of ‘terra nullius’ (this land is unoccupied and therefore free to be claimed) they each played a small but formative part in building the profound cultural divide between land and peoples that still haunts Australia today. THE EXHIBITION: Inside Out is a compelling international touring exhibition featuring forty-six miniature sculptures produced in resin using 3D printing technologies. Developments in virtual computer visualisation and integrated digital technologies are giving contemporary makers new insight and opportunities to create objects and forms which were previously impossible to produce or difficult to envisage. The exhibition is the result of collaboration between the Art Technology Coalition, the University of Technology Sydney and RMIT University in Australia along with De Montfort University, Manchester Metropolitan University and Dartington College of Arts at University College Falmouth in the United Kingdom.

Bengali handwritten character recognition using modified syntactic method

Several approaches have been proposed to recognize handwritten Bengali characters using different curve fitting algorithms and curvature analysis. In this paper, a new algorithm (Curve-fitting Algorithm) to identify various strokes of a handwritten character is developed. The curve-fitting algorithm helps recognizing various strokes of different patterns (line, quadratic curve) precisely. This reduces the error elimination burden heavily. Implementation of this Modified Syntactic Method demonstrates significant improvement in the recognition of Bengali handwritten characters.

Audio visual automatic speech recognition in vehicles

Acoustically, car cabins are extremely noisy and as a consequence, existing audio-only speech recognition systems, for voice-based control of vehicle functions such as the GPS based navigator, perform poorly. Audio-only speech recognition systems fail to make use of the visual modality of speech (eg: lip movements). As the visual modality is immune to acoustic noise, utilising this visual information in conjunction with an audio only speech recognition system has the potential to improve the accuracy of the system. The field of recognising speech using both auditory and visual inputs is known as Audio Visual Speech Recognition (AVSR). Continuous research in AVASR field has been ongoing for the past twenty-five years with notable progress being made. However, the practical deployment of AVASR systems for use in a variety of real-world applications has not yet emerged. The main reason is due to most research to date neglecting to address variabilities in the visual domain such as illumination and viewpoint in the design of the visual front-end of the AVSR system. In this paper we present an AVASR system in a real-world car environment using the AVICAR database [1], which is publicly available in-car database and we show that the use of visual speech conjunction with the audio modality is a better approach to improve the robustness and effectiveness of voice-only recognition systems in car cabin environments.

Temporal-based support vector machines for facial expression recognition

When classifying a signal, ideally we want our classifier to trigger a large response when it encounters a positive example and have little to no response for all other examples. Unfortunately in practice this does not occur with responses fluctuating, often causing false alarms. There exists a myriad of reasons why this is the case, most notably not incorporating the dynamics of the signal into the classification. In facial expression recognition, this has been highlighted as one major research question. In this paper we present a novel technique which incorporates the dynamics of the signal which can produce a strong response when the peak expression is found and essentially suppresses all other responses as much as possible. We conducted preliminary experiments on the extended Cohn-Kanade (CK+) database which shows its benefits. The ability to automatically and accurately recognize facial expressions of drivers is highly relevant to the automobile. For example, the early recognition of “surprise” could indicate that an accident is about to occur; and various safeguards could immediately be deployed to avoid or minimize injury and damage. In this paper, we conducted initial experiments on the extended Cohn-Kanade (CK+) database which shows its benefits.

Typical accuracy and quality control of a process for creating CT-Based virtual bone models

A pragmatic method for assessing the accuracy and precision of a given processing pipeline required for converting computed tomography (CT) image data of bones into representative three dimensional (3D) models of bone shapes is proposed. The method is based on coprocessing a control object with known geometry which enables the assessment of the quality of resulting 3D models. At three stages of the conversion process, distance measurements were obtained and statistically evaluated. For this study, 31 CT datasets were processed. The final 3D model of the control object contained an average deviation from reference values of −1.07±0.52 mm standard deviation (SD) for edge distances and −0.647±0.43 mm SD for parallel side distances of the control object. Coprocessing a reference object enables the assessment of the accuracy and precision of a given processing pipeline for creating CTbased 3D bone models and is suitable for detecting most systematic or human errors when processing a CT-scan. Typical errors have about the same size as the scan resolution.

Creation of a validated 3D finite element model of an ovine tibia

Introduction Ovine models are widely used in orthopaedic research. To better understand the impact of orthopaedic procedures computer simulations are necessary. 3D finite element (FE) models of bones allow implant designs to be investigated mechanically, thereby reducing mechanical testing. Hypothesis We present the development and validation of an ovine tibia FE model for use in the analysis of tibia fracture fixation plates. Material & Methods Mechanical testing of the tibia consisted of an offset 3-pt bend test with three repetitions of loading to 350N and return to 50N. Tri-axial stacked strain gauges were applied to the anterior and posterior surfaces of the bone and two rigid bodies – consisting of eight infrared active markers, were attached to the ends of the tibia. Positional measurements were taken with a FARO arm 3D digitiser. The FE model was constructed with both geometry and material properties derived from CT images of the bone. The elasticity-density relationship used for material property determination was validated separately using mechanical testing. This model was then transformed to the same coordinate system as the in vitro mechanical test and loads applied. Results Comparison between the mechanical testing and the FE model showed good correlation in surface strains (difference: anterior 2.3%, posterior 3.2%). Discussion & Conclusion This method of model creation provides a simple method for generating subject specific FE models from CT scans. The use of the CT data set for both the geometry and the material properties ensures a more accurate representation of the specific bone. This is reflected in the similarity of the surface strain results.

Understanding the Aquifier : 3D Visualisation and Groundwater Time-Series Animation in Lockyer Valley

The Lockyer Valley, southeast Queensland, hosts intensive irrigated agriculture using groundwater from over 5000 alluvial bores. A current project is considering introduction of PRW (purified recycled water) to augment groundwater supplies. To assess this, a valley-wide MODFLOW simulation model is being developed plus a new unsaturated zone flow model. To underpin these models and provide a realistic understanding of the aquifer framework a 3D visualisation model has been developed using Groundwater Visualisation System (GVS) software produced at QUT.

Multi-modal object tracking using dynamic performance metrics

Intelligent surveillance systems typically use a single visual spectrum modality for their input. These systems work well in controlled conditions, but often fail when lighting is poor, or environmental effects such as shadows, dust or smoke are present. Thermal spectrum imagery is not as susceptible to environmental effects, however thermal imaging sensors are more sensitive to noise and they are only gray scale, making distinguishing between objects difficult. Several approaches to combining the visual and thermal modalities have been proposed, however they are limited by assuming that both modalities are perfuming equally well. When one modality fails, existing approaches are unable to detect the drop in performance and disregard the under performing modality. In this paper, a novel middle fusion approach for combining visual and thermal spectrum images for object tracking is proposed. Motion and object detection is performed on each modality and the object detection results for each modality are fused base on the current performance of each modality. Modality performance is determined by comparing the number of objects tracked by the system with the number detected by each mode, with a small allowance made for objects entering and exiting the scene. The tracking performance of the proposed fusion scheme is compared with performance of the visual and thermal modes individually, and a baseline middle fusion scheme. Improvement in tracking performance using the proposed fusion approach is demonstrated. The proposed approach is also shown to be able to detect the failure of an individual modality and disregard its results, ensuring performance is not degraded in such situations.

Design and evaluation of an MRI compatible axial compression device for 3D assessment of spinal deformity and flexibility in adolescent idiopathic scoliosis

Magnetic Resonance Imaging (MRI) offers a valuable research tool for the assessment of 3D spinal deformity in AIS, however the horizontal patient position imposed by conventional scanners removes the axial compressive loading on the spine. The objective of this study was to design, construct and test an MRI compatible compression device for research into the effect of axial loading on spinal deformity using supine MRI scans. The device was evaluated by performing unloaded and loaded supine MRI scans on a series of 10 AIS patients. The patient group had a mean initial (unloaded) major Cobb angle of 43±7º, which increased to 50±9º on application of the compressive load. The 7° increase in mean Cobb angle is consistent with that reported by a previous study comparing standing versus supine posture in scoliosis patients (Torell et al, 1985. Spine 10:425-7).

Speech recognition using ad-hoc microphone arrays

While close talking microphones give the best signal quality and produce the highest accuracy from current Automatic Speech Recognition (ASR) systems, the speech signal enhanced by microphone array has been shown to be an effective alternative in a noisy environment. The use of microphone arrays in contrast to close talking microphones alleviates the feeling of discomfort and distraction to the user. For this reason, microphone arrays are popular and have been used in a wide range of applications such as teleconferencing, hearing aids, speaker tracking, and as the front-end to speech recognition systems. With advances in sensor and sensor network technology, there is considerable potential for applications that employ ad-hoc networks of microphone-equipped devices collaboratively as a virtual microphone array. By allowing such devices to be distributed throughout the users’ environment, the microphone positions are no longer constrained to traditional fixed geometrical arrangements. This flexibility in the means of data acquisition allows different audio scenes to be captured to give a complete picture of the working environment. In such ad-hoc deployment of microphone sensors, however, the lack of information about the location of devices and active speakers poses technical challenges for array signal processing algorithms which must be addressed to allow deployment in real-world applications. While not an ad-hoc sensor network, conditions approaching this have in effect been imposed in recent National Institute of Standards and Technology (NIST) ASR evaluations on distant microphone recordings of meetings. The NIST evaluation data comes from multiple sites, each with different and often loosely specified distant microphone configurations. This research investigates how microphone array methods can be applied for ad-hoc microphone arrays. A particular focus is on devising methods that are robust to unknown microphone placements in order to improve the overall speech quality and recognition performance provided by the beamforming algorithms. In ad-hoc situations, microphone positions and likely source locations are not known and beamforming must be achieved blindly. There are two general approaches that can be employed to blindly estimate the steering vector for beamforming. The first is direct estimation without regard to the microphone and source locations. An alternative approach is instead to first determine the unknown microphone positions through array calibration methods and then to use the traditional geometrical formulation for the steering vector. Following these two major approaches investigated in this thesis, a novel clustered approach which includes clustering the microphones and selecting the clusters based on their proximity to the speaker is proposed. Novel experiments are conducted to demonstrate that the proposed method to automatically select clusters of microphones (ie, a subarray), closely located both to each other and to the desired speech source, may in fact provide a more robust speech enhancement and recognition than the full array could.

Image acquisition and manipulation protocols for CT-PET fusion to improve the accuracy of gross tumour volume localisation for 3D conformal radiotherapy for lung cancer

Aims: To develop clinical protocols for acquiring PET images, performing CT-PET registration and tumour volume definition based on the PET image data, for radiotherapy for lung cancer patients and then to test these protocols with respect to levels of accuracy and reproducibility. Method: A phantom-based quality assurance study of the processes associated with using registered CT and PET scans for tumour volume definition was conducted to: (1) investigate image acquisition and manipulation techniques for registering and contouring CT and PET images in a radiotherapy treatment planning system, and (2) determine technology-based errors in the registration and contouring processes. The outcomes of the phantom image based quality assurance study were used to determine clinical protocols. Protocols were developed for (1) acquiring patient PET image data for incorporation into the 3DCRT process, particularly for ensuring that the patient is positioned in their treatment position; (2) CT-PET image registration techniques and (3) GTV definition using the PET image data. The developed clinical protocols were tested using retrospective clinical trials to assess levels of inter-user variability which may be attributed to the use of these protocols. A Siemens Somatom Open Sensation 20 slice CT scanner and a Philips Allegro stand-alone PET scanner were used to acquire the images for this research. The Philips Pinnacle3 treatment planning system was used to perform the image registration and contouring of the CT and PET images. Results: Both the attenuation-corrected and transmission images obtained from standard whole-body PET staging clinical scanning protocols were acquired and imported into the treatment planning system for the phantom-based quality assurance study. Protocols for manipulating the PET images in the treatment planning system, particularly for quantifying uptake in volumes of interest and window levels for accurate geometric visualisation were determined. The automatic registration algorithms were found to have sub-voxel levels of accuracy, with transmission scan-based CT-PET registration more accurate than emission scan-based registration of the phantom images. Respiration induced image artifacts were not found to influence registration accuracy while inadequate pre-registration over-lap of the CT and PET images was found to result in large registration errors. A threshold value based on a percentage of the maximum uptake within a volume of interest was found to accurately contour the different features of the phantom despite the lower spatial resolution of the PET images. Appropriate selection of the threshold value is dependant on target-to-background ratios and the presence of respiratory motion. The results from the phantom-based study were used to design, implement and test clinical CT-PET fusion protocols. The patient PET image acquisition protocols enabled patients to be successfully identified and positioned in their radiotherapy treatment position during the acquisition of their whole-body PET staging scan. While automatic registration techniques were found to reduce inter-user variation compared to manual techniques, there was no significant difference in the registration outcomes for transmission or emission scan-based registration of the patient images, using the protocol. Tumour volumes contoured on registered patient CT-PET images using the tested threshold values and viewing windows determined from the phantom study, demonstrated less inter-user variation for the primary tumour volume contours than those contoured using only the patient’s planning CT scans. Conclusions: The developed clinical protocols allow a patient’s whole-body PET staging scan to be incorporated, manipulated and quantified in the treatment planning process to improve the accuracy of gross tumour volume localisation in 3D conformal radiotherapy for lung cancer. Image registration protocols which factor in potential software-based errors combined with adequate user training are recommended to increase the accuracy and reproducibility of registration outcomes. A semi-automated adaptive threshold contouring technique incorporating a PET windowing protocol, accurately defines the geometric edge of a tumour volume using PET image data from a stand alone PET scanner, including 4D target volumes.