Fusion of in-vehicle sensor data to develop Intelligent Driver Training System (IDTS)

The over represented number of novice drivers involved in crashes is alarming. Driver training is one of the interventions aimed at mitigating the number of crashes that involve young drivers. To our knowledge, Advanced Driver Assistance Systems (ADAS) have never been comprehensively used in designing an intelligent driver training system. Currently, there is a need to develop and evaluate ADAS that could assess driving competencies. The aim is to develop an unsupervised system called Intelligent Driver Training System (IDTS) that analyzes crash risks in a given driving situation. In order to design a comprehensive IDTS, data is collected from the Driver, Vehicle and Environment (DVE), synchronized and analyzed. The first implementation phase of this intelligent driver training system deals with synchronizing multiple variables acquired from DVE. RTMaps is used to collect and synchronize data like GPS, vehicle dynamics and driver head movement. After the data synchronization, maneuvers are segmented out as right turn, left turn and overtake. Each maneuver is composed of several individual tasks that are necessary to be performed in a sequential manner. This paper focuses on turn maneuvers. Some of the tasks required in the analysis of ‘turn’ maneuver are: detect the start and end of the turn, detect the indicator status change, check if the indicator was turned on within a safe distance and check the lane keeping during the turn maneuver. This paper proposes a fusion and analysis of heterogeneous data, mainly involved in driving, to determine the risk factor of particular maneuvers within the drive. It also explains the segmentation and risk analysis of the turn maneuver in a drive.

Multi-spectral fusion for surveillance systems

Surveillance systems such as object tracking and abandoned object detection systems typically rely on a single modality of colour video for their input. These systems work well in controlled conditions but often fail when low lighting, shadowing, smoke, dust or unstable backgrounds are present, or when the objects of interest are a similar colour to the background. Thermal images are not affected by lighting changes or shadowing, and are not overtly affected by smoke, dust or unstable backgrounds. However, thermal images lack colour information which makes distinguishing between different people or objects of interest within the same scene difficult. ----- By using modalities from both the visible and thermal infrared spectra, we are able to obtain more information from a scene and overcome the problems associated with using either modality individually. We evaluate four approaches for fusing visual and thermal images for use in a person tracking system (two early fusion methods, one mid fusion and one late fusion method), in order to determine the most appropriate method for fusing multiple modalities. We also evaluate two of these approaches for use in abandoned object detection, and propose an abandoned object detection routine that utilises multiple modalities. To aid in the tracking and fusion of the modalities we propose a modified condensation filter that can dynamically change the particle count and features used according to the needs of the system. ----- We compare tracking and abandoned object detection performance for the proposed fusion schemes and the visual and thermal domains on their own. Testing is conducted using the OTCBVS database to evaluate object tracking, and data captured in-house to evaluate the abandoned object detection. Our results show that significant improvement can be achieved, and that a middle fusion scheme is most effective.

The mechanical response of the ovine lumbar anulus fibrosus to uniaxial, biaxial and shear loads

Analytical and computational models of the intervertebral disc (IVD) are commonly employed to enhance understanding of the biomechanics of the human spine and spinal motion segments. The accuracy of these models in predicting physiological behaviour of the spine is intrinsically reliant on the accuracy of the material constitutive representations employed to represent the spinal tissues. There is a paucity of detailed mechanical data describing the material response of the reinforced­ground matrix in the anulus fibrosus of the IVD. In the present study, the ‘reinforced­ground matrix’ was defined as the matrix with the collagen fibres embedded but not actively bearing axial load, thus incorporating the contribution of the fibre-fibre and fibre-matrix interactions. To determine mechanical parameters for the anulus ground matrix, mechanical tests were carried out on specimens of ovine anulus, under unconfined uniaxial compression, simple shear and biaxial compression. Test specimens of ovine anulus fibrosus were obtained with an adjacent layer of vertebral bone/cartilage on the superior and inferior specimen surface. Specimen geometry was such that there were no continuous collagen fibres coupling the two endplates. Samples were subdivided according to disc region - anterior, lateral and posterior - to determine the regional inhomogeneity in the anulus mechanical response. Specimens were loaded at a strain rate sufficient to avoid fluid outflow from the tissue and typical stress-strain responses under the initial load application and under repeated loading were determined for each of the three loading types. The response of the anulus tissue to the initial and repeated load cycles was significantly different for all load types, except biaxial compression in the anterior anulus. Since the maximum applied strain exceeded the damage strain for the tissue, experimental results for repeated loading reflected the mechanical ability of the tissue to carry load, subsequent to the initiation of damage. To our knowledge, this is the first study to provide experimental data describing the response of the ‘reinforced­ground matrix’ to biaxial compression. Additionally, it is novel in defining a study objective to determine the regionally inhomogeneous response of the ‘reinforced­ground matrix’ under an extensive range of loading conditions suitable for mechanical characterisation of the tissue. The results presented facilitate the development of more detailed and comprehensive constitutive descriptions for the large strain nonlinear elastic or hyperelastic response of the anulus ground matrix.

Sequential decision fusion for controlled detection errors

Information fusion in biometrics has received considerable attention. The architecture proposed here is based on the sequential integration of multi-instance and multi-sample fusion schemes. This method is analytically shown to improve the performance and allow a controlled trade-off between false alarms and false rejects when the classifier decisions are statistically independent. Equations developed for detection error rates are experimentally evaluated by considering the proposed architecture for text dependent speaker verification using HMM based digit dependent speaker models. The tuning of parameters, n classifiers and m attempts/samples, is investigated and the resultant detection error trade-off performance is evaluated on individual digits. Results show that performance improvement can be achieved even for weaker classifiers (FRR-19.6%, FAR-16.7%). The architectures investigated apply to speaker verification from spoken digit strings such as credit card numbers in telephone or VOIP or internet based applications.

A biomechanical study of top screw pullout in anterior scoliosis correction constructs

Study Design: Biomechanical testing of vertebral body screw pullout resistance with relevance to top screw pullout in endoscopic anterior scoliosis constructs. Objectives: To analyse the effect of screw positioning and angulation on pullout resistance of vertebral body screws, where the pullout takes place along a curved path as occurs in anterior scoliosis constructs. Summary of Background Data: Top screw pullout is a significant clinical problem in endoscopic anterior scoliosis surgery, with rates of up to 18% reported in the literature. Methods: A custom designed biomechanical test rig was used to perform pullout tests of Medtronic anterior vertebral screws where the pullout occurred along an arc of known radius. Using synthetic bone blocks, a range of pullout radii and screw angulations were tested, in order to determine an ‘optimal’ configuration. The optimal configuration was then compared with standard screw positioning using a series of tests on ovine vertebrae (n=29). Results: Screw angulation has a small but significant effect on pullout resistance, with maximum strength being achieved at 10 degree cephalad angulation. Combining 10 degree cephalad angulation with maximal spacing between the top two screws (maximum pullout radius) increased the pullout resistance by 88% compared to ‘standard’ screw positioning (screws inserted perpendicular to rod at mid-body height). Conclusions: The positioning of the top screw in anterior scoliosis constructs can significantly alter its pullout resistance.

Fusionless scoliosis correction using shape memory alloy staples

Severe spinal deformity in young children is a formidable challenge for optimal treatment. Standard interventions for adolescents, such as spinal deformity correction and fusion, may not be appropriate for young patients with considerable growth remaining. Alternative surgical options that provide deformity correction and protect the growth remaining in the spine are needed to treat this group of patients 1, 2. One such method is the use of shape memory alloy staples. We report our experience to date using video-assisted thoracoscopic insertion of shape memory alloy staples. A retrospective review was conducted of 13 patients with scoliosis, aged 7 to 13 years, who underwent video-assisted thoracoscopic insertion of shape memory staples. In our experience, video-assisted thoracoscopic insertion of shape memory alloy staples is a safe procedure with no complications noted. It is a reliable method of providing curve stability, however the follow up results to date indicate that the effectiveness of the procedure is greater in younger patients.

Design and evaluation of an MRI compatible axial compression device for 3D assessment of spinal deformity and flexibility

Magnetic Resonance Imaging (MRI) offers a valuable research tool for the assessment of 3D spinal deformity in AIS, however the horizontal patient position imposed by conventional scanners removes the axial compressive loading on the spine which is an important determinant of deformity shape and magnitude in standing scoliosis patients. The objective of this study was to design, construct and test an MRI compatible compression device for research into the effect of axial loading on spinal deformity using supine MRI scans. The compression device was designed and constructed, consisting of a vest worn by the patient, which was attached via straps to a pneumatically actuated footplate. An applied load of 0.5 x bodyweight was remotely controlled by a unit in the scanner operator’s console. The entire device was constructed using non-metallic components for MRI compatibility. The device was evaluated by performing unloaded and loaded supine MRI scans on a series of 10 AIS patients. The study concluded that an MRI compatible compression device had been successfully designed and constructed, providing a research tool for studies into the effect of axial loading on 3D spinal deformity in scoliosis. The 3D axially loaded MR imaging capability developed in this study will allow future research investigations of the effect of axial loading on spinal rotation, and for imaging the response of scoliotic spinal tissues to axial loading.

3D lung and rib cage reconstruction using low dose CT scans of adolescent idiopathic scoliosis pre and post anterior thoracoscopic correction

Adolescent Idiopathic Scoliosis (AIS) has been associated with reduced pulmonary function believed to be due to a restriction of lung volume by the deformed thoracic cavity. A recent study by our group examined the changes in lung volume pre and post anterior thoracoscopic scoliosis correction using pulmonary function testing (1), however the anatomical changes in ribcage shape and left/right lung volume after thoracoscopic surgery which govern overall respiratory capacity are unknown. The aim of this study was to use 3D rendering from CT scan data to compare lung and ribcage anatomical changes from pre to two years post thoracoscopic anterior scoliosis correction. The study concluded that 3D volumetric reconstruction from CT scans is a powerful means of evaluating changes in pulmonary and thoracic anatomy following surgical AIS correction. Most likely, lung volume changes following thoracoscopic scoliosis correction are multifactorial and affected by changes in height (due to residual growth), ribcage shape, diaphragm positioning, Cobb angle correction in the thoracic spine. Further analysis of the 3D reconstructions will be performed to assess how each of these factors affect lung volume in this patient cohort.

Design and evaluation of an MRI compatible axial compression device for 3D assessment of spinal deformity and flexibility in adolescent idiopathic scoliosis

Magnetic Resonance Imaging (MRI) offers a valuable research tool for the assessment of 3D spinal deformity in AIS, however the horizontal patient position imposed by conventional scanners removes the axial compressive loading on the spine. The objective of this study was to design, construct and test an MRI compatible compression device for research into the effect of axial loading on spinal deformity using supine MRI scans. The device was evaluated by performing unloaded and loaded supine MRI scans on a series of 10 AIS patients. The patient group had a mean initial (unloaded) major Cobb angle of 43±7º, which increased to 50±9º on application of the compressive load. The 7° increase in mean Cobb angle is consistent with that reported by a previous study comparing standing versus supine posture in scoliosis patients (Torell et al, 1985. Spine 10:425-7).

Development of patient-specific computer simulation tools for endoscopic scoliosis surgery

Endoscopic approaches for anterior correction of idiopathic scoliosis are a relatively new surgical technique. This paper describes the development of patient-specific finite element modelling techniques to investigate the biomechanics of single rod anterior scoliosis correction. Spinal geometry is obtained from pre-operative CT scans and material properties for osteo-ligamentous spinal tissues are based on existing literature. The techniques being developed will allow pre-surgical prediction of stresses, forces and deformations in spinal tissues, rods and screws under post-operative physiological loads.

The lived experience of families living with spinal cord disability in Northeast Thailand

The experience of disability in the global South remains relatively underreported in spite of the greater focus on disability as both an impediment to development and frequently as a result of development. This article reports a qualitative study using ethnographic techniques undertaken in the province of Khon Kaen in Northeast Thailand. The primary participants were men who had experienced a severe spinal cord injury at a time when they were breadwinners, a role which is significant in the context of a modernising state that is an active participant in a global economy. The experiences, constructions and beliefs of these men, their family carers, and other informants illustrate the complex ways in which social and cultural factors interact with the opportunities, challenges and constraints of transition to modernity. The findings, interpreted according to the 'three bodies' approach, illustrate the intersection of colonising effects, governmentality and resistance, and embodied experience in a cultural context.

Image acquisition and manipulation protocols for CT-PET fusion to improve the accuracy of gross tumour volume localisation for 3D conformal radiotherapy for lung cancer

Aims: To develop clinical protocols for acquiring PET images, performing CT-PET registration and tumour volume definition based on the PET image data, for radiotherapy for lung cancer patients and then to test these protocols with respect to levels of accuracy and reproducibility. Method: A phantom-based quality assurance study of the processes associated with using registered CT and PET scans for tumour volume definition was conducted to: (1) investigate image acquisition and manipulation techniques for registering and contouring CT and PET images in a radiotherapy treatment planning system, and (2) determine technology-based errors in the registration and contouring processes. The outcomes of the phantom image based quality assurance study were used to determine clinical protocols. Protocols were developed for (1) acquiring patient PET image data for incorporation into the 3DCRT process, particularly for ensuring that the patient is positioned in their treatment position; (2) CT-PET image registration techniques and (3) GTV definition using the PET image data. The developed clinical protocols were tested using retrospective clinical trials to assess levels of inter-user variability which may be attributed to the use of these protocols. A Siemens Somatom Open Sensation 20 slice CT scanner and a Philips Allegro stand-alone PET scanner were used to acquire the images for this research. The Philips Pinnacle3 treatment planning system was used to perform the image registration and contouring of the CT and PET images. Results: Both the attenuation-corrected and transmission images obtained from standard whole-body PET staging clinical scanning protocols were acquired and imported into the treatment planning system for the phantom-based quality assurance study. Protocols for manipulating the PET images in the treatment planning system, particularly for quantifying uptake in volumes of interest and window levels for accurate geometric visualisation were determined. The automatic registration algorithms were found to have sub-voxel levels of accuracy, with transmission scan-based CT-PET registration more accurate than emission scan-based registration of the phantom images. Respiration induced image artifacts were not found to influence registration accuracy while inadequate pre-registration over-lap of the CT and PET images was found to result in large registration errors. A threshold value based on a percentage of the maximum uptake within a volume of interest was found to accurately contour the different features of the phantom despite the lower spatial resolution of the PET images. Appropriate selection of the threshold value is dependant on target-to-background ratios and the presence of respiratory motion. The results from the phantom-based study were used to design, implement and test clinical CT-PET fusion protocols. The patient PET image acquisition protocols enabled patients to be successfully identified and positioned in their radiotherapy treatment position during the acquisition of their whole-body PET staging scan. While automatic registration techniques were found to reduce inter-user variation compared to manual techniques, there was no significant difference in the registration outcomes for transmission or emission scan-based registration of the patient images, using the protocol. Tumour volumes contoured on registered patient CT-PET images using the tested threshold values and viewing windows determined from the phantom study, demonstrated less inter-user variation for the primary tumour volume contours than those contoured using only the patient’s planning CT scans. Conclusions: The developed clinical protocols allow a patient’s whole-body PET staging scan to be incorporated, manipulated and quantified in the treatment planning process to improve the accuracy of gross tumour volume localisation in 3D conformal radiotherapy for lung cancer. Image registration protocols which factor in potential software-based errors combined with adequate user training are recommended to increase the accuracy and reproducibility of registration outcomes. A semi-automated adaptive threshold contouring technique incorporating a PET windowing protocol, accurately defines the geometric edge of a tumour volume using PET image data from a stand alone PET scanner, including 4D target volumes.

Paediatric spinal deformity research at the Mater Children's Hospital, Brisbane, Australia.

The Paediatric Spine Research group was formed in 2002 to perform high quality research into the prevention and management of spinal deformity, with an emphasis on scoliosis. The group has successfully built collaborative bridges between the scientific and research expertise at QUT, and the clinical skills and experience of the spinal orthopaedic surgeons at the Mater Children’s Hospital in Brisbane. Clinical and biomechanical research is now possible as a result of the development of detailed databases of patients who have innovative and unique surgical interventions for spinal deformity such as thoracoscopic scoliosis correction, thoracoscopic staple insertion for juvenile idiopathic scoliosis and minimally invasive growing rods. The Mater in Brisbane provides these unique datasets of spinal deformity surgery patients, whose procedures are not being performed anywhere else in the Southern Hemisphere. The most detailed is a database of thoracoscopic scoliosis correction surgery which now contains 180 patients with electronic collections of X-Rays, photographs and patient questionnaires. With ethics approval, a subset of these patients has had CT scans, and a further subset have had MRI scans with and without a compressive load to simulate the erect standing position. This database has to date contributed to 17 international refereed journal papers, a further 7 journal papers either under review or in final preparation, 53 national conference presentations and 35 international conference presentations. Major findings from selected journal publications will be presented. It is anticipated that as the surgical databases grow they will continue to provide invaluable clinical data which will feed into clinically relevant projects driven by both medical and engineering researchers whose findings will benefit spinal deformity patients and scientific knowledge worldwide.