Design of automatic vision-based inspection system for solder joint segmentation

Purpose: Computer vision has been widely used in the inspection of electronic components. This paper proposes a computer vision system for the automatic detection, localisation, and segmentation of solder joints on Printed Circuit Boards (PCBs) under different illumination conditions. Design/methodology/approach: An illumination normalization approach is applied to an image, which can effectively and efficiently eliminate the effect of uneven illumination while keeping the properties of the processed image the same as in the corresponding image under normal lighting conditions. Consequently special lighting and instrumental setup can be reduced in order to detect solder joints. These normalised images are insensitive to illumination variations and are used for the subsequent solder joint detection stages. In the segmentation approach, the PCB image is transformed from an RGB color space to a YIQ color space for the effective detection of solder joints from the background. Findings: The segmentation results show that the proposed approach improves the performance significantly for images under varying illumination conditions. Research limitations/implications: This paper proposes a front-end system for the automatic detection, localisation, and segmentation of solder joint defects. Further research is required to complete the full system including the classification of solder joint defects. Practical implications: The methodology presented in this paper can be an effective method to reduce cost and improve quality in production of PCBs in the manufacturing industry. Originality/value: This research proposes the automatic location, identification and segmentation of solder joints under different illumination conditions.

Vision-based classification of solder joint defects

Inspection of solder joints has been a critical process in the electronic manufacturing industry to reduce manufacturing cost, improve yield, and ensure product quality and reliability. The solder joint inspection problem is more challenging than many other visual inspections because of the variability in the appearance of solder joints. Although many research works and various techniques have been developed to classify defect in solder joints, these methods have complex systems of illumination for image acquisition and complicated classification algorithms. An important stage of the analysis is to select the right method for the classification. Better inspection technologies are needed to fill the gap between available inspection capabilities and industry systems. This dissertation aims to provide a solution that can overcome some of the limitations of current inspection techniques. This research proposes two inspection steps for automatic solder joint classification system. The “front-end” inspection system includes illumination normalisation, localization and segmentation. The illumination normalisation approach can effectively and efficiently eliminate the effect of uneven illumination while keeping the properties of the processed image. The “back-end” inspection involves the classification of solder joints by using Log Gabor filter and classifier fusion. Five different levels of solder quality with respect to the amount of solder paste have been defined. Log Gabor filter has been demonstrated to achieve high recognition rates and is resistant to misalignment. Further testing demonstrates the advantage of Log Gabor filter over both Discrete Wavelet Transform and Discrete Cosine Transform. Classifier score fusion is analysed for improving recognition rate. Experimental results demonstrate that the proposed system improves performance and robustness in terms of classification rates. This proposed system does not need any special illumination system, and the images are acquired by an ordinary digital camera. In fact, the choice of suitable features allows one to overcome the problem given by the use of non complex illumination systems. The new system proposed in this research can be incorporated in the development of an automated non-contact, non-destructive and low cost solder joint quality inspection system.

Spatially-explicit modelling of grassland classes - an improved method of integrating a climate-based classification model with interpolated climate surfaces

Spatially-explicit modelling of grassland classes is important to site-specific planning for improving grassland and environmental management over large areas. In this study, a climate-based grassland classification model, the Comprehensive and Sequential Classification System (CSCS) was integrated with spatially interpolated climate data to classify grassland in Gansu province, China. The study area is characterized by complex topographic features imposed by plateaus, high mountains, basins and deserts. To improve the quality of the interpolated climate data and the quality of the spatial classification over this complex topography, three linear regression methods, namely an analytic method based on multiple regression and residues (AMMRR), a modification of the AMMRR method through adding the effect of slope and aspect to the interpolation analysis (M-AMMRR) and a method which replaces the IDW approach for residue interpolation in M-AMMRR with an ordinary kriging approach (I-AMMRR), for interpolating climate variables were evaluated. The interpolation outcomes from the best interpolation method were then used in the CSCS model to classify the grassland in the study area. Climate variables interpolated included the annual cumulative temperature and annual total precipitation. The results indicated that the AMMRR and M-AMMRR methods generated acceptable climate surfaces but the best model fit and cross validation result were achieved by the I-AMMRR method. Twenty-six grassland classes were classified for the study area. The four grassland vegetation classes that covered more than half of the total study area were "cool temperate-arid temperate zonal semi-desert", "cool temperate-humid forest steppe and deciduous broad-leaved forest", "temperate-extra-arid temperate zonal desert", and "frigid per-humid rain tundra and alpine meadow". The vegetation classification map generated in this study provides spatial information on the locations and extents of the different grassland classes. This information can be used to facilitate government agencies' decision-making in land-use planning and environmental management, and for vegetation and biodiversity conservation. The information can also be used to assist land managers in the estimation of safe carrying capacities which will help to prevent overgrazing and land degradation.

Development of a Rep-inducible, BBTV-based expression system in banana

Banana bunchy top is regarded as the most important viral disease of banana, causing significant yield losses worldwide. The disease is caused by Banana bunchy top virus (BBTV), which is a circular ssDNA virus belonging to the genus Babuvirus in the family Nanoviridae. There are currently few effective control strategies for this and other ssDNA viruses. “In Plant Activation” (InPAct) is a novel technology being developed at QUT for ssDNA virus-activated suicide gene expression. The technology exploits the rolling circle replication mechanism of ssDNA viruses and is based on a unique “split” gene design such that suicide gene expression is only activated in the presence of the viral Rep. This PhD project aimed to develop a BBTV-based InPAct system as a suicide gene strategy to control BBTV. The BBTV-based InPAct vector design requires a BBTV intergenic region (IR) to be embedded within an intron in the gene expression cassette. To ensure that the BBTV IR would not interfere with intron splicing, a TEST vector was initially generated that contained the entire BBTV IR embedded within an intron in a β-glucuronidase (GUS) expression vector. Transient GUS assays in banana embryogenic cell suspensions indicated that cryptic intron splice sites were present within the IR. Transcript analysis revealed two cryptic intron splice sites in the Domain III sequence of the CR-M within the IR. Removal of the CR-M from the TEST vector resulted in an enhancement of GUS expression suggesting that the cryptic intron splice sites had been removed. An InPAct GUS vector was subsequently generated that contained the modified BBTV IR, with the CR-M (minus Domain III) repositioned within the InPAct cassette. Using transient histochemical and fluorometric GUS assays in banana embryogenic cells, the InPAct GUS vector was shown to be activated in the presence of the BBTV Rep. However, the presence of both BBTV Rep and Clink was shown to have a deleterious effect on GUS expression suggesting that these proteins were cytotoxic at the levels expressed. Analysis of replication of the InPAct vectors by Southern hybridisation revealed low levels of InPAct cassette-based episomal DNA released from the vector through the nicking/ligation activity of BBTV Rep. However, Rep-mediated episomal replicons, indicative of rolling circle replication of the released circularised cassettes, were not observed. The inability of the InPAct cassette to be replicated was further investigated. To examine whether the absence of Domain III of the CR-M was responsible, a suite of modified BBTV-based InPAct GUS vectors was constructed that contained the CR-M with the inclusion of Domain III, the CR-M with the inclusion of Domain III and additional upstream IR sequence, or no CR-M. Analysis of replication by Southern hybridisation revealed that neither the presence of Domain III, nor the entire CR-M, had an effect on replication levels. Since the InPAct cassette was significantly larger than the native BBTV genomic components (approximately 1 kb), the effect of InPAct cassette size on replication was also investigated. A suite of size variant BBTV-based vectors was constructed that increased the size of a replication competent cassette to 1.1 kbp through to 2.1 kbp.. Analysis of replication by Southern hybridisation revealed that an increase in vector size above approximately 1.5 - 1.7 kbp resulted in a decrease in replication. Following the demonstration of Rep-mediated release, circularisation and expression from the InPAct GUS vector, an InPAct vector was generated in which the uidA reporter gene was replaced with the ribonuclease-encoding suicide gene, barnase. Initially, a TEST vector was generated to assess the cytotoxicity of Barnase on banana cells. Although transient assays revealed a Barnase-induced cytotoxic effect in banana cells, the expression levels were sub-optimal. An InPAct BARNASE vector was generated and tested for BBTV Rep-activated Barnase expression using transient assays in banana embryogenic cells. High levels of background expression from the InPAct BARNASE vector made it difficult to accurately assess Rep-activated Barnase expression. Analysis of replication by Southern hybridisation revealed low levels of InPAct cassette-based episomal DNA released from the vector but no Rep-mediated episomal replicons indicative of rolling circle replication of the released circularised cassettes were again observed. Despite the inability of the InPAct vectors to replicate to enable high level gene expression, the InPAct BARNASE vector was assessed in planta for BBTV Rep-mediated activation of Barnase expression. Eleven lines of transgenic InPAct BARNASE banana plants were generated by Agrobacterium-mediated transformation and were challenged with viruliferous Pentalonia nigronervosa. At least one clonal plant in each line developed bunchy top symptoms and infection was confirmed by PCR. No localised lesions were observed on any plants, nor was there any localised GUS expression in the one InPAct GUS line challenged with viruliferous aphids. The results presented in this thesis are the first study towards the development of a BBTV-based InPAct system as a Rep-activatable suicide gene expression system to control BBTV. Although further optimisation of the vectors is necessary, the preliminary results suggest that this approach has the potential to be an effective control strategy for BBTV. The use of iterons within the InPAct vectors that are recognised by Reps from different ssDNA plant viruses may provide a broad-spectrum resistance strategy against multiple ssDNA plant viruses. Further, this technology holds great promise as a platform technology for the molecular farming of high-value proteins in vitro or in vivo through expression of the ssDNA virus Rep protein.

Model Based Estimating System for Civil Concrete Structures

An earlier CRC-CI project on ‘automatic estimating’ (AE) has shown the key benefit of model-based design methodologies in building design and construction to be the provision of timely quantitative cost evaluations. Furthermore, using AE during design improves design options, and results in improved design turn-around times, better design quality and/or lower costs. However, AEs for civil engineering structures do not exist; and research partners in the CRC-CI expressed interest in exploring the development of such a process. This document reports on these investigations. The central objective of the study was to evaluate the benefits and costs of developing an AE for concrete civil engineering works. By studying existing documents and through interviews with design engineers, contractors and estimators, we have established that current civil engineering practices (mainly roads/bridges) do not use model-based planning/design. Drawings are executed in 2D and only completed at the end of lengthy planning/design project management lifecycle stages. We have also determined that estimating plays two important, but different roles. The first is part of project management (which we have called macro level estimating). Estimating in this domain sets project budgets, controls quality delivery and contains costs. The second role is estimating during planning/design (micro level estimating). The difference between the two roles is that the former is performed at the end of various lifecycle stages, whereas the latter is performed at any suitable time during planning/design.

The user query based learning system for lifetime prediction of metallic components

Real-World Data Mining Applications generally do not end up with the creation of the models. The use of the model is the final purpose especially in prediction tasks. The problem arises when the model is built based on much more information than that the user can provide in using the model. As a result, the performance of model reduces drastically due to many missing attributes values. This paper develops a new learning system framework, called as User Query Based Learning System (UQBLS), for building data mining models best suitable for users use. We demonstrate its deployment in a real-world application of the lifetime prediction of metallic components in buildings

Bearing-only SLAM : a vision-based navigation system for autonomous robots

To navigate successfully in a previously unexplored environment, a mobile robot must be able to estimate the spatial relationships of the objects of interest accurately. A Simultaneous Localization and Mapping (SLAM) sys- tem employs its sensors to build incrementally a map of its surroundings and to localize itself in the map simultaneously. The aim of this research project is to develop a SLAM system suitable for self propelled household lawnmowers. The proposed bearing-only SLAM system requires only an omnidirec- tional camera and some inexpensive landmarks. The main advantage of an omnidirectional camera is the panoramic view of all the landmarks in the scene. Placing landmarks in a lawn field to define the working domain is much easier and more flexible than installing the perimeter wire required by existing autonomous lawnmowers. The common approach of existing bearing-only SLAM methods relies on a motion model for predicting the robot’s pose and a sensor model for updating the pose. In the motion model, the error on the estimates of object positions is cumulated due mainly to the wheel slippage. Quantifying accu- rately the uncertainty of object positions is a fundamental requirement. In bearing-only SLAM, the Probability Density Function (PDF) of landmark position should be uniform along the observed bearing. Existing methods that approximate the PDF with a Gaussian estimation do not satisfy this uniformity requirement. This thesis introduces both geometric and proba- bilistic methods to address the above problems. The main novel contribu- tions of this thesis are: 1. A bearing-only SLAM method not requiring odometry. The proposed method relies solely on the sensor model (landmark bearings only) without relying on the motion model (odometry). The uncertainty of the estimated landmark positions depends on the vision error only, instead of the combination of both odometry and vision errors. 2. The transformation of the spatial uncertainty of objects. This thesis introduces a novel method for translating the spatial un- certainty of objects estimated from a moving frame attached to the robot into the global frame attached to the static landmarks in the environment. 3. The characterization of an improved PDF for representing landmark position in bearing-only SLAM. The proposed PDF is expressed in polar coordinates, and the marginal probability on range is constrained to be uniform. Compared to the PDF estimated from a mixture of Gaussians, the PDF developed here has far fewer parameters and can be easily adopted in a probabilistic framework, such as a particle filtering system. The main advantages of our proposed bearing-only SLAM system are its lower production cost and flexibility of use. The proposed system can be adopted in other domestic robots as well, such as vacuum cleaners or robotic toys when terrain is essentially 2D.

An alginate-based hybrid system for growth factor delivery in the functional repair of large bone defects

The treatment of challenging fractures and large osseous defects presents a formidable problem for orthopaedic surgeons. Tissue engineering/regenerative medicine approaches seek to solve this problem by delivering osteogenic signals within scaffolding biomaterials. In this study, we introduce a hybrid growth factor delivery system that consists of an electrospun nanofiber mesh tube for guiding bone regeneration combined with peptide-modified alginate hydrogel injected inside the tube for sustained growth factor release. We tested the ability of this system to deliver recombinant bone morphogenetic protein-2 (rhBMP-2) for the repair of critically-sized segmental bone defects in a rat model. Longitudinal [mu]-CT analysis and torsional testing provided quantitative assessment of bone regeneration. Our results indicate that the hybrid delivery system resulted in consistent bony bridging of the challenging bone defects. However, in the absence of rhBMP-2, the use of nanofiber mesh tube and alginate did not result in substantial bone formation. Perforations in the nanofiber mesh accelerated the rhBMP-2 mediated bone repair, and resulted in functional restoration of the regenerated bone. [mu]-CT based angiography indicated that perforations did not significantly affect the revascularization of defects, suggesting that some other interaction with the tissue surrounding the defect such as improved infiltration of osteoprogenitor cells contributed to the observed differences in repair. Overall, our results indicate that the hybrid alginate/nanofiber mesh system is a promising growth factor delivery strategy for the repair of challenging bone injuries.

Adult human articular chondrocytes in a microcarrier-based culture system : expansion and redifferentiation

xpanding human chondrocytes in vitro while maintaining their ability to form cartilage remains a key challenge in cartilage tissue engineering. One promising approach to address this is to use microcarriers as substrates for chondrocyte expansion. While microcarriers have shown beneficial effects for expansion of animal and ectopic human chondrocytes, their utility has not been determined for freshly isolated adult human articular chondrocytes. Thus, we investigated the proliferation and subsequent chondrogenic differentiation of these clinically relevant cells on porous gelatin microcarriers and compared them to those expanded using traditional monolayers. Chondrocytes attached to microcarriers within 2 days and remained viable over 4 weeks of culture in spinner flasks. Cells on microcarriers exhibited a spread morphology and initially proliferated faster than cells in monolayer culture, however, with prolonged expansion they were less proliferative. Cells expanded for 1 month and enzymatically released from microcarriers formed cartilaginous tissue in micromass pellet cultures, which was similar to tissue formed by monolayer-expanded cells. Cells left attached to microcarriers did not exhibit chondrogenic capacity. Culture conditions, such as microcarrier material, oxygen tension, and mechanical stimulation require further investigation to facilitate the efficient expansion of clinically relevant human articular chondrocytes that maintain chondrogenic potential for cartilage regeneration applications.