Design and development of automatic visual inspection system for PCB manufacturing

Inspection of solder joints has been a critical process in the electronic manufacturing industry to reduce manufacturing cost, improve yield, and ensure product quality and reliability. This paper proposes two inspection modules for an automatic solder joint classification system. The “front-end” inspection system includes illumination normalisation, localisation and segmentation. The “back-end” inspection involves the classification of solder joints using the Log Gabor filter and classifier fusion. Five different levels of solder quality with respect to the amount of solder paste have been defined. The Log Gabor filter has been demonstrated to achieve high recognition rates and is resistant to misalignment. This proposed system does not need any special illumination system, and the images are acquired by an ordinary digital camera. This system could contribute to the development of automated non-contact, non-destructive and low cost solder joint quality inspection systems.

Design of automatic vision-based inspection system for solder joint segmentation

Purpose: Computer vision has been widely used in the inspection of electronic components. This paper proposes a computer vision system for the automatic detection, localisation, and segmentation of solder joints on Printed Circuit Boards (PCBs) under different illumination conditions. Design/methodology/approach: An illumination normalization approach is applied to an image, which can effectively and efficiently eliminate the effect of uneven illumination while keeping the properties of the processed image the same as in the corresponding image under normal lighting conditions. Consequently special lighting and instrumental setup can be reduced in order to detect solder joints. These normalised images are insensitive to illumination variations and are used for the subsequent solder joint detection stages. In the segmentation approach, the PCB image is transformed from an RGB color space to a YIQ color space for the effective detection of solder joints from the background. Findings: The segmentation results show that the proposed approach improves the performance significantly for images under varying illumination conditions. Research limitations/implications: This paper proposes a front-end system for the automatic detection, localisation, and segmentation of solder joint defects. Further research is required to complete the full system including the classification of solder joint defects. Practical implications: The methodology presented in this paper can be an effective method to reduce cost and improve quality in production of PCBs in the manufacturing industry. Originality/value: This research proposes the automatic location, identification and segmentation of solder joints under different illumination conditions.

Automatic solder joint defect classification using the Log-Gabor filter

This paper proposes the validity of a Gabor filter bank for feature extraction of solder joint images on Printed Circuit Boards (PCBs). A distance measure based on the Mahalanobis Cosine metric is also presented for classification of five different types of solder joints. From the experimental results, this methodology achieved high accuracy and a well generalised performance. This can be an effective method to reduce cost and improve quality in the production of PCBs in the manufacturing industry.

Vision-based classification of solder joint defects

Inspection of solder joints has been a critical process in the electronic manufacturing industry to reduce manufacturing cost, improve yield, and ensure product quality and reliability. The solder joint inspection problem is more challenging than many other visual inspections because of the variability in the appearance of solder joints. Although many research works and various techniques have been developed to classify defect in solder joints, these methods have complex systems of illumination for image acquisition and complicated classification algorithms. An important stage of the analysis is to select the right method for the classification. Better inspection technologies are needed to fill the gap between available inspection capabilities and industry systems. This dissertation aims to provide a solution that can overcome some of the limitations of current inspection techniques. This research proposes two inspection steps for automatic solder joint classification system. The “front-end” inspection system includes illumination normalisation, localization and segmentation. The illumination normalisation approach can effectively and efficiently eliminate the effect of uneven illumination while keeping the properties of the processed image. The “back-end” inspection involves the classification of solder joints by using Log Gabor filter and classifier fusion. Five different levels of solder quality with respect to the amount of solder paste have been defined. Log Gabor filter has been demonstrated to achieve high recognition rates and is resistant to misalignment. Further testing demonstrates the advantage of Log Gabor filter over both Discrete Wavelet Transform and Discrete Cosine Transform. Classifier score fusion is analysed for improving recognition rate. Experimental results demonstrate that the proposed system improves performance and robustness in terms of classification rates. This proposed system does not need any special illumination system, and the images are acquired by an ordinary digital camera. In fact, the choice of suitable features allows one to overcome the problem given by the use of non complex illumination systems. The new system proposed in this research can be incorporated in the development of an automated non-contact, non-destructive and low cost solder joint quality inspection system.

Automatic inspection of metallic surface defects using genetic algorithms

This paper is concerned with the problem of automatic inspection of metallic surface using machine vision. An experimental system has been developed to take images of external metallic surfaces and an intelligent approach based on morphology and genetic algorithms is proposed to detect structural defects on bumpy metallic surfaces. The approach employs genetic algorithms to automatically learn morphology processing parameters such as structuring elements and defect segmentation threshold. This paper describes the detailed procedures which include encoding scheme, genetic operation and evaluation function.

The proposed method has been implemented and tested on a number of metallic surfaces. The results suggest that the method can provide an accurate identification to the defects and can be developed into a viable commercial visual inspection system.

Machine vision system for automatic inspection of surface defects in aluminum die casting

A camera based machine vision system for the automatic inspection of surface defects in aluminum die casting is presented. The system uses a hybrid image processing algorithm based on mathematic morphology to detect defects with different sizes and shapes. The defect inspection algorithm consists of two parts. One is a parameter learning algorithm, in which a genetic algorithm is used to extract optimal structuring element parameters, and segmentation and noise removal thresholds. The second part is a defect detection algorithm, in which the parameters obtained by a genetic algorithm are used for morphological operations. The machine vision system has been applied in an industrial setting to detect two types of casting defects: parts mix-up and any defects on the surface of castings. The system performs with a 99% or higher accuracy for both part mix-up and defect detection and is currently used in industry as part of normal production.

Machine vision system for automatic inspection of surface defects in aluminium die casting

A machine vision system is presented for the automatic inspection of surface defects in aluminium die casting. The system uses a hybrid image processing algorithm based on mathematic morphology to detect defects with different sizes and shapes. The defect inspection algorithm consists of two parts. One is a parameter learning algorithm, in which a genetic algorithm is used to extract optimal structuring element parameters, and segmentation and noise removal thresholds. The second part is a defect detection algorithm, in which the parameters obtained by a genetic algorithm are used for morphological operations. The machine vision system has been applied in an industrial setting to detect two types of casting defects: parts mix-up and any defects on the surface of castings. The system performs with a 99% or higher accuracy for both part mix-up and defect detection and is currently used in industry as part of normal production.

Sistema para formação e avaliação dos operadores das máquinas de inspeção ótica automática

A montagem de circuitos eletrónicos é um processo extremamente complexo, e como tal muito difícil de controlar. Ao longo do processo produtivo, é colocada solda no PCB (printed circuit board), seguidamente são colocados os componentes eletrónicos que serão depois soldados através de um sistema de convecção, sendo por fim inspecionados todos os componentes, com o intuito de detetar eventuais falhas no circuito. Esta inspeção é efetuada por uma máquina designada por AOI (automatic optical inspection), que através da captura de várias imagens do PCB, analisa cada uma, utilizando algoritmos de processamento de imagem como forma de verificar a presença, colocação e soldadura de todos os componentes. Um dos grandes problemas na classificação dos defeitos relaciona-se com a quantidade de defeitos mal classificados que passam para os processos seguintes, por análise errada por parte dos operadores. Assim, apenas com uma formação adequada, realizada continuamente, é possível garantir uma menor taxa de falhas por parte dos operadores e consequentemente um aumento na qualidade dos produtos. Através da implementação da metodologia Gage R&R para atributos, que é parte integrante da estratégia “six sigma” foi possível analisar a aptidão dos operadores, com base na repetição aleatória de várias imagens. Foi desenvolvido um software que implementa esta metodologia na formação dos operadores das máquinas AOI, de forma a verificar a sua aptidão, tendo como objetivo a melhoria do seu desempenho futuro, através da medição e quantificação das dificuldades de cada pessoa. Com esta nova sistemática foi mais fácil entender a necessidade de formação de cada operador, pois com a constante evolução dos componentes eletrónicos e com o surgimento de novos componentes, estão implícitas novas dificuldades para os operadores neste tipo de tarefa. Foi também possível reduzir o número de defeitos mal classificados de forma significativa, através da aposta na formação com o auxílio do software desenvolvido.

Advanced predictive quality control strategy involving different facilities

There are many industries that use highly technological solutions to improve quality in all of their products. The steel industry is one example. Several automatic surface-inspection systems are used in the steel industry to identify various types of defects and to help operators decide whether to accept, reroute, or downgrade the material, subject to the assessment process. This paper focuses on promoting a strategy that considers all defects in an integrated fashion. It does this by managing the uncertainty about the exact position of a defect due to different process conditions by means of Gaussian additive influence functions. The relevance of the approach is in making possible consistency and reliability between surface inspection systems. The results obtained are an increase in confidence in the automatic inspection system and an ability to introduce improved prediction and advanced routing models. The prediction is provided to technical operators to help them in their decision-making process. It shows the increase in improvement gained by reducing the 40 % of coils that are downgraded at the hot strip mill because of specific defects. In addition, this technology facilitates an increase of 50 % in the accuracy of the estimate of defect survival after the cleaning facility in comparison to the former approach. The proposed technology is implemented by means of software-based, multi-agent solutions. It makes possible the independent treatment of information, presentation, quality analysis, and other relevant functions.

Inspection of teats by colour image analysis for automatic milking systems

C.R. Bull, N.J.B. McFarlane, R. Zwiggelaar, C.J. Allen and T.T. Mottram, 'Inspection of teats by colour image analysis for automatic milking systems', Computers and Electronics in Agriculture 15 (1), 15-26 (1996)

Towards automatic power line detection for a UAV surveillance system using pulse coupled neural filter and an improved Hough transform

Spatial information captured from optical remote sensors on board unmanned aerial vehicles (UAVs) has great potential in automatic surveillance of electrical infrastructure. For an automatic vision-based power line inspection system, detecting power lines from a cluttered background is one of the most important and challenging tasks. In this paper, a novel method is proposed, specifically for power line detection from aerial images. A pulse coupled neural filter is developed to remove background noise and generate an edge map prior to the Hough transform being employed to detect straight lines. An improved Hough transform is used by performing knowledge-based line clustering in Hough space to refine the detection results. The experiment on real image data captured from a UAV platform demonstrates that the proposed approach is effective for automatic power line detection.

Automatic tracking, super-resolution and recognition of human faces from surveillance video

Identifying an individual from surveillance video is a difficult, time consuming and labour intensive process. The proposed system aims to streamline this process by filtering out unwanted scenes and enhancing an individual's face through super-resolution. An automatic face recognition system is then used to identify the subject or present the human operator with likely matches from a database. A person tracker is used to speed up the subject detection and super-resolution process by tracking moving subjects and cropping a region of interest around the subject's face to reduce the number and size of the image frames to be super-resolved respectively. In this paper, experiments have been conducted to demonstrate how the optical flow super-resolution method used improves surveillance imagery for visual inspection as well as automatic face recognition on an Eigenface and Elastic Bunch Graph Matching system. The optical flow based method has also been benchmarked against the ``hallucination'' algorithm, interpolation methods and the original low-resolution images. Results show that both super-resolution algorithms improved recognition rates significantly. Although the hallucination method resulted in slightly higher recognition rates, the optical flow method produced less artifacts and more visually correct images suitable for human consumption.

Compensation of unmodeled aircraft dynamics in airborne inspection of linear infrastructure assets

Fixed-wing aircraft equipped with downward pointing cameras and/or LiDAR can be used for inspecting approximately piecewise linear assets such as oil-gas pipelines, roads and power-lines. Automatic control of such aircraft is important from a productivity and safety point of view (long periods of precision manual flight at low-altitude is not considered reasonable from a safety perspective). This paper investigates the effect of any unwanted coupling between guidance and autopilot loops (typically caused by unmodeled delays in the aircraft’s response), and the specific impact of any unwanted dynamics on the performance of aircraft undertaking inspection of piecewise linear corridor assets (such as powerlines). Simulation studies and experimental flight tests are used to demonstrate the benefits of a simple compensator in mitigating the unwanted lateral oscillatory behaviour (or coupling) that is caused by unmodeled time constants in the aircraft dynamics.