|Computers, Materials & Continua |
Algorithmic Scheme for Concurrent Detection and Classification of Printed Circuit Board Defects
Faculty of Engineering, Rajamangala University of Technology Thanyaburi (Main Campus), Khlong Luang, Pathum Thani, 12110, Thailand
*Corresponding Author: Jakkree Srinonchat. Email: email@example.com
Received: 07 February 2021; Accepted: 31 August 2021
Abstract: An ideal printed circuit board (PCB) defect inspection system can detect defects and classify PCB defect types. Existing defect inspection technologies can identify defects but fail to classify all PCB defect types. This research thus proposes an algorithmic scheme that can detect and categorize all 14-known PCB defect types. In the proposed algorithmic scheme, fuzzy c-means clustering is used for image segmentation via image subtraction prior to defect detection. Arithmetic and logic operations, the circle hough transform (CHT), morphological reconstruction (MR), and connected component labeling (CCL) are used in defect classification. The algorithmic scheme achieves 100% defect detection and 99.05% defect classification accuracies. The novelty of this research lies in the concurrent use of CHT, MR, and CCL algorithms to accurately detect and classify all 14-known PCB defect types and determine the defect characteristics such as the location, area, and nature of defects. This information is helpful in electronic parts manufacturing for finding the root causes of PCB defects and appropriately adjusting the manufacturing process. Moreover, the algorithmic scheme can be integrated into machine vision to streamline the manufacturing process, improve the PCB quality, and lower the production cost.
Keywords: : PCB inspection; PCB defect types; defect detection; defect classification; image processing
In electronic component manufacturing, a system for defect inspection of printed circuit boards (PCB) prevents defective PCBs from advancing to the subsequent stage. Because of higher efficiency, accuracy, and productivity compared to human inspectors, PCB defect inspection technologies based on automatic image-processing techniques have been increasingly deployed.
There are three PCB defect inspection approaches: referential, non-referential, and hybrid . PCB defects are detected in the referential approach by comparing a PCB image against a template PCB image. At the same time, in the non-referential method, a pre-defined mathematical rule is established to detect PCB defects. As for the hybrid approach, it uses a combination of referential and non-referential methods. However, the mentioned PCB defect inspection approaches mainly concentrate on defect detection, which cannot identify the root causes of defects. Therefore, researchers need to work in both PCB defect detection and PCB defect classification simultaneously.
In PCB fabrication, there are 14-known PCB defect types: Breakout, Pin hole, Open circuit, Under etch, Mouse bite, Missing conductor, Spur, Short circuit, Wrong size hole, Conductor too close, Spurious copper, Excessive short, Missing hole, and Over etch . PCB defects consist of two groups: functional and cosmetic defects . Functional defects cause the PCB to malfunction, whereas cosmetic defects affect the PCB appearance and might impair the performance of the PCB. Fig. 1 illustrates a template PCB and a defective PCB containing 14-known defect types. In , the 14-known PCB defects are further categorized by processes (i.e., drilling and etching) and characteristics (i.e., missing copper and excess copper), as shown in Tab. 1.
In [4–8], image subtraction is used to detect PCB defects, and arithmetic operations are utilized to classify defects. Meanwhile, in [9–11], image subtraction and morphological operations are applied to detect and classify PCB defects. However, these techniques can classify only four to eight defect types (out of 14-known defect types). Convolutional neural network (CNN) methods are employed to classify PCB defects in [12–14], but their classification capabilities are low. In , all 14-known defect types are initially categorized into five classes, and a classification algorithm is applied to classify defect types. However, the algorithm can only separate the defects into five categories and cannot satisfy 14 types. Moreover, the work of  is then improved by applying a segmentation exercise, which is described in [16–18]. These works can only expand five classes to seven classes, and in , the number is improved to 11 categories. However, they still cannot identify all 14 defect types. Since each type of PCB defect occurs due to different causes and processes, the classification of 14 defect types can help determine the cause of the error and therefore reduce long-term production costs.
An image segmentation algorithm based on mathematical and morphological tools is proposed to divide individual PCB images into four segments: hole, pad, thick line, and thin line. The algorithm is usually employed as a pre-processing step for classification tasks . In , a k-nearest neighbor algorithm is utilized to classify the defects, and the algorithm can only classify all 14-known defect types with 70%–80% accuracy. Nonetheless, the high-accuracy defect classification procedure is essential for an inspection system during PCB production processes.
Accordingly, this study proposes an algorithmic scheme to detect and classify all 14-known PCB defect types with at least 90% accuracy. In the proposed defect inspection system, a fuzzy c-means clustering algorithm is used in image segmentation. Image subtraction is utilized for defect detection, and the circle Hough transform (CHT), morphological reconstruction (MR), and connected component labelling (CCL) algorithms are employed in defect classification.
2 The Algorithmic Scheme for PCB Defect Detection and Classification
In this research, the PCB detection and classification system uses the referential approach based on 14-known PCB defect types. The proposed system consists of four steps: pre-processing, image segmentation (Section 2.1), defect detection, and defect classification (Section 2.2).
Pre-Processing and Image Segmentation
Fig. 2 illustrates the flowchart of the pre-processing and image segmentation processes. During the pre-processing stage, RGB color images (containing PCB defects) are converted into grayscale images and median-filtered to remove the noise from images. An affine transform is used to scale, rotate, and translate the median-filtered images before image segmentation using fuzzy c-means clustering (i.e., the constituent images of an input image).
The RGB color images are converted into grayscale images by individual pixels following Eq. (1), where R, G, and B represent red, green, and blue channels in the color image, respectively:
Then, median filtering is used to remove the noise from images, whereby the median grayscale is assigned to individual pixels in the image . The median greyscale is calculated by Eq. (2) as follows:
where is the initial grayscale, is the median greyscale, and W is the 3 × 3 kernel, of which m and n are the number of columns and rows, respectively.
Furthermore, the affine transform algorithm is used to scale, rotate, and transform the median-filtered images. The affine transform algorithm is mathematically expressed in Eq. (3), consisting of six degrees of freedom: and for transformation, and for scaling, and and for rotation :
where and are the affine-transform and the median-filtered images, respectively.
Next, in image segmentation, the fuzzy c-means clustering algorithm is utilized to separate circuit parts, including holes, pads, and traces from the background. The algorithm uses the nearest distance between any given pixel and its cluster centroid using the Euclidean distance as described in Eqs. (4)–(6) :
where J is the nearest distance between a given pixel and its cluster centroid, is a data member of the cluster center, is the centroid of the cluster j, k is the number of clusters, n is the total number of data, and is the data at pixel i and cluster .
In addition, the fuzzy c-means clustering algorithm divides each median-filtered image (i.e., input image) into 32 rectangular-shaped segmental images. The segmental images are subsequently designated into clusters according to standard features and characteristics. The constituent images (or segmental images) belonging to various clusters then undergo defect detection and classification.
In this study, 500 input images contain various degrees of 14-known PCB defects. In segmentation, each input image is divided into 32 segmental images using fuzzy c-means clustering, generating 16,000 segmental images. The segmental images are individually assigned to clusters according to common characteristics, resulting in 100 clusters. For verification purposes, this study utilizes 10,000 segmental images for testing the defect detection and classification algorithm.
Defect Detection and Classification
Fig. 3 illustrates the flowchart of detection and classification of 14-known PCB defects. In defect detection, image subtraction is utilized to compare the segmental images against the reference image (i.e., the defect-free segmental image). A negative value () indicates excess copper (Eq. (7)) and a positive value () means missing copper (Eq. (8)). A negative or positive subtraction value stands for defective PCB, whereas a zero-subtraction value signifies non-defective PCB, as follows:
where is the reference PCB image, and is the defect-containing segmental PCB image.
The PCB defect types with a positive subtraction value include Breakout, Pin hole, Open circuit, Mouse bite, Missing conductor, and Over etch. In contrast, those with a negative subtraction value contain Under etch, Spur, Short circuit, Wrong size hole, Spurious copper, Excessive short, and Missing hole. The Conductor too close has either a positive or negative subtraction value, thereby presenting classification challenges.
CHT, MR, and CCL algorithms are utilized along with image subtraction to enhance defect detection. Specifically, the CHT algorithm detects and classifies circular patterns by transforming a set of feature points in the image space into a set of accumulated votes in parameter space. The votes at each feature point are accumulated in an accumulator array for all parameter combinations. The highest number of votes in the array elements indicates the presence of a circular pattern. A circle is described in Eq. (9), and its parameters are defined in Eqs. (10) and (11) . The hole pad pattern of the defect-containing segmental images () is determined in Eqs. (9)–(11):
where and are the center coordinates, r is the circle radius, and is the rotational angle of the segmental image (0–360°).
The MR algorithm sequentially classifies segmental images by features (i.e., holes, pads, and traces) and then transforms the connected background pixels into foreground pixels until the object boundaries are reached. The MR algorithm for flood-filling is expressed in Eq. (12), where denotes the segmental image and is the flood-fill segmental image :
The CCL algorithm scans segmental images, groups pixels into components by pixel connectivity, and assigns labels to individual pixels until the boundaries are reached . In this research, binary CCL (i.e., 0 and 1) combines the connected copper traces into one group. The number of groups and sizes of the groups are used to classify defect types.
The Algorithmic Scheme of Defect Detection and Defect Classification
The inputs of the algorithmic scheme are , , , , , and , and the outputs are the 14 known PCB defects (denoted as D01-D14). The experiment use the computer hardware with an Intel i5-8250U CPU @ 1.60 GHz and 8 GB RAM. The sequential steps of the algorithmic scheme are as follows:
Step 1: Find excess copper in hole pads using .
Step 2: Apply the CCL algorithm to validate, and CHT classifies the defect as either a Wrong size hole (D09) or a Missing hole (D13).
Step 3: Find missing copper in hole pads using .
Step 4: Apply the CCL algorithm to validate, and CHT classifies the defect as Breakout (D01).
Step 5: Find Pin hole defect (D02) from .
Step 6: Or else, the defect is an Excessive short (D12).
Step 7: Apply the CCL algorithm to determine the number and size of groups.
Step 8: Add negative pixels to the defect-containing segmental image and re-apply the CCL algorithm to determine the number and size of groups.
Step 9: Comparing Steps 7 and 8. If the number and size of groups decrease, the defect is a Short circuit defect (D08). Otherwise, the defect is a Spurious copper defect (D11).
Step 10: Add positive pixels to the defect-containing segmental image and re-apply the CCL algorithm to determine the number and size of groups.
Step 11: Comparing Steps 7 and 10. If the number of groups increases and the group size is smaller than the square of the trace width, the defect is an Open circuit defect (D03). If the number of groups increases and the size of the group is larger than the square of the trace width, it is a Missing conductor defect (D06).
Step 12: If the number of groups remains unchanged, proceed to steps 13–17.
Step 13: If the defect has negative pixels and is asymmetric, it is a Spur defect (D07).
Step 14: If the defect has positive pixels and is asymmetric, it is a Mouse bite defect (D05).
Step 15: If the defect has either negative or positive pixels and is symmetric, it is a Conductor too close defect (D10).
Step 16: If the defect has negative pixels and is symmetric, it is an Under etch defect (D04).
Step 17: If the defect has positive pixels and is symmetric, it is an Over etch defect (D14).
Step 18: If the defect does not fall into any of the 14 categories, it is another defect (D00).
Step 19: About Steps 1–18, all 14-known PCB defect types are detected and classified.
3 Experimental Results and Discussion
PCB Input and Segmental Images
As previously stated, 500 input images contain various degrees of 14-known PCB defects. The input images are segmented using the fuzzy c-means clustering algorithm. In the segmentation, each input image is divided into 32 segmental images, resulting in 16,000 segmental images. The segmental images are individually assigned to one of 100 clusters according to their common characteristics. In the meanwhile, this study uses 10,000 segmental images to test the algorithmic scheme for defect detection and classification. Fig. 4 depicts an input image and corresponding defect-containing segmental images.
Image Segmentation Performance
This study utilizes fuzzy c-means clustering in image segmentation. Before that, simulations are carried out to determine the optimal algorithmic scheme among four alternative image segmentation algorithms, i.e., global thresholding, adaptive thresholding, k-means clustering, and fuzzy c-means clustering. Besides, the performance metrics include intersection over union (IoU), accuracy, precision, recall, and elapse time. The IoU, accuracy, precision, and recall are calculated by Eqs. (13)–(16) . Fig. 5 compares the performance (mean values) of four alternative image segmentation algorithms, and Tab. 2 tabulates their respective standard deviations (SD). The SD is calculated following Eq. (17).
where , , , and are the number of true positives, true negatives, false positives, and false negatives, respectively.
where is a value in the data set, is the mean of the data set, and N is the number of data points in the population.
In Fig. 5, fuzzy c-means clustering achieves the highest precision (99.34%), accuracy (98.62%), recall (98.17%), and IoU (0.9574). Nonetheless, the elapsed time of fuzzy c-means clustering is longer than the other segmentation algorithms, which can be overcome by upgrading the computer processing unit. As shown in Tab. 2, the standard deviations are relatively small, suggesting minimal deviation from the means.
Defect Detection Performance
The image subtraction algorithm is utilized to detect defects on PCB by comparing the segmental images of input images against the reference image (i.e., defect-free segmental image). A negative or positive subtraction value indicates a defective PCB, whereas a zero-subtraction value signifies a non-defective PCB.
The PCB defect types with a positive subtraction value include Breakout, Pin hole, Open circuit, Mouse bite, Missing conductor, and Over etch. In contrast, those with a negative subtraction value include Under etch, Spur, Short circuit, Wrong size hole, Spurious copper, Excessive short, and Missing hole. The Conductor too close defect has either a positive or negative subtraction value. Figs. 6a–6c illustrate an example of defect detection results using image subtraction, with the positive values (as represented by white-colored marks) indicating detected defects. The image subtraction algorithm can achieve 100% defect detection accuracy.
Defect Classification Performance
CHT, MR, and CCL algorithms are utilized to classify PCB defects according to the 14-known PCB defect types (D01-D14). As discussed above, the defect classification is proposed sequentially and terminated when all 14-known defects are algorithmically classified. The experimental results show that the algorithmic scheme can achieve about 99.05% defect classification accuracy.
Fig. 7a illustrates the 14-known PCB defect types, and Figs. 7b–7o illustrate Breakout (D01), Pin hole (D02), Open circuit (D03), Under etch (D04), Mouse bite (D05), Missing conductor (D06), Spur (D07), Short circuit (D08), Wrong size hole (D09), Conductor too close (D10), Spurious copper (D11), Excessive short (D12), Missing hole (D13), and Over etch (D14), respectively.
This research validates the proposed algorithmic scheme by comparing against previously developed methods that including the well-known conventional operations [5–7,9], the convolutional neural networks [11,12], and the k-nearest neighbor method . In Tab. 3, one finds that in [5–7,9,11,12] can identify only some types of defects, while  and the proposed algorithm can detect all 14 defect types. Considering the accuracy perspective, it is clear that the accuracy rate of  is about 70%−80%, while our accuracy rate is 96.5%−100%.
As the proposed method resides at the nexus of the CHT, MR and CCL algorithms, the features of these algorithms allow the performance of the proposed algorithmic scheme to detect defects more comprehensively and more accurately than other methods, as shown in Tab. 3.
This study proposes the PCB defect inspection system using a referential approach. The novelty of this work lies in the concurrent use of arithmetic and logical operations, as well as CHT, MR, and CCL algorithms. This algorithm effectively classifies all 14-known PCB defect types, unlike conventional techniques that can only classify a few kinds of defects or cannot achieve high efficiency. The experimental results reveal that the proposed algorithmic scheme can achieve 100% defect detection efficiency and 99.05% defect classification accuracy, demonstrating that our proposed algorithmic scheme has superior performance to other existing PCB defect classification methods. Given the impressive detection and classification performance, the algorithmic scheme can be integrated into machine vision to streamline the production, improve PCB quality, and lower the production cost as human inspectors will no longer be required for defect classification.
The PCB architecture continues to develop and acquires more complicated structures. The currently designed algorithm may also be required to catch up with the growth of PCB developments. Hence, future work will be focused on detecting and classifying defects occurring during PCB production that exhibits more complex structures and processes.
Acknowledgement: The authors would like to express sincere gratitude to the Signal Processing Research Laboratory, the Faculty of Engineering, Rajamangala University of Technology Thanyaburi, for insight and expertise. Appreciation also goes to Dr. Kan Kanjana of KB Systems (Thailand) Co., Ltd., for valuable comments and suggestions. We also thank TopEdit (https://www.topeditsci.com) for its linguistic assistance during the preparation of this manuscript.
Funding Statement: This research is supported by the National Research Council of Thailand (NRCT). Project ID: 618211.
Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.
|This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.|