Color Normalization for Robust Automatic Bill of Materials Generation and Visual Inspection of PCBs

Abstract

A Bill of Materials (BoM) is the list of all components present on a Printed Circuit Board (PCB). BoMs are useful for multiple forms of failure analysis and hardware assurance. In this paper, we build upon previous work and present an updated framework to automatically extract a BoM from optical images of PCBs in order to keep up to date with technological advancements. This is accomplished by revising the framework to emphasize the role of machine learning and by incorporating domain knowledge of PCB design and hardware Trojans. For accurate machine learning methods, it is critical that the input PCB images are normalized. Hence, we explore the effect of imaging conditions (e.g. camera type, lighting intensity, and lighting color) on component classification, before and after color correction. This is accomplished by collecting PCB images under a variety of imaging conditions and conducting a linear discriminant analysis before and after color checker profile correction, a method commonly used in photography. This paper shows color correction can effectively reduce the intraclass variance of different PCB components, which results in a higher component classification accuracy. This is extremely desirable for machine learning methods, as increased prior knowledge can decrease the number of ground truth images necessary for training. Finally, we detail the future work for data normalization for more accurate automatic BoM extraction. Index Terms – automatic visual inspection; PCB reverse engineering; PCB competitor analysis; hardware assurance; bill of materials

FPIC: A Novel Semantic Dataset for Optical PCB Assurance

“…A portion of the dataset was collected with widely varying illumination and camera sensor parameters in order to study the effects of normalization on the raw data [47].…”

PCB Component Detection Using Computer Vision for Hardware Assurance

“…Over the years, physical inspection in the optical domain has become a popular approach within the community due to its mostly non-contact and non-destructive nature [3]. Traditionally, a subject matter expert (SME) performs visual inspections of PCBs under certain controlled conditions.…”

“…Application of an automatic Bill of Materials extraction method to tackle this problem was introduced in [3]. The proposed framework of materials extraction for PCB assurance (as shown in Figure 1) involves two major steps: (1) imaging modality, and (2) image analysis.…”

“…In this paper, we focus on the image analysis step where the goal is to detect and identify the components in a PCB with high accuracy. While developing an efficient system for detecting components automatically, it should be kept in mind that the method needs to be fast and highly accurate to meet the requirements of critical applications (e.g., military and biomedical applications) [3]. In addition, there are a wide variety of factors and challenges which needs special attention while developing an automated PCB assurance system.…”

“…Additionally, in the case of foreign, competitor, and/or malicious technologies, the components may be intentionally obfuscated with uncommon designs, camouflaging, and misleading or absent silkscreen labels. Moreover, factors related to image acquisition systems can also significantly affect the performance of developed component detection and identification algorithm [3]. Unfortunately, such challenges are very difficult to overcome with existing traditional computer vision (CV) and image understanding methods for component detection and identification, alone [4].…”