Cascaded foreign object detection in manufacturing processes using convolutional neural networks and synthetic data generation methodology

Tang, Jixiang; Zhou, Huan; Wang, Tiankui; Zhang, Jin; Wang, Youli; Wang, Xuanyin

doi:10.1007/s10845-022-01976-3

Cited by 16 publications

(4 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some researchers 14,15 also adopt methods based on deep learning to achieve tobacco grading. Focusing on the detection of foreign objects in tobacco production, Tang et al 16 solve the scarcity of images with foreign objects by synthetic data generation methodology and complete the detection of foreign objects through a cascade convolutional neural network (CNN). Chao et al 17 implement the color sorting table method and gray threshold method to inspect multiple foreign objects on the conveyor belt.…”

Section: Related Workmentioning

confidence: 99%

Visual detection of residual cut tobacco based on multi-feature fusion and Markov random field

Liu

Zhang

et al. 2022

J. Electron. Imag.

Self Cite

View full text Add to dashboard Cite

Image segmentation has been commonly used for segmenting regions of interest in the industrial field. However, segmenting small objects in a reflective background is a complex task that limits the performance of image segmentation in the industry. We take residual cut tobacco in industrial production as an object and propose an automatic detection method. This method overcomes the influence of reflective background and achieves accurate detection of small-area residual cut tobacco. The proposed method combines multi-feature fusion method and Markov random field model to construct a coarse-to-fine segmentation framework. The coarse part is used to eliminate the effect of specular highlights and specular reflection. The fine part is used to detect and clear small regions of residual cut tobacco that are wrongly segmented by the coarse part. We collect 2411 images in the real industrial production line for evaluation. The experimental results show that the proposed method yields an averaged precision and mean intersection over union of 95.85% and 86.71%, which outperforms some detection methods based on deep learning.

show abstract

Section: Related Workmentioning

confidence: 99%

Visual detection of residual cut tobacco based on multi-feature fusion and Markov random field

Liu

Zhang

et al. 2022

J. Electron. Imag.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Additionally, synthetic data are more sustainable and can be generated in potentially unlimited numbers with pre-annotation, which would otherwise be required to be added manually to the realworld data [33]. Synthetic data have been successfully demonstrated to enhance model performance [31][32][33][34][35]. Additionally, it is important to highlight that the Medicines and Healthcare products Agency (MHRA) has introduced two synthetic datasets that have played a significant role in advancing medical technologies in the fight against COVID-19 and cardiovascular disease.…”

Section: Introductionmentioning

confidence: 99%

Virtually Possible: Enhancing Quality Control of 3D-Printed Medicines with Machine Vision Trained on Photorealistic Images

Sun,

Alkahtani,

Gaisford

et al. 2023

Pharmaceutics

View full text Add to dashboard Cite

Three-dimensional (3D) printing is an advanced pharmaceutical manufacturing technology, and concerted efforts are underway to establish its applicability to various industries. However, for any technology to achieve widespread adoption, robustness and reliability are critical factors. Machine vision (MV), a subset of artificial intelligence (AI), has emerged as a powerful tool to replace human inspection with unprecedented speed and accuracy. Previous studies have demonstrated the potential of MV in pharmaceutical processes. However, training models using real images proves to be both costly and time consuming. In this study, we present an alternative approach, where synthetic images were used to train models to classify the quality of dosage forms. We generated 200 photorealistic virtual images that replicated 3D-printed dosage forms, where seven machine learning techniques (MLTs) were used to perform image classification. By exploring various MV pipelines, including image resizing and transformation, we achieved remarkable classification accuracies of 80.8%, 74.3%, and 75.5% for capsules, tablets, and films, respectively, for classifying stereolithography (SLA)-printed dosage forms. Additionally, we subjected the MLTs to rigorous stress tests, evaluating their scalability to classify over 3000 images and their ability to handle irrelevant images, where accuracies of 66.5% (capsules), 72.0% (tablets), and 70.9% (films) were obtained. Moreover, model confidence was also measured, and Brier scores ranged from 0.20 to 0.40. Our results demonstrate promising proof of concept that virtual images exhibit great potential for image classification of SLA-printed dosage forms. By using photorealistic virtual images, which are faster and cheaper to generate, we pave the way for accelerated, reliable, and sustainable AI model development to enhance the quality control of 3D-printed medicines.

show abstract

“…However, the accuracy of this method is greatly affected by the location of the target and the surrounding environment due to the instability of the color value. Tang et al [10]. proposed a cascaded foreign object detector (CFOD) based on convolutional neural networks (CNN) for the same industrial situation and achieved the localization and detection of tobacco parcels through cascaded networks.…”

Section: Introductionmentioning

confidence: 99%

Few-Shot Learning for Small Impurities in Tobacco Stems With Improved YOLOv7

Xue

et al. 2023

IEEE Access

View full text Add to dashboard Cite

With the increase of public concern about health and smoking, the authorities have gradually tightened the control of tar content in cigarettes, making reconstituted tobacco a growing concern for tobacco companies. Tobacco stems are used as the main raw material for reconstituted tobacco, but they contain a large number of small broken impurities mainly from cigarette butts, which are difficult to remove efficiently by air selection and manual methods. Detection schemes for cigarette butt impurities based on computer vision and deep learning are still difficult. The scarcity of images containing foreign impurities in cigarette butts and the small size of impurities limit the efficient application of deep learning algorithms. In view of the small impurities' characteristics, this paper optimizes the model structure of the YOLOv7 algorithm, and only retains the two detection head structures with high feature resolution, which reduces the model parameters by 29.68%. Using online data augmentation and transfer learning, the difficulty of small sample datasets is overcome. After the CutMix, Mosaic, Affine transformation, Copy-paste data augmentation in this paper, the model precision is increased by 6.95%, and the recall rate is increased by 10.51%. Detection FPS has been increased from 99 FPS to 111 FPS. Precision and recall rate reached 97.21% and 92.11%. Compared with YOLOv4_csp, the precision is in-creased by 11.58%, and the recall rate is increased by 0.48%. It shows that the improved YOLOv7xs model has the potential for wide application in small target recognition. At the same time, it has shown the potential to avoid the harm of toxic substances produced by cigarette impurities in the combustion process and promotes the application of computer vision and deep learning in industrial production.

show abstract

Cascaded foreign object detection in manufacturing processes using convolutional neural networks and synthetic data generation methodology

Cited by 16 publications

References 34 publications

Visual detection of residual cut tobacco based on multi-feature fusion and Markov random field

Visual detection of residual cut tobacco based on multi-feature fusion and Markov random field

Virtually Possible: Enhancing Quality Control of 3D-Printed Medicines with Machine Vision Trained on Photorealistic Images

Few-Shot Learning for Small Impurities in Tobacco Stems With Improved YOLOv7

Contact Info

Product

Resources

About