Deep learning-powered visual inspection for metal surfaces: A systemic image-centric approach for small size defect detection

Abstract

Small-size defect detection is important in the manufacturing industry for example, missing a hairline crack of less than 3 mm (length and width) on a cylinder head may cause part failure. Computer vision techniques have shown promise in automating manual inspection processes. Computer vision applies deep learning (DL) algorithms where multiple processing layers are used to extract high-level features from a large dataset. However, using DL algorithms is challenging in lieu of several factors, such as varying illumination, difficulty localizing a defect due to its size, defect accessibility, incorrect annotations, etc. Real factory data on small-size defects were obtained from our industry partner, John Deere. The experiments showed that when humans and DL agree on a decision, their combined chance of being correct is above 90%, whereas when they disagree, the probability of human inspectors being correct drops to 63.3%. This indicates that DL favorably complements manual inspection. We then considered using DL algorithms for small-size defect detection. Researchers have studied metal surface defect detection on limited data using several DL algorithms with a main focus on comparing algorithm performance on a benchmark (GC10 DET) dataset with the assumption of the benchmark data annotation as the ground truth. No conclusive studies on the impact of data validation on DL models trained on limited data have been conducted to the best of our knowledge. We considered improving the data quality and found that improvements in GC10DET data yielded a 10-30% performance increase compared to existing approaches under the same conditions. The more data a DL model is trained with, the better it will recognize a new small-size defect. During the current transition of industries from manual to automated inspection, data is scarce. Previous works presented the best-performing results on small data and may not represent the true quality of the results. In contrast, we used the bootstrapping technique to achieve 95% confidence in the performance of a DL algorithm (a 95% recall and mAP were achieved). Lastly, we evaluated a popular template matching approach (multi-template matching) against a popular DL algorithm (YOLOv5) and found that the latter performs significantly better. This dissertation will have a broad impact on the several phases towards automated inspection of metal surface defects ranging from data preparation, model synthesis, and ensuring repeatability and reproducibility in real industrial settings.