Automated Defect Detection: An End-to-End Pipeline from Data to Deployment

Abstract

Defects are a natural part of every manufacturing process and can lead to financial losses as well as pose substantial safety risks if not efficiently screened for. The recent advent of advanced computer vision algorithms present a real-time means for high-accuracy defect screening, but run into serious training bottlenecks due to the limited size of defect data sets. Additionally, potentially fluctuating machine and material conditions means that the defect in question can change, necessitating a re-training of the entire model. This can be a highly costly process and also lead to catastrophic forgetting of the first defective set. This thesis presents a custom end-to-end pipeline for automated defect detection, surmounting the aforementioned obstacles. The data set was augmented using both classical methods and a generative adversarial network. A custom object detection algorithm was then built that substantially outperforms YOLO and Faster-RCNN for the task at hand. A binary network classifier with a custom loss function to punish false negatives was created, achieving both a high classification accuracy and an ability to remember prior defect types, thereby removing the need for costly retraining. These computer vision components were then implemented in both simulation and real-world deployment, using pybullet and a Franka PID-control robot arm respectively. Using DPPG and PPO reinforcement learning methods, the dynamic behavior of object grasping was optimized via simulation for the context of defect detection. Finally, experimental trials were conducted with this complete pipeline, demonstrating robustness to false negatives as well as a 84.4\% overall success rate.