Augmenting human-guided progressive learning with machine vision systems for robust surface defect detection

Machine vision systems commonly utilize Convolutional Neural Networks (CNNs) for in-line surface defect detection of manufactured components. The prediction abilities of vision-based inspection systems deteriorate with time as the defect detection model trained on fixed image datasets fails to accommodate deviations. This paper proposes a human-guided progressive learning approach that systematically imparts learning of new features to the CNN-powered vision-based defect detection system. The approach augments the surface defect detection model with human intelligence, using an intuitive user interface to address model drift. The human expert monitors the trained model performance under specific conditions leading to the change of characteristics during implementation, identifies misclassifications, and initiates re-training. The algorithm accumulates misclassified data till a pre-defined threshold level is reached or a human expert terminates inspection. The misclassified results merge with the original datasets for progressive re-training using a strategy similar to the base model development. The present work utilizes pre-trained CNN Efficientnet-b0 to develop the surface defect detection model for tapered roller inspection through transfer learning. It is concluded that the progressive re-training improves defect detection performance and reduces misclassifications. The Matthews Correlation Coefficient (MCC) score, derived from the confusion matrix, showed improvement from 0.6 to 0.82 after four iterations. A cross-model benchmarking study is also performed to show the versatility of the proposed approach. The present work demonstrated that the human-guided progressive learning approach can provide adaptability to vision-based surface defect detection utilizing deep learning algorithms and enhance system performance during real-time implementation.