Neural Network Compression for Visual Quality Control in Industrial Edge Computing
Resumo
Ensuring that customers receive defect-free products is a big challenge for industries. Visual inspection is a method that can retain defective products before they leave the production line. Deep Learning has enabled us to automate this process using very complex neural networks. However, edge devices are resource-constrained, and the deployment of very deep models may not be feasible. To address this problem, our methodology investigated the application of Knowledge Distillation and Network Quantization to reduce the requirement for resources while trying not to lose performance, enabling deployment on edge computing devices. Knowledge Distillation consists of training a small model (student) under the guidance of a complex one (teacher). Network Quantization reduces the numerical precision of the weights to further decrease the storage size of the model. Our methodology was evaluated in the MVTecAD, DML, and VAD datasets. The results were compared between the teacher and student models. To measure the benefits brought by compressed techniques, we also fine-tuned the student model without the teacher’s guidance and compared it to our approach. The results show that our methodology reduced the model size to 5.5% of the teacher’s size to the best-case scenario, being 36 times faster, while preserving similar performance to the complex model for most objects.Referências
G. Pang, C. Shen, L. Cao, and A. V. D. Hengel, “Deep learning for anomaly detection: A review,” ACM computing surveys (CSUR), vol. 54, no. 2, pp. 1–38, 2021.
J. Liu, G. Xie, J. Wang, S. Li, C. Wang, F. Zheng, and Y. Jin, “Deep industrial image anomaly detection: A survey,” Machine Intelligence Research, vol. 21, no. 1, pp. 104–135, 2024.
Z. Li, H. Li, and L. Meng, “Model compression for deep neural networks: A survey,” Computers, vol. 12, no. 3, p. 60, 2023.
M. A. Ponti, F. P. dos Santos, L. S. Ribeiro, and G. B. Cavallari, “Training deep networks from zero to hero: avoiding pitfalls and going beyond,” in 2021 34th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE, 2021, pp. 9–16.
W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge computing: Vision and challenges,” IEEE internet of things journal, vol. 3, no. 5, pp. 637–646, 2016.
A. Lopes, F. P. dos Santos, D. de Oliveira, M. Schiezaro, and H. Pedrini, “Computer vision model compression techniques for embedded systems: A survey,” Computers & Graphics, vol. 123, p. 104015, 2024.
L. Ruff, J. R. Kauffmann, R. A. Vandermeulen, G. Montavon, W. Samek, M. Kloft, T. G. Dietterich, and K.-R. Müller, “A unifying review of deep and shallow anomaly detection,” Proceedings of the IEEE, vol. 109, no. 5, pp. 756–795, 2021.
Z. Liu, Y. Zhou, Y. Xu, and Z. Wang, “Simplenet: A simple network for image anomaly detection and localization,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 20 402–20 411.
C.-L. Li, K. Sohn, J. Yoon, and T. Pfister, “Cutpaste: Self-supervised learning for anomaly detection and localization,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 9664–9674.
P. Mishra, R. Verk, D. Fornasier, C. Piciarelli, and G. L. Foresti, “Vt-adl: A vision transformer network for image anomaly detection and localization,” in 2021 IEEE 30th International Symposium on Industrial Electronics (ISIE). IEEE, 2021, pp. 01–06.
A. Aboah, “A vision-based system for traffic anomaly detection using deep learning and decision trees,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 4207–4212.
A. S. Khan, Z. Ahmad, J. Abdullah, and F. Ahmad, “A spectrogram image-based network anomaly detection system using deep convolutional neural network,” IEEE access, vol. 9, pp. 87 079–87 093, 2021.
H. S. Mputu, A. Abdel-Mawgood, A. Shimada, and M. S. Sayed, “Tomato quality classification based on transfer learning feature extraction and machine learning algorithm classifiers,” IEEE Access, 2024.
M. Rudolph, T. Wehrbein, B. Rosenhahn, and B. Wandt, “Asymmetric student-teacher networks for industrial anomaly detection,” in Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2023, pp. 2592–2602.
W. Zhang, C. Xie, and L. Xu, “Knowledge distillation-based multi-scale feature matching for industrial anomaly detection in intelligent manufacturing,” in 2024 IEEE International Conference on e-Business Engineering (ICEBE), 2024, pp. 46–52.
X. Chen, L. Cao, S. Zhang, X. Zheng, and Y. Zhang, “Breaking the bias: Recalibrating the attention of industrial anomaly detection,” arXiv preprint arXiv:2412.08189, 2024.
J. Chen and X. Ran, “Deep learning with edge computing: A review,” pp. 1655–1674, 8 2019.
X. Wang, Y. Han, V. C. Leung, D. Niyato, X. Yan, and X. Chen, “Convergence of edge computing and deep learning: A comprehensive survey,” pp. 869–904, 4 2020.
J.-H. Luo, H. Zhang, H.-Y. Zhou, C.-W. Xie, J. Wu, and W. Lin, “Thinet: Pruning cnn filters for a thinner net,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 10, pp. 2525–2538, 2019.
A. G. Howard, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861, 2017.
J. Gu, Z. Wang, J. Kuen, L. Ma, A. Shahroudy, B. Shuai, T. Liu, X. Wang, G. Wang, J. Cai, and T. Chen, “Recent advances in convolutional neural networks,” Pattern Recognition, vol. 77, pp. 354–377, 2018. [Online]. Available: [link]
F. P. dos Santos and M. A. Ponti, “Alignment of local and global features from multiple layers of convolutional neural network for image classification,” in 2019 32nd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE, 2019, pp. 241–248.
J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, “How transferable are features in deep neural networks?” in Advances in neural information processing systems, 2014, pp. 3320–3328.
L. Jiao and J. Zhao, “A survey on the new generation of deep learning in image processing,” IEEE Access, vol. 7, pp. 172 231–172 263, 2019.
Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, “A survey of convolutional neural networks: analysis, applications, and prospects,” IEEE transactions on neural networks and learning systems, vol. 33, no. 12, pp. 6999–7019, 2021.
K. He, X. Zhang, S. Ren, and J. Sun, “Identity mappings in deep residual networks,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14. Springer, 2016, pp. 630–645.
S. O. Haroon-Sulyman, S. S. Kamaruddin, F. K. Ahmad, S. Abdul-Rahman, and N. I. Aziz, “Techniques for mitigating the vanishing gradient problem in deep neural networks,” in Knowledge Management International Conference. Springer, 2025, pp. 428–438.
A. Howard, M. Sandler, G. Chu, L.-C. Chen, B. Chen, M. Tan, W. Wang, Y. Zhu, R. Pang, V. Vasudevan et al., “Searching for mobilenetv3,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 1314–1324.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248–255.
J. Gou, B. Yu, S. J. Maybank, and D. Tao, “Knowledge distillation: A survey,” International Journal of Computer Vision, vol. 129, pp. 1789–1819, 6 2021.
G. Hinton, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015.
T. Kim, J. Oh, N. Kim, S. Cho, and S.-Y. Yun, “Comparing kullback-leibler divergence and mean squared error loss in knowledge distillation,” arXiv preprint arXiv:2105.08919, 2021.
T. Liang, J. Glossner, L. Wang, S. Shi, and X. Zhang, “Pruning and quantization for deep neural network acceleration: A survey,” Neurocomputing, vol. 461, pp. 370–403, 2021.
P. Bergmann, M. Fauser, D. Sattlegger, and C. Steger, “Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 9592–9600.
Karakurai Inc., “Real-world dataset for anomaly detection,” 2025, acessado em: 23 de maio de 2025. [Online]. Available: [link]
A. Baitieva, D. Hurych, V. Besnier, and O. Bernard, “Supervised anomaly detection for complex industrial images,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 17 754–17 762.
M. Sokolova and G. Lapalme, “A systematic analysis of performance measures for classification tasks,” Information processing & management, vol. 45, no. 4, pp. 427–437, 2009.
D. P. Kingma, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
D. Masters and C. Luschi, “Revisiting small batch training for deep neural networks,” arXiv preprint arXiv:1804.07612, 2018.
D. Walsh, “Occam’s razor: A principle of intellectual elegance,” American Philosophical Quarterly, vol. 16, no. 3, pp. 241–244, 1979.
S. Geman, E. Bienenstock, and R. Doursat, “Neural networks and the bias/variance dilemma,” Neural computation, vol. 4, no. 1, pp. 1–58, 1992.
J. Liu, G. Xie, J. Wang, S. Li, C. Wang, F. Zheng, and Y. Jin, “Deep industrial image anomaly detection: A survey,” Machine Intelligence Research, vol. 21, no. 1, pp. 104–135, 2024.
Z. Li, H. Li, and L. Meng, “Model compression for deep neural networks: A survey,” Computers, vol. 12, no. 3, p. 60, 2023.
M. A. Ponti, F. P. dos Santos, L. S. Ribeiro, and G. B. Cavallari, “Training deep networks from zero to hero: avoiding pitfalls and going beyond,” in 2021 34th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE, 2021, pp. 9–16.
W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge computing: Vision and challenges,” IEEE internet of things journal, vol. 3, no. 5, pp. 637–646, 2016.
A. Lopes, F. P. dos Santos, D. de Oliveira, M. Schiezaro, and H. Pedrini, “Computer vision model compression techniques for embedded systems: A survey,” Computers & Graphics, vol. 123, p. 104015, 2024.
L. Ruff, J. R. Kauffmann, R. A. Vandermeulen, G. Montavon, W. Samek, M. Kloft, T. G. Dietterich, and K.-R. Müller, “A unifying review of deep and shallow anomaly detection,” Proceedings of the IEEE, vol. 109, no. 5, pp. 756–795, 2021.
Z. Liu, Y. Zhou, Y. Xu, and Z. Wang, “Simplenet: A simple network for image anomaly detection and localization,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 20 402–20 411.
C.-L. Li, K. Sohn, J. Yoon, and T. Pfister, “Cutpaste: Self-supervised learning for anomaly detection and localization,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 9664–9674.
P. Mishra, R. Verk, D. Fornasier, C. Piciarelli, and G. L. Foresti, “Vt-adl: A vision transformer network for image anomaly detection and localization,” in 2021 IEEE 30th International Symposium on Industrial Electronics (ISIE). IEEE, 2021, pp. 01–06.
A. Aboah, “A vision-based system for traffic anomaly detection using deep learning and decision trees,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 4207–4212.
A. S. Khan, Z. Ahmad, J. Abdullah, and F. Ahmad, “A spectrogram image-based network anomaly detection system using deep convolutional neural network,” IEEE access, vol. 9, pp. 87 079–87 093, 2021.
H. S. Mputu, A. Abdel-Mawgood, A. Shimada, and M. S. Sayed, “Tomato quality classification based on transfer learning feature extraction and machine learning algorithm classifiers,” IEEE Access, 2024.
M. Rudolph, T. Wehrbein, B. Rosenhahn, and B. Wandt, “Asymmetric student-teacher networks for industrial anomaly detection,” in Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2023, pp. 2592–2602.
W. Zhang, C. Xie, and L. Xu, “Knowledge distillation-based multi-scale feature matching for industrial anomaly detection in intelligent manufacturing,” in 2024 IEEE International Conference on e-Business Engineering (ICEBE), 2024, pp. 46–52.
X. Chen, L. Cao, S. Zhang, X. Zheng, and Y. Zhang, “Breaking the bias: Recalibrating the attention of industrial anomaly detection,” arXiv preprint arXiv:2412.08189, 2024.
J. Chen and X. Ran, “Deep learning with edge computing: A review,” pp. 1655–1674, 8 2019.
X. Wang, Y. Han, V. C. Leung, D. Niyato, X. Yan, and X. Chen, “Convergence of edge computing and deep learning: A comprehensive survey,” pp. 869–904, 4 2020.
J.-H. Luo, H. Zhang, H.-Y. Zhou, C.-W. Xie, J. Wu, and W. Lin, “Thinet: Pruning cnn filters for a thinner net,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 10, pp. 2525–2538, 2019.
A. G. Howard, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861, 2017.
J. Gu, Z. Wang, J. Kuen, L. Ma, A. Shahroudy, B. Shuai, T. Liu, X. Wang, G. Wang, J. Cai, and T. Chen, “Recent advances in convolutional neural networks,” Pattern Recognition, vol. 77, pp. 354–377, 2018. [Online]. Available: [link]
F. P. dos Santos and M. A. Ponti, “Alignment of local and global features from multiple layers of convolutional neural network for image classification,” in 2019 32nd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE, 2019, pp. 241–248.
J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, “How transferable are features in deep neural networks?” in Advances in neural information processing systems, 2014, pp. 3320–3328.
L. Jiao and J. Zhao, “A survey on the new generation of deep learning in image processing,” IEEE Access, vol. 7, pp. 172 231–172 263, 2019.
Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, “A survey of convolutional neural networks: analysis, applications, and prospects,” IEEE transactions on neural networks and learning systems, vol. 33, no. 12, pp. 6999–7019, 2021.
K. He, X. Zhang, S. Ren, and J. Sun, “Identity mappings in deep residual networks,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14. Springer, 2016, pp. 630–645.
S. O. Haroon-Sulyman, S. S. Kamaruddin, F. K. Ahmad, S. Abdul-Rahman, and N. I. Aziz, “Techniques for mitigating the vanishing gradient problem in deep neural networks,” in Knowledge Management International Conference. Springer, 2025, pp. 428–438.
A. Howard, M. Sandler, G. Chu, L.-C. Chen, B. Chen, M. Tan, W. Wang, Y. Zhu, R. Pang, V. Vasudevan et al., “Searching for mobilenetv3,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 1314–1324.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248–255.
J. Gou, B. Yu, S. J. Maybank, and D. Tao, “Knowledge distillation: A survey,” International Journal of Computer Vision, vol. 129, pp. 1789–1819, 6 2021.
G. Hinton, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015.
T. Kim, J. Oh, N. Kim, S. Cho, and S.-Y. Yun, “Comparing kullback-leibler divergence and mean squared error loss in knowledge distillation,” arXiv preprint arXiv:2105.08919, 2021.
T. Liang, J. Glossner, L. Wang, S. Shi, and X. Zhang, “Pruning and quantization for deep neural network acceleration: A survey,” Neurocomputing, vol. 461, pp. 370–403, 2021.
P. Bergmann, M. Fauser, D. Sattlegger, and C. Steger, “Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 9592–9600.
Karakurai Inc., “Real-world dataset for anomaly detection,” 2025, acessado em: 23 de maio de 2025. [Online]. Available: [link]
A. Baitieva, D. Hurych, V. Besnier, and O. Bernard, “Supervised anomaly detection for complex industrial images,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 17 754–17 762.
M. Sokolova and G. Lapalme, “A systematic analysis of performance measures for classification tasks,” Information processing & management, vol. 45, no. 4, pp. 427–437, 2009.
D. P. Kingma, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
D. Masters and C. Luschi, “Revisiting small batch training for deep neural networks,” arXiv preprint arXiv:1804.07612, 2018.
D. Walsh, “Occam’s razor: A principle of intellectual elegance,” American Philosophical Quarterly, vol. 16, no. 3, pp. 241–244, 1979.
S. Geman, E. Bienenstock, and R. Doursat, “Neural networks and the bias/variance dilemma,” Neural computation, vol. 4, no. 1, pp. 1–58, 1992.
Publicado
30/09/2025
Como Citar
NOUCHI, Eliton Katsuhiro; SANTOS, Fernando Pereira dos.
Neural Network Compression for Visual Quality Control in Industrial Edge Computing. In: WORKSHOP DE APLICAÇÕES INDUSTRIAIS - CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), 38. , 2025, Salvador/BA.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 309-316.
