Lightweight CNNs for Edge Vision: A Cross-Benchmark Study of Robustness and Efficiency
Resumo
Selecting lightweight CNNs for edge vision from clean accuracy alone can lead to suboptimal deployments, since practical performance also depends on corruption robustness, latency, throughput, and hardware constraints. We compare MobileNetV3-Small, EfficientNet-B0, and MCUNet on CIFAR-10, CIFAR-100, and Wake Vision under the CIFAR-C corruption-error protocol and float32 TensorFlow Lite CPU-only inference on two Android phones and a Raspberry Pi Zero 2 W. EfficientNet-B0 attains the highest clean accuracy and lowest absolute mCE (42.82–66.33); MobileNetV3-Small is the most efficient (P90 from 1.83 ms, 158.63 images/s, 3.58–3.78 MB). Friedman and Nemenyi tests confirm this ranking.Referências
Alf, L. M., Hoffmann, R. B., Müller, C., and Griebler, D. (2023). Análise da execução de algoritmos de aprendizado de máquina em dispositivos embarcados. In Anais do XXIV SSCAD, pages 61–72.
Alves da Silva, I. P. C., Pereira, E. A. M., Barboza, E. d. A., dos Santos Neto, B. F., and Ribeiro, M. d. M. (2025). Evaluating the impact of compression techniques on the robustness of cnns under natural corruptions. In IEEE Int. Conf. on Machine Learning and Applications (ICMLA). IEEE.
Andrade, G. V. d., Santos, S. R. d., Silva, I. P. C. A. d., Pereira, E. A. M., and Barboza, E. d. A. (2025). From code to field: Evaluating the robustness of convolutional neural networks for disease diagnosis in mango leaves. In Intelligent Systems (BRACIS 2025), Part IV, volume 16182 of LNAI, pages 291–305. Springer.
Baller, S. P., Jindal, A., Chadha, M., and Gerndt, M. (2021). DeepEdgeBench: Benchmarking deep neural networks on edge devices. In IEEE Int. Conf. on Cloud Engineering (IC2E), pages 20–30.
Banbury, C., Njor, E., Garavagno, A. M., Mazumder, M., et al. (2026). Wake vision: A tailored dataset and benchmark suite for TinyML computer vision applications. Journal of Data-centric Machine Learning Research, 3.
Banbury, C., Reddi, V. J., Torelli, P., Holleman, J., et al. (2021). MLPerf tiny benchmark. In NeurIPS Datasets and Benchmarks Track, volume 1.
Benavoli, A., Corani, G., and Mangili, F. (2016). Should we really use post-hoc tests based on mean-ranks? Journal of Machine Learning Research, 17(5):1–10.
Burgert, T., Stoll, O., Rota, P., and Demir, B. (2025). ImageNet-trained cnns are not biased towards texture: Revisiting feature reliance through controlled suppression. In Advances in Neural Information Processing Systems, volume 38.
Demšar, J. (2006). Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7:1–30.
Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., et al. (2019). ImageNet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations.
Google (2025). XNNPACK: High-efficiency floating-point neural network inference operators for mobile, server, and web. [link]. Software library, accessed 2026-08-18.
Hendrycks, D., Basart, S., Mu, N., Kadavath, S., et al. (2021). The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8340–8349.
Hendrycks, D. and Dietterich, T. (2019). Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations.
Hendrycks, D., Mu, N., Cubuk, E. D., Zoph, B., et al. (2020). AugMix: A simple data processing method to improve robustness and uncertainty. In International Conference on Learning Representations.
Howard, A., Sandler, M., Chu, G., Chen, L.-C., et al. (2019). Searching for mobilenetv3. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1314–1324.
Jung, S., Lukasik, J., and Keuper, M. (2023). Neural architecture design and robustness: A dataset. In International Conference on Learning Representations.
Korkmaz, O. E. (2025). Revisiting convolutional design for efficient cnn architectures in edge-aware applications. Scientific Reports, 15:44351.
Krizhevsky, A. (2009). Learning multiple layers of features from tiny images. Technical Report TR-2009, University of Toronto.
Lin, J., Chen, W.-M., Cai, H., Gan, C., and Han, S. (2021). MCUNetV2: Memory-efficient patch-based inference for tiny deep learning. In Advances in Neural Information Processing Systems, volume 34, pages 2346–2358.
Lin, J., Chen, W.-M., Lin, Y., Cohn, J., et al. (2020). Mcunet: Tiny deep learning on iot devices. Advances in Neural Information Processing Systems, 33:11711–11722.
Michaelis, C., Mitzkus, B., Geirhos, R., Rusak, E., et al. (2019). Benchmarking robustness in object detection: Autonomous driving when winter is coming. arXiv preprint arXiv:1907.07484.
Miranda Filho, R., Matias, P., and de Freitas, R. (2025). HarpIA: a tool for comparative analysis of embedded AI models on android. In Anais do XXII ENIAC, pages 1938–1949.
Pereira, E., Barboza, E., Araújo, Í., dos Santos, S., et al. (2026). Robustness in TinyML: A systematic literature review. ACM Transactions on Embedded Computing Systems.
Rojas-Carrasco, P. and Guinaldo, M. (2026). Multi-objective evaluation of lightweight AI models on low-cost edge devices using pareto fronts. Applied Sciences, 16(8):3679.
Shahriar, T. (2025). Comparative analysis of lightweight CNNs for resource-constrained devices: Predictive performance, efficiency trade-offs, and initialization effects. arXiv preprint arXiv:2505.03303.
Soares, B., Silva, W., Ponciano, G., Stefanie, B., et al. (2024). Embedded neural networks for identifying spodoptera frugiperda in corn plantations. In Anais do XXI ENIAC, pages 376–387.
Tan, M. and Le, Q. V. (2019). Efficientnet: Rethinking model scaling for convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 6105–6114. PMLR.
Tang, S., Gong, R., Wang, Y., Liu, A., et al. (2021). RobustART: Benchmarking robustness on architecture design and training techniques. arXiv preprint arXiv:2109.05211.
TensorFlow Authors (2024). Tensorflow lite: Deploy machine learning models on mobile and embedded devices. [link]. Accessed: 2026-04-18.
Wang, S., Veldhuis, R., Brune, C., and Strisciuglio, N. (2024). A survey on the robustness of computer vision models against common corruptions. arXiv preprint arXiv:2305.06024v4.
Zhang, L. L., Han, S., Wei, J., Zheng, N., et al. (2021). nn-Meter: Towards accurate latency prediction of deep-learning model inference on diverse edge devices. In Proc. ACM MobiSys, pages 81–93.
Alves da Silva, I. P. C., Pereira, E. A. M., Barboza, E. d. A., dos Santos Neto, B. F., and Ribeiro, M. d. M. (2025). Evaluating the impact of compression techniques on the robustness of cnns under natural corruptions. In IEEE Int. Conf. on Machine Learning and Applications (ICMLA). IEEE.
Andrade, G. V. d., Santos, S. R. d., Silva, I. P. C. A. d., Pereira, E. A. M., and Barboza, E. d. A. (2025). From code to field: Evaluating the robustness of convolutional neural networks for disease diagnosis in mango leaves. In Intelligent Systems (BRACIS 2025), Part IV, volume 16182 of LNAI, pages 291–305. Springer.
Baller, S. P., Jindal, A., Chadha, M., and Gerndt, M. (2021). DeepEdgeBench: Benchmarking deep neural networks on edge devices. In IEEE Int. Conf. on Cloud Engineering (IC2E), pages 20–30.
Banbury, C., Njor, E., Garavagno, A. M., Mazumder, M., et al. (2026). Wake vision: A tailored dataset and benchmark suite for TinyML computer vision applications. Journal of Data-centric Machine Learning Research, 3.
Banbury, C., Reddi, V. J., Torelli, P., Holleman, J., et al. (2021). MLPerf tiny benchmark. In NeurIPS Datasets and Benchmarks Track, volume 1.
Benavoli, A., Corani, G., and Mangili, F. (2016). Should we really use post-hoc tests based on mean-ranks? Journal of Machine Learning Research, 17(5):1–10.
Burgert, T., Stoll, O., Rota, P., and Demir, B. (2025). ImageNet-trained cnns are not biased towards texture: Revisiting feature reliance through controlled suppression. In Advances in Neural Information Processing Systems, volume 38.
Demšar, J. (2006). Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7:1–30.
Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., et al. (2019). ImageNet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations.
Google (2025). XNNPACK: High-efficiency floating-point neural network inference operators for mobile, server, and web. [link]. Software library, accessed 2026-08-18.
Hendrycks, D., Basart, S., Mu, N., Kadavath, S., et al. (2021). The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8340–8349.
Hendrycks, D. and Dietterich, T. (2019). Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations.
Hendrycks, D., Mu, N., Cubuk, E. D., Zoph, B., et al. (2020). AugMix: A simple data processing method to improve robustness and uncertainty. In International Conference on Learning Representations.
Howard, A., Sandler, M., Chu, G., Chen, L.-C., et al. (2019). Searching for mobilenetv3. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1314–1324.
Jung, S., Lukasik, J., and Keuper, M. (2023). Neural architecture design and robustness: A dataset. In International Conference on Learning Representations.
Korkmaz, O. E. (2025). Revisiting convolutional design for efficient cnn architectures in edge-aware applications. Scientific Reports, 15:44351.
Krizhevsky, A. (2009). Learning multiple layers of features from tiny images. Technical Report TR-2009, University of Toronto.
Lin, J., Chen, W.-M., Cai, H., Gan, C., and Han, S. (2021). MCUNetV2: Memory-efficient patch-based inference for tiny deep learning. In Advances in Neural Information Processing Systems, volume 34, pages 2346–2358.
Lin, J., Chen, W.-M., Lin, Y., Cohn, J., et al. (2020). Mcunet: Tiny deep learning on iot devices. Advances in Neural Information Processing Systems, 33:11711–11722.
Michaelis, C., Mitzkus, B., Geirhos, R., Rusak, E., et al. (2019). Benchmarking robustness in object detection: Autonomous driving when winter is coming. arXiv preprint arXiv:1907.07484.
Miranda Filho, R., Matias, P., and de Freitas, R. (2025). HarpIA: a tool for comparative analysis of embedded AI models on android. In Anais do XXII ENIAC, pages 1938–1949.
Pereira, E., Barboza, E., Araújo, Í., dos Santos, S., et al. (2026). Robustness in TinyML: A systematic literature review. ACM Transactions on Embedded Computing Systems.
Rojas-Carrasco, P. and Guinaldo, M. (2026). Multi-objective evaluation of lightweight AI models on low-cost edge devices using pareto fronts. Applied Sciences, 16(8):3679.
Shahriar, T. (2025). Comparative analysis of lightweight CNNs for resource-constrained devices: Predictive performance, efficiency trade-offs, and initialization effects. arXiv preprint arXiv:2505.03303.
Soares, B., Silva, W., Ponciano, G., Stefanie, B., et al. (2024). Embedded neural networks for identifying spodoptera frugiperda in corn plantations. In Anais do XXI ENIAC, pages 376–387.
Tan, M. and Le, Q. V. (2019). Efficientnet: Rethinking model scaling for convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 6105–6114. PMLR.
Tang, S., Gong, R., Wang, Y., Liu, A., et al. (2021). RobustART: Benchmarking robustness on architecture design and training techniques. arXiv preprint arXiv:2109.05211.
TensorFlow Authors (2024). Tensorflow lite: Deploy machine learning models on mobile and embedded devices. [link]. Accessed: 2026-04-18.
Wang, S., Veldhuis, R., Brune, C., and Strisciuglio, N. (2024). A survey on the robustness of computer vision models against common corruptions. arXiv preprint arXiv:2305.06024v4.
Zhang, L. L., Han, S., Wei, J., Zheng, N., et al. (2021). nn-Meter: Towards accurate latency prediction of deep-learning model inference on diverse edge devices. In Proc. ACM MobiSys, pages 81–93.
Publicado
19/10/2026
Como Citar
SANTOS, Saulo Roberto dos; PEREIRA, Emanuel Adler Medeiros; BARBOZA, Erick de Andrade; SILVA, Itallo Patrick Castro Alves da; RIBEIRO, Márcio de Medeiros; SANTOS NETO, Baldoino Fonseca dos.
Lightweight CNNs for Edge Vision: A Cross-Benchmark Study of Robustness and Efficiency. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 23. , 2026, Cuiabá/MT.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 3083-3094.
ISSN 2763-9061.
DOI: https://doi.org/10.5753/eniac.2026.31118.
