OACE: Multi-criteria Assessment of Assertiveness and Cost for Deep Learning Models
Abstract
The rapid expansion of Deep Learning (DL) has intensified the need for more rigorous evaluation methodologies for implementing solutions in real-world scenarios. In view of this, this work presents Optimized Assertiveness-Cost Evaluation (OACE), a method based on Multi-Criteria Decision Making that integrates assertiveness and computational cost criteria into a single parameterizable function, aiming to systematize the evaluation of DL models. To demonstrate its effectiveness, an experiment was conducted with Random Walk and the CIFAR-10 database, evaluating five DL architectures in a balanced scenario. The findings identified MobileNetV2 as the best model for the defined scenario, with a score of 𝑆𝜙 (𝑚) = 0.9541, surpassing MobileNetB0 by 38%, due to its superior efficiency.
Keywords:
Deep Learning, Assertiveness, Computational Cost, Multi-Criteria Methods, Performance Evaluation
References
Sidra Aslam and Ali B Nassif. 2023. Deep learning based CIFAR-10 classification. In Advances in Science and Engineering Technology International Conferences. IEEE.
Lu Bai,Weixing Ji, Qinyuan Li, Xilai Yao,Wei Xin, andWanyi Zhu. 2022. Dnnabacus: Toward accurate computational cost prediction for deep neural networks. arXiv preprint arXiv:2205.12095 (2022).
Siddhant Bhadauria, Dharmendra Bisht, T Poongodi, and Suman Yadav. 2022. Assertive vision using deep learning and LSTM. In 2022 2nd International Conference on Innovative Practices in Technology and Management (ICIPTM). IEEE.
Alfredo Canziani, Adam Paszke, and Eugenio Culurciello. 2016. An analysis of deep neural network models for practical applications. arXiv preprint (2016).
Maarten V de Hoop, Daniel Z Huang, Elizabeth Qian, and Andrew M Stuart. 2022. The cost-accuracy trade-off in operator learning with neural networks. arXiv preprint arXiv:2203.13181 (2022).
Salmani et al. 2023. Reconciling High Accuracy, Cost-Efficiency, and Low Latency of Inference Serving Systems. In Proceedings of the 3rd Workshop on Machine Learning and Systems (Rome, Italy). 78–86.
Christian Gianoglio, Edoardo Ragusa, Paolo Gastaldo, and Maurizio Valle. 2021. A novel learning strategy for the trade-off between accuracy and computational cost: a touch modalities classification case study. IEEE Sensors Journal 22 (2021).
Vishu Gupta, Youjia Li, Alec Peltekian, Muhammed Nur Talha Kilic, Wei-keng Liao, Alok Choudhary, and Ankit Agrawal. 2024. Simultaneously improving accuracy and computational cost under parametric constraints in materials property prediction tasks. Journal of Cheminformatics 16, 1 (2024), 17.
Anil Jadhav and Rajendra Sonar. 2009. Analytic Hierarchy Process (AHP), Weighted Scoring Method (WSM), and Hybrid Knowledge Based System (HKBS) for Software Selection: A Comparative Study. In Second International Conference on Emerging Trends in Engineering and Technology, ICETET-09. IEEE, 991–997.
Alex Krizhevsky. 2009. Learning Multiple Layers of Features from Tiny Images. Technical Report. University of Toronto. [link] Technical Report.
Hasmat Malik, Gopal Chaudhary, and Smriti Srivastava. 2022. Digital transformation through advances in artificial intelligence and machine learning.
Sonia et al. Mijwil, Aggarwal. 2022. Has the Future Started? The Current Growth of ArtificialIntelligence, Machine Learning, and Deep Learning. Iraqi Journal for Computer Science and Mathematics 3, 1 (2022), 13.
Gireen Naidu, Tranos Zuva, and Elias Mmbongeni Sibanda. 2023. A review of evaluation metrics in machine learning algorithms. In Computer science on-line conference. Springer, 15–25.
Ebenezer Oluwasakin, Thomas Torku, S Tingting, Ahmeed Yinusa, S Hamdan, Samir Poudel, N Hasan, J Vargas, and K Poudel. 2023. Minimization of high computational cost in data preprocessing and modeling using MPI4Py. Machine Learning with Applications 13 (2023), 100483.
KARL PEARSON. 1905. The Problem of the Random Walk. Nature 72, 1865 (07 1905), 294. DOI: 10.1038/072294b0
Lyanh Pinto, André Alves, Adriano Dos Santos, Flávio Moura, Walter Oliveira, Jefferson Morais, Roberto de Oliveira, Diego Cardoso, and Marcos Seruffo. 2024. Optimized Assertiveness-Cost Evaluation: An Innovative Performance Measuring Method for Machine Learning Models. In 2024 IEEE LA-CCI. IEEE, 1–6.
Joao Schuler, Santiago Romani, Mohamed Abdel-Nasser, Hatem Rashwan, and Domenec Puig. 2022. Grouped pointwise convolutions reduce parameters in convolutional neural networks. In Mendel, Vol. 28. 23–31.
Xin-She Yang. 2020. Nature-inspired optimization algorithms. Academic Press.
Lu Bai,Weixing Ji, Qinyuan Li, Xilai Yao,Wei Xin, andWanyi Zhu. 2022. Dnnabacus: Toward accurate computational cost prediction for deep neural networks. arXiv preprint arXiv:2205.12095 (2022).
Siddhant Bhadauria, Dharmendra Bisht, T Poongodi, and Suman Yadav. 2022. Assertive vision using deep learning and LSTM. In 2022 2nd International Conference on Innovative Practices in Technology and Management (ICIPTM). IEEE.
Alfredo Canziani, Adam Paszke, and Eugenio Culurciello. 2016. An analysis of deep neural network models for practical applications. arXiv preprint (2016).
Maarten V de Hoop, Daniel Z Huang, Elizabeth Qian, and Andrew M Stuart. 2022. The cost-accuracy trade-off in operator learning with neural networks. arXiv preprint arXiv:2203.13181 (2022).
Salmani et al. 2023. Reconciling High Accuracy, Cost-Efficiency, and Low Latency of Inference Serving Systems. In Proceedings of the 3rd Workshop on Machine Learning and Systems (Rome, Italy). 78–86.
Christian Gianoglio, Edoardo Ragusa, Paolo Gastaldo, and Maurizio Valle. 2021. A novel learning strategy for the trade-off between accuracy and computational cost: a touch modalities classification case study. IEEE Sensors Journal 22 (2021).
Vishu Gupta, Youjia Li, Alec Peltekian, Muhammed Nur Talha Kilic, Wei-keng Liao, Alok Choudhary, and Ankit Agrawal. 2024. Simultaneously improving accuracy and computational cost under parametric constraints in materials property prediction tasks. Journal of Cheminformatics 16, 1 (2024), 17.
Anil Jadhav and Rajendra Sonar. 2009. Analytic Hierarchy Process (AHP), Weighted Scoring Method (WSM), and Hybrid Knowledge Based System (HKBS) for Software Selection: A Comparative Study. In Second International Conference on Emerging Trends in Engineering and Technology, ICETET-09. IEEE, 991–997.
Alex Krizhevsky. 2009. Learning Multiple Layers of Features from Tiny Images. Technical Report. University of Toronto. [link] Technical Report.
Hasmat Malik, Gopal Chaudhary, and Smriti Srivastava. 2022. Digital transformation through advances in artificial intelligence and machine learning.
Sonia et al. Mijwil, Aggarwal. 2022. Has the Future Started? The Current Growth of ArtificialIntelligence, Machine Learning, and Deep Learning. Iraqi Journal for Computer Science and Mathematics 3, 1 (2022), 13.
Gireen Naidu, Tranos Zuva, and Elias Mmbongeni Sibanda. 2023. A review of evaluation metrics in machine learning algorithms. In Computer science on-line conference. Springer, 15–25.
Ebenezer Oluwasakin, Thomas Torku, S Tingting, Ahmeed Yinusa, S Hamdan, Samir Poudel, N Hasan, J Vargas, and K Poudel. 2023. Minimization of high computational cost in data preprocessing and modeling using MPI4Py. Machine Learning with Applications 13 (2023), 100483.
KARL PEARSON. 1905. The Problem of the Random Walk. Nature 72, 1865 (07 1905), 294. DOI: 10.1038/072294b0
Lyanh Pinto, André Alves, Adriano Dos Santos, Flávio Moura, Walter Oliveira, Jefferson Morais, Roberto de Oliveira, Diego Cardoso, and Marcos Seruffo. 2024. Optimized Assertiveness-Cost Evaluation: An Innovative Performance Measuring Method for Machine Learning Models. In 2024 IEEE LA-CCI. IEEE, 1–6.
Joao Schuler, Santiago Romani, Mohamed Abdel-Nasser, Hatem Rashwan, and Domenec Puig. 2022. Grouped pointwise convolutions reduce parameters in convolutional neural networks. In Mendel, Vol. 28. 23–31.
Xin-She Yang. 2020. Nature-inspired optimization algorithms. Academic Press.
Published
2025-11-10
How to Cite
PINTO, Lyanh Vinicios Lopes; SERUFFO, Marcos Cesar da Rocha.
OACE: Multi-criteria Assessment of Assertiveness and Cost for Deep Learning Models. In: UNDERGRADUATE RESEARCH CONTEST - BRAZILIAN SYMPOSIUM ON MULTIMEDIA AND THE WEB (WEBMEDIA), 31. , 2025, Rio de Janeiro/RJ.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2025
.
p. 53-56.
ISSN 2596-1683.
DOI: https://doi.org/10.5753/webmedia_estendido.2025.15959.
