Towards the Evolution of Tools for Detecting Vulnerabilities in Smart Contracts: A Case Study of Mythril and Slither
Abstract
Blockchain is an innovative technology applied in various areas such as finance, record management, electronic voting, and gaming. Transactions on the blockchain are often executed by smart contracts, which are considered critical in terms of security. There are several tools designed to automatically identify vulnerabilities in smart contracts; however, as previous studies have shown, the effectiveness of these tools is usually low. Therefore, identifying vulnerabilities in smart contracts through automation remains a significant challenge. In this study, we investigate the evolution of two important tools for detecting vulnerabilities in smart contracts: Mythril and Slither, evaluating their effectiveness in identifying vulnerabilities and whether they have evolved compared to earlier versions. To do this, we ran the tools with the latest versions on a set of 69 smart contracts previously analyzed in an earlier study. The results were compared with the already classified vulnerabilities and subjected to manual validation to assess their accuracy. The experiments show that Mythril demonstrated improvements in reducing false positives, while Slither improved the detection of Access Control-related flaws. However, the tools showed limitations in their evolution for certain types of vulnerabilities. These findings reinforce the ongoing need for the continuous improvement and evaluation of automated security analysis tools for smart contracts.
References
Belotti, M., Božić, N., Pujolle, G., and Secci, S. (2019). A vademecum on blockchain technologies: When, which, and how. IEEE Communications Surveys & Tutorials, 21(4):3796–3838.
Bhushan, B., Sinha, P., Sagayam, K. M., et al. (2021). Untangling blockchain technology: A survey on state of the art, security threats, privacy services, applications and future research directions. Computers & Electrical Engineering, 90:106897.
Buterin, V. et al. (2014). A next-generation smart contract and decentralized application platform. White paper, 3(37).
Chess, B. and McGraw, G. (2004). Static analysis for security. IEEE Security & Privacy, 2(6):76–79.
Chess, B. and West, J. (2007). Secure programming with static analysis. Pearson Education.
Di Angelo, M. and Salzer, G. (2019). A survey of tools for analyzing Ethereum smart contracts. In 2019 IEEE International Conference on Decentralized Applications and Infrastructures (DAPPCON), pages 69–78. IEEE.
Di Pierro, M. (2017). What is the blockchain? Computing in Science & Engineering, 19(5):92–95.
Durieux, T., Ferreira, J. F., Abreu, R., and Cruz, P. (2020). Empirical review of automated analysis tools on 47,587 Ethereum smart contracts. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, pages 530–541.
Eshghie, M., Artho, C., and Gurov, D. (2021). Dynamic vulnerability detection on smart contracts using machine learning. In Proceedings of the 25th International Conference on Evaluation and Assessment in Software Engineering, pages 305–312.
Feist, J., Grieco, G., and Groce, A. (2019). Slither: A static analysis framework for smart contracts. In 2019 IEEE/ACM 2nd International Workshop on Emerging Trends in Software Engineering for Blockchain (WETSEB), pages 8–15. IEEE.
Ferreira, J. F., Cruz, P., Durieux, T., and Abreu, R. (2020). SmartBugs: A framework to analyze Solidity smart contracts. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering, pages 1349–1352.
Guo, Y. and Liang, C. (2016). Blockchain application and outlook in the banking industry. Financial Innovation, 2:1–12.
Ibba, G., Aufiero, S., Neykova, R., Bartolucci, S., Ortu, M., Tonelli, R., and Destefanis, G. (2024). A curated Solidity smart contracts repository of metrics and vulnerability. In Proceedings of the 20th International Conference on Predictive Models and Data Analytics in Software Engineering, pages 32–41.
Liao, J.-W., Tsai, T.-T., He, C.-K., and Tien, C.-W. (2019). SoliAudit: Smart contract vulnerability assessment based on machine learning and fuzz testing. In 2019 Sixth International Conference on Internet of Things: Systems, Management and Security (IOTSMS), pages 458–465. IEEE.
Maesa, D. D. F. and Mori, P. (2020). Blockchain 3.0 applications survey. Journal of Parallel and Distributed Computing, 138:99–114.
Munir, S. and Reichenbach, C. (2023). TODLER: A transaction ordering dependency analyzer for Ethereum smart contracts. In 2023 IEEE/ACM 6th International Workshop on Emerging Trends in Software Engineering for Blockchain (WETSEB), pages 9–16. IEEE.
Nakamoto, S. (2008). Bitcoin: A peer-to-peer electronic cash system. Decentralized Business Review, page 21260.
Prechtel, D., Groß, T., and Müller, T. (2019). Evaluating spread of ‘gasless send’ in Ethereum smart contracts. In 2019 10th IFIP International Conference on New Technologies, Mobility and Security (NTMS), pages 1–6. IEEE.
Qian, P., Liu, Z., He, Q., Zimmermann, R., and Wang, X. (2020). Towards automated reentrancy detection for smart contracts based on sequential models. IEEE Access, 8:19685–19695.
Singh, S., Hosen, A. S., and Yoon, B. (2021). Blockchain security attacks, challenges, and solutions for the future distributed IoT network. IEEE Access, 9:13938–13959.
Tsankov, P., Dan, A., Drachsler-Cohen, D., Gervais, A., Buenzli, F., and Vechev, M. (2018). Securify: Practical security analysis of smart contracts. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pages 67–82.
Ye, J., Ma, M., Peng, T., Peng, Y., and Xue, Y. (2019). Towards automated generation of bug benchmark for smart contracts. In 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pages 184–187. IEEE.
Zheng, Z., Zhang, N., Su, J., Zhong, Z., Ye, M., and Chen, J. (2023). Turn the rudder: A beacon of reentrancy detection for smart contracts on Ethereum. arXiv preprint arXiv:2303.13770.
