Automated Fuzz Driver Generation for Java via Search-Based Testing: A Preliminary Study
Resumo
Coverage-guided fuzzing has proven to be remarkably effective in uncovering security and reliability issues across large software systems. However, its adoption in practice remains hindered by the need for manually crafted fuzz drivers, whose construction is labor-intensive, error-prone, and often unrealistic for open-source software, where thousands of libraries expose complex and interdependent APIs. In this paper, we present arrabida, a novel approach to automating the generation of fuzz drivers for Java through encapsulated controlled execution contexts and orchestrated use of APIs, such as unit test cases. arrabida integrates search-based test generation (via EvoSuite) with static analysis, transforming the synthesized tests into Jazzer-compatible fuzz drivers, which are compilable and deployable in OSS-Fuzz. We evaluated arrabida on three real-world Java projects from OSS-Fuzz, by comparing generated drivers with their manually written counterparts in terms of iteration throughput, coverage, feature exploration, and crash detection. Our results show that arrabida drivers achieve comparable results to manually written drivers, exceeding coverage and number of iterations, while discovering unique program features and behaviors. arrabida triggered 11 unique crashes.Referências
Babić, D., Bucur, S., Chen, Y., Ivančić, F., King, T., Kusano, M., Lemieux, C., Szekeres, L., and Wang, W. (2019). FUDGE: Fuzz driver generation at scale. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE ’19), pages 975–985, Tallinn, Estonia. ACM.
Böhme, M., Cadar, C., and Roychoudhury, A. (2021). Fuzzing: Challenges and reflections. IEEE Software, 38(3):79–86.
Campos, J., Panichella, A., and Fraser, G. (2019). EvoSuite at the SBST 2019 Tool Competition. In 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing (SBST), pages 29–32.
Chen, P., Xie, Y., Lyu, Y., Wang, Y., and Chen, H. (2023). Hopper: Interpretative fuzzing for libraries. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, CCS ’23, page 1600–1614, New York, NY, USA. Association for Computing Machinery.
Fraser, G. and Arcuri, A. (2011). Evosuite: automatic test suite generation for object-oriented software. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering, ESEC/FSE ’11, page 416–419, New York, NY, USA. Association for Computing Machinery.
Fraser, G. and Arcuri, A. (2014a). A Large-Scale Evaluation of Automated Unit Test Generation Using EvoSuite. ACM Trans. Softw. Eng. Methodol., 24(2).
Fraser, G. and Arcuri, A. (2014b). Evosuite at the second unit testing tool competition. In Vos, T. E., Lakhotia, K., and Bauersfeld, S., editors, Future Internet Testing, pages 95–100, Cham. Springer International Publishing.
Fraser, G. and Arcuri, A. (2016). EvoSuite at the SBST 2016 tool competition. In Proceedings of the 9th International Workshop on Search-Based Software Testing, SBST ’16, page 33–36, New York, NY, USA. Association for Computing Machinery.
Fraser, G. and Arcuri, A. (2021). Evosuite. [link]. Accessed: 28 Aug 2025.
Gao, W., Pham, V.-T., Liu, D., Chang, O., Murray, T., and Rubinstein, B. I. (2023). Beyond the coverage plateau: A comprehensive study of fuzz blockers (registered report). In Proceedings of the 2nd International Fuzzing Workshop, FUZZING 2023, page 47–55, New York, NY, USA. Association for Computing Machinery.
Ispoglou, K. K., Austin, D., Mohan, V., and Payer, M. (2020). Fuzzgen: automatic fuzzer generation. In Proceedings of the 29th USENIX Conference on Security Symposium, SEC’20, USA. USENIX Association.
Jahangirova, G. and Terragni, V. (2023). Sbft tool competition 2023 - java test case generation track. In 2023 IEEE/ACM International Workshop on Search-Based and Fuzz Testing (SBFT), pages 61–64.
Jeong, B., Jang, J., Yi, H., Moon, J., Kim, J., Jeon, I., Kim, T., Shim, W., and Hwang, Y. H. (2023). Utopia: Automatic generation of fuzz driver using unit tests. In 2023 IEEE Symposium on Security and Privacy (SP), pages 2676–2692.
Jung, J., Tong, S., Hu, H., Lim, J., Jin, Y., and Kim, T. (2021). Winnie : Fuzzing windows applications with harness synthesis and fast cloning. In Proceedings of the 2021 Network and Distributed System Security Symposium (NDSS 2021).
LCC, G. (2024). Typespecfuzzer. [link]. Accessed: 28 Aug 2025.
Lyu, Y., Xie, Y., Chen, P., and Chen, H. (2024). Prompt fuzzing for fuzz driver generation. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, CCS ’24, page 3793–3807, New York, NY, USA. Association for Computing Machinery.
Manès, V. J., Han, H., Han, C., Cha, S. K., Egele, M., Schwartz, E. J., and Woo, M. (2021). The art, science, and engineering of fuzzing: A survey. IEEE Transactions on Software Engineering, 47(11):2312–2331.
Panichella, A., Campos, J., and Fraser, G. (2020). EvoSuite at the SBST 2020 Tool Competition. In Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW’20, page 549–552, New York, NY, USA. Association for Computing Machinery.
Serebryany, K. (2017). OSS-Fuzz - google’s continuous fuzzing service for open source software. Vancouver, BC. USENIX Association.
Shamshiri, S., Just, R., Rojas, J. M., Fraser, G., McMinn, P., and Arcuri, A. (2015). Do Automatically Generated Unit Tests Find Real Faults? An Empirical Study of Effectiveness and Challenges. In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 201–211.
SquareDev (2024). Javapoet. [link]. Accessed: 28 Aug 2025.
Team, G. O. S. S. (2023). Ai-powered fuzzing: Breaking the bug hunting barrier. Google Security Blog. Accessed: 24 August 2025.
Zhang, C., Lin, X., Li, Y., Xue, Y., Xie, J., Chen, H., Ying, X., Wang, J., and Liu, Y. (2021). APICraft: Fuzz driver generation for closed-source sdk libraries. In Proceedings of the 30th USENIX Security Symposium, pages 2811–2828. USENIX Association.
Zhang, C., Zheng, Y., Bai, M., Li, Y., Ma, W., Xie, X., Li, Y., Sun, L., and Liu, Y. (2024). How effective are they? exploring large language model based fuzz driver generation. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2024, page 1223–1235, New York, NY, USA. Association for Computing Machinery.
Zhu, X., Wen, S., Camtepe, S., and Xiang, Y. (2022). Fuzzing: A survey for roadmap. ACM Comput. Surv., 54(11s).
Böhme, M., Cadar, C., and Roychoudhury, A. (2021). Fuzzing: Challenges and reflections. IEEE Software, 38(3):79–86.
Campos, J., Panichella, A., and Fraser, G. (2019). EvoSuite at the SBST 2019 Tool Competition. In 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing (SBST), pages 29–32.
Chen, P., Xie, Y., Lyu, Y., Wang, Y., and Chen, H. (2023). Hopper: Interpretative fuzzing for libraries. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, CCS ’23, page 1600–1614, New York, NY, USA. Association for Computing Machinery.
Fraser, G. and Arcuri, A. (2011). Evosuite: automatic test suite generation for object-oriented software. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering, ESEC/FSE ’11, page 416–419, New York, NY, USA. Association for Computing Machinery.
Fraser, G. and Arcuri, A. (2014a). A Large-Scale Evaluation of Automated Unit Test Generation Using EvoSuite. ACM Trans. Softw. Eng. Methodol., 24(2).
Fraser, G. and Arcuri, A. (2014b). Evosuite at the second unit testing tool competition. In Vos, T. E., Lakhotia, K., and Bauersfeld, S., editors, Future Internet Testing, pages 95–100, Cham. Springer International Publishing.
Fraser, G. and Arcuri, A. (2016). EvoSuite at the SBST 2016 tool competition. In Proceedings of the 9th International Workshop on Search-Based Software Testing, SBST ’16, page 33–36, New York, NY, USA. Association for Computing Machinery.
Fraser, G. and Arcuri, A. (2021). Evosuite. [link]. Accessed: 28 Aug 2025.
Gao, W., Pham, V.-T., Liu, D., Chang, O., Murray, T., and Rubinstein, B. I. (2023). Beyond the coverage plateau: A comprehensive study of fuzz blockers (registered report). In Proceedings of the 2nd International Fuzzing Workshop, FUZZING 2023, page 47–55, New York, NY, USA. Association for Computing Machinery.
Ispoglou, K. K., Austin, D., Mohan, V., and Payer, M. (2020). Fuzzgen: automatic fuzzer generation. In Proceedings of the 29th USENIX Conference on Security Symposium, SEC’20, USA. USENIX Association.
Jahangirova, G. and Terragni, V. (2023). Sbft tool competition 2023 - java test case generation track. In 2023 IEEE/ACM International Workshop on Search-Based and Fuzz Testing (SBFT), pages 61–64.
Jeong, B., Jang, J., Yi, H., Moon, J., Kim, J., Jeon, I., Kim, T., Shim, W., and Hwang, Y. H. (2023). Utopia: Automatic generation of fuzz driver using unit tests. In 2023 IEEE Symposium on Security and Privacy (SP), pages 2676–2692.
Jung, J., Tong, S., Hu, H., Lim, J., Jin, Y., and Kim, T. (2021). Winnie : Fuzzing windows applications with harness synthesis and fast cloning. In Proceedings of the 2021 Network and Distributed System Security Symposium (NDSS 2021).
LCC, G. (2024). Typespecfuzzer. [link]. Accessed: 28 Aug 2025.
Lyu, Y., Xie, Y., Chen, P., and Chen, H. (2024). Prompt fuzzing for fuzz driver generation. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, CCS ’24, page 3793–3807, New York, NY, USA. Association for Computing Machinery.
Manès, V. J., Han, H., Han, C., Cha, S. K., Egele, M., Schwartz, E. J., and Woo, M. (2021). The art, science, and engineering of fuzzing: A survey. IEEE Transactions on Software Engineering, 47(11):2312–2331.
Panichella, A., Campos, J., and Fraser, G. (2020). EvoSuite at the SBST 2020 Tool Competition. In Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW’20, page 549–552, New York, NY, USA. Association for Computing Machinery.
Serebryany, K. (2017). OSS-Fuzz - google’s continuous fuzzing service for open source software. Vancouver, BC. USENIX Association.
Shamshiri, S., Just, R., Rojas, J. M., Fraser, G., McMinn, P., and Arcuri, A. (2015). Do Automatically Generated Unit Tests Find Real Faults? An Empirical Study of Effectiveness and Challenges. In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 201–211.
SquareDev (2024). Javapoet. [link]. Accessed: 28 Aug 2025.
Team, G. O. S. S. (2023). Ai-powered fuzzing: Breaking the bug hunting barrier. Google Security Blog. Accessed: 24 August 2025.
Zhang, C., Lin, X., Li, Y., Xue, Y., Xie, J., Chen, H., Ying, X., Wang, J., and Liu, Y. (2021). APICraft: Fuzz driver generation for closed-source sdk libraries. In Proceedings of the 30th USENIX Security Symposium, pages 2811–2828. USENIX Association.
Zhang, C., Zheng, Y., Bai, M., Li, Y., Ma, W., Xie, X., Li, Y., Sun, L., and Liu, Y. (2024). How effective are they? exploring large language model based fuzz driver generation. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2024, page 1223–1235, New York, NY, USA. Association for Computing Machinery.
Zhu, X., Wen, S., Camtepe, S., and Xiang, Y. (2022). Fuzzing: A survey for roadmap. ACM Comput. Surv., 54(11s).
Publicado
11/05/2026
Como Citar
FERNANDES, Leo; CARVALHO, Juvenal; CATARINO, André; CAMPOS, José; SOARES, Elvys; ABREU, Rui.
Automated Fuzz Driver Generation for Java via Search-Based Testing: A Preliminary Study. In: CONGRESSO IBERO-AMERICANO EM ENGENHARIA DE SOFTWARE (CIBSE), 29. , 2026, Recife/PE.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2026
.
p. 373-387.
