Mutation Testing to Support the Security Testing of Android Applications
Resumo
The Android system has seen considerable growth in its vulnerability landscape due to an extensive application catalog catering to many user needs, many of which are security sensitive. This growth leads to an ever-increasing concern about security robustness; hence, security testing Android apps has gained substantial prominence in recent years. Many security professionals and tools specialize in security testing Android applications, but the quality of testing procedures varies significantly. In this paper, we present a preliminary study exploring the use of Mutation Testing to support Android security testing. We propose novel mutation operators, implement them in code, and conduct an experiment to evaluate their resemblance to real-world vulnerabilities.We test our mutants using a well-known open-source tool named mobsfscan. Our results indicate the adequacy of our operators for supporting security testing. Moreover, we reveal a potential design flaw in mobsfscan.
Palavras-chave:
Software Testing, Mutation Testing, Security Testing, Android, Mutation Operator
Referências
Hilmi Abdullah and Subhi R. M. Zeebaree. 2021. Android Mobile Applications Vulnerabilities and Prevention Methods: A Review. In 2021 2nd Information Technology To Enhance e-learning and Other Application (IT-ELA). 148–153. DOI: 10.1109/IT-ELA52201.2021.9773615
Ajin Abraham. 2024. mobsfscan. Retrieved July 2, 2024 from [link]
Yasemin Acar, Michael Backes, Sven Bugiel, Sascha Fahl, Patrick McDaniel, and Matthew Smith. 2016. SoK: Lessons Learned from Android Security Research for Appified Software Platforms. In 2016 IEEE Symposium on Security and Privacy (SP). 433–451. DOI: 10.1109/SP.2016.33
Saket Acharya, Umashankar Rawat, Roheet Bhatnagar, and Bharat Bhushan. 2022. A Comprehensive Review of Android Security: Threats, Vulnerabilities, Malware Detection, and Analysis. Sec. and Commun. Netw. 2022 (jan 2022), 34 pages. DOI: 10.1155/2022/7775917
Amit Seal Ami, Kaushal Kafle, Kevin Moran, Adwait Nadkarni, and Denys Poshyvanyk. 2021. Systematic Mutation-Based Evaluation of the Soundness of Security-Focused Android Static Analysis Techniques. 24, 3, Article 15 (feb 2021), 37 pages. DOI: 10.1145/3439802
AndroidDevelopers. 2024. android:debuggable. Retrieved February 5, 2024 from [link]
Gaurang Bhatnagar, Rujul Gandhi, and Sergey Toshin. 2022. InsecureShop. Retrieved July 2, 2024 from [link]
Richard Bonett, Kaushal Kafle, Kevin Moran, Adwait Nadkarni, and Denys Poshyvanyk. 2018. Discovering Flaws in Security-Focused Static Analysis Tools for Android Using Systematic Mutation. In Proceedings of the 27th USENIX Conference on Security Symposium (Baltimore, MD, USA) (SEC’18). USENIX Association, USA, 1263–1280.
M. Delamaro, J. Maldonado, and M. Jino. 2016. Introdução Ao Teste De Software–2. ed. Elsevier.
Lin Deng, Nariman Mirzaei, Paul Ammann, and Jeff Offutt. 2015. Towards mutation analysis of Android apps. In 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW). 1–10. DOI: 10.1109/ICSTW.2015.7107450
Lin Deng and A. Jefferson Offutt. 2018. Experimental Evaluation of Redundancy in Android Mutation Testing. Int. J. Softw. Eng. Knowl. Eng. 28 (2018), 1597–1618.
Lin Deng and A. Jefferson Offutt. 2018. Reducing the Cost of Android Mutation Testing. In International Conference on Software Engineering and Knowledge Engineering.
Lin Deng, Jeff Offutt, Paul Ammann, and Nariman Mirzaei. 2017. Mutation operators for testing Android apps. Information and Software Technology 81 (2017), 154–168. [link]
Lin Deng, Jeff Offutt, and David Samudio. 2017. Is Mutation Analysis Effective at Testing Android Apps?. In 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS). 86–93. DOI: 10.1109/QRS.2017.19
Android Developers. 2023.. Retrieved January 11, 2023 from [link]
Android Developers. 2023. android:exported. Retrieved February 5, 2023 from [link]
Android Developers. 2023. App Manifest Overview. Retrieved January 11, 2023 from [link]
Android Developers. 2023. Application Fundamentals. Retrieved January 4, 2023 from [link]
Android Developers. 2023. Intent. Retrieved January 11, 2023 from [link]
Android Developers. 2024.. Retrieved February 5, 2024 from [link]
Android Developers. 2024. Cleartext / Plaintext HTTP. Retrieved February 5, 2024 from [link]
Android Developers. 2024. Hardcoded Cryptographic Secrets. Retrieved February 5, 2024 from [link]
Android Developers. 2024. Pending intents. Retrieved February 5, 2024 from [link]
Android Developers. 2024. Tapjacking. Retrieved February 5, 2024 from [link]
J. Drake, P. Fora, Z. Lanier, C. Mulliner, S. Ridley, and G.Wicherski. 2014. Android Hacker’s Handbook. Wiley.
Camilo Escobar-Velásquez, Michael Osorio-Riaño, and Mario Linares-Vásquez. 2019. MutAPK: Source-Codeless Mutant Generation for Android Apps. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). 1090–1093. DOI: 10.1109/ASE.2019.00109
R. Finkbine. 2003. Usage of mutation testing as a measure of test suite robustness. In Digital Avionics Systems Conference, 2003. DASC ’03. The 22nd (Indianapolis, IN, USA). IEEE. DOI: 10.1109/DASC.2003.1245826
Google. 2023. Common risks. Retrieved May 1, 2023 from [link]
Jalal B. Hur and Jawwad A. Shamsi. 2017. A survey on security issues, vulnerabilities and attacks in Android based smartphone. In 2017 International Conference on Information and Communication Technologies (ICICT). 40–46. DOI: 10.1109/ICICT.2017.8320163
Oversecured Inc. 2020. ovaa. Retrieved July 2, 2024 from [link]
Reyhaneh Jabbarvand and Sam Malek. 2017. μDroid: An Energy-Aware Mutation Testing Framework for Android. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (Paderborn, Germany) (ESEC/FSE 2017). Association for Computing Machinery, New York, NY, USA, 208–219. DOI: 10.1145/3106237.3106244
Aseem Jakhar. 2016. diva-android. Retrieved July 2, 2024 from [link]
Jignesh Joshi and Chandresh Parekh. 2016. Android smartphone vulnerabilities: A survey. In 2016 International Conference on Advances in Computing, Communication, Automation (ICACCA) (Spring). 1–5. DOI: 10.1109/ICACCA.2016.7578857
X. Li, L. Yu, and X.P. Luo. 2017. Chapter 7 - On Discovering Vulnerabilities in Android Applications. In Mobile Security and Privacy, Man Ho Au and Kim-Kwang Raymond Choo (Eds.). Syngress, Boston, 155–166. [link]
Mario Linares-Vásquez, Gabriele Bavota, Michele Tufano, Kevin Moran, Massimiliano Di Penta, Christopher Vendome, Carlos Bernal-Cárdenas, and Denys Poshyvanyk. 2017. Enabling Mutation Testing for Android Apps (ESEC/FSE 2017). Association for Computing Machinery, New York, NY, USA, 233–244. DOI: 10.1145/3106237.3106275
Jian Liu, Xusheng Xiao, Lihua Xu, Liang Dou, and Andy Podgurski. 2020. Droid-Mutator: An Effective Mutation Analysis Tool for Android Applications. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings (Seoul, South Korea) (ICSE ’20). Association for Computing Machinery, New York, NY, USA, 77–80. DOI: 10.1145/3377812.3382134
Eduardo Luna and Omar El Ariss. 2018. Edroid: A Mutation Tool for Android Apps. In 2018 6th International Conference in Software Engineering Research and Innovation (CONISOFT). 99–108. DOI: 10.1109/CONISOFT.2018.8645883
Yu-Seung Ma, Jeff Offutt, and Yong Rae Kwon. 2005. MuJava: an automated class mutation system. Software Testing, Verification and Reliability 15, 2 (2005), 97–133. arXiv: [link] [link]
Jonas Mayer and Florentin Wieser. 2024. finstergram. Retrieved July 2, 2024 from [link]
MITRE. 2023. Retrieved November 18, 2022 from [link]
Kevin Moran, Michele Tufano, Carlos Bernal-Cárdenas, Mario Linares-Vásquez, Gabriele Bavota, Christopher Vendome, Massimiliano Di Penta, and Denys Poshyvanyk. 2018. MDroid+: A Mutation Testing Framework for Android. In 2018 IEEE/ACM 40th International Conference on Software Engineering: Companion (ICSE-Companion). 33–36.
OWASP. 2016. MASTG-Hacking-Playground. Retrieved July 2, 2024 from [link]
OWASP. 2023. OWASP Mobile Top 10. Retrieved March 21, 2023 from [link]
OWASP. 2024. Reference applications. Retrieved July 2, 2024 from [link]
Ana C. R. Paiva, João M. E. P. Gouveia, Jean-David Elizabeth, and Márcio E. Delamaro. 2019. Testing When Mobile Apps Go to Background and Come Back to Foreground. In 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). 102–111. DOI: 10.1109/ICSTW.2019.00038
Satish Patnayak. 2019. AndroGoat. Retrieved July 2, 2024 from [link]
Cyber Security and Privacy Foundation (breakthesec). 2015. DodoVulnerableBank. Retrieved July 2, 2024 from [link]
Abhinav Sejpal and Karan Sawhney. 2015. Digitalbank. Retrieved July 2, 2024 from [link]
Dinesh Shetty, Anant Shrivastava, and Dark Cowling. 2014. Insecure-Bankv2. Retrieved July 2, 2024 from [link]
Henrique Neves Silva. 2020. Uma Abordagem de Teste de Mutação para Avaliar a Acessibilidade de Aplicações Android. Retrieved July 15, 2024 from [link] Master thesis.
Henrique Neves Silva, Jackson Prado Lima, Silvia Regina Vergilio, and Andre Takeshi Endo. 2022. A mapping study on mutation testing for mobile applications. Software Testing, Verification and Reliability 32, 8 (2022), e1801. [link]
Statista. 2023. Number of available applications in the Google Play Store from December 2009 to December 2023. Retrieved July 3, 2024 from [link]
Macario Polo Usaola, Gonzalo Rojas, Isyed Rodríguez, and Suilen Hernández. 2017. An Architecture for the Development of Mutation Operators. In 2017 IEEE International Conference on Software Testing, Verification and ValidationWorkshops (ICSTW). 143–148. DOI: 10.1109/ICSTW.2017.31
Eduardo Vasconcelos. 2024. seed-vulns. Retrieved July 2, 2024 from [link]
Ajin Abraham. 2024. mobsfscan. Retrieved July 2, 2024 from [link]
Yasemin Acar, Michael Backes, Sven Bugiel, Sascha Fahl, Patrick McDaniel, and Matthew Smith. 2016. SoK: Lessons Learned from Android Security Research for Appified Software Platforms. In 2016 IEEE Symposium on Security and Privacy (SP). 433–451. DOI: 10.1109/SP.2016.33
Saket Acharya, Umashankar Rawat, Roheet Bhatnagar, and Bharat Bhushan. 2022. A Comprehensive Review of Android Security: Threats, Vulnerabilities, Malware Detection, and Analysis. Sec. and Commun. Netw. 2022 (jan 2022), 34 pages. DOI: 10.1155/2022/7775917
Amit Seal Ami, Kaushal Kafle, Kevin Moran, Adwait Nadkarni, and Denys Poshyvanyk. 2021. Systematic Mutation-Based Evaluation of the Soundness of Security-Focused Android Static Analysis Techniques. 24, 3, Article 15 (feb 2021), 37 pages. DOI: 10.1145/3439802
AndroidDevelopers. 2024. android:debuggable. Retrieved February 5, 2024 from [link]
Gaurang Bhatnagar, Rujul Gandhi, and Sergey Toshin. 2022. InsecureShop. Retrieved July 2, 2024 from [link]
Richard Bonett, Kaushal Kafle, Kevin Moran, Adwait Nadkarni, and Denys Poshyvanyk. 2018. Discovering Flaws in Security-Focused Static Analysis Tools for Android Using Systematic Mutation. In Proceedings of the 27th USENIX Conference on Security Symposium (Baltimore, MD, USA) (SEC’18). USENIX Association, USA, 1263–1280.
M. Delamaro, J. Maldonado, and M. Jino. 2016. Introdução Ao Teste De Software–2. ed. Elsevier.
Lin Deng, Nariman Mirzaei, Paul Ammann, and Jeff Offutt. 2015. Towards mutation analysis of Android apps. In 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW). 1–10. DOI: 10.1109/ICSTW.2015.7107450
Lin Deng and A. Jefferson Offutt. 2018. Experimental Evaluation of Redundancy in Android Mutation Testing. Int. J. Softw. Eng. Knowl. Eng. 28 (2018), 1597–1618.
Lin Deng and A. Jefferson Offutt. 2018. Reducing the Cost of Android Mutation Testing. In International Conference on Software Engineering and Knowledge Engineering.
Lin Deng, Jeff Offutt, Paul Ammann, and Nariman Mirzaei. 2017. Mutation operators for testing Android apps. Information and Software Technology 81 (2017), 154–168. [link]
Lin Deng, Jeff Offutt, and David Samudio. 2017. Is Mutation Analysis Effective at Testing Android Apps?. In 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS). 86–93. DOI: 10.1109/QRS.2017.19
Android Developers. 2023.
Android Developers. 2023. android:exported. Retrieved February 5, 2023 from [link]
Android Developers. 2023. App Manifest Overview. Retrieved January 11, 2023 from [link]
Android Developers. 2023. Application Fundamentals. Retrieved January 4, 2023 from [link]
Android Developers. 2023. Intent. Retrieved January 11, 2023 from [link]
Android Developers. 2024.
Android Developers. 2024. Cleartext / Plaintext HTTP. Retrieved February 5, 2024 from [link]
Android Developers. 2024. Hardcoded Cryptographic Secrets. Retrieved February 5, 2024 from [link]
Android Developers. 2024. Pending intents. Retrieved February 5, 2024 from [link]
Android Developers. 2024. Tapjacking. Retrieved February 5, 2024 from [link]
J. Drake, P. Fora, Z. Lanier, C. Mulliner, S. Ridley, and G.Wicherski. 2014. Android Hacker’s Handbook. Wiley.
Camilo Escobar-Velásquez, Michael Osorio-Riaño, and Mario Linares-Vásquez. 2019. MutAPK: Source-Codeless Mutant Generation for Android Apps. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). 1090–1093. DOI: 10.1109/ASE.2019.00109
R. Finkbine. 2003. Usage of mutation testing as a measure of test suite robustness. In Digital Avionics Systems Conference, 2003. DASC ’03. The 22nd (Indianapolis, IN, USA). IEEE. DOI: 10.1109/DASC.2003.1245826
Google. 2023. Common risks. Retrieved May 1, 2023 from [link]
Jalal B. Hur and Jawwad A. Shamsi. 2017. A survey on security issues, vulnerabilities and attacks in Android based smartphone. In 2017 International Conference on Information and Communication Technologies (ICICT). 40–46. DOI: 10.1109/ICICT.2017.8320163
Oversecured Inc. 2020. ovaa. Retrieved July 2, 2024 from [link]
Reyhaneh Jabbarvand and Sam Malek. 2017. μDroid: An Energy-Aware Mutation Testing Framework for Android. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (Paderborn, Germany) (ESEC/FSE 2017). Association for Computing Machinery, New York, NY, USA, 208–219. DOI: 10.1145/3106237.3106244
Aseem Jakhar. 2016. diva-android. Retrieved July 2, 2024 from [link]
Jignesh Joshi and Chandresh Parekh. 2016. Android smartphone vulnerabilities: A survey. In 2016 International Conference on Advances in Computing, Communication, Automation (ICACCA) (Spring). 1–5. DOI: 10.1109/ICACCA.2016.7578857
X. Li, L. Yu, and X.P. Luo. 2017. Chapter 7 - On Discovering Vulnerabilities in Android Applications. In Mobile Security and Privacy, Man Ho Au and Kim-Kwang Raymond Choo (Eds.). Syngress, Boston, 155–166. [link]
Mario Linares-Vásquez, Gabriele Bavota, Michele Tufano, Kevin Moran, Massimiliano Di Penta, Christopher Vendome, Carlos Bernal-Cárdenas, and Denys Poshyvanyk. 2017. Enabling Mutation Testing for Android Apps (ESEC/FSE 2017). Association for Computing Machinery, New York, NY, USA, 233–244. DOI: 10.1145/3106237.3106275
Jian Liu, Xusheng Xiao, Lihua Xu, Liang Dou, and Andy Podgurski. 2020. Droid-Mutator: An Effective Mutation Analysis Tool for Android Applications. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings (Seoul, South Korea) (ICSE ’20). Association for Computing Machinery, New York, NY, USA, 77–80. DOI: 10.1145/3377812.3382134
Eduardo Luna and Omar El Ariss. 2018. Edroid: A Mutation Tool for Android Apps. In 2018 6th International Conference in Software Engineering Research and Innovation (CONISOFT). 99–108. DOI: 10.1109/CONISOFT.2018.8645883
Yu-Seung Ma, Jeff Offutt, and Yong Rae Kwon. 2005. MuJava: an automated class mutation system. Software Testing, Verification and Reliability 15, 2 (2005), 97–133. arXiv: [link] [link]
Jonas Mayer and Florentin Wieser. 2024. finstergram. Retrieved July 2, 2024 from [link]
MITRE. 2023. Retrieved November 18, 2022 from [link]
Kevin Moran, Michele Tufano, Carlos Bernal-Cárdenas, Mario Linares-Vásquez, Gabriele Bavota, Christopher Vendome, Massimiliano Di Penta, and Denys Poshyvanyk. 2018. MDroid+: A Mutation Testing Framework for Android. In 2018 IEEE/ACM 40th International Conference on Software Engineering: Companion (ICSE-Companion). 33–36.
OWASP. 2016. MASTG-Hacking-Playground. Retrieved July 2, 2024 from [link]
OWASP. 2023. OWASP Mobile Top 10. Retrieved March 21, 2023 from [link]
OWASP. 2024. Reference applications. Retrieved July 2, 2024 from [link]
Ana C. R. Paiva, João M. E. P. Gouveia, Jean-David Elizabeth, and Márcio E. Delamaro. 2019. Testing When Mobile Apps Go to Background and Come Back to Foreground. In 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). 102–111. DOI: 10.1109/ICSTW.2019.00038
Satish Patnayak. 2019. AndroGoat. Retrieved July 2, 2024 from [link]
Cyber Security and Privacy Foundation (breakthesec). 2015. DodoVulnerableBank. Retrieved July 2, 2024 from [link]
Abhinav Sejpal and Karan Sawhney. 2015. Digitalbank. Retrieved July 2, 2024 from [link]
Dinesh Shetty, Anant Shrivastava, and Dark Cowling. 2014. Insecure-Bankv2. Retrieved July 2, 2024 from [link]
Henrique Neves Silva. 2020. Uma Abordagem de Teste de Mutação para Avaliar a Acessibilidade de Aplicações Android. Retrieved July 15, 2024 from [link] Master thesis.
Henrique Neves Silva, Jackson Prado Lima, Silvia Regina Vergilio, and Andre Takeshi Endo. 2022. A mapping study on mutation testing for mobile applications. Software Testing, Verification and Reliability 32, 8 (2022), e1801. [link]
Statista. 2023. Number of available applications in the Google Play Store from December 2009 to December 2023. Retrieved July 3, 2024 from [link]
Macario Polo Usaola, Gonzalo Rojas, Isyed Rodríguez, and Suilen Hernández. 2017. An Architecture for the Development of Mutation Operators. In 2017 IEEE International Conference on Software Testing, Verification and ValidationWorkshops (ICSTW). 143–148. DOI: 10.1109/ICSTW.2017.31
Eduardo Vasconcelos. 2024. seed-vulns. Retrieved July 2, 2024 from [link]
Publicado
30/09/2024
Como Citar
VASCONCELOS, Eduardo S. M. de; DELAMARO, Marcio E.; SOUZA, Simone R. S..
Mutation Testing to Support the Security Testing of Android Applications. In: SIMPÓSIO BRASILEIRO DE TESTES DE SOFTWARE SISTEMÁTICO E AUTOMATIZADO (SAST), 9. , 2024, Curitiba/PR.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2024
.
p. 29-38.
DOI: https://doi.org/10.5753/sast.2024.3773.
