skip to main content
10.1145/3569902.3570187acmotherconferencesArticle/Chapter ViewAbstractPublication PagesladcConference Proceedingsconference-collections
research-article

Evaluation of SMT solvers in abstraction-based software model checking

Authors Info & Claims
Published:17 January 2023Publication History

ABSTRACT

The verification of safety-critical software systems is an algorithmically complex task, which often utilizes SMT (Satisfiability Modulo Theory) solvers for computing abstractions and refinements. It follows that the performance of SMT solvers impacts the performance of the abstraction-refinement-based verification process. Moreover, SMT solvers algorithmically affect the refinement process with either Craig interpolation or unsatisfiable cores. We evaluated eight versions of six different SMT solvers on software verification benchmarks to evaluate their effect on software model checking.

References

  1. Zsófia Ádám, Levente Bajczi, Mihály Dobos-Kovács, Ákos Hajdu, and Vince Molnár. 2022. Theta: portfolio of CEGAR-based analyses with dynamic algorithm selection (Competition Contribution). In Tools and Algorithms for the Construction and Analysis of Systems, Dana Fisman and Grigore Rosu (Eds.). Springer International Publishing, Cham, 474–478.Google ScholarGoogle Scholar
  2. Haniel Barbosa, Clark Barrett, Martin Brain, Gereon Kremer, Hanna Lachnitt, Makai Mann, Abdalrhman Mohamed, Mudathir Mohamed, Aina Niemetz, Andres Nötzli, Alex Ozdemir, Mathias Preiner, Andrew Reynolds, Ying Sheng, Cesare Tinelli, and Yoni Zohar. 2022. cvc5: A Versatile and Industrial-Strength SMT Solver. In Tools and Algorithms for the Construction and Analysis of Systems, Dana Fisman and Grigore Rosu (Eds.). Springer International Publishing, Cham, 415–442.Google ScholarGoogle Scholar
  3. Clark Barrett, Christopher L. Conway, Morgan Deters, Liana Hadarean, Dejan Jovanović, Tim King, Andrew Reynolds, and Cesare Tinelli. 2011. CVC4. In Computer Aided Verification, Ganesh Gopalakrishnanand Shaz Qadeer (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 171–177.Google ScholarGoogle Scholar
  4. Clark Barrett, Aaron Stump, Cesare Tinelli, 2010. The smt-lib standard: Version 2.0. In Proceedings of the 8th international workshop on satisfiability modulo theories (Edinburgh, England), Vol. 13. 14.Google ScholarGoogle Scholar
  5. Clark Barrett and Cesare Tinelli. 2018. Satisfiability Modulo Theories. Springer International Publishing, Cham, 305–343. https://doi.org/10.1007/978-3-319-10575-8_11Google ScholarGoogle ScholarCross RefCross Ref
  6. Dirk Beyer. 2012. Competition on Software Verification. In Tools and Algorithms for the Construction and Analysis of Systems, Cormac Flanaganand Barbara König (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 504–524.Google ScholarGoogle Scholar
  7. Dirk Beyer, Thomas A. Henzinger, and Grégory Théoduloz. 2007. Configurable Software Verification: Concretizing the Convergence of Model Checking and Program Analysis. In Computer Aided Verification, Werner Damm and Holger Hermanns (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 504–518.Google ScholarGoogle Scholar
  8. Dirk Beyer and Stefan Löwe. 2013. Explicit-State Software Model Checking Based on CEGAR and Interpolation. In Fundamental Approaches to Software Engineering, Vittorio Cortellessa and Dániel Varró (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 146–162.Google ScholarGoogle Scholar
  9. Angelo Brillout, Daniel Kroening, Philipp Rümmer, and Thomas Wahl. 2011. An Interpolating Sequent Calculus for Quantifier-Free Presburger Arithmetic. Journal of Automated Reasoning 47 (2011), 341–367. Issue 4. http://dx.doi.org/10.1007/s10817-011-9237-yGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  10. Roberto Bruttomesso, Alessandro Cimatti, Anders Franzén, Alberto Griggio, and Roberto Sebastiani. 2008. The MathSAT 4 SMT Solver. In Computer Aided Verification, Aarti Gupta and Sharad Malik (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 299–303.Google ScholarGoogle Scholar
  11. Jürgen Christ, Jochen Hoenicke, and Alexander Nutz. 2012. SMTInterpol: An Interpolating SMT Solver. In Model Checking Software, Alastair Donaldson and David Parker (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 248–254.Google ScholarGoogle Scholar
  12. Edmund Clarke, Orna Grumberg, Somesh Jha, Yuan Lu, and Helmut Veith. 2000. Counterexample-Guided Abstraction Refinement. In Computer Aided Verification, E. Allen Emerson and Aravinda Prasad Sistla (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 154–169.Google ScholarGoogle ScholarCross RefCross Ref
  13. Edmund M. Clarke, Thomas A. Henzinger, and Helmut Veith. 2018. Introduction to Model Checking. In Handbook of Model Checking, Edmund M. Clarke, Thomas A. Henzinger, Helmut Veith, and Roderick Bloem (Eds.). Springer International Publishing, Cham, 1–26. https://doi.org/10.1007/978-3-319-10575-8_1Google ScholarGoogle ScholarCross RefCross Ref
  14. Leonardo de Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In Tools and Algorithms for the Construction and Analysis of Systems, C. R. Ramakrishnanand Jakob Rehof (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 337–340.Google ScholarGoogle Scholar
  15. Daniel Dietsch, Matthias Heizmann, Betim Musa, Alexander Nutz, and Andreas Podelski. 2017. Craig vs. Newton in Software Model Checking. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (Paderborn, Germany) (ESEC/FSE 2017). Association for Computing Machinery, New York, NY, USA, 487–497. https://doi.org/10.1145/3106237.3106307Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Mihály Dobos-Kovács, Ákos Hajdu, and András Vörös. 2021. Bitvector Support in the Theta Formal Verification Framework. In 2021 10th Latin-American Symposium on Dependable Computing (LADC). IEEE, 01–08. https://doi.org/10.1109/LADC53747.2021.9672595Google ScholarGoogle ScholarCross RefCross Ref
  17. Ákos Hajdu and Zoltán Micskei. 2020. Efficient Strategies for CEGAR-based Model Checking. Journal of Automated Reasoning 64, 6 (2020), 1051–1091. https://doi.org/10.1007/s10817-019-09535-xGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  18. Ranjit Jhala, Andreas Podelski, and Andrey Rybalchenko. 2018. Predicate Abstraction for Program Verification. Springer International Publishing, Cham, 447–491. https://doi.org/10.1007/978-3-319-10575-8_15Google ScholarGoogle ScholarCross RefCross Ref
  19. Nikita Malyshev, Irina Dudina, Daniil Kutz, Alexander Novikov, and Sergey Vartanov. 2019. SMT Solvers in Application to Static and Dynamic Symbolic Execution: A Case Study. In 2019 Ivannikov Ispras Open Conference (ISPRAS). IEEE, 9–15.Google ScholarGoogle ScholarCross RefCross Ref
  20. Kenneth L. McMillan. 2005. Applications of Craig Interpolants in Model Checking. In Tools and Algorithms for the Construction and Analysis of Systems. Springer Berlin Heidelberg, Berlin, Heidelberg, 1–12.Google ScholarGoogle Scholar
  21. Sabino Francesco Roselli, Kristofer Bengtsson, and Knut Åkesson. 2018. SMT solvers for job-shop scheduling problems: Models comparison and performance evaluation. In 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE). IEEE, 547–552.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Tamás Tóth, Ákos Hajdu, András Vörös, Zoltán Micskei, and István Majzik. 2017. Theta: a Framework for Abstraction Refinement-Based Model Checking. In Proceedings of the 17th Conference on Formal Methods in Computer-Aided Design, Daryl Stewart and Georg Weissenbacher (Eds.). IEEE, 176–179. https://doi.org/10.23919/FMCAD.2017.8102257Google ScholarGoogle ScholarCross RefCross Ref
  23. Tjark Weber, Sylvain Conchon, David Déharbe, Matthias Heizmann, Aina Niemetz, and Giles Reger. 2019. The SMT competition 2015–2018. Journal on Satisfiability, Boolean Modeling and Computation 11, 1(2019), 221–259.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Evaluation of SMT solvers in abstraction-based software model checking

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Other conferences
        LADC '22: Proceedings of the 11th Latin-American Symposium on Dependable Computing
        November 2022
        167 pages
        ISBN:9781450397377
        DOI:10.1145/3569902

        Copyright © 2022 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 17 January 2023

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed limited
      • Article Metrics

        • Downloads (Last 12 months)21
        • Downloads (Last 6 weeks)1

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format .

      View HTML Format