Skip to main content

Logic-Based Explanations for Linear Support Vector Classifiers with Reject Option

  • Conference paper
  • First Online:
Intelligent Systems (BRACIS 2023)

Abstract

Support Vector Classifier (SVC) is a well-known Machine Learning (ML) model for linear classification problems. It can be used in conjunction with a reject option strategy to reject instances that are hard to correctly classify and delegate them to a specialist. This further increases the confidence of the model. Given this, obtaining an explanation of the cause of rejection is important to not blindly trust the obtained results. While most of the related work has developed means to give such explanations for machine learning models, to the best of our knowledge none have done so for when reject option is present. We propose a logic-based approach with formal guarantees on the correctness and minimality of explanations for linear SVCs with reject option. We evaluate our approach by comparing it to Anchors, which is a heuristic algorithm for generating explanations. Obtained results show that our proposed method gives shorter explanations with reduced time cost. Furthermore, although our approach is demonstrated with linear SVCs, it can be easily adapted to other classifiers with reject option, such as neural networks and random forests.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://archive.ics.uci.edu/ml/datasets.php.

  2. 2.

    https://www.kaggle.com/datasets/uciml/pima-indians-diabetes-database.

  3. 3.

    https://github.com/scikit-learn/scikit-learn/tree/main/sklearn/datasets/data.

  4. 4.

    https://github.com/franciscomateus0119/Logic-based-Explanations-for-Linear-Support-Vector-Classifiers-with-Reject-Option.

  5. 5.

    https://github.com/coin-or/Cbc.

  6. 6.

    https://github.com/coin-or/pulp.

References

  1. Audemard, G., Bellart, S., Bounia, L., Koriche, F., Lagniez, J.M., Marquis, P.: On preferred abductive explanations for decision trees and random forests. In: 31st IJCAI, pp. 643–650 (2022)

    Google Scholar 

  2. Chlaoua, R., Meraoumia, A., Aiadi, K.E., Korichi, M.: Deep learning for finger-knuckle-print identification system based on PCANet and SVM classifier. Evol. Syst. 10(2), 261–272 (2019)

    Article  Google Scholar 

  3. Chow, C.: On optimum recognition error and reject tradeoff. IEEE Trans. Inf. Theory 16(1), 41–46 (1970)

    Article  MATH  Google Scholar 

  4. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20, 273–297 (1995)

    Article  MATH  Google Scholar 

  5. De Oliveira, A.C., Gomes, J.P.P., Neto, A.R.R., de Souza, A.H.: Efficient minimal learning machines with reject option. In: 5th BRACIS, pp. 397–402. IEEE (2016)

    Google Scholar 

  6. Guyon, I., Weston, J., Barnhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machines. Mach. Learn. 46, 389–422 (2002)

    Article  MATH  Google Scholar 

  7. Ignatiev, A.: Towards trustable explainable AI. In: IJCAI, pp. 5154–5158 (2020)

    Google Scholar 

  8. Ignatiev, A., Izza, Y., Stuckey, P.J., Marques-Silva, J.: Using MaxSAT for efficient explanations of tree ensembles. In: AAAI, vol. 36, pp. 3776–3785 (2022)

    Google Scholar 

  9. Ignatiev, A., Narodytska, N., Marques-Silva, J.: Abduction-based explanations for machine learning models. In: AAAI, vol. 33, pp. 1511–1519 (2019)

    Google Scholar 

  10. Ignatiev, A., Narodytska, N., Marques-Silva, J.: On formal reasoning about explanations. In: RCRA (2020)

    Google Scholar 

  11. Izza, Y., Ignatiev, A., Marques-Silva, J.: On explaining decision trees. arXiv preprint arXiv:2010.11034 (2020)

  12. Krause, J., Dasgupta, A., Swartz, J., Aphinyanaphongs, Y., Bertini, E.: A workflow for visual diagnostics of binary classifiers using instance-level explanations (2017)

    Google Scholar 

  13. Kroening, D., Strichman, O.: Decision Procedures. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-50497-0

  14. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  15. Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  16. Marques-Silva, J., Gerspacher, T., Cooper, M., Ignatiev, A., Narodytska, N.: Explaining Naive Bayes and other linear classifiers with polynomial time and delay. NeurIPS 33, 20590–20600 (2020)

    Google Scholar 

  17. Mesquita, D.P., Rocha, L.S., Gomes, J.P.P., Rocha Neto, A.R.: Classification with reject option for software defect prediction. Appl. Soft Comput. 49, 1085–1093 (2016)

    Google Scholar 

  18. Ribeiro, M.T., Singh, S., Guestrin, C.: “Why should I trust you?” Explaining the predictions of any classifier. In: 22nd ACM SIGKDD, pp. 1135–1144 (2016)

    Google Scholar 

  19. Ribeiro, M.T., Singh, S., Guestrin, C.: Anchors: high-precision model-agnostic explanations. In: AAAI, vol. 32 (2018)

    Google Scholar 

  20. Richhariya, B., Tanveer, M., Rashid, A.H., Initiative, A.D.N., et al.: Diagnosis of Alzheimer’s disease using universum support vector machine based recursive feature elimination (USVM-RFE). Biomed. Signal Process. Control 59, 101903 (2020)

    Article  Google Scholar 

  21. Vapnik, V.: Statistical Learning Theory. Wiley (1998)

    Google Scholar 

  22. Weld, D.S., Bansal, G.: Intelligible artificial intelligence. CoRR abs/1803.04263 (2018)

    Google Scholar 

  23. Yang, W., Si, Y., Wang, D., Guo, B.: Automatic recognition of arrhythmia based on principal component analysis network and linear support vector machine. Comput. Biol. Med. 101, 22–32 (2018)

    Article  Google Scholar 

Download references

Acknowledgments

The authors thank FUNCAP and CNPq for partially supporting our research work.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Francisco Mateus Rocha .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rocha, F.M., Rocha, T.A., Ribeiro, R.P.F., Rocha, A.R. (2023). Logic-Based Explanations for Linear Support Vector Classifiers with Reject Option. In: Naldi, M.C., Bianchi, R.A.C. (eds) Intelligent Systems. BRACIS 2023. Lecture Notes in Computer Science(), vol 14195. Springer, Cham. https://doi.org/10.1007/978-3-031-45368-7_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-45368-7_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-45367-0

  • Online ISBN: 978-3-031-45368-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics