Nawshin Mannan Proma
ABSTRACT
Autonomous vehicles (AV) have the potential to improve road transport, but faults in the autonomous driving software can result in serious accidents. To assess the safety of AV driving software, we need to consider the wide variety and diversity of situations that it may encounter. Explicit situation coverage has previously been presented, but its usefulness has received a little empirical scrutiny. In this study, we evaluate a situation coverage based safety testing approach by comparing the performance of random and situation coverage-based test generation in terms of its ability to detect seeded faults in our ego AV at a road intersection under diverse environmental conditions. Our results suggest that this implementation of situation coverage, at least, does not provide an advantage over random generation.
- [1] [n. d.]. https://cocodataset.org/Google Scholar
- Rob Alexander, Heather Rebecca Hawkins, and Andrew John Rae. 2015. Situation coverage–a coverage criterion for testing autonomous robots. (2015).Google Scholar
- Paul Ammann and Jeff Offutt. 2016. Introduction to software testing. Cambridge University Press.Google Scholar
- Anneliese Andrews, Mahmoud Abdelgawad, and Ahmed Gario. 2015. Active World Model for Testing Autonomous Systems Using CEFSM.. In MoDeVVa@ MoDELS. 1–10.Google Scholar
- Anneliese Andrews, Mahmoud Abdelgawad, and Ahmed Gario. 2015. Towards world model-based test generation in autonomous systems. In 2015 3rd International Conference on Model-Driven Engineering and Software Development (MODELSWARD). IEEE, 1–12.Google Scholar
- Anneliese Andrews, Mahmoud Abdelgawad, and Ahmed Gario. 2016. World model for testing autonomous systems using petri nets. In 2016 IEEE 17th International Symposium on High Assurance Systems Engineering (HASE). IEEE, 65–69.Google ScholarDigital Library
- Aren A Babikian. 2020. Automated generation of test scenario models for the system-level safety assurance of autonomous vehicles. In Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings. 1–7.Google ScholarDigital Library
- [8] Carla-Simulator. [n. d.]. https://github.com/carla-simulator/scenario_runnerGoogle Scholar
- Daniel J Fagnant and Kara Kockelman. 2015. Preparing a nation for autonomous vehicles: opportunities, barriers and policy recommendations. Transportation Research Part A: Policy and Practice 77 (2015), 167–181.Google ScholarCross Ref
- Francesca M Favarò, Nazanin Nader, Sky O Eurich, Michelle Tripp, and Naresh Varadaraju. 2017. Examining accident reports involving autonomous vehicles in California. PLoS one 12, 9 (2017), e0184952.Google ScholarCross Ref
- Heather Hawkins and Rob Alexander. 2019. Situation Coverage Testing for a Simulated Autonomous Car–an Initial Case Study. arXiv preprint arXiv:1911.06501 (2019).Google Scholar
- Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017).Google Scholar
- Jonathan Huang, Vivek Rathod, Chen Sun, Menglong Zhu, Anoop Korattikara, Alireza Fathi, Ian Fischer, Zbigniew Wojna, Yang Song, Sergio Guadarrama, 2017. Speed/accuracy trade-offs for modern convolutional object detectors. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7310–7311.Google ScholarCross Ref
- H. Kinsley. 2017. Object detection with tensorflow - self driving cars p.17. https://www.youtube.com/watch?v=UAXulqzn5PsGoogle Scholar
- Philip Koopman and Michael Wagner. 2017. Autonomous Vehicle Safety: An Interdisciplinary Challenge. IEEE Intelligent Transportation Systems Magazine 9, 1 (2017), 90–96. https://doi.org/10.1109/MITS.2016.2583491Google ScholarCross Ref
- Benjamin Lesage and Rob Alexander. 2021. SASSI: safety analysis using simulation-based situation coverage for cobot systems. In Computer Safety, Reliability, and Security: 40th International Conference, SAFECOMP 2021, York, UK, September 8–10, 2021, Proceedings 40. Springer, 195–209.Google Scholar
- Greig Mordue, Anders Yeung, and Fan Wu. 2020. The looming challenges of regulating high level autonomous vehicles. Transportation research part A: policy and practice 132 (2020), 174–187.Google Scholar
- Zaid Tahir and Rob Alexander. 2022. Intersection focused Situation Coverage-based Verification and Validation Framework for Autonomous Vehicles Implemented in CARLA. In Modelling and Simulation for Autonomous Systems: 8th International Conference, MESAS 2021, Virtual Event, October 13–14, 2021, Revised Selected Papers. Springer, 191–212.Google ScholarDigital Library
- Simon Ulbrich, Till Menzel, Andreas Reschka, Fabian Schuldt, and Markus Maurer. 2015. Defining and substantiating the terms scene, situation, and scenario for automated driving. In 2015 IEEE 18th international conference on intelligent transportation systems. IEEE, 982–988.Google ScholarDigital Library
- Oliver Zendel, Wolfgang Herzner, and Markus Murschitz. 2013. Vitro-model based vision testing for robustness. In IEEE ISR 2013. IEEE, 1–6.Google ScholarCross Ref
Index Terms
- Systematic Situation Coverage versus Random Situation Coverage for Safety Testing in an Autonomous Car Simulation
Recommendations
Intersection Focused Situation Coverage-Based Verification and Validation Framework for Autonomous Vehicles Implemented in CARLA
Modelling and Simulation for Autonomous SystemsAbstractAutonomous Vehicles (AVs) i.e., self-driving cars, operate in a safety-critical domain, since errors in the autonomous driving software can lead to huge losses. Statistically, road intersections which are a part of the AVs operational design ...
SASSI: Safety Analysis Using Simulation-Based Situation Coverage for Cobot Systems
Computer Safety, Reliability, and SecurityAbstractAssessing the safety of collaborative robot (cobot) systems is a difficult task due to the myriad of possible interactions between robots and operators, and the potential for injury to the operators. Using a situation coverage approach we can ...
The Hexagonal Spindle Model for Human Situation Awareness While Autonomous Driving
HCI in Mobility, Transport, and Automotive SystemsAbstractOver the years, many automobiles have been installed with automated functions (Debernard et al. [5]). Traditional automotive manufacturers in addition to newly emerging IT companies have been recently developing autonomous driving functions. ...
Comments