High-quality scenario-based testing for autonomous cars
Trustworthy operation of Automated Vehicles (AV) in a safe way is a major bottleneck for commercial introduction of this new technology. Consumers must feel safe with AVs or they will not purchase them, and developers need to build and approve methodologies to assure safety of AVs.
Validation aims to demonstrate that the techniques for AV meet the requirements of the users. For systems deploying components that make use of Artificial Intelligence technologies such as machine learning, this task is particularly hard because established engineering approaches like classical software verification and validation tests cannot directly be applied.
In BEST we develop a novel methodology and technology for deriving tests from databases of pre-recorded drives. The main research goals of BEST include the provision of quantitative guarantees on the required quality of tests and the development of methods for better test case generation.
A popular suggestion to test autonomous vehicles is to randomly pick test cases for virtual testing from pre-recorded drives. However, huge amounts of data are necessary to achieve a sufficiently high coverage of all potential situations. Some studies suggest that recordings of at least 6.6 billion kilometres will be needed.
Moreover, the direct reuse of recorded drives for testing purposes as well as the random approach are questionable undertakings. In terms of random testing, the search space simply is too large. In terms of directly reusing recorded drives, the quality of pre-recorded test cases is system-specific: recorded test cases may be “good” test cases for one system (system version or variant) and useless for another. Consequently, system-specific test cases have to be generated.
The approach taken in the BEST project is to analyse databases of recorded driving or traffic situations and to use clustering techniques to mine various abstract scenarios, so-called scenario types. System-specific tests are then generated by searching for particular parameters settings in a scenario type that may lead to the system violating certain safety requirements, such as not obeying a minimal safety distance between cars.
01.10.2019 – 30.09.2022
