SBFT Tool Competition 2024 - CPS-UAV Test Case Generation Track
•
0 gefällt mir•6 views
Sajad Khatiri, Prasun Saurabh, Timothy Zimmermann, Charith Munasinghe, Christian Birchler, Sebastiano Panichella: SBFT Tool Competition 2024 - CPS-UAV Test Case Generation Track 17th International Workshop on Search-Based and Fuzz Testing
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Mehr von Sebastiano Panichella
Mehr von Sebastiano Panichella (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
SBFT Tool Competition 2024 - CPS-UAV Test Case Generation Track
- 1. SBFT Tool Competition 2024 CPS-UAV Test Case Generation Track IEEE/ACM International Workshop on Search-Based and Fuzz Testing (SBFT) 2024 15 April 2024 Sajad Khatiri Prasun Saurabh Timothy Zimmermann Charith Munasinghe Christian Birchler Sebastiano Panichella
- 2. • ` Unmanned Aerial Vehicles (UAVs) 2
- 3. 3 Sample PX4 Flight Log
- 4. Aerialist UAV Test Bench 4 "Simulation-based testing of unmanned aerial vehicles with Aerialist“, ICSE 2024 Khatiri, Sajad, Sebastiano Panichella, and Paolo Tonella UAV Config. Env. Config. Commands Expectation Test Description
- 5. Given an autonomous UAV flight mission Generate test cases that violate safety distance to the obstacles by placing obstacles in the environment 5 Test Generation
- 6. Competition Rules • Use the provided platform for test definition • Test Generation using a Search-based approach • Place up to 4 box-shaped obstacles • Size (length, width, height) • Position (x, y, z) • Orientation (r) • Obstacles should • Keep the mission physically possible • Fit in the predefined area • Be Taller than the flight altitude (10m) 6
- 7. Evaluation • 6 competing test generation tools (7 submitted) • 1 baseline approach • Generated tests for 6 flight missions • With 200 Simulation Budget • Using our K8S evaluation platform • Report the failing ones • 20 Tests from each test suite were evaluated • Simulated 5 times • Assign points to each execution • Assign score to each test case • Estimate Test Suite Score 7
- 8. Tests Count 8 0 20 40 60 80 100 120 140 160 mission 2 mission 3 mission4 mission 5 mission 6 mission 7 TAIiST Surrealist AmbieGen DeepHyperion-UAV CAMBA TUMB WOGAN-UAV
- 9. Test Suite Score 9 0.00 2.00 4.00 6.00 8.00 10.00 12.00 14.00 16.00 18.00 mission 2 mission 3 mission 4 mission 5 mission 6 mission 7 TAIiST Surrealist AmbieGen DeepHyperion-UAV CAMBA TUMB WOGAN-UAV
- 10. Ranking 10 0.00 10.00 20.00 30.00 40.00 50.00 60.00 WOGAN-UAV TUMB CAMBA DeepHyperion-UAV AmbieGen TAIiST Surrealist [baseline] Total Score
- 11. 11 0.00 10.00 20.00 30.00 40.00 50.00 60.00 WOGAN-UAV TUMB CAMBA DeepHyperion-UAV AmbieGen TAIiST Surrealist [baseline] Total Score github.com/skhatiri/UAV-Testing-Competition github.com/skhatiri/Aerialist mazr@zhaw.ch Sajad Khatiri ICSE Demonstration Presentation on Aerialist Time: Friday 19.4, 15:15 Room: Fernando Pessoa