Black-Box System Testing of Real-Time Embedded Systems Using Random and Search-based Testing
Abstract
Testing real-time embedded systems (RTES) is in many
ways challenging. Thousands of test cases can be potentially executed on
an industrial RTES. Given the magnitude of testing at the system level,
only a fully automated approach can really scale up to test industrial
RTES. In this paper we take a black-box approach and model the RTES
environment using the UML/MARTE international standard. Our main
motivation is to provide a more practical approach to the model-based
testing of RTES by allowing system testers, who are often not familiar
with the system design but know the application domain well-enough, to
model the environment to enable test automation. Environment models can
support the automation of three tasks: the code generation of an
environment simulator, the selection of test cases, and the evaluation
of their expected results (oracles). In this paper, we focus on the
second task (test case selection) and investigate three test automation
strategies using inputs from UML/MARTE environment models: Random
Testing (baseline), Adaptive Random Testing, and Search-Based Testing
(using Genetic Algorithms). Based on one industrial case study and three
artificial systems, we show how, in general, no technique is better than
the others. Which test selection technique to use is determined by the
failure rate (testing stage) and the execution time of test cases.
Finally, we propose a practical process to combine the use of all three
test strategies.
Origin | Files produced by the author(s) |
---|
Loading...