TTCN-3 Bibliography
TTCN-3 Bibliography |
|
 |
 |
 |
|
Vega, D. 2008, June 3–6, TTCN-3 data analyzer using constraint programming. Unpublished paper presented at ETSI TTCN-3 User Conference 2008, Madrid (Spain). Added by: Deleted user (20/08/2008, 15:13) |
| Resource type: Conference Paper BibTeX citation key: Vega View all bibliographic details  |
Categories: General Creators: Vega Publisher: MTP, ETSI (Madrid (Spain)) Collection: ETSI TTCN-3 User Conference 2008 |
Views: 55/2616
|
| Abstract |
|
Assessing the test quality, producing and identifying effective tests are challenging tasks and debated subjects in the research and industry. In this respect, a framework for analyzing different quality aspects of test specifications has been provided in. It proposes a quality model for test specifications derived from the ISO/IEC 9126 quality model. Various test metrics have been already developed measuring selected aspects but they are still in their infancy or regard test quality aspects rather at programming language level than at test specific expectations. The test coverage which is the measure of test completeness on different levels can be obtained easier, e.g. the extent to which a test specification covers system requirements, system model, system code, etc. Hence, test coverage can be analyzed with the help of different metrics. In the white-box (structural) testing, testing code metrics such as statement, branch coverage, etc. indicate the degree to which the test specification covers the system code. In many cases, only the system interfaces are available (e.g. black-box testing), either provided as interface specification, documentation or formally described. With the advances of model-based system development, a model of the system is provided as well. Consequently, black-box (functional) system model coverage metrics are in use and usually adopted from code coverage metrics, e.g.: state coverage, transition coverage and alike. This presentation introduces the idea of analyzing and refining the test data variance computation as a method to determine the quality of TTCN-3 tests. Data variance characterizes the test data distribution over the Test System Interface (TSI). The coverage at the TSI level may be assessed by means of a semantic similarity derived by using a partitioning method of the stimuli space. TTCN-3 template distance constitutes the criterion to quantify the variance: “similar” and “different enough” test data. Due to the high flexibility of the language, the stimuli templates to be compared may not contain concrete values but also variables, user defined or language specific function calls, testcase and module parameters, expressions, values which are known at runtime only, etc. Therefore, we combine Constraint Programming (CP) - a software technology for effective solving problems - with static analysis of TTCN-3 test suites targeting a realistic template solving. We name template constraint an expression whose value (or domain restrictions) can only be determined by looking at the execution paths in the analyzed test behavior. To carry out quantitative estimates, i.e. come to concrete values of the distance metrics, a template solver method is required. This is mostly applicable for the cases where complex tree-like structured templates (e.g. record types) define the set of stimuli for a specific port, of a specific TTCN-3 type. In the presented approach, these templates are subject of distance computation. Due to the recursive definition of the template distance for structured types (e.g. for record types we consider n-Dimensional Euclidian Distance), only the concrete values corresponding to the fields of basic types are necessary. Since the distance computation in a TTCN-3 test suite is achieved along a static analysis, we applied Symbolic Execution which assumes that instead of supplying the normal inputs to a program (e.g. numbers), one supplies symbols representing arbitrary values within a specific domain or constrained values. In white-box testing, the Symbolic Execution method uses the Control Flow Graph (CFG) of a program, which is a graph abstraction of the program, and symbolically executes the program by selecting only one execution path from the CFG. Tools such as the well known Java PathFinder (JPF) have successfully adopted this technique to prove the correctness of a program. Following this method, we built first the CFG associated to a TTCN-3 testcase whose behavior encapsulates a selected stimuli-message involved in the distance calculation. A TTCN-3 symbolic values solver has been realized with the help of CFG and constraint programming methods (CP) targeting the domain reduction for a specific template field. Added by: Deleted user |