Combinatorial Interaction Testing of Software Product Lines: A Mapping Study

Software Product Line (SPL) is a software engineering paradigm that is inspired by the concept of reusability of common features, formulated for different software product. Complete testing on entire SPL is known to be unfeasible, due to the very large number of possible products to be produced, configured using a subset or all possible features in the SPL. This paper reports a systematic mapping study (SMS) of relevant primary studies as the evidence on the application of Combinatorial Interaction Testing (CIT) for SPL. In CIT, one has to construct a covering array, which is a set of configurations having valid feature combinations and every combination of t features appears at least once in the array. This is also known as t-wise testing. By following the systematic mapping study guidelines, we have selected and filtered 44 primary studies for review. The most prominent CIT techniques in aiding SPL testing are those based on greedy algorithms followed by meta-heuristics algorithms. The motivation of SPL testing is to anticipate the feature interaction problem, in which the majority of the works were reported to leverage test configuration selection approach, while some employed test configuration prioritization approach. Numerous works have been reported, but only few works managed to demonstrate their scalability, as most primary studies only deal with low strength (t is less than 4) of t-wise testing.