A System Analysis Study Comparing Reverse Engineered Combinatorial Testing to Expert Judgment

The Lockheed Martin F-16 ventral fin redesign-combinatorial test study effort was to demonstrate how Combinatorial Testing (CT) could have been applied to system hardware design flaw analysis. The historic analysis was able to determine a set of combinations, which isolated the problem and tested a solution. However, the original effort was expensive, time consuming, and required highly specialized knowledge from the expert to be effective. The new study was an effort to understand if combinatorial test could be applied to similar situations using the original data but conducted by a less senior person without an expert's knowledgebase. The situation and CT approaches are detailed in this paper. In the study, a series of iterations created combinatorial test cases which could have "replicated" the original highly optimized and successful test program, without the expert.