Analytically Quantifying Gains in the Test and Evaluation Process through Capabilities-Based Analysis

Military operating environments are increasingly diverse and technically challenging. Fielding relevant weapons systems to meet the demands of this environment is increasingly difficult, prompting policy shifts that mandate a focus on systems capable of combating a wide threat range. The Capabilities-Based Test and Evaluation (CBT&E) construct is the Department of the Navy's effort to concentrate on integrated system design with the objective of satisfying a particular operational response (capability) under a robust range of operating conditions. One aspect of CBT&E is the increased employment of advanced mathematical and statistical techniques in the Test and Evaluation (T&E) process. This study illustrates advantages of incorporating these invaluable techniques, like Design of Experiments (DOE) and Modeling and Simulation (M&S), within the T&E process. We also suggest a general methodology for approaching test plan design, presented via a notional scenario in which a complex system must defend a forward outpost. We found through statistical analysis that the application of DOE concepts to the System Under Test (SUT) throughout three primary phases of T&E quantifiably improved the accomplishment of the selected Measure of Effectiveness (MOE). The original document contains color images.