Re-estimation of software reliability after maintenance

The common practice of reusing test cases for regression testing is incompatible with estimating the reliability of modified software, because dependencies between software changes and previous test results lead to estimation bias. Statistical testing with random operational inputs can produce valid reliability estimates, but its cost may be excessive unless the nature of software changes is considered. We propose an economical way of estimating the reliability of modified software under certain common circumstances. It calls for updating a previous reliability estimate by estimating differences in the behavior of successive software versions. The older version is used as a relative oracle to reduce the number of executions that must be checked manually for conformance to requirements. Our approach is justified using only elementary probability and does not require reliability growth assumptions.