An Independent Evaluation of the Rome Laboratory Framework for Certification of Reusable Software Components

In this effort, Underwriters Laboratories (UL) conducted an independent review of the Rome Laboratory framework for certification of reusable software components from the perspective of its potential application by a third party test laboratory. The review consisted of two parts: a desk review and a trial application. The desk review considered the goals and objectives of the framework, its costs and benefits, its standardization potential, and technology transfer issues. In the trial application, an asset was selected and the default certification process was applied. A scenario for the development and use of the asset was created to provide the context from which certification concerns, criteria, and requirements were identified. Benchmark components, including instrumented code, models and fault sets, were created for the asset to provide expected results to access the results of the procedures specified by the default process. UL determined the framework is built upon clear and sound research. UL found that the framework's effort to associate techniques with defect types is a necessary approach, however, they recommended more detailed analysis of the relationships between techniques and defects before techniques can be recommended with confidence. UL plans to pursue this research in an experimental lab being established at UL. They plan to broaden the framework's certification model and position components of the framework for incorporation into certification standards.