Fuzzy test model for performance evaluation matrix of service operating systems

•We proposed a novel approach for PEM of service operating systems.•We developed a Buckley fuzzy test to collects fuzzy linguistic data credibly.•The confidence intervals construct membership functions to reduce sampling error.•The fuzzy testing for satisfaction indices identifies items in need of improvement.•Pair-wise comparisons fuzzy test of influence indices prioritize items to improve. The performance evaluation matrix is a useful and convenient tool for collecting user opinions and identifying service items that are in need of improvement in operating systems. It has been used in many studies looking for ways to increase the response willingness of customers and solve the problem of fuzziness in user opinions. Likert scales are simple and can increase the response willingness of customers and the efficiency of data collection. In contrast, fuzzy linguistic scales can more accurately capture the voice of the customer, but responding to them is a more complex matter and lowers the response willingness of customers. Confidence intervals can lower the risk of misjudgment caused by sampling errors in user opinion collection; the smaller the confidence intervals are, the lower the risk of misjudgment is. We therefore developed the Buckley fuzzy test methods that maintain the simplicity offered by Likert scales in collecting data. We used the indices confidence intervals derived from the data to construct fuzzy membership functions and develop our fuzzy test methods. Clearly, those approaches solve the problem of fuzziness in customer opinions without affecting the response willingness of customers. Those methods are grounded in confidence intervals, which reduce the risk of misjudgment caused by sampling errors. Furthermore, when confidence intervals are relatively wide due to cost and time considerations, the Buckley fuzzy test methods overcome this limitation by swiftly identifying service items that are in need of improvement in an operating system, as well as prioritizing service items using pair-wise comparisons of the influence indices of the service items. This enables the identification of the service items that should take priority when resources are limited. Finally, we used a case study involving a computer-assisted language learning system to demonstrate the applicability of the proposed approach.