A Novel Approach to Automate IoT Testing of Gateways and Devices

The software (firmware) testing for Internet of Things (IoT) usually involves multiple communication protocols and different hardware devices. Together with complex user scenarios/ environments and frequently updated firmware, it can be time-consuming and error-prone to build test environments and conduct the testing. To alleviate these problems, this paper presents an approach and an automated IoT Testing Tool, called IoT3, which automatically builds test environments, executes tests for IoT gateways, and performs system compatibility testing. For gateway testing, IoT3 supports keyword-driven testing method, facilitating the development and maintenance of test scripts. In addition, a mock device method is proposed to simulate the changes of environment conditions, such as temperature, humidity, and the packet loss of Bluetooth connections. For compatibility testing, IoT3 allows a test engineer to select a target environment, a particular version combination of the IoT devices (gateways, sensors, or Apps), and performs a full compatibility testing for the devices. To evaluate the effectiveness of the approach, an industry case study was conducted. The results indicate that, as compared to manual testing, using the proposed approach (IoT3) can save testing time, reduce human efforts, improve test coverage, and also detect more defects.