IoTBench: A data centrical and configurable IoT benchmark suite

As the Internet of Things (IoT) industry expands, the demand for microprocessors and microcontrollers used in IoT systems has increased steadily. Benchmarks provide a valuable reference for processor evaluation. Different IoT application scenarios face different data scales, dimensions, and types. However, the current popular benchmarks only evaluate the processor’s performance under fixed data formats. These benchmarks cannot adapt to the fragmented scenarios faced by processors. This paper proposes a new benchmark, namely IoTBench. The IoTBench workloads cover three types of algorithms commonly used in IoT applications: matrix processing, list operation, and convolution. Moreover, IoTBench divides the data space into different evaluation subspaces according to the data scales, data types, and data dimensions. We analyze the impact of different data types, data dimensions, and data scales on processor performance and compare ARM with RISC-V and MinorCPU with O3CPU using IoTBench. We also explored the performance of processors with different architecture configurations in different evaluation subspaces and found the optimal architecture of different evaluation subspaces. The specifications, source code, and results are publicly available from https://www.benchcouncil.org/iotbench/. •We propose and implement IoTBench, which covers three types of algorithms commonly used in IoT applications: matrix processing, list operation, and convolution.•We analyze the impact of different data types, data dimensions, and data scales on the processor performance, and compare ARM with RISC-V and MinorCPU with O3CPU using IoTBench.•We explored the performance of processors with different architecture configurations in different evaluation subspaces and find the optimal architecture of different evaluation subspaces.