SIMTAM: Generation Diversity Test Programs for FPGA Simulation Tools Testing Via Timing Area Mutation

Field-Programmable Gate Array (FPGA) timing simulation is essential in electronic circuit design, allowing for the verification of timing characteristics like delays and clock frequencies. However, bugs in timing simulation tools can lead to inaccurate results, potentially causing designers to miss critical issues in chip performance. Traditional testing methods often fall short in thoroughly assessing these tools, as current FPGA testing primarily focuses on synthesis and behavioral simulation, neglecting timing aspects. To address this issue, we propose SIMTAM for testing timing simulation tools. Specifically, SIMTAM consists of three components: equivalent delay region construction, diversity program segment generation, and differential testing. Given a seed circuit design file written by hardware description language such as Verilog, the delay region construction component randomly identifies delay structures for inertial delay in the design file to construct equivalent delay sleep regions. In the sleep region, the simulator skips the signal pulse whose width is less than the specified delay, thus ensuring the equivalence of the variations. The diversity program segment generation component combines Verilog expressions using generation operators and inject them into the sleep region to generate diverse design files. The differential testing component compares the seed and variant design files to find compilation inconsistency issues. In five months, SIMTAM reported 16 bugs to developers in two popular timing simulation tools Iverilog and Vivado; ten of which are confirmed.