The NAIL Accelerator Interface Layer for Low Latency FPGA Offload

We present the NAIL Accelerator Interface Layer, a framework for offloading accelerated computations to Field Programmable Gate Arrays served across the network. NAIL has been specifically optimised for latency sensitive applications and has been deployed and sustained for a variety of high-scale operational workloads. It allows compute acceleration benefits of Field Programmable Gate Arrays (FPGAs) to be more easily exploited through a flexible host communication layer. Multiple accelerators with independent streams are supported with demonstrated scalability to large numbers of concurrent tasks. Virtualisation and management is incorporated into the hardware to do away with the typical overheads and design complexities of software-managed offload frameworks. As a result, it achieves very low latencies of 4 microseconds for inline requests to below 10 microseconds for larger request sizes. NAIL has been developed and deployed in a challenging industrial setting and is now released to the wider community as open source.