An FPGA-based Integrated MapReduce Accelerator Platform

MapReduce is a programming framework for distributed systems that is used to automatically parallelize and schedule the tasks to distributed resources. MapReduce is widely used in data centers to process enterprise databases and Big Data. This paper presents a novel MapReduce accelerator platform based on FPGAs that can be used to speedup the processing of the MapReduce data. The proposed platform consists of specialized hardware accelerators for the Map tasks and a shared configurable accelerator for the Reduce tasks. The hardware accelerators for the Map tasks are developed using a modified source-to-source High-level Synthesis (HLS) tool while the Reduce accelerator is based on a novel hashing scheme. The proposed scheme is implemented, mapped and evaluated to a Virtex 7 FGPA. The performance evaluation is based on a benchmark suite that represent typical MapReduce applications and it shows that the proposed scheme can achieve up to 2 orders of magnitude energy reduction compared to General Purpose Processors (GPPs).