Scheduling hardware resources for offloading functions in a heterogeneous computing system

A heterogeneous computing system can include a host memory and a host processor. The host memory is configured to maintain a write task queue and a read task queue. The host processor is coupled to the host memory and a processing device. The host processor is adapted to store write tasks in the write task queue. The write tasks cause transfer of input data to the processing device. The processing device is adapted to perform offloaded functions. The host processor is adapted to store read tasks in the read task queue. The read tasks cause transfer of results from the offloaded functions from the processing device. The host processor is further adapted to maintain a number of direct memory access (DMA) worker threads corresponding to concurrent data transfer capability of the processing device. Each DMA worker thread is preconfigured to execute tasks from the write task queue or the read task queue.