SDN-based fog and cloud interplay for stream processing

This paper focuses on SDN-based approaches for deploying stream processing workloads on heterogeneous environments comprising wide-area networks, cloud and fog resources. Stream processing applications impose strict latency requirements to operate appropriately. Deploying workloads in the fog reduces unnecessary delays, but its computational resources may not handle all the tasks. On the other hand, offloading the tasks to the cloud is constrained by limited network resources and involves additional transmission delays that exceed latency thresholds. Adaptive workload deployment may solve these issues by ensuring that resource and latency requirements are satisfied for all the data streams processed by an application. This paper’s main contribution consists of dynamic workload placement algorithms operating on stream processing requests with latency constraints. Provisioning of computing infrastructure exploits the interplay between fog and cloud under limited network capacity. The algorithms aim to maximize the ratio of successfully handled requests by effectively utilizing available resources while meeting application latency constraints. Experiments demonstrate that the goal can be achieved by detailed analysis of requests and ensuring balanced computing and network resources utilization. As a result, up to 30% improvement over the reference algorithms in success rate is observed. •Dynamic workload deployment is highly affected by resource allocation algorithms.•Transmission latency is a key factor of the operator allocation process.•Considering both resource utilization and request execution graph yields best results.•Two of the proposed algorithms provide both optimal resource usage and success rates.