Demo: Scaling on the Edge – A Benchmarking Suite for Human-in-the-Loop Applications

Benchmarking human-in-the-loop appli-cations running on edge computing in-frastructure is complex given their nature,which heavily depends on the actions takenby the human user. This limits reproducibil-ity as well as feasibility of performance eval-uations. We propose a methodology andpresent a benchmarking suite that can ad-dress these challenges. Our core idea restson recording traces of these applicationswhich are played out in a controlled fashionbased on an underlying model of human be-havior. The traces are exposed to the origi-nal backend compute process of the respec-tive human-in-the-loop application, gener-ating realistic feedback. This allows for anautomated system which greatly simplifiesbenchmarking large scale scenarios.