Aggregation Dynamics in Overlay Networks and Their Implications for Self-Organized Distributed Applications

In this paper, we investigate the global self-aggregation dynamics arising from local decision-based rewiring of an overlay network, used as an abstraction for an autonomic service-oriented architecture. We measure the ability of a selected set of local rules to foster self-organization of what is originally a random graph into a structured network. Scalability issues with respect to the key parameters of system size and diversity are extensively discussed. Conflicting goals are introduced, in the form of a population of nodes actively seeking to acquire neighbours of a type different from their own, resulting in decreased local homogeneity. We show that a 'secondary' self-organization process ensues, whereby nodes spontaneously cluster according to their implicit objective. Finally, we introduce dynamic goals by making the preferred neighbour type a function of the local characteristics of a simulated workload. We demonstrate that in this context, an overlay rewiring process based purely on local decisions and interactions can result in efficient load-balancing without central planning. We conclude by discussing the implications of our findings for the design of future distributed applications, the likely influence of other factors and of extreme parameter values on the ability of the system to self-organize and the potential improvements to our framework.