Centralized and decentralized asynchronous optimization of stochastic discrete-event systems

We propose and analyze centralized and decentralized asynchronous control structures for the parametric optimization of stochastic discrete-event systems (DES) consisting of K distributed components. We use a stochastic approximation type of optimization scheme driven by gradient estimates of a global performance measure with respect to local control parameters. The estimates are obtained in distributed and asynchronous fashion at the K components based on local state information only. We identify two verifiable conditions for the estimators and show that if they, and some additional technical conditions, are satisfied, our centralized optimization schemes, as well as the fully decentralized asynchronous one we propose, all converge to a global optimum in a weak sense. All schemes have the additional property of using the entire state history, not just the part included in the interval since the last control update; thus, no system data are wasted. We include an application of our approach to a well-known stochastic scheduling problem and show explicit numerical results using some recently developed gradient estimators.