Recoverable asynchronous message driven processing in a multi-node system

An approach provides for recovery of message driven directors responsible for dynamically allocating resources within a multi-node database system. The receipt of a message, the action taken in response to it, and other messages sent in furtherance of the action, are performed within a single transaction. In this way, the messages and other data are left in a state consistent with the message and action taken in response being fully processed or not processed at all. A standby director, stepping in for a crashed director, inherits a state consistent with where a crashed director last successfully completed an action. The approach insulates directors from the special perils of performing high latency actions by delegating the performance of such actions to a job, (i.e. another process).