Debugging in data parallel computations

The debugging of a program in a data parallel environment. A connection is established between a debugging module and a process of the data parallel environment. The connection causes the data parallel environment to notify the debugging module of certain events as they occur in the execution of the process. Upon notification of such an event, the process execution is paused, and the debugging module may query the data parallel environment for information regarding the process at the device independent virtual machine layer. Upon completion of this querying, the process may then resume execution. This may occur repeatedly if multiple events are encountered.