Using multi-modal input to control multiple objects on a display

Disclosed is a modular system with several modal input devices, each modal input controlling a separate item on the display. Each input device retrieves a stream of modal input data from the user. The inputs are interpreted by modal interpretation logic that can interpret each of the retrieved modal input data streams into a corresponding of set of actions. Each modal input device is assigned to control one of the displayed objects by modal pairing logic so that each corresponding set of actions sent to the allocated object. Each set of actions is then used by modal control logic to be control each displayed object by its assigned set of actions. The modal pairing logic may be operable to dynamically change the assignment of the modal input devices and their corresponding display objects. The system may have input management logic to manage at least two modal input devices as a group, allowing the user to provide simultaneous control of two or more objects.