Systems and methods for performing a device action based on a detected gesture

Systems and methods for performing an action based on a detected gesture are provided. The systems and methods provided herein detect a direction of an initial touchless gesture and process subsequent touchless gestures based on the direction of the initial touchless gesture. The systems and methods translate a coordinate system related to a user device and a gesture library based on the detected direction such that subsequent touchless gestures can be processed based on the detected direction. The systems and methods allow a user to make a touchless gesture over a device to interact with the device independent of the orientation of the device since the direction of the initial gesture can set the coordinate system or context for subsequent gesture detection.