Systems and methods for identifying and tracking physical objects during a robotic surgical procedure

Systems and methods are disclosed comprising a robotic device, an instrument attachable to the robotic device to treat tissue, a vision device attached to the robotic device or instrument, and one or more controllers. The vision device generates vision data sets captured from multiple perspectives of the physical object enabled by the vision device moving in a plurality of degrees of freedom during movement of the robotic device. The controller(s) have at least one processor and are in communication with the vision device. The controller(s) associate a virtual object with the physical object based on one or more features of the physical object identifiable in the vision data sets. The virtual object at least partially defines a virtual boundary defining a constraint on movement of the robotic device relative to the physical object. In some cases, movement of the robotic device is actively constrained by using the virtual boundary.