Handheld Input Devices with Sleeves

A system may include an electronic device such as a head-mounted device and a handheld input device for controlling the electronic device. The handheld input device may include a stylus and a removable sleeve on the stylus. The input-output capabilities of the handheld input device may be shared between the stylus and the removable sleeve. The stylus may include touch sensor circuitry, a force-sensitive tip, and a motion sensor. The sleeve may include conductors for translating touch input on the sleeve to the touch sensor circuitry on the stylus, a deformable member for translating forces on the sleeve to the force-sensitive tip of the stylus, and visual markers that can be detected by an external camera and used with motion sensor data from the stylus to track the handheld input device. The removable sleeve may include haptic output devices and a battery and may be attached to an item without electronics.