Video presentation based on multiple-flying camera to provide continuous and complementary images for teleoperation

To enhance visual information for the teleoperators of construction machines, we previously developed a video system that automatically controls the angle and zoom of multiple environmental cameras on the ground. However, in general, ground-fixed camera systems (GCS) have issues regarding the quality of the images, i.e., frequent image switching and insufficient viewpoint adjustment. Toward solving these problems, this study proposes a fundamental design method for a video presentation system using multiple flying cameras and investigates teleoperators' responses in terms of behavioral and mental aspects. The teleoperators need to rapidly understand the environment and precisely control the machine, so the video presentation system requires continuity to project consistent images by avoiding frequent changes in capturing parameters. It also requires complementarity between images to provide three-dimensional (3D) information with less mental rotation and translation. To investigate the feasibility of the framework of a multi-flying camera system (FCS), we implemented a prototype and ran experiments in which a construction machine in a virtual reality simulator moves over uneven terrain and carries objects. We found that FCS could reduce the work time and subjective workload and increase the subjective usability compared to a conventional GCS. Moreover, we could obtain the gaze distribution and view transition characteristics as well as their individual differences. These results will contribute to establishing a fundamental design method for FCS. •A video presentation system uses multiple flying cameras that are dynamically placed in the air.•The system adopts the tracking mode and projects one camera image onto one corresponding view.•The system adopts a cab view, an overhead view, and two manipulator views.•The proposed system reduced work time and subjective workload.