Integrating probabilistic logic and multimodal spatial concepts for efficient robotic object search in home environments

Our study introduces a novel approach that combined probabilistic logic and multimodal spatial concepts to enable a robot to efficiently acquire place–object relationships in a new home environment with few learning iterations. By leveraging probabilistic logic, which employs predicate logic with probability values, we represent common-sense knowledge of the place–object relationships. The integration of logical inference and cross-modal inference to calculate conditional probabilities across different modalities enables the robot to infer object locations even when their likely locations are undefined. To evaluate the effectiveness of our method, we conducted simulation experiments and compared the results with three baselines: multimodal spatial concepts only, common-sense knowledge only, and common-sense knowledge and multimodal spatial concepts combined. By comparing the number of room visits required by the robot to locate 24 objects, we demonstrated the improved performance of our approach. For search tasks including objects whose locations were undefined, the findings demonstrate that our method reduced the learning cost by a factor of 1.6 compared to the baseline methods. Additionally, we conducted a qualitative analysis in a real-world environment to examine the impact of integrating the two inferences and identified the scenarios that influence changes in the task success rate.