Visual interaction method and system for robot with body, terminal and medium

The invention relates to a visual interaction method and system for a robot with a body, a terminal and a medium, and the method comprises the steps: obtaining visual image data of the robot, the visual image data comprises an RGB image, a depth image and an infrared image; inputting the visual imag...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
1. Verfasser: ZHANG YONGPAN
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention relates to a visual interaction method and system for a robot with a body, a terminal and a medium, and the method comprises the steps: obtaining visual image data of the robot, the visual image data comprises an RGB image, a depth image and an infrared image; inputting the visual image data into a pre-trained image processing model for processing to obtain feature point data of the visual image data; performing deep fusion on the feature point data of the visual image data by using a multi-modal feature fusion algorithm to generate spatial perception data of the robot surrounding environment; based on the spatial perception data, performing three-dimensional reconstruction and scene analysis to obtain an identification analysis result; according to the recognition analysis result, an interaction scheme of the robot is generated, and the interaction scheme comprises path planning, interaction actions with the object and voice intonations; and adjusting a behavior strategy of the robot according