Autonomous UAV Visual Navigation Using an Improved Deep Reinforcement Learning

In recent years, unmanned aerial vehicles (UAVs) have grown in popularity for a variety of purposes, including parcel delivery, search operations for missing persons, and surveillance. However, autonomously navigating UAVs in dynamic environments is a challenging task due to the presence of moving o...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE access 2024, Vol.12, p.79967-79977
Hauptverfasser: Samma, Hussein, El-Ferik, Sami
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In recent years, unmanned aerial vehicles (UAVs) have grown in popularity for a variety of purposes, including parcel delivery, search operations for missing persons, and surveillance. However, autonomously navigating UAVs in dynamic environments is a challenging task due to the presence of moving objects like pedestrians. In addition, traditional deep reinforcement learning approaches suffer from slow learning rates in dynamic situations and they need substantial training data. To improve learning performance, the present study proposed an enhanced deep reinforcement learning approach that encompasses two distinct learning stages namely the reinforced and self-supervised. In the reinforced learning stage, the deep Q-learning network (DQN) has been implemented and trained guided by the loss in the bellmen equation. On the other hand, the self-supervised stage is responsible for fine-tuning the backbone layers of DQN and it was directed by the contrastive loss function. The main benefit of incorporating the self-supervised stage is to speed up the encoding of the input scene captured by the UAV camera. To further enhance the navigation performances, an obstacle detection model was embedded to reduce UAV collisions. For experimental analysis, we have utilized an outdoor UAV-simulated environment called Blocks. This environment contains stationary objects that mimic buildings, as well as moving pedestrians. The study undertaken indicates that the implementation of the self-supervised stage led to significant improvements in navigation performance. Specifically, the simulated UAV was able to navigate longer distances in the correct direction toward the goal point. Moreover, the conducted analysis shows a significant navigation performance as compared with other DQN-based approaches like double DQN and dueling DQN.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2024.3409780