Keypoint Detection Technique for Image-Based Visual Servoing of Manipulators

This paper introduces an innovative keypoint detection technique based on Convolutional Neural Networks (CNNs) to enhance the performance of existing Deep Visual Servoing (DVS) models. To validate the convergence of the Image-Based Visual Servoing (IBVS) algorithm, real-world experiments utilizing f...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:arXiv.org 2024-09
Hauptverfasser: Amiri, Niloufar, Wang, Guanghui, Janabi-Sharifi, Farrokh
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title arXiv.org
container_volume
creator Amiri, Niloufar
Wang, Guanghui
Janabi-Sharifi, Farrokh
description This paper introduces an innovative keypoint detection technique based on Convolutional Neural Networks (CNNs) to enhance the performance of existing Deep Visual Servoing (DVS) models. To validate the convergence of the Image-Based Visual Servoing (IBVS) algorithm, real-world experiments utilizing fiducial markers for feature detection are conducted before designing the CNN-based feature detector. To address the limitations of fiducial markers, the novel feature detector focuses on extracting keypoints that represent the corners of a more realistic object compared to fiducial markers. A dataset is generated from sample data captured by the camera mounted on the robot end-effector while the robot operates randomly in the task space. The samples are automatically labeled, and the dataset size is increased by flipping and rotation. The CNN model is developed by modifying the VGG-19 pre-trained on the ImageNet dataset. While the weights in the base model remain fixed, the fully connected layer's weights are updated to minimize the mean absolute error, defined based on the deviation of predictions from the real pixel coordinates of the corners. The model undergoes two modifications: replacing max-pooling with average-pooling in the base model and implementing an adaptive learning rate that decreases during epochs. These changes lead to a 50 percent reduction in validation loss. Finally, the trained model's reliability is assessed through k-fold cross-validation.
format Article
fullrecord <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_3108437777</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>3108437777</sourcerecordid><originalsourceid>FETCH-proquest_journals_31084377773</originalsourceid><addsrcrecordid>eNqNjMsKwjAURIMgWLT_cMF1IU1a27UvFHVlcVtCva0pNal5CP69WfgBzmYWZ85MSMQ4T5MyY2xGYmt7SilbFSzPeUTOJ_yMWioHW3TYOKkVVNg8lHx5hFYbOD5Fh8laWLzDTVovBriieQenA93CRSg5-kE4beyCTFsxWIx_PSfL_a7aHJLR6HBnXd1rb1RANU9pmfEihP-3-gJRKD2S</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3108437777</pqid></control><display><type>article</type><title>Keypoint Detection Technique for Image-Based Visual Servoing of Manipulators</title><source>Free E- Journals</source><creator>Amiri, Niloufar ; Wang, Guanghui ; Janabi-Sharifi, Farrokh</creator><creatorcontrib>Amiri, Niloufar ; Wang, Guanghui ; Janabi-Sharifi, Farrokh</creatorcontrib><description>This paper introduces an innovative keypoint detection technique based on Convolutional Neural Networks (CNNs) to enhance the performance of existing Deep Visual Servoing (DVS) models. To validate the convergence of the Image-Based Visual Servoing (IBVS) algorithm, real-world experiments utilizing fiducial markers for feature detection are conducted before designing the CNN-based feature detector. To address the limitations of fiducial markers, the novel feature detector focuses on extracting keypoints that represent the corners of a more realistic object compared to fiducial markers. A dataset is generated from sample data captured by the camera mounted on the robot end-effector while the robot operates randomly in the task space. The samples are automatically labeled, and the dataset size is increased by flipping and rotation. The CNN model is developed by modifying the VGG-19 pre-trained on the ImageNet dataset. While the weights in the base model remain fixed, the fully connected layer's weights are updated to minimize the mean absolute error, defined based on the deviation of predictions from the real pixel coordinates of the corners. The model undergoes two modifications: replacing max-pooling with average-pooling in the base model and implementing an adaptive learning rate that decreases during epochs. These changes lead to a 50 percent reduction in validation loss. Finally, the trained model's reliability is assessed through k-fold cross-validation.</description><identifier>EISSN: 2331-8422</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Algorithms ; Artificial neural networks ; Automation ; Corners ; Datasets ; End effectors ; Image enhancement ; Robots ; Task space ; Visual control</subject><ispartof>arXiv.org, 2024-09</ispartof><rights>2024. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>780,784</link.rule.ids></links><search><creatorcontrib>Amiri, Niloufar</creatorcontrib><creatorcontrib>Wang, Guanghui</creatorcontrib><creatorcontrib>Janabi-Sharifi, Farrokh</creatorcontrib><title>Keypoint Detection Technique for Image-Based Visual Servoing of Manipulators</title><title>arXiv.org</title><description>This paper introduces an innovative keypoint detection technique based on Convolutional Neural Networks (CNNs) to enhance the performance of existing Deep Visual Servoing (DVS) models. To validate the convergence of the Image-Based Visual Servoing (IBVS) algorithm, real-world experiments utilizing fiducial markers for feature detection are conducted before designing the CNN-based feature detector. To address the limitations of fiducial markers, the novel feature detector focuses on extracting keypoints that represent the corners of a more realistic object compared to fiducial markers. A dataset is generated from sample data captured by the camera mounted on the robot end-effector while the robot operates randomly in the task space. The samples are automatically labeled, and the dataset size is increased by flipping and rotation. The CNN model is developed by modifying the VGG-19 pre-trained on the ImageNet dataset. While the weights in the base model remain fixed, the fully connected layer's weights are updated to minimize the mean absolute error, defined based on the deviation of predictions from the real pixel coordinates of the corners. The model undergoes two modifications: replacing max-pooling with average-pooling in the base model and implementing an adaptive learning rate that decreases during epochs. These changes lead to a 50 percent reduction in validation loss. Finally, the trained model's reliability is assessed through k-fold cross-validation.</description><subject>Algorithms</subject><subject>Artificial neural networks</subject><subject>Automation</subject><subject>Corners</subject><subject>Datasets</subject><subject>End effectors</subject><subject>Image enhancement</subject><subject>Robots</subject><subject>Task space</subject><subject>Visual control</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><recordid>eNqNjMsKwjAURIMgWLT_cMF1IU1a27UvFHVlcVtCva0pNal5CP69WfgBzmYWZ85MSMQ4T5MyY2xGYmt7SilbFSzPeUTOJ_yMWioHW3TYOKkVVNg8lHx5hFYbOD5Fh8laWLzDTVovBriieQenA93CRSg5-kE4beyCTFsxWIx_PSfL_a7aHJLR6HBnXd1rb1RANU9pmfEihP-3-gJRKD2S</recordid><startdate>20240920</startdate><enddate>20240920</enddate><creator>Amiri, Niloufar</creator><creator>Wang, Guanghui</creator><creator>Janabi-Sharifi, Farrokh</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20240920</creationdate><title>Keypoint Detection Technique for Image-Based Visual Servoing of Manipulators</title><author>Amiri, Niloufar ; Wang, Guanghui ; Janabi-Sharifi, Farrokh</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-proquest_journals_31084377773</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Algorithms</topic><topic>Artificial neural networks</topic><topic>Automation</topic><topic>Corners</topic><topic>Datasets</topic><topic>End effectors</topic><topic>Image enhancement</topic><topic>Robots</topic><topic>Task space</topic><topic>Visual control</topic><toplevel>online_resources</toplevel><creatorcontrib>Amiri, Niloufar</creatorcontrib><creatorcontrib>Wang, Guanghui</creatorcontrib><creatorcontrib>Janabi-Sharifi, Farrokh</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science &amp; Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Access via ProQuest (Open Access)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Amiri, Niloufar</au><au>Wang, Guanghui</au><au>Janabi-Sharifi, Farrokh</au><format>book</format><genre>document</genre><ristype>GEN</ristype><atitle>Keypoint Detection Technique for Image-Based Visual Servoing of Manipulators</atitle><jtitle>arXiv.org</jtitle><date>2024-09-20</date><risdate>2024</risdate><eissn>2331-8422</eissn><abstract>This paper introduces an innovative keypoint detection technique based on Convolutional Neural Networks (CNNs) to enhance the performance of existing Deep Visual Servoing (DVS) models. To validate the convergence of the Image-Based Visual Servoing (IBVS) algorithm, real-world experiments utilizing fiducial markers for feature detection are conducted before designing the CNN-based feature detector. To address the limitations of fiducial markers, the novel feature detector focuses on extracting keypoints that represent the corners of a more realistic object compared to fiducial markers. A dataset is generated from sample data captured by the camera mounted on the robot end-effector while the robot operates randomly in the task space. The samples are automatically labeled, and the dataset size is increased by flipping and rotation. The CNN model is developed by modifying the VGG-19 pre-trained on the ImageNet dataset. While the weights in the base model remain fixed, the fully connected layer's weights are updated to minimize the mean absolute error, defined based on the deviation of predictions from the real pixel coordinates of the corners. The model undergoes two modifications: replacing max-pooling with average-pooling in the base model and implementing an adaptive learning rate that decreases during epochs. These changes lead to a 50 percent reduction in validation loss. Finally, the trained model's reliability is assessed through k-fold cross-validation.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier EISSN: 2331-8422
ispartof arXiv.org, 2024-09
issn 2331-8422
language eng
recordid cdi_proquest_journals_3108437777
source Free E- Journals
subjects Algorithms
Artificial neural networks
Automation
Corners
Datasets
End effectors
Image enhancement
Robots
Task space
Visual control
title Keypoint Detection Technique for Image-Based Visual Servoing of Manipulators
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-24T23%3A03%3A59IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=document&rft.atitle=Keypoint%20Detection%20Technique%20for%20Image-Based%20Visual%20Servoing%20of%20Manipulators&rft.jtitle=arXiv.org&rft.au=Amiri,%20Niloufar&rft.date=2024-09-20&rft.eissn=2331-8422&rft_id=info:doi/&rft_dat=%3Cproquest%3E3108437777%3C/proquest%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=3108437777&rft_id=info:pmid/&rfr_iscdi=true