BCI Control of a Robotic arm based on SSVEP with Moving Stimuli for Reach and grasp Tasks

Brain-computer interface (BCI) provides a novel technology for patients and healthy human subjects to control a robotic arm. Currently, BCI control of a robotic arm to complete the reaching and grasping tasks in an unstructured environment is still challenging because the current BCI technology does...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE journal of biomedical and health informatics 2023-08, Vol.27 (8), p.1-12
Hauptverfasser: Ai, Jikun, Meng, Jianjun, Mai, Ximing, Zhu, Xiangyang
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 12
container_issue 8
container_start_page 1
container_title IEEE journal of biomedical and health informatics
container_volume 27
creator Ai, Jikun
Meng, Jianjun
Mai, Ximing
Zhu, Xiangyang
description Brain-computer interface (BCI) provides a novel technology for patients and healthy human subjects to control a robotic arm. Currently, BCI control of a robotic arm to complete the reaching and grasping tasks in an unstructured environment is still challenging because the current BCI technology does not meet the requirement of manipulating a multi-degree robotic arm accurately and robustly. BCI based on steady-state visual evoked potential (SSVEP) could output a high information transfer rate; however, the conventional SSVEP paradigm failed to control a robotic arm to move continuously and accurately because the users have to switch their gaze between the flickering stimuli and the target frequently. This study proposed a novel SSVEP paradigm in which the flickering stimuli were attached to the robotic arm's gripper and moved with it. First, an offline experiment was designed to investigate the effects of moving flickering stimuli on the SSVEP's responses and decoding accuracy. After that, contrast experiments were conducted, and twelve subjects were recruited to participate in a robotic arm control experiment using both the paradigm one (P1, with moving flickering stimuli) and the paradigm two (P2, conventional fixed flickering stimuli) using a block randomization design to balance their sequences. Double blinks were used to trigger the grasping action asynchronously whenever the subjects were confident that the position of the robotic arm's gripper was accurate enough. Experimental results showed that the paradigm P1 with moving flickering stimuli provided a much better control performance than the conventional paradigm P2 in completing a reaching and grasping task in an unstructured environment. Subjects' subjective feedback scored by a NASA-TLX mental workload scale also corroborated the BCI control performance. The results of this study suggest that the proposed control interface based on SSVEP BCI provides a better solution for robotic arm control to complete the accurate reaching and grasping tasks.
doi_str_mv 10.1109/JBHI.2023.3277612
format Article
fullrecord <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_crossref_primary_10_1109_JBHI_2023_3277612</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>10128977</ieee_id><sourcerecordid>2847967462</sourcerecordid><originalsourceid>FETCH-LOGICAL-c350t-7b89efbf664a23d6e319c864ee23d2a926cfd877420dc7e00eaf788bfd6a49943</originalsourceid><addsrcrecordid>eNpdkE1LJDEQhoMoKuoPEJYl4MXLjEmlzcdxHUadRVEcFTw16e6KtnZ3xqTbZf_9ZphRFnOppHjqpfIQcsjZmHNmTn6fXc7GwECMBSglOWyQXeBSjwCY3vy8c5PtkIMYX1k6OrWM3CY7QgFjXMAueTqbzOjEd33wDfWOWnrnC9_XJbWhpYWNWFHf0fn8cXpL_9T9C732H3X3TOd93Q5NTZ0P9A5t-UJtV9HnYOOC3tv4FvfJlrNNxIN13SMP59P7yeXo6uZiNvl1NSrFKetHqtAGXeGkzCyISqLgptQyQ0wvsAZk6SqtVAasKhUyhtYprQtXSZsZk4k9crzKXQT_PmDs87aOJTaN7dAPMYf0ayUFGJXQo2_oqx9Cl7ZLVKaMVJmERPEVVQYfY0CXL0Ld2vA35yxfqs-X6vOl-nytPs38XCcPRYvV18Sn6AT8WAE1Iv4XyEEbpcQ_6qWE7g</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2847967462</pqid></control><display><type>article</type><title>BCI Control of a Robotic arm based on SSVEP with Moving Stimuli for Reach and grasp Tasks</title><source>IEEE Electronic Library (IEL)</source><creator>Ai, Jikun ; Meng, Jianjun ; Mai, Ximing ; Zhu, Xiangyang</creator><creatorcontrib>Ai, Jikun ; Meng, Jianjun ; Mai, Ximing ; Zhu, Xiangyang</creatorcontrib><description>Brain-computer interface (BCI) provides a novel technology for patients and healthy human subjects to control a robotic arm. Currently, BCI control of a robotic arm to complete the reaching and grasping tasks in an unstructured environment is still challenging because the current BCI technology does not meet the requirement of manipulating a multi-degree robotic arm accurately and robustly. BCI based on steady-state visual evoked potential (SSVEP) could output a high information transfer rate; however, the conventional SSVEP paradigm failed to control a robotic arm to move continuously and accurately because the users have to switch their gaze between the flickering stimuli and the target frequently. This study proposed a novel SSVEP paradigm in which the flickering stimuli were attached to the robotic arm's gripper and moved with it. First, an offline experiment was designed to investigate the effects of moving flickering stimuli on the SSVEP's responses and decoding accuracy. After that, contrast experiments were conducted, and twelve subjects were recruited to participate in a robotic arm control experiment using both the paradigm one (P1, with moving flickering stimuli) and the paradigm two (P2, conventional fixed flickering stimuli) using a block randomization design to balance their sequences. Double blinks were used to trigger the grasping action asynchronously whenever the subjects were confident that the position of the robotic arm's gripper was accurate enough. Experimental results showed that the paradigm P1 with moving flickering stimuli provided a much better control performance than the conventional paradigm P2 in completing a reaching and grasping task in an unstructured environment. Subjects' subjective feedback scored by a NASA-TLX mental workload scale also corroborated the BCI control performance. The results of this study suggest that the proposed control interface based on SSVEP BCI provides a better solution for robotic arm control to complete the accurate reaching and grasping tasks.</description><identifier>ISSN: 2168-2194</identifier><identifier>EISSN: 2168-2208</identifier><identifier>DOI: 10.1109/JBHI.2023.3277612</identifier><identifier>PMID: 37200132</identifier><identifier>CODEN: IJBHA9</identifier><language>eng</language><publisher>United States: IEEE</publisher><subject>Brain ; Brain-computer interface (BCI) ; Brain-Computer Interfaces ; Computer applications ; Electroencephalography - methods ; Evoked Potentials, Visual ; Grasping ; Human-computer interface ; Humans ; Implants ; Information transfer ; Manipulators ; moving stimuli ; Photic Stimulation ; reach and grasp ; Robot arms ; Robot control ; Robot kinematics ; robotic arm ; Robotic Surgical Procedures ; Robotics ; Robots ; Steady-state visual evoked potential (SSVEP) ; Stimuli ; Switches ; Task analysis ; Technology ; Visual evoked potentials ; Visualization</subject><ispartof>IEEE journal of biomedical and health informatics, 2023-08, Vol.27 (8), p.1-12</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c350t-7b89efbf664a23d6e319c864ee23d2a926cfd877420dc7e00eaf788bfd6a49943</citedby><cites>FETCH-LOGICAL-c350t-7b89efbf664a23d6e319c864ee23d2a926cfd877420dc7e00eaf788bfd6a49943</cites><orcidid>0000-0003-4914-6636 ; 0000-0003-0813-652X ; 0000-0002-3416-1764</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/10128977$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>315,781,785,797,27929,27930,54763</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/10128977$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/37200132$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Ai, Jikun</creatorcontrib><creatorcontrib>Meng, Jianjun</creatorcontrib><creatorcontrib>Mai, Ximing</creatorcontrib><creatorcontrib>Zhu, Xiangyang</creatorcontrib><title>BCI Control of a Robotic arm based on SSVEP with Moving Stimuli for Reach and grasp Tasks</title><title>IEEE journal of biomedical and health informatics</title><addtitle>JBHI</addtitle><addtitle>IEEE J Biomed Health Inform</addtitle><description>Brain-computer interface (BCI) provides a novel technology for patients and healthy human subjects to control a robotic arm. Currently, BCI control of a robotic arm to complete the reaching and grasping tasks in an unstructured environment is still challenging because the current BCI technology does not meet the requirement of manipulating a multi-degree robotic arm accurately and robustly. BCI based on steady-state visual evoked potential (SSVEP) could output a high information transfer rate; however, the conventional SSVEP paradigm failed to control a robotic arm to move continuously and accurately because the users have to switch their gaze between the flickering stimuli and the target frequently. This study proposed a novel SSVEP paradigm in which the flickering stimuli were attached to the robotic arm's gripper and moved with it. First, an offline experiment was designed to investigate the effects of moving flickering stimuli on the SSVEP's responses and decoding accuracy. After that, contrast experiments were conducted, and twelve subjects were recruited to participate in a robotic arm control experiment using both the paradigm one (P1, with moving flickering stimuli) and the paradigm two (P2, conventional fixed flickering stimuli) using a block randomization design to balance their sequences. Double blinks were used to trigger the grasping action asynchronously whenever the subjects were confident that the position of the robotic arm's gripper was accurate enough. Experimental results showed that the paradigm P1 with moving flickering stimuli provided a much better control performance than the conventional paradigm P2 in completing a reaching and grasping task in an unstructured environment. Subjects' subjective feedback scored by a NASA-TLX mental workload scale also corroborated the BCI control performance. The results of this study suggest that the proposed control interface based on SSVEP BCI provides a better solution for robotic arm control to complete the accurate reaching and grasping tasks.</description><subject>Brain</subject><subject>Brain-computer interface (BCI)</subject><subject>Brain-Computer Interfaces</subject><subject>Computer applications</subject><subject>Electroencephalography - methods</subject><subject>Evoked Potentials, Visual</subject><subject>Grasping</subject><subject>Human-computer interface</subject><subject>Humans</subject><subject>Implants</subject><subject>Information transfer</subject><subject>Manipulators</subject><subject>moving stimuli</subject><subject>Photic Stimulation</subject><subject>reach and grasp</subject><subject>Robot arms</subject><subject>Robot control</subject><subject>Robot kinematics</subject><subject>robotic arm</subject><subject>Robotic Surgical Procedures</subject><subject>Robotics</subject><subject>Robots</subject><subject>Steady-state visual evoked potential (SSVEP)</subject><subject>Stimuli</subject><subject>Switches</subject><subject>Task analysis</subject><subject>Technology</subject><subject>Visual evoked potentials</subject><subject>Visualization</subject><issn>2168-2194</issn><issn>2168-2208</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><sourceid>EIF</sourceid><recordid>eNpdkE1LJDEQhoMoKuoPEJYl4MXLjEmlzcdxHUadRVEcFTw16e6KtnZ3xqTbZf_9ZphRFnOppHjqpfIQcsjZmHNmTn6fXc7GwECMBSglOWyQXeBSjwCY3vy8c5PtkIMYX1k6OrWM3CY7QgFjXMAueTqbzOjEd33wDfWOWnrnC9_XJbWhpYWNWFHf0fn8cXpL_9T9C732H3X3TOd93Q5NTZ0P9A5t-UJtV9HnYOOC3tv4FvfJlrNNxIN13SMP59P7yeXo6uZiNvl1NSrFKetHqtAGXeGkzCyISqLgptQyQ0wvsAZk6SqtVAasKhUyhtYprQtXSZsZk4k9crzKXQT_PmDs87aOJTaN7dAPMYf0ayUFGJXQo2_oqx9Cl7ZLVKaMVJmERPEVVQYfY0CXL0Ld2vA35yxfqs-X6vOl-nytPs38XCcPRYvV18Sn6AT8WAE1Iv4XyEEbpcQ_6qWE7g</recordid><startdate>20230801</startdate><enddate>20230801</enddate><creator>Ai, Jikun</creator><creator>Meng, Jianjun</creator><creator>Mai, Ximing</creator><creator>Zhu, Xiangyang</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7QF</scope><scope>7QO</scope><scope>7QQ</scope><scope>7SC</scope><scope>7SE</scope><scope>7SP</scope><scope>7SR</scope><scope>7TA</scope><scope>7TB</scope><scope>7U5</scope><scope>8BQ</scope><scope>8FD</scope><scope>F28</scope><scope>FR3</scope><scope>H8D</scope><scope>JG9</scope><scope>JQ2</scope><scope>K9.</scope><scope>KR7</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>NAPCQ</scope><scope>P64</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0003-4914-6636</orcidid><orcidid>https://orcid.org/0000-0003-0813-652X</orcidid><orcidid>https://orcid.org/0000-0002-3416-1764</orcidid></search><sort><creationdate>20230801</creationdate><title>BCI Control of a Robotic arm based on SSVEP with Moving Stimuli for Reach and grasp Tasks</title><author>Ai, Jikun ; Meng, Jianjun ; Mai, Ximing ; Zhu, Xiangyang</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c350t-7b89efbf664a23d6e319c864ee23d2a926cfd877420dc7e00eaf788bfd6a49943</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Brain</topic><topic>Brain-computer interface (BCI)</topic><topic>Brain-Computer Interfaces</topic><topic>Computer applications</topic><topic>Electroencephalography - methods</topic><topic>Evoked Potentials, Visual</topic><topic>Grasping</topic><topic>Human-computer interface</topic><topic>Humans</topic><topic>Implants</topic><topic>Information transfer</topic><topic>Manipulators</topic><topic>moving stimuli</topic><topic>Photic Stimulation</topic><topic>reach and grasp</topic><topic>Robot arms</topic><topic>Robot control</topic><topic>Robot kinematics</topic><topic>robotic arm</topic><topic>Robotic Surgical Procedures</topic><topic>Robotics</topic><topic>Robots</topic><topic>Steady-state visual evoked potential (SSVEP)</topic><topic>Stimuli</topic><topic>Switches</topic><topic>Task analysis</topic><topic>Technology</topic><topic>Visual evoked potentials</topic><topic>Visualization</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Ai, Jikun</creatorcontrib><creatorcontrib>Meng, Jianjun</creatorcontrib><creatorcontrib>Mai, Ximing</creatorcontrib><creatorcontrib>Zhu, Xiangyang</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>Aluminium Industry Abstracts</collection><collection>Biotechnology Research Abstracts</collection><collection>Ceramic Abstracts</collection><collection>Computer and Information Systems Abstracts</collection><collection>Corrosion Abstracts</collection><collection>Electronics &amp; Communications Abstracts</collection><collection>Engineered Materials Abstracts</collection><collection>Materials Business File</collection><collection>Mechanical &amp; Transportation Engineering Abstracts</collection><collection>Solid State and Superconductivity Abstracts</collection><collection>METADEX</collection><collection>Technology Research Database</collection><collection>ANTE: Abstracts in New Technology &amp; Engineering</collection><collection>Engineering Research Database</collection><collection>Aerospace Database</collection><collection>Materials Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>ProQuest Health &amp; Medical Complete (Alumni)</collection><collection>Civil Engineering Abstracts</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>Nursing &amp; Allied Health Premium</collection><collection>Biotechnology and BioEngineering Abstracts</collection><collection>MEDLINE - Academic</collection><jtitle>IEEE journal of biomedical and health informatics</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Ai, Jikun</au><au>Meng, Jianjun</au><au>Mai, Ximing</au><au>Zhu, Xiangyang</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>BCI Control of a Robotic arm based on SSVEP with Moving Stimuli for Reach and grasp Tasks</atitle><jtitle>IEEE journal of biomedical and health informatics</jtitle><stitle>JBHI</stitle><addtitle>IEEE J Biomed Health Inform</addtitle><date>2023-08-01</date><risdate>2023</risdate><volume>27</volume><issue>8</issue><spage>1</spage><epage>12</epage><pages>1-12</pages><issn>2168-2194</issn><eissn>2168-2208</eissn><coden>IJBHA9</coden><abstract>Brain-computer interface (BCI) provides a novel technology for patients and healthy human subjects to control a robotic arm. Currently, BCI control of a robotic arm to complete the reaching and grasping tasks in an unstructured environment is still challenging because the current BCI technology does not meet the requirement of manipulating a multi-degree robotic arm accurately and robustly. BCI based on steady-state visual evoked potential (SSVEP) could output a high information transfer rate; however, the conventional SSVEP paradigm failed to control a robotic arm to move continuously and accurately because the users have to switch their gaze between the flickering stimuli and the target frequently. This study proposed a novel SSVEP paradigm in which the flickering stimuli were attached to the robotic arm's gripper and moved with it. First, an offline experiment was designed to investigate the effects of moving flickering stimuli on the SSVEP's responses and decoding accuracy. After that, contrast experiments were conducted, and twelve subjects were recruited to participate in a robotic arm control experiment using both the paradigm one (P1, with moving flickering stimuli) and the paradigm two (P2, conventional fixed flickering stimuli) using a block randomization design to balance their sequences. Double blinks were used to trigger the grasping action asynchronously whenever the subjects were confident that the position of the robotic arm's gripper was accurate enough. Experimental results showed that the paradigm P1 with moving flickering stimuli provided a much better control performance than the conventional paradigm P2 in completing a reaching and grasping task in an unstructured environment. Subjects' subjective feedback scored by a NASA-TLX mental workload scale also corroborated the BCI control performance. The results of this study suggest that the proposed control interface based on SSVEP BCI provides a better solution for robotic arm control to complete the accurate reaching and grasping tasks.</abstract><cop>United States</cop><pub>IEEE</pub><pmid>37200132</pmid><doi>10.1109/JBHI.2023.3277612</doi><tpages>12</tpages><orcidid>https://orcid.org/0000-0003-4914-6636</orcidid><orcidid>https://orcid.org/0000-0003-0813-652X</orcidid><orcidid>https://orcid.org/0000-0002-3416-1764</orcidid></addata></record>
fulltext fulltext_linktorsrc
identifier ISSN: 2168-2194
ispartof IEEE journal of biomedical and health informatics, 2023-08, Vol.27 (8), p.1-12
issn 2168-2194
2168-2208
language eng
recordid cdi_crossref_primary_10_1109_JBHI_2023_3277612
source IEEE Electronic Library (IEL)
subjects Brain
Brain-computer interface (BCI)
Brain-Computer Interfaces
Computer applications
Electroencephalography - methods
Evoked Potentials, Visual
Grasping
Human-computer interface
Humans
Implants
Information transfer
Manipulators
moving stimuli
Photic Stimulation
reach and grasp
Robot arms
Robot control
Robot kinematics
robotic arm
Robotic Surgical Procedures
Robotics
Robots
Steady-state visual evoked potential (SSVEP)
Stimuli
Switches
Task analysis
Technology
Visual evoked potentials
Visualization
title BCI Control of a Robotic arm based on SSVEP with Moving Stimuli for Reach and grasp Tasks
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-11T10%3A05%3A56IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=BCI%20Control%20of%20a%20Robotic%20arm%20based%20on%20SSVEP%20with%20Moving%20Stimuli%20for%20Reach%20and%20grasp%20Tasks&rft.jtitle=IEEE%20journal%20of%20biomedical%20and%20health%20informatics&rft.au=Ai,%20Jikun&rft.date=2023-08-01&rft.volume=27&rft.issue=8&rft.spage=1&rft.epage=12&rft.pages=1-12&rft.issn=2168-2194&rft.eissn=2168-2208&rft.coden=IJBHA9&rft_id=info:doi/10.1109/JBHI.2023.3277612&rft_dat=%3Cproquest_RIE%3E2847967462%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2847967462&rft_id=info:pmid/37200132&rft_ieee_id=10128977&rfr_iscdi=true