Assessment of Robotic Console Skills (ARCS): construct validity of a novel global rating scale for technical skills in robotically assisted surgery
Background Skill assessment during robotically assisted surgery remains challenging. While the popularity of the Global Evaluative Assessment of Robotics Skills (GEARS) has grown, its lack of discrimination between independent console skills limits its usefulness. The purpose of this study was to ev...
Gespeichert in:
| Veröffentlicht in: | Surgical endoscopy 2018, Vol.32 (1), p.526-535 |
|---|---|
| Hauptverfasser: | , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 535 |
|---|---|
| container_issue | 1 |
| container_start_page | 526 |
| container_title | Surgical endoscopy |
| container_volume | 32 |
| creator | Liu, May Purohit, Shreya Mazanetz, Joshua Allen, Whitney Kreaden, Usha S. Curet, Myriam |
| description | Background
Skill assessment during robotically assisted surgery remains challenging. While the popularity of the Global Evaluative Assessment of Robotics Skills (GEARS) has grown, its lack of discrimination between independent console skills limits its usefulness. The purpose of this study was to evaluate construct validity and interrater reliability of a novel assessment designed to overcome this limitation.
Methods
We created the Assessment of Robotic Console Skills (ARCS), a global rating scale with six console skill domains. Fifteen volunteers who were console surgeons for 0 (“novice”), 1–100 (“intermediate”), or >100 (“experienced”) robotically assisted procedures performed three standardized tasks. Three blinded raters scored the task videos using ARCS, with a 5-point Likert scale for each skill domain. Scores were analyzed for evidence of construct validity and interrater reliability.
Results
Group demographics were indistinguishable except for the number of robotically assisted procedures performed (
p
= 0.001). The mean scores of experienced subjects exceeded those of novices in dexterity (3.8 > 1.4,
p
1.8,
p
2.2,
p
1.9,
p
= 0.001), and force sensitivity (4.3 > 2.6,
p
1.4,
p
= 0.002), field of view (2.8 > 1.8,
p
= 0.021), instrument visualization (3.2 > 2.2,
p
= 0.045), manipulator workspace (3.1 > 1.9,
p
= 0.004), and force sensitivity (3.7 > 2.6,
p
= 0.033). The mean scores of experienced subjects exceeded those of intermediates in dexterity (3.8 > 2.8,
p
= 0.003), field of view (4.1 > 2.8,
p
3.2,
p
= 0.044). Rater agreement in each domain demonstrated statistically significant concordance (
p
|
| doi_str_mv | 10.1007/s00464-017-5694-7 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_1915346360</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>1915346360</sourcerecordid><originalsourceid>FETCH-LOGICAL-c372t-bcba877ac6e6ca62f338ff1d9411fba39db6503b52e8087883b466df5900f9803</originalsourceid><addsrcrecordid>eNp1kc2KFDEUhYMoTjv6AG4k4GZclCaVVH7cNY1_MCDM6DqkUkmbMV0Zc1MD_Ry-sGlqFBFchSTfOfdyDkLPKXlNCZFvgBAueEeo7AaheScfoA3lrO_6nqqHaEM0I10vNT9DTwBuSMM1HR6js14JIQcuNujnFsADHPxccQ74Ko-5Rod3eYacPL7-HlMCfLG92l2_eotde65lcRXf2RSnWI8nkcVzvvMJ71MebcLF1jjvMTjbDEIuuHr3bY7timG1izMu6yCb0hFbgAjVTxiWsvfl-BQ9CjaBf3Z_nqOv79992X3sLj9_-LTbXnaOyb52oxutktI64YWzog-MqRDopDmlYbRMT6MYCBuH3iuipFJs5EJMYdCEBK0IO0cXq-9tyT8WD9UcIjifkp19XsDQFhbjgokT-vIf9CYvZW7bNUq3OJXkrFF0pVzJAMUHc1viwZajocScGjNrY6Y1Zk6NGdk0L-6dl_Hgpz-K3xU1oF8BaF9zy-ev0f91_QVu8qJ5</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>1992868743</pqid></control><display><type>article</type><title>Assessment of Robotic Console Skills (ARCS): construct validity of a novel global rating scale for technical skills in robotically assisted surgery</title><source>SpringerNature Journals</source><creator>Liu, May ; Purohit, Shreya ; Mazanetz, Joshua ; Allen, Whitney ; Kreaden, Usha S. ; Curet, Myriam</creator><creatorcontrib>Liu, May ; Purohit, Shreya ; Mazanetz, Joshua ; Allen, Whitney ; Kreaden, Usha S. ; Curet, Myriam</creatorcontrib><description><![CDATA[Background
Skill assessment during robotically assisted surgery remains challenging. While the popularity of the Global Evaluative Assessment of Robotics Skills (GEARS) has grown, its lack of discrimination between independent console skills limits its usefulness. The purpose of this study was to evaluate construct validity and interrater reliability of a novel assessment designed to overcome this limitation.
Methods
We created the Assessment of Robotic Console Skills (ARCS), a global rating scale with six console skill domains. Fifteen volunteers who were console surgeons for 0 (“novice”), 1–100 (“intermediate”), or >100 (“experienced”) robotically assisted procedures performed three standardized tasks. Three blinded raters scored the task videos using ARCS, with a 5-point Likert scale for each skill domain. Scores were analyzed for evidence of construct validity and interrater reliability.
Results
Group demographics were indistinguishable except for the number of robotically assisted procedures performed (
p
= 0.001). The mean scores of experienced subjects exceeded those of novices in dexterity (3.8 > 1.4,
p
< 0.001), field of view (4.1 > 1.8,
p
< 0.001), instrument visualization (3.9 > 2.2,
p
< 0.001), manipulator workspace (3.6 > 1.9,
p
= 0.001), and force sensitivity (4.3 > 2.6,
p
< 0.001). The mean scores of intermediate subjects exceeded those of novices in dexterity (2.8 > 1.4,
p
= 0.002), field of view (2.8 > 1.8,
p
= 0.021), instrument visualization (3.2 > 2.2,
p
= 0.045), manipulator workspace (3.1 > 1.9,
p
= 0.004), and force sensitivity (3.7 > 2.6,
p
= 0.033). The mean scores of experienced subjects exceeded those of intermediates in dexterity (3.8 > 2.8,
p
= 0.003), field of view (4.1 > 2.8,
p
< 0.001), and instrument visualization (3.9 > 3.2,
p
= 0.044). Rater agreement in each domain demonstrated statistically significant concordance (
p
< 0.05).
Conclusions
We present strong evidence for construct validity and interrater reliability of ARCS. Our study shows that learning curves for some console skills plateau faster than others. Therefore, ARCS may be more useful than GEARS to evaluate distinct console skills. Future studies will examine why some domains did not adequately differentiate between subjects and applications for intraoperative use.]]></description><identifier>ISSN: 0930-2794</identifier><identifier>EISSN: 1432-2218</identifier><identifier>DOI: 10.1007/s00464-017-5694-7</identifier><identifier>PMID: 28667546</identifier><language>eng</language><publisher>New York: Springer US</publisher><subject>Abdominal Surgery ; Gastroenterology ; Gynecology ; Hepatology ; Medicine ; Medicine & Public Health ; New Technology ; Proctology ; Skills ; Surgery ; Validation studies</subject><ispartof>Surgical endoscopy, 2018, Vol.32 (1), p.526-535</ispartof><rights>Springer Science+Business Media, LLC 2017</rights><rights>Surgical Endoscopy is a copyright of Springer, (2017). All Rights Reserved.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c372t-bcba877ac6e6ca62f338ff1d9411fba39db6503b52e8087883b466df5900f9803</citedby><cites>FETCH-LOGICAL-c372t-bcba877ac6e6ca62f338ff1d9411fba39db6503b52e8087883b466df5900f9803</cites><orcidid>0000-0003-4709-5417</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://link.springer.com/content/pdf/10.1007/s00464-017-5694-7$$EPDF$$P50$$Gspringer$$H</linktopdf><linktohtml>$$Uhttps://link.springer.com/10.1007/s00464-017-5694-7$$EHTML$$P50$$Gspringer$$H</linktohtml><link.rule.ids>315,781,785,27928,27929,41492,42561,51323</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/28667546$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Liu, May</creatorcontrib><creatorcontrib>Purohit, Shreya</creatorcontrib><creatorcontrib>Mazanetz, Joshua</creatorcontrib><creatorcontrib>Allen, Whitney</creatorcontrib><creatorcontrib>Kreaden, Usha S.</creatorcontrib><creatorcontrib>Curet, Myriam</creatorcontrib><title>Assessment of Robotic Console Skills (ARCS): construct validity of a novel global rating scale for technical skills in robotically assisted surgery</title><title>Surgical endoscopy</title><addtitle>Surg Endosc</addtitle><addtitle>Surg Endosc</addtitle><description><![CDATA[Background
Skill assessment during robotically assisted surgery remains challenging. While the popularity of the Global Evaluative Assessment of Robotics Skills (GEARS) has grown, its lack of discrimination between independent console skills limits its usefulness. The purpose of this study was to evaluate construct validity and interrater reliability of a novel assessment designed to overcome this limitation.
Methods
We created the Assessment of Robotic Console Skills (ARCS), a global rating scale with six console skill domains. Fifteen volunteers who were console surgeons for 0 (“novice”), 1–100 (“intermediate”), or >100 (“experienced”) robotically assisted procedures performed three standardized tasks. Three blinded raters scored the task videos using ARCS, with a 5-point Likert scale for each skill domain. Scores were analyzed for evidence of construct validity and interrater reliability.
Results
Group demographics were indistinguishable except for the number of robotically assisted procedures performed (
p
= 0.001). The mean scores of experienced subjects exceeded those of novices in dexterity (3.8 > 1.4,
p
< 0.001), field of view (4.1 > 1.8,
p
< 0.001), instrument visualization (3.9 > 2.2,
p
< 0.001), manipulator workspace (3.6 > 1.9,
p
= 0.001), and force sensitivity (4.3 > 2.6,
p
< 0.001). The mean scores of intermediate subjects exceeded those of novices in dexterity (2.8 > 1.4,
p
= 0.002), field of view (2.8 > 1.8,
p
= 0.021), instrument visualization (3.2 > 2.2,
p
= 0.045), manipulator workspace (3.1 > 1.9,
p
= 0.004), and force sensitivity (3.7 > 2.6,
p
= 0.033). The mean scores of experienced subjects exceeded those of intermediates in dexterity (3.8 > 2.8,
p
= 0.003), field of view (4.1 > 2.8,
p
< 0.001), and instrument visualization (3.9 > 3.2,
p
= 0.044). Rater agreement in each domain demonstrated statistically significant concordance (
p
< 0.05).
Conclusions
We present strong evidence for construct validity and interrater reliability of ARCS. Our study shows that learning curves for some console skills plateau faster than others. Therefore, ARCS may be more useful than GEARS to evaluate distinct console skills. Future studies will examine why some domains did not adequately differentiate between subjects and applications for intraoperative use.]]></description><subject>Abdominal Surgery</subject><subject>Gastroenterology</subject><subject>Gynecology</subject><subject>Hepatology</subject><subject>Medicine</subject><subject>Medicine & Public Health</subject><subject>New Technology</subject><subject>Proctology</subject><subject>Skills</subject><subject>Surgery</subject><subject>Validation studies</subject><issn>0930-2794</issn><issn>1432-2218</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2018</creationdate><recordtype>article</recordtype><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><recordid>eNp1kc2KFDEUhYMoTjv6AG4k4GZclCaVVH7cNY1_MCDM6DqkUkmbMV0Zc1MD_Ry-sGlqFBFchSTfOfdyDkLPKXlNCZFvgBAueEeo7AaheScfoA3lrO_6nqqHaEM0I10vNT9DTwBuSMM1HR6js14JIQcuNujnFsADHPxccQ74Ko-5Rod3eYacPL7-HlMCfLG92l2_eotde65lcRXf2RSnWI8nkcVzvvMJ71MebcLF1jjvMTjbDEIuuHr3bY7timG1izMu6yCb0hFbgAjVTxiWsvfl-BQ9CjaBf3Z_nqOv79992X3sLj9_-LTbXnaOyb52oxutktI64YWzog-MqRDopDmlYbRMT6MYCBuH3iuipFJs5EJMYdCEBK0IO0cXq-9tyT8WD9UcIjifkp19XsDQFhbjgokT-vIf9CYvZW7bNUq3OJXkrFF0pVzJAMUHc1viwZajocScGjNrY6Y1Zk6NGdk0L-6dl_Hgpz-K3xU1oF8BaF9zy-ev0f91_QVu8qJ5</recordid><startdate>2018</startdate><enddate>2018</enddate><creator>Liu, May</creator><creator>Purohit, Shreya</creator><creator>Mazanetz, Joshua</creator><creator>Allen, Whitney</creator><creator>Kreaden, Usha S.</creator><creator>Curet, Myriam</creator><general>Springer US</general><general>Springer Nature B.V</general><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>3V.</scope><scope>7RV</scope><scope>7X7</scope><scope>7XB</scope><scope>88E</scope><scope>8AO</scope><scope>8FI</scope><scope>8FJ</scope><scope>8FK</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>BENPR</scope><scope>CCPQU</scope><scope>FYUFA</scope><scope>GHDGH</scope><scope>K9.</scope><scope>KB0</scope><scope>M0S</scope><scope>M1P</scope><scope>NAPCQ</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0003-4709-5417</orcidid></search><sort><creationdate>2018</creationdate><title>Assessment of Robotic Console Skills (ARCS): construct validity of a novel global rating scale for technical skills in robotically assisted surgery</title><author>Liu, May ; Purohit, Shreya ; Mazanetz, Joshua ; Allen, Whitney ; Kreaden, Usha S. ; Curet, Myriam</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c372t-bcba877ac6e6ca62f338ff1d9411fba39db6503b52e8087883b466df5900f9803</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2018</creationdate><topic>Abdominal Surgery</topic><topic>Gastroenterology</topic><topic>Gynecology</topic><topic>Hepatology</topic><topic>Medicine</topic><topic>Medicine & Public Health</topic><topic>New Technology</topic><topic>Proctology</topic><topic>Skills</topic><topic>Surgery</topic><topic>Validation studies</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Liu, May</creatorcontrib><creatorcontrib>Purohit, Shreya</creatorcontrib><creatorcontrib>Mazanetz, Joshua</creatorcontrib><creatorcontrib>Allen, Whitney</creatorcontrib><creatorcontrib>Kreaden, Usha S.</creatorcontrib><creatorcontrib>Curet, Myriam</creatorcontrib><collection>PubMed</collection><collection>CrossRef</collection><collection>ProQuest Central (Corporate)</collection><collection>Nursing & Allied Health Database</collection><collection>Health & Medical Collection</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>Medical Database (Alumni Edition)</collection><collection>ProQuest Pharma Collection</collection><collection>Hospital Premium Collection</collection><collection>Hospital Premium Collection (Alumni Edition)</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>Proquest Central</collection><collection>ProQuest One Community College</collection><collection>Health Research Premium Collection</collection><collection>Health Research Premium Collection (Alumni)</collection><collection>ProQuest Health & Medical Complete (Alumni)</collection><collection>Nursing & Allied Health Database (Alumni Edition)</collection><collection>Health & Medical Collection (Alumni Edition)</collection><collection>Medical Database</collection><collection>Nursing & Allied Health Premium</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>MEDLINE - Academic</collection><jtitle>Surgical endoscopy</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Liu, May</au><au>Purohit, Shreya</au><au>Mazanetz, Joshua</au><au>Allen, Whitney</au><au>Kreaden, Usha S.</au><au>Curet, Myriam</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Assessment of Robotic Console Skills (ARCS): construct validity of a novel global rating scale for technical skills in robotically assisted surgery</atitle><jtitle>Surgical endoscopy</jtitle><stitle>Surg Endosc</stitle><addtitle>Surg Endosc</addtitle><date>2018</date><risdate>2018</risdate><volume>32</volume><issue>1</issue><spage>526</spage><epage>535</epage><pages>526-535</pages><issn>0930-2794</issn><eissn>1432-2218</eissn><abstract><![CDATA[Background
Skill assessment during robotically assisted surgery remains challenging. While the popularity of the Global Evaluative Assessment of Robotics Skills (GEARS) has grown, its lack of discrimination between independent console skills limits its usefulness. The purpose of this study was to evaluate construct validity and interrater reliability of a novel assessment designed to overcome this limitation.
Methods
We created the Assessment of Robotic Console Skills (ARCS), a global rating scale with six console skill domains. Fifteen volunteers who were console surgeons for 0 (“novice”), 1–100 (“intermediate”), or >100 (“experienced”) robotically assisted procedures performed three standardized tasks. Three blinded raters scored the task videos using ARCS, with a 5-point Likert scale for each skill domain. Scores were analyzed for evidence of construct validity and interrater reliability.
Results
Group demographics were indistinguishable except for the number of robotically assisted procedures performed (
p
= 0.001). The mean scores of experienced subjects exceeded those of novices in dexterity (3.8 > 1.4,
p
< 0.001), field of view (4.1 > 1.8,
p
< 0.001), instrument visualization (3.9 > 2.2,
p
< 0.001), manipulator workspace (3.6 > 1.9,
p
= 0.001), and force sensitivity (4.3 > 2.6,
p
< 0.001). The mean scores of intermediate subjects exceeded those of novices in dexterity (2.8 > 1.4,
p
= 0.002), field of view (2.8 > 1.8,
p
= 0.021), instrument visualization (3.2 > 2.2,
p
= 0.045), manipulator workspace (3.1 > 1.9,
p
= 0.004), and force sensitivity (3.7 > 2.6,
p
= 0.033). The mean scores of experienced subjects exceeded those of intermediates in dexterity (3.8 > 2.8,
p
= 0.003), field of view (4.1 > 2.8,
p
< 0.001), and instrument visualization (3.9 > 3.2,
p
= 0.044). Rater agreement in each domain demonstrated statistically significant concordance (
p
< 0.05).
Conclusions
We present strong evidence for construct validity and interrater reliability of ARCS. Our study shows that learning curves for some console skills plateau faster than others. Therefore, ARCS may be more useful than GEARS to evaluate distinct console skills. Future studies will examine why some domains did not adequately differentiate between subjects and applications for intraoperative use.]]></abstract><cop>New York</cop><pub>Springer US</pub><pmid>28667546</pmid><doi>10.1007/s00464-017-5694-7</doi><tpages>10</tpages><orcidid>https://orcid.org/0000-0003-4709-5417</orcidid></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0930-2794 |
| ispartof | Surgical endoscopy, 2018, Vol.32 (1), p.526-535 |
| issn | 0930-2794 1432-2218 |
| language | eng |
| recordid | cdi_proquest_miscellaneous_1915346360 |
| source | SpringerNature Journals |
| subjects | Abdominal Surgery Gastroenterology Gynecology Hepatology Medicine Medicine & Public Health New Technology Proctology Skills Surgery Validation studies |
| title | Assessment of Robotic Console Skills (ARCS): construct validity of a novel global rating scale for technical skills in robotically assisted surgery |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-17T06%3A09%3A49IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Assessment%20of%20Robotic%20Console%20Skills%20(ARCS):%20construct%20validity%20of%20a%20novel%20global%20rating%20scale%20for%20technical%20skills%20in%20robotically%20assisted%20surgery&rft.jtitle=Surgical%20endoscopy&rft.au=Liu,%20May&rft.date=2018&rft.volume=32&rft.issue=1&rft.spage=526&rft.epage=535&rft.pages=526-535&rft.issn=0930-2794&rft.eissn=1432-2218&rft_id=info:doi/10.1007/s00464-017-5694-7&rft_dat=%3Cproquest_cross%3E1915346360%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=1992868743&rft_id=info:pmid/28667546&rfr_iscdi=true |