Evaluation of an Algorithm for Automatic Grading of Forum Messages in MOOC Discussion Forums
Discussion forums are a valuable source of information in educational platforms such as Massive Open Online Courses (MOOCs), as users can exchange opinions or even help other students in an asynchronous way, contributing to the sustainability of MOOCs even with low interaction from the instructor. T...
Gespeichert in:
| Veröffentlicht in: | Sustainability 2021-08, Vol.13 (16), p.9364 |
|---|---|
| Hauptverfasser: | , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | |
|---|---|
| container_issue | 16 |
| container_start_page | 9364 |
| container_title | Sustainability |
| container_volume | 13 |
| creator | Perez-Nicolas, Raquel L Alario-Hoyos, Carlos Estevez-Ayres, Iria Moreno-Marcos, Pedro Manuel Munoz-Merino, Pedro J Delgado Kloos, Carlos |
| description | Discussion forums are a valuable source of information in educational platforms such as Massive Open Online Courses (MOOCs), as users can exchange opinions or even help other students in an asynchronous way, contributing to the sustainability of MOOCs even with low interaction from the instructor. Therefore, the use of the forum messages to get insights about students’ performance in a course is interesting. This article presents an automatic grading approach that can be used to assess learners through their interactions in the forum. The approach is based on the combination of three dimensions: (1) the quality of the content of the interactions, (2) the impact of the interactions, and (3) the user’s activity in the forum. The evaluation of the approach compares the assessment by experts with the automatic assessment obtaining a high accuracy of 0.8068 and Normalized Root Mean Square Error (NRMSE) of 0.1799, which outperforms previous existing approaches. Future research work can try to improve the automatic grading by the training of the indicators of the approach depending on the MOOCs or the combination with text mining techniques. |
| doi_str_mv | 10.3390/su13169364 |
| format | Article |
| fullrecord | <record><control><sourceid>gale_proqu</sourceid><recordid>TN_cdi_gale_incontextgauss_ISR_A811449427</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><galeid>A811449427</galeid><sourcerecordid>A811449427</sourcerecordid><originalsourceid>FETCH-LOGICAL-c368t-f31f46bf8e9b2e602b6959a145c7de3ae3b5d93969eb3f42d43aec410db61cbb3</originalsourceid><addsrcrecordid>eNpVkd9LwzAQx4soOOZe_AsCPil0Jk2aNY9lujnYGPjjTShpmtSMtplJKvrfmzpBd_dwx93ne8dxUXSJ4BRjBm9djzCiDFNyEo0SOEMxgik8_ZefRxPndjAYxoghOope7z9403OvTQeMArwDeVMbq_1bC5SxIO-9aUNbgKXlle7qgVoY27dgI53jtXRAd2Cz3c7BnXaid24Y9UO4i-hM8cbJyW8cRy-L--f5Q7zeLlfzfB0LTDMfK4wUoaXKJCsTSWFSUpYyjkgqZpXEXOIyrRhmlMkSK5JUJNQEQbAqKRJlicfR1WHu3pr3Xjpf7Exvu7CySNIsCdIsnQVqeqBq3shCd8p4y0XwSrZamE4qHep5hhAhjCSD4PpIEBgvP33Nw5HF6unxmL05sMIa56xUxd7qltuvAsFi-E7x9x38DSUagEQ</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2582939857</pqid></control><display><type>article</type><title>Evaluation of an Algorithm for Automatic Grading of Forum Messages in MOOC Discussion Forums</title><source>MDPI - Multidisciplinary Digital Publishing Institute</source><source>EZB Electronic Journals Library</source><creator>Perez-Nicolas, Raquel L ; Alario-Hoyos, Carlos ; Estevez-Ayres, Iria ; Moreno-Marcos, Pedro Manuel ; Munoz-Merino, Pedro J ; Delgado Kloos, Carlos</creator><creatorcontrib>Perez-Nicolas, Raquel L ; Alario-Hoyos, Carlos ; Estevez-Ayres, Iria ; Moreno-Marcos, Pedro Manuel ; Munoz-Merino, Pedro J ; Delgado Kloos, Carlos</creatorcontrib><description>Discussion forums are a valuable source of information in educational platforms such as Massive Open Online Courses (MOOCs), as users can exchange opinions or even help other students in an asynchronous way, contributing to the sustainability of MOOCs even with low interaction from the instructor. Therefore, the use of the forum messages to get insights about students’ performance in a course is interesting. This article presents an automatic grading approach that can be used to assess learners through their interactions in the forum. The approach is based on the combination of three dimensions: (1) the quality of the content of the interactions, (2) the impact of the interactions, and (3) the user’s activity in the forum. The evaluation of the approach compares the assessment by experts with the automatic assessment obtaining a high accuracy of 0.8068 and Normalized Root Mean Square Error (NRMSE) of 0.1799, which outperforms previous existing approaches. Future research work can try to improve the automatic grading by the training of the indicators of the approach depending on the MOOCs or the combination with text mining techniques.</description><identifier>ISSN: 2071-1050</identifier><identifier>EISSN: 2071-1050</identifier><identifier>DOI: 10.3390/su13169364</identifier><language>eng</language><publisher>Basel: MDPI AG</publisher><subject>Accuracy ; Algorithms ; Comparative analysis ; Data mining ; Education ; Electronic discussion groups ; Evaluation ; Linguistics ; Massive open online courses ; Online instruction ; Reputations ; Social aspects ; Students ; Sustainability ; Technology application</subject><ispartof>Sustainability, 2021-08, Vol.13 (16), p.9364</ispartof><rights>COPYRIGHT 2021 MDPI AG</rights><rights>2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c368t-f31f46bf8e9b2e602b6959a145c7de3ae3b5d93969eb3f42d43aec410db61cbb3</citedby><cites>FETCH-LOGICAL-c368t-f31f46bf8e9b2e602b6959a145c7de3ae3b5d93969eb3f42d43aec410db61cbb3</cites><orcidid>0000-0003-0835-1414 ; 0000-0002-3082-0814 ; 0000-0002-1047-5398 ; 0000-0002-2552-4674 ; 0000-0003-4093-3705</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>315,781,785,27926,27927</link.rule.ids></links><search><creatorcontrib>Perez-Nicolas, Raquel L</creatorcontrib><creatorcontrib>Alario-Hoyos, Carlos</creatorcontrib><creatorcontrib>Estevez-Ayres, Iria</creatorcontrib><creatorcontrib>Moreno-Marcos, Pedro Manuel</creatorcontrib><creatorcontrib>Munoz-Merino, Pedro J</creatorcontrib><creatorcontrib>Delgado Kloos, Carlos</creatorcontrib><title>Evaluation of an Algorithm for Automatic Grading of Forum Messages in MOOC Discussion Forums</title><title>Sustainability</title><description>Discussion forums are a valuable source of information in educational platforms such as Massive Open Online Courses (MOOCs), as users can exchange opinions or even help other students in an asynchronous way, contributing to the sustainability of MOOCs even with low interaction from the instructor. Therefore, the use of the forum messages to get insights about students’ performance in a course is interesting. This article presents an automatic grading approach that can be used to assess learners through their interactions in the forum. The approach is based on the combination of three dimensions: (1) the quality of the content of the interactions, (2) the impact of the interactions, and (3) the user’s activity in the forum. The evaluation of the approach compares the assessment by experts with the automatic assessment obtaining a high accuracy of 0.8068 and Normalized Root Mean Square Error (NRMSE) of 0.1799, which outperforms previous existing approaches. Future research work can try to improve the automatic grading by the training of the indicators of the approach depending on the MOOCs or the combination with text mining techniques.</description><subject>Accuracy</subject><subject>Algorithms</subject><subject>Comparative analysis</subject><subject>Data mining</subject><subject>Education</subject><subject>Electronic discussion groups</subject><subject>Evaluation</subject><subject>Linguistics</subject><subject>Massive open online courses</subject><subject>Online instruction</subject><subject>Reputations</subject><subject>Social aspects</subject><subject>Students</subject><subject>Sustainability</subject><subject>Technology application</subject><issn>2071-1050</issn><issn>2071-1050</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><recordid>eNpVkd9LwzAQx4soOOZe_AsCPil0Jk2aNY9lujnYGPjjTShpmtSMtplJKvrfmzpBd_dwx93ne8dxUXSJ4BRjBm9djzCiDFNyEo0SOEMxgik8_ZefRxPndjAYxoghOope7z9403OvTQeMArwDeVMbq_1bC5SxIO-9aUNbgKXlle7qgVoY27dgI53jtXRAd2Cz3c7BnXaid24Y9UO4i-hM8cbJyW8cRy-L--f5Q7zeLlfzfB0LTDMfK4wUoaXKJCsTSWFSUpYyjkgqZpXEXOIyrRhmlMkSK5JUJNQEQbAqKRJlicfR1WHu3pr3Xjpf7Exvu7CySNIsCdIsnQVqeqBq3shCd8p4y0XwSrZamE4qHep5hhAhjCSD4PpIEBgvP33Nw5HF6unxmL05sMIa56xUxd7qltuvAsFi-E7x9x38DSUagEQ</recordid><startdate>20210801</startdate><enddate>20210801</enddate><creator>Perez-Nicolas, Raquel L</creator><creator>Alario-Hoyos, Carlos</creator><creator>Estevez-Ayres, Iria</creator><creator>Moreno-Marcos, Pedro Manuel</creator><creator>Munoz-Merino, Pedro J</creator><creator>Delgado Kloos, Carlos</creator><general>MDPI AG</general><scope>AAYXX</scope><scope>CITATION</scope><scope>ISR</scope><scope>4U-</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><orcidid>https://orcid.org/0000-0003-0835-1414</orcidid><orcidid>https://orcid.org/0000-0002-3082-0814</orcidid><orcidid>https://orcid.org/0000-0002-1047-5398</orcidid><orcidid>https://orcid.org/0000-0002-2552-4674</orcidid><orcidid>https://orcid.org/0000-0003-4093-3705</orcidid></search><sort><creationdate>20210801</creationdate><title>Evaluation of an Algorithm for Automatic Grading of Forum Messages in MOOC Discussion Forums</title><author>Perez-Nicolas, Raquel L ; Alario-Hoyos, Carlos ; Estevez-Ayres, Iria ; Moreno-Marcos, Pedro Manuel ; Munoz-Merino, Pedro J ; Delgado Kloos, Carlos</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c368t-f31f46bf8e9b2e602b6959a145c7de3ae3b5d93969eb3f42d43aec410db61cbb3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Accuracy</topic><topic>Algorithms</topic><topic>Comparative analysis</topic><topic>Data mining</topic><topic>Education</topic><topic>Electronic discussion groups</topic><topic>Evaluation</topic><topic>Linguistics</topic><topic>Massive open online courses</topic><topic>Online instruction</topic><topic>Reputations</topic><topic>Social aspects</topic><topic>Students</topic><topic>Sustainability</topic><topic>Technology application</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Perez-Nicolas, Raquel L</creatorcontrib><creatorcontrib>Alario-Hoyos, Carlos</creatorcontrib><creatorcontrib>Estevez-Ayres, Iria</creatorcontrib><creatorcontrib>Moreno-Marcos, Pedro Manuel</creatorcontrib><creatorcontrib>Munoz-Merino, Pedro J</creatorcontrib><creatorcontrib>Delgado Kloos, Carlos</creatorcontrib><collection>CrossRef</collection><collection>Gale In Context: Science</collection><collection>University Readers</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central UK/Ireland</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>Access via ProQuest (Open Access)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><jtitle>Sustainability</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Perez-Nicolas, Raquel L</au><au>Alario-Hoyos, Carlos</au><au>Estevez-Ayres, Iria</au><au>Moreno-Marcos, Pedro Manuel</au><au>Munoz-Merino, Pedro J</au><au>Delgado Kloos, Carlos</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Evaluation of an Algorithm for Automatic Grading of Forum Messages in MOOC Discussion Forums</atitle><jtitle>Sustainability</jtitle><date>2021-08-01</date><risdate>2021</risdate><volume>13</volume><issue>16</issue><spage>9364</spage><pages>9364-</pages><issn>2071-1050</issn><eissn>2071-1050</eissn><abstract>Discussion forums are a valuable source of information in educational platforms such as Massive Open Online Courses (MOOCs), as users can exchange opinions or even help other students in an asynchronous way, contributing to the sustainability of MOOCs even with low interaction from the instructor. Therefore, the use of the forum messages to get insights about students’ performance in a course is interesting. This article presents an automatic grading approach that can be used to assess learners through their interactions in the forum. The approach is based on the combination of three dimensions: (1) the quality of the content of the interactions, (2) the impact of the interactions, and (3) the user’s activity in the forum. The evaluation of the approach compares the assessment by experts with the automatic assessment obtaining a high accuracy of 0.8068 and Normalized Root Mean Square Error (NRMSE) of 0.1799, which outperforms previous existing approaches. Future research work can try to improve the automatic grading by the training of the indicators of the approach depending on the MOOCs or the combination with text mining techniques.</abstract><cop>Basel</cop><pub>MDPI AG</pub><doi>10.3390/su13169364</doi><orcidid>https://orcid.org/0000-0003-0835-1414</orcidid><orcidid>https://orcid.org/0000-0002-3082-0814</orcidid><orcidid>https://orcid.org/0000-0002-1047-5398</orcidid><orcidid>https://orcid.org/0000-0002-2552-4674</orcidid><orcidid>https://orcid.org/0000-0003-4093-3705</orcidid><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 2071-1050 |
| ispartof | Sustainability, 2021-08, Vol.13 (16), p.9364 |
| issn | 2071-1050 2071-1050 |
| language | eng |
| recordid | cdi_gale_incontextgauss_ISR_A811449427 |
| source | MDPI - Multidisciplinary Digital Publishing Institute; EZB Electronic Journals Library |
| subjects | Accuracy Algorithms Comparative analysis Data mining Education Electronic discussion groups Evaluation Linguistics Massive open online courses Online instruction Reputations Social aspects Students Sustainability Technology application |
| title | Evaluation of an Algorithm for Automatic Grading of Forum Messages in MOOC Discussion Forums |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-17T19%3A22%3A55IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-gale_proqu&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Evaluation%20of%20an%20Algorithm%20for%20Automatic%20Grading%20of%20Forum%20Messages%20in%20MOOC%20Discussion%20Forums&rft.jtitle=Sustainability&rft.au=Perez-Nicolas,%20Raquel%20L&rft.date=2021-08-01&rft.volume=13&rft.issue=16&rft.spage=9364&rft.pages=9364-&rft.issn=2071-1050&rft.eissn=2071-1050&rft_id=info:doi/10.3390/su13169364&rft_dat=%3Cgale_proqu%3EA811449427%3C/gale_proqu%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2582939857&rft_id=info:pmid/&rft_galeid=A811449427&rfr_iscdi=true |