Rater variation in pragmatic assessment: The impact of the linguistic background on peer-assessment and self-assessment
The present study investigates variability among raters from different linguistic backgrounds, who evaluated the pragmatic performance of English language learners with varying native languages (L1s) by using both self- and peer-assessments. To this end, written discourse completion task (WDCT) samp...
Gespeichert in:
| Veröffentlicht in: | Lodz papers in pragmatics 2020-07, Vol.16 (1), p.67-85 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 85 |
|---|---|
| container_issue | 1 |
| container_start_page | 67 |
| container_title | Lodz papers in pragmatics |
| container_volume | 16 |
| creator | Sonnenburg-Winkler, Sunni L. Eslami, Zohreh R. Derakhshan, Ali |
| description | The present study investigates variability among raters from different linguistic backgrounds, who evaluated the pragmatic performance of English language learners with varying native languages (L1s) by using both self- and peer-assessments. To this end, written discourse completion task (WDCT) samples of requesting speech acts from 10 participants were collected. Thereafter, the participants were asked to assess their peers’ WDCTs before assessing their own samples using the same rating scale. The raters were further asked to provide an explanation for their rating decisions. Findings indicate that there may indeed be a link between a rater’s language background and their scoring patterns, although the results regarding peer- and self-assessment are mixed. There are both similarities and differences in the participants’ use of pragmatic norms and social rules in evaluating appropriateness. |
| doi_str_mv | 10.1515/lpp-2020-0004 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2451525271</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2451525271</sourcerecordid><originalsourceid>FETCH-LOGICAL-c2317-b1bb17a75563817e0b3d595e1b559fc8933c305e1c698c74d8111557f95fdcea3</originalsourceid><addsrcrecordid>eNptkM9LwzAYhoMoOOaO3gOeo_maJml3k-EvGAgyzyFN0xrt2pq0jv33pk6YB0_53o_newMPQpdAr4EDv2n6niQ0oYRSmp6gGWR5RtKUidOfmRMBVJyjRQiuoFQkmZSZmKHdix6sx1_aOz24rsWuxb3X9TYmg3UINoStbYcl3rxZ7La9NgPuKjzE1Li2Hl2YwEKbj9p3Y1vi2NFb68nxFuu4Drap_uwu0Fmlm2AXv-8cvd7fbVaPZP388LS6XROTMJCkgKIAqSXngmUgLS1YyXNuoeA8r0yWM2YYjdmIPDMyLTMA4FxWOa9KYzWbo6tDb--7z9GGQb13o2_jlypJo7eEJxIiRQ6U8V0I3laq926r_V4BVZNeFfWqSa-a9EZ-eeB3uon6Slv7cR-HY_m_dyBASPYNxvuCPQ</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2451525271</pqid></control><display><type>article</type><title>Rater variation in pragmatic assessment: The impact of the linguistic background on peer-assessment and self-assessment</title><source>De Gruyter journals</source><creator>Sonnenburg-Winkler, Sunni L. ; Eslami, Zohreh R. ; Derakhshan, Ali</creator><creatorcontrib>Sonnenburg-Winkler, Sunni L. ; Eslami, Zohreh R. ; Derakhshan, Ali</creatorcontrib><description>The present study investigates variability among raters from different linguistic backgrounds, who evaluated the pragmatic performance of English language learners with varying native languages (L1s) by using both self- and peer-assessments. To this end, written discourse completion task (WDCT) samples of requesting speech acts from 10 participants were collected. Thereafter, the participants were asked to assess their peers’ WDCTs before assessing their own samples using the same rating scale. The raters were further asked to provide an explanation for their rating decisions. Findings indicate that there may indeed be a link between a rater’s language background and their scoring patterns, although the results regarding peer- and self-assessment are mixed. There are both similarities and differences in the participants’ use of pragmatic norms and social rules in evaluating appropriateness.</description><identifier>ISSN: 1895-6106</identifier><identifier>EISSN: 1898-4436</identifier><identifier>DOI: 10.1515/lpp-2020-0004</identifier><language>eng</language><publisher>Lodz: De Gruyter</publisher><subject>English as a second language learning ; English language learners ; Language assessment ; linguistic background ; Measures ; Peer assessment ; Pragmatics ; rater variation ; Self evaluation ; self-assessment ; Speech acts</subject><ispartof>Lodz papers in pragmatics, 2020-07, Vol.16 (1), p.67-85</ispartof><rights>2020 Walter de Gruyter GmbH, Berlin/Boston</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c2317-b1bb17a75563817e0b3d595e1b559fc8933c305e1c698c74d8111557f95fdcea3</citedby><cites>FETCH-LOGICAL-c2317-b1bb17a75563817e0b3d595e1b559fc8933c305e1c698c74d8111557f95fdcea3</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://www.degruyter.com/document/doi/10.1515/lpp-2020-0004/pdf$$EPDF$$P50$$Gwalterdegruyter$$H</linktopdf><linktohtml>$$Uhttps://www.degruyter.com/document/doi/10.1515/lpp-2020-0004/html$$EHTML$$P50$$Gwalterdegruyter$$H</linktohtml><link.rule.ids>314,776,780,27901,27902,66497,68281</link.rule.ids></links><search><creatorcontrib>Sonnenburg-Winkler, Sunni L.</creatorcontrib><creatorcontrib>Eslami, Zohreh R.</creatorcontrib><creatorcontrib>Derakhshan, Ali</creatorcontrib><title>Rater variation in pragmatic assessment: The impact of the linguistic background on peer-assessment and self-assessment</title><title>Lodz papers in pragmatics</title><description>The present study investigates variability among raters from different linguistic backgrounds, who evaluated the pragmatic performance of English language learners with varying native languages (L1s) by using both self- and peer-assessments. To this end, written discourse completion task (WDCT) samples of requesting speech acts from 10 participants were collected. Thereafter, the participants were asked to assess their peers’ WDCTs before assessing their own samples using the same rating scale. The raters were further asked to provide an explanation for their rating decisions. Findings indicate that there may indeed be a link between a rater’s language background and their scoring patterns, although the results regarding peer- and self-assessment are mixed. There are both similarities and differences in the participants’ use of pragmatic norms and social rules in evaluating appropriateness.</description><subject>English as a second language learning</subject><subject>English language learners</subject><subject>Language assessment</subject><subject>linguistic background</subject><subject>Measures</subject><subject>Peer assessment</subject><subject>Pragmatics</subject><subject>rater variation</subject><subject>Self evaluation</subject><subject>self-assessment</subject><subject>Speech acts</subject><issn>1895-6106</issn><issn>1898-4436</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><sourceid>8G5</sourceid><sourceid>BENPR</sourceid><sourceid>GUQSH</sourceid><sourceid>M2O</sourceid><recordid>eNptkM9LwzAYhoMoOOaO3gOeo_maJml3k-EvGAgyzyFN0xrt2pq0jv33pk6YB0_53o_newMPQpdAr4EDv2n6niQ0oYRSmp6gGWR5RtKUidOfmRMBVJyjRQiuoFQkmZSZmKHdix6sx1_aOz24rsWuxb3X9TYmg3UINoStbYcl3rxZ7La9NgPuKjzE1Li2Hl2YwEKbj9p3Y1vi2NFb68nxFuu4Drap_uwu0Fmlm2AXv-8cvd7fbVaPZP388LS6XROTMJCkgKIAqSXngmUgLS1YyXNuoeA8r0yWM2YYjdmIPDMyLTMA4FxWOa9KYzWbo6tDb--7z9GGQb13o2_jlypJo7eEJxIiRQ6U8V0I3laq926r_V4BVZNeFfWqSa-a9EZ-eeB3uon6Slv7cR-HY_m_dyBASPYNxvuCPQ</recordid><startdate>20200728</startdate><enddate>20200728</enddate><creator>Sonnenburg-Winkler, Sunni L.</creator><creator>Eslami, Zohreh R.</creator><creator>Derakhshan, Ali</creator><general>De Gruyter</general><general>Walter de Gruyter GmbH</general><scope>AAYXX</scope><scope>CITATION</scope><scope>3V.</scope><scope>7T9</scope><scope>7XB</scope><scope>88I</scope><scope>8FK</scope><scope>8G5</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AIMQZ</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>GNUQQ</scope><scope>GUQSH</scope><scope>HCIFZ</scope><scope>LIQON</scope><scope>M2O</scope><scope>M2P</scope><scope>MBDVC</scope><scope>PADUT</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>Q9U</scope></search><sort><creationdate>20200728</creationdate><title>Rater variation in pragmatic assessment: The impact of the linguistic background on peer-assessment and self-assessment</title><author>Sonnenburg-Winkler, Sunni L. ; Eslami, Zohreh R. ; Derakhshan, Ali</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c2317-b1bb17a75563817e0b3d595e1b559fc8933c305e1c698c74d8111557f95fdcea3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>English as a second language learning</topic><topic>English language learners</topic><topic>Language assessment</topic><topic>linguistic background</topic><topic>Measures</topic><topic>Peer assessment</topic><topic>Pragmatics</topic><topic>rater variation</topic><topic>Self evaluation</topic><topic>self-assessment</topic><topic>Speech acts</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Sonnenburg-Winkler, Sunni L.</creatorcontrib><creatorcontrib>Eslami, Zohreh R.</creatorcontrib><creatorcontrib>Derakhshan, Ali</creatorcontrib><collection>CrossRef</collection><collection>ProQuest Central (Corporate)</collection><collection>Linguistics and Language Behavior Abstracts (LLBA)</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>Science Database (Alumni Edition)</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>Research Library (Alumni Edition)</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>ProQuest One Literature</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>ProQuest Central Student</collection><collection>Research Library Prep</collection><collection>SciTech Premium Collection</collection><collection>ProQuest One Literature - U.S. Customers Only</collection><collection>Research Library</collection><collection>Science Database</collection><collection>Research Library (Corporate)</collection><collection>Research Library China</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central Basic</collection><jtitle>Lodz papers in pragmatics</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Sonnenburg-Winkler, Sunni L.</au><au>Eslami, Zohreh R.</au><au>Derakhshan, Ali</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Rater variation in pragmatic assessment: The impact of the linguistic background on peer-assessment and self-assessment</atitle><jtitle>Lodz papers in pragmatics</jtitle><date>2020-07-28</date><risdate>2020</risdate><volume>16</volume><issue>1</issue><spage>67</spage><epage>85</epage><pages>67-85</pages><issn>1895-6106</issn><eissn>1898-4436</eissn><abstract>The present study investigates variability among raters from different linguistic backgrounds, who evaluated the pragmatic performance of English language learners with varying native languages (L1s) by using both self- and peer-assessments. To this end, written discourse completion task (WDCT) samples of requesting speech acts from 10 participants were collected. Thereafter, the participants were asked to assess their peers’ WDCTs before assessing their own samples using the same rating scale. The raters were further asked to provide an explanation for their rating decisions. Findings indicate that there may indeed be a link between a rater’s language background and their scoring patterns, although the results regarding peer- and self-assessment are mixed. There are both similarities and differences in the participants’ use of pragmatic norms and social rules in evaluating appropriateness.</abstract><cop>Lodz</cop><pub>De Gruyter</pub><doi>10.1515/lpp-2020-0004</doi><tpages>19</tpages></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 1895-6106 |
| ispartof | Lodz papers in pragmatics, 2020-07, Vol.16 (1), p.67-85 |
| issn | 1895-6106 1898-4436 |
| language | eng |
| recordid | cdi_proquest_journals_2451525271 |
| source | De Gruyter journals |
| subjects | English as a second language learning English language learners Language assessment linguistic background Measures Peer assessment Pragmatics rater variation Self evaluation self-assessment Speech acts |
| title | Rater variation in pragmatic assessment: The impact of the linguistic background on peer-assessment and self-assessment |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-10T09%3A05%3A49IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Rater%20variation%20in%20pragmatic%20assessment:%20The%20impact%20of%20the%20linguistic%20background%20on%20peer-assessment%20and%20self-assessment&rft.jtitle=Lodz%20papers%20in%20pragmatics&rft.au=Sonnenburg-Winkler,%20Sunni%20L.&rft.date=2020-07-28&rft.volume=16&rft.issue=1&rft.spage=67&rft.epage=85&rft.pages=67-85&rft.issn=1895-6106&rft.eissn=1898-4436&rft_id=info:doi/10.1515/lpp-2020-0004&rft_dat=%3Cproquest_cross%3E2451525271%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2451525271&rft_id=info:pmid/&rfr_iscdi=true |