Improving Peer Assessment Validity and Reliability Through a Fuzzy Coherence Measure

Classical evaluation methods, assessments, exams, and so forth accentuate the perception of one against all, professor versus learners. Including students in the assessment process, allows transforming the professor from an opponent to a critical friend, with the role of helping students to recognize both their strengths and weaknesses. However, assessing peers is not as easy as it may appear, since reliability and validity issues still represent a major concern. To improve reliability, this study proposes a consensus-reaching measure that reduces the impact of inconsistent evaluations (harsh or overvalued) through minimizing iteratively the sum of weighted incoherencies. Furthermore, it improves validity by computing the similarity of each assessment to the professor's evaluation. Moreover, it is easier for students to assess with linguistic grades "Good," "Fair," etc., than with classical numbers, hence fuzzy logic is used to encompass the inherent uncertainty of students' assessments. This pilot study included 23 students in the 4th year project management course where learners assessed their peers' oral presentations. Results suggest that the used measures permit achieving high reliability even with students from different cultures.