Assessing the Validity and Reliability of a Rubric-Based Assessment

Abstract

This paper presents a self-evaluation study of the Final Year Project II (FYPII) report rubric, examining its validity, reliability, and effectiveness in supporting student learning. While rubrics are widely used in education, concerns remain about their fairness and consistency in subjective assessment. This study employed a validity checklist, an explicitness check, an interrater survey, and an awareness survey to evaluate the rubric’s performance. Although it met all validity criteria, ambiguous terms affected reliability. The interrater survey revealed discrepancies among raters, leading to the proposal of four rating rules to enhance consistency. Despite facilitating learning, the rubric showed a gap between students’ understanding of the criteria and their ability to produce quality work. Effective supervisor supervision was identified as crucial in bridging this gap. These findings highlight that a well-designed rubric alone is insufficient—proper implementation is essential to ensure meaningful assessment without overburdening evaluators.