MINIMIZING RATERS’ BIAS IN ASSESSING WRITING PERFORMANCE

Measuring writing performance, which is usually subjective can increase many interpretations which may not the same to the constructs. This problem is overcome with inter-rater reliability among raters to get an agreement which not always relating to the construct of the test and surely threaten their rating task unreliable. One of the threats of the construct validity is construct-irrelevant variance which includes dimensions beyond the construct and enables rater to measure the test taker's response too stringent or lenient. This is called bias because it distorts the test results and therefore the conclusion of the test taker's response based on scores is less valid. Rater bias can be caused by the halo effect that raters do not distinguish between different aspects of a composition. Another frequent bias is raters using middle of the scale. They are wary to use the two ends of the scale of the scoring rubric criteria. These errors are usually committed by novice raters. During rating task, they attempted to judge the criteria while that they are reading. However, the expert raters do not focus so much on certain things when as they read, but allow them to communicate with the texts on a more personal level. Then they evaluate the text more generally as a whole after they finish reading. Thus, they need to form a rater training to help novice developing a sense of the standards, as well as developing ways of approaching the rating task. Rater training also helps to improve intra-rater rather than inter-rater reliability and there is a correlation between training/experience and rating task. An Authentic rater training introduced by Kim (2016) which is characterized as a rater-¬centered bottom¬-up approach can be applied.