On Bayesian Analysis of Multirater Ordinal Data: An Application to Automated Essay Grading Academic Article

abstract

• A framework is proposed for the analysis of ordinal categorical data when ratings from several judges are available. I emphasize the tasks of estimating latent trait characteristics of individual items, regressing these latent traits on observed covariates, and comparing the performance of raters. The model is illustrated in the design and evaluation of an automated essay grader. This grader is based on a regression of variables, obtained from a grammar checker, on essay scores estimated from a panel of experts. The performance of the grader is evaluated relative to human graders, and implications on the reliability and repeatability of both automated and human raters is investigated. Copyright 1996 Taylor & Francis Group, LLC.

authors

• Johnson, Valen

published proceedings

• Journal of the American Statistical Association

author list (cited authors)

• Johnson, V. E.

citation count

• 58

complete list of authors

• Johnson, Valen E

publication date

• January 1996

publisher

• Taylor & Francis  Publisher

published in

• JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION  Journal

Digital Object Identifier (DOI)

• 10.1080/01621459.1996.10476662

start page

• 42

end page

• 51

volume

• 91

issue

• 433

URL

• http%3A%2F%2Fdx.doi.org%2F10.1080%2F01621459.1996.10476662