A Hierarchical IRT Model for Identifying Group Level Aberrant Growth to Detect Cheating

Abstract

As cheating on high-stakes tests continues to be an issue for standardized testing, approaches for detecting cheating proliferate. Approaches to cheating detection vary, with common strategies being detecting unusual levels of wrong-to-right erasures (e.g., Wollack, Cohen, & Eckerly, 2015), similarity of answer patterns (Karabatsos, 2003), and aberrant improvement over time (e.g., Bishop & Egan, 2016). However, the majority of research focuses on detecting cheating at the individual level. As recent events have shown (e.g., the Atlanta cheating scandal), cheating at the group level is also a threat to the validity of decision made from scores on high-stakes standardized tests. The present study adapts the Bayesian Hierarchical Linear Model (BHLM) introduced in Skorupski & Egan (2013, 2014) and further developed in Skorupski, Fitzpatrick, and Egan (2016) to detect group-level aberrance within an IRT framework. Since many testing companies use a latent trait model to estimate examinee ability, this method may prove more compatible with operational testing programs’ current approach to scaling.

Publication
Poster presented at the Conference on Test Security, Madison, WI.
Date
Links