Assessing model fit for the Dynamic Learning Maps alternate assessment using a Bayesian estimation

Abstract

As diagnostic classification models become more widely used in operational assessments, it is important to be able effectively evaluate model fit. In this paper, we examine a new method for evaluating model fit using Bayesian model estimation and posterior predictive model checks in the context of the Dynamic Learning Maps® Alternate Assessment System. Our findings suggest that posterior predictive model checks are a methodologically sound way to estimate model fit. However, more work is needed to understand the sensitivity and specificity of these indices. Additionally, more work is needed to evaluate practical significance of model misfit, compared to the statistical significance.

Date