Measuring the reliability of student mastery classifications at multiple levels of reporting

April 1, 2018


Methods for estimating reliability for aggregated mastery classifications.


April 15, 2018


2:45 PM – 3:00 PM


New York, NY


National Council on Measurement in Education Annual Meeting

As the use of diagnostic assessment systems transitions from research applications to large-scale assessments for accountability purposes, reliability methods that provide evidence at each level of reporting must are needed. The purpose of this paper is to summarize one simulation-based method for estimating and reporting reliability for an operational, large-scale, diagnostic assessment system. This assessment system reports the results and associated reliability evidence at the individual skill level for each academic content standard and broader content strands. The system also summarizes results for the overall subject using achievement levels, which are often included in state accountability metrics. Results are summarized as measures of association between true and estimated mastery status for each level of reporting.

Posted on:
April 1, 2018
1 minute read, 115 words
talk conference psychometrics
See Also:
A Simulated Retest Method for Estimating Classification Reliability
Technical Evidence for Diagnostic Assessments
ATLAS Report Branding