Empirical Methods for Evaluating Maps

Illustrations and Results

April 1, 2019

Abstract

Methods for using empirical data to validate a proposed learning map structure.

Date

April 6, 2019

Time

4:10 PM – 6:10 PM

Location

Toronto, Canada

Event

National Council on Measurement in Education Annual Meeting

Learning progressions and learning map structures are increasingly being used as the basis for the design of large-scale assessments. Of critical importance to these designs is the validity of the map structure used to build the assessments. Most commonly, evidence for the validity of a map structure comes from procedural evidence gathered during the learning map creation process (e.g., research literature, external reviews, etc.). However, it is also important to provide support for the validity of the map structure with empirical evidence using data gathered from the assessment. In this paper, we propose a framework for the empirical validation of learning maps and progressions using diagnostic classification models. Three methods are proposed within this framework that provide different levels of model assumptions and types of inferences. The framework is then applied to the Dynamic Learning Maps® (DLM®) alternate assessment system to illustrate the utility and limitations of each method. Results show that each of the proposed methods have some limitations, but are able to provide complementary information for the evaluation of the proposed structure of content standards (Essential Elements) in the DLM assessment.

Posted on:
April 1, 2019
Length:
1 minute read, 183 words
Categories:
talk conference psychometrics
Tags:
DLM
See Also:
Technical Evidence for Diagnostic Assessments
ATLAS Report Branding
Psychometric Considerations for Learning Maps-Based Assessments