I am thrilled to announce the release of measr 1.0.0. The goal of measr is to provide a user-friendly interface for estimating and evaluating diagnostic classification models (DCMs; also called cognitive diagnostic models [CDMs]). This is a major release to mark the conclusion of the initial development work that was funded by the Institute of Education Sciences. Importantly, this does not mean that measr is going dormant! We are still actively developing measr, so we’ll continue to add new features and respond to bug reports.
You can install the updated version of measr from CRAN with:
install.packages("measr")This blog post will highlight the major improvements, including improved documentation and vignettes, as well as a some minor updates.
Improved Documentation
The focus of recent work on measr has been to improve documentation. To that end, the existing vignettes have been updated, and several new vignettes have been added.
First, the getting started vignette has been updated to include additional installation information. This should help reduce the friction for setting up the components needed for measr, namely, installing Stan. Because measr interfaces with Stan to estimate DCMs, a working Stan installation is needed for the package to function properly. The updated vignette includes helpful guidance for installing both the rstan and cmdstanr packages for using Stan. Only one of the two is required, but both are supported so you can choose whichever Stan front end you prefer.
A new vignette on model evaluation has been added to demonstrate different methods for assessing the performance of a DCM after it has been estimated. This includes the M2 statistic (Liu et al., 2016), as well as posterior predictive model checks for evaluating both overall and item-level model fit (Sinharay & Almond, 2007; Thompson, 2019). To demonstrate how these methods are implemented for measr, a simulated data set is used where we know how well different types of DCMs should fit. The existing model estimation vignette was also updated to use this same simulated data set so that we can compare parameter estimates derived from measr to the known parameter values that were used to generate the data.
Because the model estimation and evaluation vignettes now use simulated data, we also added a case study to walk through a DCM-based analysis from start to finish using a real data set. In this case study, data from the Examination for the Certificate of Proficiency in English (ECPE), which has been widely used in the DCM literature (e.g., Templin & Hoffman, 2013). We start with exploratory analyses, then we estimate a DCM, examine parameter estimates, and interpret model fit and reliability analyses.
Finally, example uses have been added for all functions included in measr. These examples can be found on the documentation pages for each function. All functions are indexed on the reference page, which has been reorganized to group functions with related functionality.
Minor Changes
There were also a number of minor improvements:
- The Stan code for estimating models has been updated to be compatible with the new array syntax.
- We published an article on measr in the Journal of Open Source Software (Thompson, 2023), and the preferred citation when using measr has been updated.
For a complete list of changes, check out the changelog.
Additional Resources
If you are interested in learing more about measr and diagnostic models, I taught a workshop in June at StanCon 2023. The workshop provided an overview of diagnostic models and their use cases and then walked through how to estimate and evaluate different models using measr. All of the workshop materials are available online, including slides, exercises, and solutions.
Finally, if you are interested in attending a workshop in person, I’ll be teaching another workshop in April at the 2024 annual meeting of the National Conference on Measurement in Education (NCME). The materials from this workshop will also be made available online for those who are unable to attend. For more information on the conference and register, see the conference website.
Acknowledgments
The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305D210045 to the University of Kansas. The opinions expressed are those of the authors and do not represent the views of the the Institute or the U.S. Department of Education.
Featured photo by patricia serna on Unsplash.