Selected Publications

As the use of diagnostic assessment systems transitions from research applications to large-scale assessments for accountability purposes, reliability methods that provide evidence at each level of reporting must are needed. The purpose of this paper is to summarize one simulation-based method for estimating and reporting reliability for an operational, large-scale, diagnostic assessment system. This assessment system reports the results and associated reliability evidence at the individual skill level for each academic content standard and broader content strands. The system also summarizes results for the overall subject using achievement levels, which are often included in state accountability metrics. Results are summarized as measures of association between true and estimated mastery status for each level of reporting.
NCME

This graphic shows the probability of providing a correct response to an item in a multidimensional item response theory (MIRT) model. In the 1PL, 2PL, and 3PL MIRT models, the choice of compensatory parameterizations can greatly affect the probability of a correct response.
EM:IP

Recent Posts

More Posts

A while back, I was inspired by this Twitter exchange to create a bot that would tweet out tidyverse related material. Armchair Business Man Suggestion: Ever thought about having a Twitter account that automatocally tweets each new topic accompanied with link to said post ala Reddit? @mxlearn and other forums? Would be super useful for the #rstats commune. — Sudo | Science📊 (@AgentZeroNine) November 14, 2017 Last week I finally had enough time to sit and put some work in to get this idea up and running!

CONTINUE READING

This time last year, I submitted a graphic to the Educational Measurement: Issues and Practice (EM:IP) cover showcase competition. In April at the annual National Council on Measurement in Education conference, it was announced that I was one of four winners that would be featured on the cover of EM:IP this year. Earlier this week, the issue with my graphic was released! The graphic demonstrates how different levels of compensation in multidimensional item response theory models (MIRT).

CONTINUE READING

This is the final post in the tidy sports analytics series, in which I’ve been using play-by-play from the 2016 NFL season to demonstrate the power of the tidyverse. Previously, I’ve discussed: Part 1: Data manipulation using dplyr; Part 2: Data reshaping and tidying using tidyr; Part 3: Data visualization using ggplot2. This post doesn’t feature any new data analysis. Instead, I want to use this last post to talk about the tidyverse more generally and cover some of other advantages of using these packages for data analysis.

CONTINUE READING

Projects

Evaluating Model Estimation Processes for Diagnostic Classification Models

PhD dissertation submitted to the University of Kansas

jayhawkdown

Template for writing theses at the University of Kansas using rmarkdown and bookdown.

Soccer predictions using Bayesian mixed effects models

Models developed for the prediction of individual games, European domestic leagues, and the UEFA Champions League.

Publications

More Publications

  • Using simulation to evaluate retest reliability of diagnostic assessment results

    Details

  • Measuring the reliability of diagnostic mastery classifications at multiple levels of reporting

    Details

  • A hierarchical model for identifying group-level aberrant growth to detect cheating

    Details

  • Construct Irrelevant Variance

    Details PDF

  • Visualizing different levels of compensation in multidimensional item response theory models

    Details PDF PNG Cover

Recent & Upcoming Talks

More Talks