A Q-matrix validation method for the sequential GDINA model

Abstract

Cognitive diagnosis models (CDMs) have received increasing attention recently. A central component for most CDMs is the Q-matrix (Tatsuoka, 1983), which specifies the association between items and attributes of interest. The development of the Q-matrix is typically based on expert judgment; however, this process tends to be subjective and therefore, is prone to errors. Studies have found that the misspecifications in the Q-matrix may lead to inaccurate attribute classifications. To address this issue, a number of Q-matrix validation approaches aiming to detect and correct the misspecifications in the Q-matrix have been developed in the literature. Nevertheless, until now, there is no Q-matrix validation method available for the CDMs developed for polytomously scored items. Recently, a sequential generalized deterministic inputs, noisy and gate model (sequential GDINA; Ma, de la Torre, & Sun, 2015) has been developed for graded responses. It assumes that item categories are attained in a sequential manner, and that attributes are associated with item categories explicitly. In this study, a Qmatrix validation approach using the Wald test in a forward manner was proposed for this model, to detect and correct the misspecifications in the attribute and category association. Various conditions including sample sizes, item quality, and the proportion of misspecification were controlled in this study. The falsenegative and false-positive rates were examined. Preliminary results showed that the proposed Q-matrix validation method had low false negative rates. The false positive rates were also low even when 10% elements in the Q-matrix were mis-specified.