An optimal implementation of the GDI Q-matrix validation method

Abstract

In the context of cognitive diagnosis models, a Q-matrix reflects the correspondence between attributes and items. The construction of this Q-matrix is typically theoretically based, most of the time relying on domain experts. This approach is subjective and may lead to misspecifications in the Q-matrix. All this will negatively affect the attribute classification accuracy. In response, several methods of empirical Q-matrix validation have been developed with the aim of correcting misspecified entries in a Q-matrix. One of these methods is the general discrimination index (GDI) method proposed by de la Torre and Chiu (2016). All items with a proportion of variance accounted for (PVAF) lower than a predetermined cutoff for PVAF are modified. In this method, the Q-matrix is assumed to be correct. This assumption is likely to be violated by the problems indicated above. A possible solution to this problem is to apply the method iteratively. Hence, this study investigates the iterative application of the GDI method where only one item is modified at each step of the iterative procedure, and the cutoff for PVAF is updated considering the new parameter estimates. The performance of this implementation of the GDI method was assessed by means of Monte Carlo simulations. Results showed that the performance of the GDI method improved when the application was iterative at the item level and was used in conjunction with an appropriate cutoff point. This was more noticeable when the original Q-matrix misspecification rate was high.