Assessment of Differential Statement Functioning in Ipsative Tests with Multidimensional Forced-Choice Items

Abstract

Ipsative tests with multidimensional forced-choice (MFC) items have been widely used to assess career interest, values, and personality to prevent response biases. Recently, there has been a surge of interest in developing item response theory models for MFC items. In reality, a statement in an MFC item may have different utilities for different groups, which is referred to as differential statement functioning (DSF). However, few studies have been investigated methods for detecting DSF owing to the challenges related to the features of ipsative tests. In this study, three methods were adapted for DSF assessment in MFC items: equal-mean-utility (EMU), all-other-statement (AOS), and constant-statement (CS). Simulation studies were conducted to evaluate the recovery of parameters and the performance of the proposed methods. Results showed that statement parameters and DSF parameters were well recovered for all the three methods when the test did not contain any DSF statement. When the test contained one or more DSF statements, only the CS method yielded accurate estimates. With respect to DSF assessment, both the EMU method using the bootstrap standard error and the AOS method performed appropriately so long as the test did not contain any DSF statement. The CS method performed well in cases where one or more DSF-free statements were chosen as an anchor. The longer the anchor statements, the higher the power of DSF detection.