Improving the quality of ability estimates through multidimensional scoring and incorporation of ancillary variables

Abstract

For one reason or another, various sources of information, namely, ancillary variables and correlational structure of the latent abilities, which are usually available in most testing situations, are ignored in ability estimation. A general model that incorporates these sources of information is proposed in this article. The model has a general formulation that allows incorporation of either source or both sources of information in scoring the examinees using various item response models and subsumes the traditional method of expected a posteriori as a special case. Results show that using the different sources of information singly or simultaneously provides better ability estimates (i.e., higher correlation with the true abilities and smaller posterior variance and mean squared error). The optimal condition occurs when several short tests measuring highly correlated abilities that also correlate highly with the covariates are used. Markov chain Monte Carlo parameter estimation algorithms corresponding to the different model formulations are also developed. Simulated and actual data are analyzed to establish the usefulness and feasibility of the proposed models. Several practical considerations in using these models are also discussed. © 2009 SAGE Publications.