File Download

There are no files associated with this item.

  Links for fulltext
     (May Require Subscription)
Supplementary

Article: Validity and Reliability of Situational Judgement Test Scores: A New Approach Based on Cognitive Diagnosis Models

TitleValidity and Reliability of Situational Judgement Test Scores: A New Approach Based on Cognitive Diagnosis Models
Authors
Keywordscognitive diagnosis models
Issue Date2016
Citation
Organizational Research Methods, 2016, v. 19, n. 3, p. 506-532 How to Cite?
Abstract© The Author(s) 2016.Conventional methods for assessing the validity and reliability of situational judgment test (SJT) scores have proven to be inadequate. For example, factor analysis techniques typically lead to nonsensical solutions, and assumptions underlying Cronbach’s alpha coefficient are violated due to the multidimensional nature of SJTs. In the current article, we describe how cognitive diagnosis models (CDMs) provide a new approach that not only overcomes these limitations but that also offers extra advantages for scoring and better understanding SJTs. The analysis of the Q-matrix specification, model fit, and model parameter estimates provide a greater wealth of information than traditional procedures do. Our proposal is illustrated using data taken from a 23-item SJT that presents situations about student-related issues. Results show that CDMs are useful tools for scoring tests, like SJTs, in which multiple knowledge, skills, abilities, and other characteristics are required to correctly answer the items. SJT classifications were reliable and significantly related to theoretically relevant variables. We conclude that CDM might help toward the exploration of the nature of the constructs underlying SJT, one of the principal challenges in SJT research.
Persistent Identifierhttp://hdl.handle.net/10722/228249
ISSN
2015 Impact Factor: 4.727
2015 SCImago Journal Rankings: 5.705

 

DC FieldValueLanguage
dc.contributor.authorSorrel, Miguel A.-
dc.contributor.authorOlea, Julio-
dc.contributor.authorAbad, Francisco J.-
dc.contributor.authorde la Torre, Jimmy-
dc.contributor.authorAguado, David-
dc.contributor.authorLievens, Filip-
dc.date.accessioned2016-08-01T06:45:34Z-
dc.date.available2016-08-01T06:45:34Z-
dc.date.issued2016-
dc.identifier.citationOrganizational Research Methods, 2016, v. 19, n. 3, p. 506-532-
dc.identifier.issn1094-4281-
dc.identifier.urihttp://hdl.handle.net/10722/228249-
dc.description.abstract© The Author(s) 2016.Conventional methods for assessing the validity and reliability of situational judgment test (SJT) scores have proven to be inadequate. For example, factor analysis techniques typically lead to nonsensical solutions, and assumptions underlying Cronbach’s alpha coefficient are violated due to the multidimensional nature of SJTs. In the current article, we describe how cognitive diagnosis models (CDMs) provide a new approach that not only overcomes these limitations but that also offers extra advantages for scoring and better understanding SJTs. The analysis of the Q-matrix specification, model fit, and model parameter estimates provide a greater wealth of information than traditional procedures do. Our proposal is illustrated using data taken from a 23-item SJT that presents situations about student-related issues. Results show that CDMs are useful tools for scoring tests, like SJTs, in which multiple knowledge, skills, abilities, and other characteristics are required to correctly answer the items. SJT classifications were reliable and significantly related to theoretically relevant variables. We conclude that CDM might help toward the exploration of the nature of the constructs underlying SJT, one of the principal challenges in SJT research.-
dc.languageeng-
dc.relation.ispartofOrganizational Research Methods-
dc.subjectcognitive diagnosis models-
dc.titleValidity and Reliability of Situational Judgement Test Scores: A New Approach Based on Cognitive Diagnosis Models-
dc.typeArticle-
dc.description.natureLink_to_subscribed_fulltext-
dc.identifier.doi10.1177/1094428116630065-
dc.identifier.scopuseid_2-s2.0-84974527970-
dc.identifier.volume19-
dc.identifier.issue3-
dc.identifier.spage506-
dc.identifier.epage532-
dc.identifier.eissn1552-7425-

Export via OAI-PMH Interface in XML Formats


OR


Export to Other Non-XML Formats