Cueing effects of item writing flaws in multiple-choice questions

Abstract

Background: Multiple-choice questions are frequently used in high-stakes assessments across health science disciplines. Many test items, however, contain cues to the correct answer allowing students without the requisite knowledge to correctly answer the test item. The purpose of this study was to examine the cueing effect of five common item-writing flaws: word repeats in the question stem and the correct option, the longest option is correct, use of absolute terms in the options, use of ‘all of the above’ and use of ‘none of the above’ as options. Summary of work: We reviewed 3623 test items used in one school of nursing over a 7-year period. Questions were evaluated for 19 frequently occurring item-writing flaws, including five of the above identified flaws. We compared the proportion correct and the item discrimination indices of the flawed items with unflawed items. Summary of results: Items containing the identified item-writing flaws were significantly less difficult and less discriminating than unflawed items. Students were more likely to select the correct answer in questions that contained cues. Conclusions: Cueing in multiple-choice questions is common and the presence of cueing flaws in test items would enhance student guessing on multiple-choice tests. Take-home messages: Adequate training in item-writing is recommended for all faculty members who are responsible for developing tests. Peer review prior to test administration to identify cuing flaws may also improve the quality of test items and reduce cueing effects.