Designing and implementing standards based, summative competency tests

Abstract

Assessment of clinical skills competency remains critical to progress determination and gate-keeping responsibilities of dental education providers. In apprenticeship models of clinical practice the novice learns by observation, practice, trial and error and the expert guides and provides specific and in the moment feedback or grading. Much, however, remains fuzzy and implicit in such judgment frameworks with the instability of expert and novice judgment processes only becoming apparent in high-stakes assessments (Wyatt-Smith & Klenowski, 2012). Undergraduate dental students (n = 24) and clinical teachers (n = 6) consented to interviews on formative and summative clinical skills assessments. Thematic analysis indicated areas of difference between student and staff views. Staff found explicit assessment criteria distributed prior to formative clinical assessments to be useful for consistency of task expectations. Despite these, for summative assessments, holistic judgments were influenced by prior knowledge of the examinee. Non-task specific factors such as knowledge of the student’s ability, amount of experience in the clinical skill were indicated as influencing these holistic judgments. Students found task specifications to be clear; however, tutor judgments were perceived as sometimes variable, even amongst the same task and with double marking. The role of standards based specifications for individual clinical skills procedures was indicated as helpful in bringing both the novice closer to understanding the subtle and nuanced decision-making process that their expert assessors apply, and in giving assessors more guidance when making summative decision making. The aim of this presentation is to outline with outcomes for: defining the scope of summative assessments, the process of writing and creating standards descriptors for competency testing including, what and when to asses, moderation of standards and implementation to stakeholders. Implications will be drawn for reconsidering clinical assessments to resolve the tension between generic outcomes-based approaches and behaviorist, skill-based competencies.